2023-11-06T01:11:40.270 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-06T01:11:40.271 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} --archive /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641 --name yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi --owner scheduled_yuriw@teuthology --verbose -- /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641/orig.config.yaml 2023-11-06T01:11:40.332 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-06T01:11:40.422 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641 branch: reef-release description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} email: yweinste@redhat.com first_in_suite: false job_id: '7448641' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: reef-release ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef-release sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1834 sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: reef-release suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-05_23:44:49 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3084822 2023-11-06T01:11:40.423 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa; will attempt to use it 2023-11-06T01:11:40.424 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa/tasks 2023-11-06T01:11:40.424 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-06T01:11:40.426 INFO:teuthology.task.internal:Checking packages... 2023-11-06T01:11:40.453 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5' 2023-11-06T01:11:40.454 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-06T01:11:40.455 INFO:teuthology.packaging:ref: None 2023-11-06T01:11:40.456 INFO:teuthology.packaging:tag: None 2023-11-06T01:11:40.456 INFO:teuthology.packaging:branch: reef-release 2023-11-06T01:11:40.456 INFO:teuthology.packaging:sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T01:11:40.457 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef-release 2023-11-06T01:11:40.700 INFO:teuthology.task.internal:Found packages for ceph version 18.2.0-1181.gf7e9f9af 2023-11-06T01:11:40.701 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-06T01:11:40.708 INFO:teuthology.task.internal:no buildpackages task found 2023-11-06T01:11:40.708 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-06T01:11:40.739 INFO:teuthology.task.internal:Saving configuration 2023-11-06T01:11:40.761 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-06T01:11:40.767 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T01:11:40.797 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 01:05:43.993804', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-06T01:11:40.820 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 01:05:43.995175', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-06T01:11:40.843 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 01:05:43.996383', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-06T01:11:40.844 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-06T01:11:40.850 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-11-06T01:11:40.850 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-11-06T01:11:40.851 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-11-06T01:11:40.851 INFO:teuthology.run_tasks:Running task console_log... 2023-11-06T01:11:40.975 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8276b36c10>, signals=[15]) 2023-11-06T01:11:40.977 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-06T01:11:40.983 INFO:teuthology.task.internal:Opening connections... 2023-11-06T01:11:40.984 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2023-11-06T01:11:40.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:11:41.097 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2023-11-06T01:11:41.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:11:41.165 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2023-11-06T01:11:41.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:11:41.230 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-06T01:11:41.236 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2023-11-06T01:11:41.258 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2023-11-06T01:11:41.259 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2023-11-06T01:11:41.319 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Red Hat Enterprise Linux" 2023-11-06T01:11:41.319 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8.6 (Ootpa)" 2023-11-06T01:11:41.319 INFO:teuthology.orchestra.run.smithi053.stdout:ID="rhel" 2023-11-06T01:11:41.320 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="fedora" 2023-11-06T01:11:41.320 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8.6" 2023-11-06T01:11:41.320 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2023-11-06T01:11:41.320 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-06T01:11:41.320 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2023-11-06T01:11:41.321 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-06T01:11:41.321 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.redhat.com/" 2023-11-06T01:11:41.321 INFO:teuthology.orchestra.run.smithi053.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-06T01:11:41.321 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-06T01:11:41.321 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:11:41.322 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-06T01:11:41.322 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-06T01:11:41.322 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-06T01:11:41.322 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-06T01:11:41.323 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2023-11-06T01:11:41.344 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2023-11-06T01:11:41.366 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2023-11-06T01:11:41.366 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-11-06T01:11:41.426 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Red Hat Enterprise Linux" 2023-11-06T01:11:41.426 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8.6 (Ootpa)" 2023-11-06T01:11:41.426 INFO:teuthology.orchestra.run.smithi062.stdout:ID="rhel" 2023-11-06T01:11:41.426 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="fedora" 2023-11-06T01:11:41.427 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8.6" 2023-11-06T01:11:41.427 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2023-11-06T01:11:41.427 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-06T01:11:41.427 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-11-06T01:11:41.427 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-06T01:11:41.428 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.redhat.com/" 2023-11-06T01:11:41.428 INFO:teuthology.orchestra.run.smithi062.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-06T01:11:41.428 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-06T01:11:41.428 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:11:41.428 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-06T01:11:41.428 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-06T01:11:41.429 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-06T01:11:41.429 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-06T01:11:41.429 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-11-06T01:11:41.452 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2023-11-06T01:11:41.473 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2023-11-06T01:11:41.474 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2023-11-06T01:11:41.531 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="Red Hat Enterprise Linux" 2023-11-06T01:11:41.532 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="8.6 (Ootpa)" 2023-11-06T01:11:41.532 INFO:teuthology.orchestra.run.smithi163.stdout:ID="rhel" 2023-11-06T01:11:41.532 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="fedora" 2023-11-06T01:11:41.532 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="8.6" 2023-11-06T01:11:41.533 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el8" 2023-11-06T01:11:41.533 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-06T01:11:41.533 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2023-11-06T01:11:41.533 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-06T01:11:41.533 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.redhat.com/" 2023-11-06T01:11:41.534 INFO:teuthology.orchestra.run.smithi163.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-06T01:11:41.534 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-06T01:11:41.534 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:11:41.534 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-06T01:11:41.535 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-06T01:11:41.535 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-06T01:11:41.535 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-06T01:11:41.536 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2023-11-06T01:11:41.560 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-06T01:11:41.571 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-06T01:11:41.606 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-06T01:11:41.606 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2023-11-06T01:11:41.612 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2023-11-06T01:11:41.614 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2023-11-06T01:11:41.633 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-06T01:11:41.640 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-06T01:11:41.640 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2023-11-06T01:11:41.672 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2023-11-06T01:11:41.675 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2023-11-06T01:11:41.708 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-06T01:11:41.806 INFO:teuthology.run_tasks:Running task kernel... 2023-11-06T01:11:41.851 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-11-06T01:11:41.852 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-11-06T01:11:41.853 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-11-06T01:11:41.853 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-06T01:11:41.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T01:11:41.874 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2023-11-06T01:11:41.935 INFO:teuthology.orchestra.run.smithi053.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-06T01:11:41.936 INFO:teuthology.task.kernel:Running kernel on smithi053: 4.18.0-372.9.1.el8.x86_64 2023-11-06T01:11:41.936 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum install -y kernel 2023-11-06T01:11:43.178 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-06T01:11:43.179 INFO:teuthology.orchestra.run.smithi053.stdout:Unable to read consumer identity 2023-11-06T01:11:44.760 INFO:teuthology.orchestra.run.smithi053.stdout:Copr repo for python3-asyncssh owned by ceph 11 kB/s | 3.5 kB 00:00 2023-11-06T01:11:45.444 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-11-06T01:11:49.194 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 480 kB/s | 24 kB 00:00 2023-11-06T01:11:49.913 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-06T01:11:49.973 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2023-11-06T01:11:49.974 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2023-11-06T01:11:49.975 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2023-11-06T01:11:50.097 DEBUG:teuthology.orchestra.run.smithi053:> echo no | sudo yum reinstall kernel || true 2023-11-06T01:11:50.446 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-06T01:11:50.447 INFO:teuthology.orchestra.run.smithi053.stdout:Unable to read consumer identity 2023-11-06T01:11:50.728 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:01 ago on Mon 06 Nov 2023 01:11:49 AM UTC. 2023-11-06T01:11:50.771 INFO:teuthology.orchestra.run.smithi053.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-06T01:11:50.807 INFO:teuthology.orchestra.run.smithi053.stderr:Error: No packages marked for reinstall. 2023-11-06T01:11:50.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum reinstall -y kernel || true 2023-11-06T01:11:51.201 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-06T01:11:51.202 INFO:teuthology.orchestra.run.smithi053.stdout:Unable to read consumer identity 2023-11-06T01:11:51.475 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:02 ago on Mon 06 Nov 2023 01:11:49 AM UTC. 2023-11-06T01:11:51.520 INFO:teuthology.orchestra.run.smithi053.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-06T01:11:51.538 INFO:teuthology.orchestra.run.smithi053.stderr:Error: No packages marked for reinstall. 2023-11-06T01:11:51.588 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2023-11-06T01:11:51.672 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-06T01:11:51.672 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-06T01:11:51.672 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-06T01:11:51.673 DEBUG:teuthology.parallel:result is None 2023-11-06T01:11:51.673 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-06T01:11:51.681 INFO:teuthology.task.internal:Creating test directory... 2023-11-06T01:11:51.681 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-06T01:11:51.684 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-06T01:11:51.687 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-06T01:11:51.707 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-06T01:11:51.722 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-06T01:11:51.748 INFO:teuthology.task.internal:Creating archive directory... 2023-11-06T01:11:51.748 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-06T01:11:51.750 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-06T01:11:51.757 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-06T01:11:51.793 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-06T01:11:51.800 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-06T01:11:51.801 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-06T01:11:51.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T01:11:51.844 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-06T01:11:51.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T01:11:51.863 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-06T01:11:51.866 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-06T01:11:51.886 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-06T01:11:51.900 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T01:11:51.912 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T01:11:51.941 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T01:11:51.958 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T01:11:51.960 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T01:11:51.976 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T01:11:51.977 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-06T01:11:51.985 INFO:teuthology.task.internal:Configuring sudo... 2023-11-06T01:11:51.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-06T01:11:51.988 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-06T01:11:51.991 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-06T01:11:52.020 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-06T01:11:52.028 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-06T01:11:52.029 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-06T01:11:52.056 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-06T01:11:52.062 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-06T01:11:52.080 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T01:11:52.156 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T01:11:52.205 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T01:11:52.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T01:11:52.332 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:11:52.333 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-06T01:11:52.399 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T01:11:52.449 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T01:11:52.500 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T01:11:52.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T01:11:52.632 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-11-06T01:11:52.632 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-06T01:11:52.702 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T01:11:52.757 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T01:11:52.811 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T01:11:52.900 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T01:11:52.944 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-11-06T01:11:52.945 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-06T01:11:53.013 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2023-11-06T01:11:53.016 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2023-11-06T01:11:53.019 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2023-11-06T01:11:53.083 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-06T01:11:53.095 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-06T01:11:53.108 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-06T01:11:53.502 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-06T01:11:53.511 INFO:teuthology.task.internal:Starting timer... 2023-11-06T01:11:53.512 INFO:teuthology.run_tasks:Running task pcp... 2023-11-06T01:11:53.556 INFO:teuthology.run_tasks:Running task selinux... 2023-11-06T01:11:53.575 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-11-06T01:11:53.576 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2023-11-06T01:11:53.646 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2023-11-06T01:11:53.648 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2023-11-06T01:11:53.850 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2023-11-06T01:11:53.852 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2023-11-06T01:11:53.913 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: 2023-11-06T01:11:53.915 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-06T01:11:53.915 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2023-11-06T01:11:53.945 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2023-11-06T01:11:53.945 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2023-11-06T01:11:53.970 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2023-11-06T01:11:53.971 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2023-11-06T01:11:53.992 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2023-11-06T01:11:53.993 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi163.front.sepia.ceph.com': 'permissive'} 2023-11-06T01:11:53.993 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-06T01:11:54.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T01:11:54.024 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-06T01:11:54.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T01:11:54.053 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-06T01:11:54.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T01:11:54.082 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-06T01:11:54.082 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2023-11-06T01:11:54.112 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2023-11-06T01:11:54.141 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2023-11-06T01:11:54.169 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-06T01:11:54.179 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-06T01:11:54.180 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-06T01:11:54.202 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-06T01:11:54.204 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi053.front.sepia.ceph.com,smithi062.front.sepia.ceph.com,smithi163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-06T01:21:20.323 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com')] 2023-11-06T01:21:20.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-11-06T01:21:20.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:21:20.396 DEBUG:teuthology.orchestra.run.smithi053:> true 2023-11-06T01:21:20.468 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-11-06T01:21:20.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-11-06T01:21:20.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:21:20.540 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-11-06T01:21:20.608 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-11-06T01:21:20.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-11-06T01:21:20.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:21:20.678 DEBUG:teuthology.orchestra.run.smithi163:> true 2023-11-06T01:21:20.747 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-11-06T01:21:20.748 INFO:teuthology.run_tasks:Running task clock... 2023-11-06T01:21:20.759 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-06T01:21:20.760 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-06T01:21:20.760 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T01:21:20.763 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-06T01:21:20.763 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T01:21:20.766 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-06T01:21:20.766 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T01:21:20.796 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-06T01:21:20.797 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-06T01:21:20.812 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-06T01:21:20.812 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-06T01:21:20.818 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-06T01:21:20.832 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-06T01:21:20.876 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2023-11-06T01:21:20.878 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2023-11-06T01:21:20.888 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2023-11-06T01:21:20.891 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2023-11-06T01:21:20.892 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2023-11-06T01:21:20.903 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-06T01:21:20.904 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2023-11-06T01:21:20.907 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-06T01:21:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-06T01:21:20.918 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-06T01:21:20.922 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-06T01:21:20.932 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-06T01:21:20.994 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2023-11-06T01:21:20.995 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2023-11-06T01:21:20.997 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-06T01:21:20.998 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2023-11-06T01:21:20.998 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:20.998 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:20.998 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:20.999 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:21.000 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-06T01:21:21.001 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2023-11-06T01:21:21.001 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:21.001 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:21.001 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:21.001 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:21.007 INFO:teuthology.orchestra.run.smithi163.stderr:bash: ntpq: command not found 2023-11-06T01:21:21.011 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-06T01:21:21.011 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2023-11-06T01:21:21.011 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:21.011 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:21.011 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:21.012 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T01:21:21.013 INFO:teuthology.run_tasks:Running task pexec... 2023-11-06T01:21:21.023 INFO:teuthology.task.pexec:Executing custom commands... 2023-11-06T01:21:21.023 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi053.front.sepia.ceph.com 2023-11-06T01:21:21.024 DEBUG:teuthology.orchestra.run.smithi053:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-06T01:21:21.024 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi062.front.sepia.ceph.com 2023-11-06T01:21:21.025 DEBUG:teuthology.orchestra.run.smithi062:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-06T01:21:21.025 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi163.front.sepia.ceph.com 2023-11-06T01:21:21.025 DEBUG:teuthology.orchestra.run.smithi163:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-06T01:21:21.425 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:21.432 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:21.434 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:22.240 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:39 ago on Mon 06 Nov 2023 01:19:42 AM UTC. 2023-11-06T01:21:22.278 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:32 ago on Mon 06 Nov 2023 01:19:49 AM UTC. 2023-11-06T01:21:22.288 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:36 ago on Mon 06 Nov 2023 01:19:45 AM UTC. 2023-11-06T01:21:24.006 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2023-11-06T01:21:24.007 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2023-11-06T01:21:24.007 INFO:teuthology.orchestra.run.smithi163.stdout: Package Architecture Version Repository Size 2023-11-06T01:21:24.008 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2023-11-06T01:21:24.008 INFO:teuthology.orchestra.run.smithi163.stdout:Resetting modules: 2023-11-06T01:21:24.008 INFO:teuthology.orchestra.run.smithi163.stdout: container-tools 2023-11-06T01:21:24.008 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:21:24.009 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2023-11-06T01:21:24.009 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2023-11-06T01:21:24.009 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:21:24.277 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-11-06T01:21:24.279 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-11-06T01:21:24.279 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2023-11-06T01:21:24.279 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-11-06T01:21:24.279 INFO:teuthology.orchestra.run.smithi062.stdout:Resetting modules: 2023-11-06T01:21:24.279 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools 2023-11-06T01:21:24.280 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:21:24.280 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-11-06T01:21:24.280 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-11-06T01:21:24.280 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:21:24.292 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2023-11-06T01:21:24.293 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2023-11-06T01:21:24.294 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2023-11-06T01:21:24.294 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2023-11-06T01:21:24.294 INFO:teuthology.orchestra.run.smithi053.stdout:Resetting modules: 2023-11-06T01:21:24.294 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools 2023-11-06T01:21:24.295 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:21:24.295 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2023-11-06T01:21:24.295 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2023-11-06T01:21:24.295 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:21:24.975 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:42 ago on Mon 06 Nov 2023 01:19:42 AM UTC. 2023-11-06T01:21:25.286 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:35 ago on Mon 06 Nov 2023 01:19:49 AM UTC. 2023-11-06T01:21:25.312 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:39 ago on Mon 06 Nov 2023 01:19:45 AM UTC. 2023-11-06T01:21:27.361 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2023-11-06T01:21:27.870 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:28.130 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-11-06T01:21:28.155 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2023-11-06T01:21:28.634 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:28.689 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:46 ago on Mon 06 Nov 2023 01:19:42 AM UTC. 2023-11-06T01:21:28.754 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:29.494 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:44 ago on Mon 06 Nov 2023 01:19:45 AM UTC. 2023-11-06T01:21:29.577 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:40 ago on Mon 06 Nov 2023 01:19:49 AM UTC. 2023-11-06T01:21:30.481 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2023-11-06T01:21:30.575 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================================================ 2023-11-06T01:21:30.575 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2023-11-06T01:21:30.575 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================================================ 2023-11-06T01:21:30.576 INFO:teuthology.orchestra.run.smithi163.stdout:Installing group/module packages: 2023-11-06T01:21:30.576 INFO:teuthology.orchestra.run.smithi163.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-06T01:21:30.576 INFO:teuthology.orchestra.run.smithi163.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-06T01:21:30.576 INFO:teuthology.orchestra.run.smithi163.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-06T01:21:30.577 INFO:teuthology.orchestra.run.smithi163.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-06T01:21:30.577 INFO:teuthology.orchestra.run.smithi163.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-06T01:21:30.577 INFO:teuthology.orchestra.run.smithi163.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-06T01:21:30.577 INFO:teuthology.orchestra.run.smithi163.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-06T01:21:30.578 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2023-11-06T01:21:30.578 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-06T01:21:30.578 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-06T01:21:30.578 INFO:teuthology.orchestra.run.smithi163.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-06T01:21:30.579 INFO:teuthology.orchestra.run.smithi163.stdout:Installing module profiles: 2023-11-06T01:21:30.579 INFO:teuthology.orchestra.run.smithi163.stdout: container-tools/common 2023-11-06T01:21:30.579 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:21:30.579 INFO:teuthology.orchestra.run.smithi163.stdout:Enabling module streams: 2023-11-06T01:21:30.580 INFO:teuthology.orchestra.run.smithi163.stdout: container-tools rhel8 2023-11-06T01:21:30.580 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:21:30.580 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2023-11-06T01:21:30.580 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================================================ 2023-11-06T01:21:30.580 INFO:teuthology.orchestra.run.smithi163.stdout:Install 10 Packages 2023-11-06T01:21:30.581 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:21:30.581 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 18 M 2023-11-06T01:21:30.582 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 65 M 2023-11-06T01:21:30.582 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2023-11-06T01:21:30.647 INFO:teuthology.orchestra.run.smithi163.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 361 kB/s | 25 kB 00:00 2023-11-06T01:21:30.672 INFO:teuthology.orchestra.run.smithi163.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 434 kB/s | 41 kB 00:00 2023-11-06T01:21:30.697 INFO:teuthology.orchestra.run.smithi163.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 786 kB/s | 94 kB 00:00 2023-11-06T01:21:30.722 INFO:teuthology.orchestra.run.smithi163.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 656 kB/s | 49 kB 00:00 2023-11-06T01:21:30.756 INFO:teuthology.orchestra.run.smithi163.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-11-06T01:21:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-11-06T01:21:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-11-06T01:21:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 26 MB/s | 2.2 MB 00:00 2023-11-06T01:21:31.007 INFO:teuthology.orchestra.run.smithi163.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 31 MB/s | 7.0 MB 00:00 2023-11-06T01:21:31.116 INFO:teuthology.orchestra.run.smithi163.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-11-06T01:21:31.117 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2023-11-06T01:21:31.117 INFO:teuthology.orchestra.run.smithi163.stdout:Total 34 MB/s | 18 MB 00:00 2023-11-06T01:21:31.318 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2023-11-06T01:21:31.336 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2023-11-06T01:21:31.336 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2023-11-06T01:21:31.435 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2023-11-06T01:21:31.517 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2023-11-06T01:21:31.521 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2023-11-06T01:21:31.522 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-11-06T01:21:31.538 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================================ 2023-11-06T01:21:31.538 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2023-11-06T01:21:31.539 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================================ 2023-11-06T01:21:31.539 INFO:teuthology.orchestra.run.smithi053.stdout:Installing group/module packages: 2023-11-06T01:21:31.539 INFO:teuthology.orchestra.run.smithi053.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-06T01:21:31.539 INFO:teuthology.orchestra.run.smithi053.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-06T01:21:31.539 INFO:teuthology.orchestra.run.smithi053.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-06T01:21:31.540 INFO:teuthology.orchestra.run.smithi053.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-06T01:21:31.540 INFO:teuthology.orchestra.run.smithi053.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-06T01:21:31.540 INFO:teuthology.orchestra.run.smithi053.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-06T01:21:31.540 INFO:teuthology.orchestra.run.smithi053.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-06T01:21:31.540 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2023-11-06T01:21:31.540 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-06T01:21:31.541 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-06T01:21:31.541 INFO:teuthology.orchestra.run.smithi053.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-06T01:21:31.541 INFO:teuthology.orchestra.run.smithi053.stdout:Installing module profiles: 2023-11-06T01:21:31.541 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools/common 2023-11-06T01:21:31.541 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:21:31.542 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling module streams: 2023-11-06T01:21:31.542 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools rhel8 2023-11-06T01:21:31.542 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:21:31.542 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2023-11-06T01:21:31.542 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================================ 2023-11-06T01:21:31.543 INFO:teuthology.orchestra.run.smithi053.stdout:Install 10 Packages 2023-11-06T01:21:31.543 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:21:31.543 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 18 M 2023-11-06T01:21:31.544 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 65 M 2023-11-06T01:21:31.544 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2023-11-06T01:21:31.607 INFO:teuthology.orchestra.run.smithi053.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 374 kB/s | 25 kB 00:00 2023-11-06T01:21:31.628 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================================ 2023-11-06T01:21:31.628 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-11-06T01:21:31.628 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================================ 2023-11-06T01:21:31.628 INFO:teuthology.orchestra.run.smithi062.stdout:Installing group/module packages: 2023-11-06T01:21:31.629 INFO:teuthology.orchestra.run.smithi062.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-06T01:21:31.629 INFO:teuthology.orchestra.run.smithi062.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-06T01:21:31.629 INFO:teuthology.orchestra.run.smithi062.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-06T01:21:31.629 INFO:teuthology.orchestra.run.smithi062.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-06T01:21:31.630 INFO:teuthology.orchestra.run.smithi062.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-06T01:21:31.630 INFO:teuthology.orchestra.run.smithi062.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-06T01:21:31.630 INFO:teuthology.orchestra.run.smithi062.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-06T01:21:31.630 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-11-06T01:21:31.630 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-06T01:21:31.631 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-06T01:21:31.631 INFO:teuthology.orchestra.run.smithi062.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-06T01:21:31.631 INFO:teuthology.orchestra.run.smithi062.stdout:Installing module profiles: 2023-11-06T01:21:31.631 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools/common 2023-11-06T01:21:31.632 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:21:31.632 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling module streams: 2023-11-06T01:21:31.632 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools rhel8 2023-11-06T01:21:31.632 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:21:31.632 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-11-06T01:21:31.633 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================================ 2023-11-06T01:21:31.633 INFO:teuthology.orchestra.run.smithi062.stdout:Install 10 Packages 2023-11-06T01:21:31.633 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:21:31.634 INFO:teuthology.orchestra.run.smithi053.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 446 kB/s | 41 kB 00:00 2023-11-06T01:21:31.634 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 18 M 2023-11-06T01:21:31.635 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 65 M 2023-11-06T01:21:31.635 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-11-06T01:21:31.649 INFO:teuthology.orchestra.run.smithi053.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 865 kB/s | 94 kB 00:00 2023-11-06T01:21:31.675 INFO:teuthology.orchestra.run.smithi053.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 736 kB/s | 49 kB 00:00 2023-11-06T01:21:31.699 INFO:teuthology.orchestra.run.smithi062.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-11-06T01:21:31.724 INFO:teuthology.orchestra.run.smithi062.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 266 kB/s | 25 kB 00:00 2023-11-06T01:21:31.733 INFO:teuthology.orchestra.run.smithi053.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.1 MB/s | 524 kB 00:00 2023-11-06T01:21:31.758 INFO:teuthology.orchestra.run.smithi053.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.8 MB/s | 149 kB 00:00 2023-11-06T01:21:31.774 INFO:teuthology.orchestra.run.smithi062.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 284 kB/s | 41 kB 00:00 2023-11-06T01:21:31.776 INFO:teuthology.orchestra.run.smithi053.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.9 MB/s | 213 kB 00:00 2023-11-06T01:21:31.799 INFO:teuthology.orchestra.run.smithi062.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 492 kB/s | 49 kB 00:00 2023-11-06T01:21:31.833 INFO:teuthology.orchestra.run.smithi062.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.7 MB/s | 524 kB 00:00 2023-11-06T01:21:31.858 INFO:teuthology.orchestra.run.smithi062.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-11-06T01:21:31.868 INFO:teuthology.orchestra.run.smithi053.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-11-06T01:21:31.875 INFO:teuthology.orchestra.run.smithi062.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 5.0 MB/s | 213 kB 00:00 2023-11-06T01:21:31.993 INFO:teuthology.orchestra.run.smithi053.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 30 MB/s | 7.0 MB 00:00 2023-11-06T01:21:32.002 INFO:teuthology.orchestra.run.smithi062.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 17 MB/s | 2.2 MB 00:00 2023-11-06T01:21:32.037 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2023-11-06T01:21:32.076 INFO:teuthology.orchestra.run.smithi053.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-11-06T01:21:32.077 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2023-11-06T01:21:32.077 INFO:teuthology.orchestra.run.smithi053.stdout:Total 34 MB/s | 18 MB 00:00 2023-11-06T01:21:32.208 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-06T01:21:32.231 INFO:teuthology.orchestra.run.smithi062.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 19 MB/s | 7.0 MB 00:00 2023-11-06T01:21:32.285 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2023-11-06T01:21:32.306 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2023-11-06T01:21:32.306 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2023-11-06T01:21:32.343 INFO:teuthology.orchestra.run.smithi062.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 14 MB/s | 8.0 MB 00:00 2023-11-06T01:21:32.343 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-11-06T01:21:32.343 INFO:teuthology.orchestra.run.smithi062.stdout:Total 26 MB/s | 18 MB 00:00 2023-11-06T01:21:32.418 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-06T01:21:32.499 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2023-11-06T01:21:32.504 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2023-11-06T01:21:32.552 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-11-06T01:21:32.573 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-11-06T01:21:32.573 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-11-06T01:21:32.613 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-06T01:21:32.768 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-11-06T01:21:32.772 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-11-06T01:21:32.822 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-06T01:21:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2023-11-06T01:21:33.196 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-06T01:21:33.245 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-06T01:21:33.346 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-11-06T01:21:33.416 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-06T01:21:33.536 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-06T01:21:33.647 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-06T01:21:33.725 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-06T01:21:33.842 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-06T01:21:33.955 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-06T01:21:33.967 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-06T01:21:34.178 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-06T01:21:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-06T01:21:34.582 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-06T01:21:35.065 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-06T01:21:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-06T01:21:35.283 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-06T01:21:35.396 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-06T01:21:35.442 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-06T01:21:35.628 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-06T01:21:35.895 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-06T01:21:35.895 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-06T01:21:35.895 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-06T01:21:35.897 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-06T01:21:35.897 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-06T01:21:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-06T01:21:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-06T01:21:36.345 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-06T01:21:36.480 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-06T01:21:36.581 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-06T01:21:36.690 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-06T01:21:36.789 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-06T01:21:36.960 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-06T01:21:36.960 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-06T01:21:36.961 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-06T01:21:36.961 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-06T01:21:36.961 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-06T01:21:36.961 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-06T01:21:36.961 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-06T01:21:36.962 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-06T01:21:36.962 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-06T01:21:36.962 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-06T01:21:36.965 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-06T01:21:37.072 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-06T01:21:37.072 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:54 ago on Mon 06 Nov 2023 01:19:42 AM UTC. 2023-11-06T01:21:37.250 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-06T01:21:37.251 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-06T01:21:37.251 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-06T01:21:37.251 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-06T01:21:37.251 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-06T01:21:37.251 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-06T01:21:37.252 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-06T01:21:37.252 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-06T01:21:37.252 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-06T01:21:37.252 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-06T01:21:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-06T01:21:38.143 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:52 ago on Mon 06 Nov 2023 01:19:45 AM UTC. 2023-11-06T01:21:38.419 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-06T01:21:38.419 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:48 ago on Mon 06 Nov 2023 01:19:49 AM UTC. 2023-11-06T01:21:39.660 INFO:teuthology.orchestra.run.smithi163.stdout:Installed products updated. 2023-11-06T01:21:39.854 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:21:39.854 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2023-11-06T01:21:39.854 INFO:teuthology.orchestra.run.smithi163.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:39.855 INFO:teuthology.orchestra.run.smithi163.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-06T01:21:39.855 INFO:teuthology.orchestra.run.smithi163.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:39.855 INFO:teuthology.orchestra.run.smithi163.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-06T01:21:39.856 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-06T01:21:39.856 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-06T01:21:39.856 INFO:teuthology.orchestra.run.smithi163.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:39.856 INFO:teuthology.orchestra.run.smithi163.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:39.856 INFO:teuthology.orchestra.run.smithi163.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-06T01:21:39.857 INFO:teuthology.orchestra.run.smithi163.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-06T01:21:39.857 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:21:39.857 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2023-11-06T01:21:40.008 DEBUG:teuthology.parallel:result is None 2023-11-06T01:21:41.141 INFO:teuthology.orchestra.run.smithi053.stdout:Installed products updated. 2023-11-06T01:21:41.330 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-11-06T01:21:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:21:41.356 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2023-11-06T01:21:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-06T01:21:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-06T01:21:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-06T01:21:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-06T01:21:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-06T01:21:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-06T01:21:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:21:41.358 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2023-11-06T01:21:41.500 DEBUG:teuthology.parallel:result is None 2023-11-06T01:21:41.541 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:21:41.541 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-11-06T01:21:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-06T01:21:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-06T01:21:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-06T01:21:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-06T01:21:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-06T01:21:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-06T01:21:41.544 INFO:teuthology.orchestra.run.smithi062.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-06T01:21:41.544 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:21:41.544 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-11-06T01:21:41.687 DEBUG:teuthology.parallel:result is None 2023-11-06T01:21:41.687 INFO:teuthology.run_tasks:Running task install... 2023-11-06T01:21:41.697 DEBUG:teuthology.task.install:project ceph 2023-11-06T01:21:41.698 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'}} 2023-11-06T01:21:41.698 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'} 2023-11-06T01:21:41.698 INFO:teuthology.task.install:Using flavor: default 2023-11-06T01:21:41.704 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-06T01:21:41.705 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-06T01:21:41.705 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5', 'tag': None, 'wait_for_package': False} 2023-11-06T01:21:41.706 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T01:21:41.708 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5', 'tag': None, 'wait_for_package': False} 2023-11-06T01:21:41.709 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T01:21:41.710 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5', 'tag': None, 'wait_for_package': False} 2023-11-06T01:21:41.710 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T01:21:41.877 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/ 2023-11-06T01:21:41.878 INFO:teuthology.task.install.rpm:Package version is 18.2.0-1181.gf7e9f9af 2023-11-06T01:21:42.088 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/ 2023-11-06T01:21:42.088 INFO:teuthology.task.install.rpm:Package version is 18.2.0-1181.gf7e9f9af 2023-11-06T01:21:42.094 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/ 2023-11-06T01:21:42.094 INFO:teuthology.task.install.rpm:Package version is 18.2.0-1181.gf7e9f9af 2023-11-06T01:21:42.313 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-06T01:21:42.313 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-11-06T01:21:42.314 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-06T01:21:42.347 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-06T01:21:42.347 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-06T01:21:42.419 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-06T01:21:42.437 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-06T01:21:42.438 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-11-06T01:21:42.438 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-06T01:21:42.471 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-06T01:21:42.472 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-06T01:21:42.500 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-06T01:21:42.528 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-06T01:21:42.528 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:21:42.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-06T01:21:42.543 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-06T01:21:42.562 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-06T01:21:42.562 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-06T01:21:42.570 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2023-11-06T01:21:42.570 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2023-11-06T01:21:42.624 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-06T01:21:42.635 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-06T01:21:42.691 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2023-11-06T01:21:42.693 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2023-11-06T01:21:42.717 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-06T01:21:42.786 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2023-11-06T01:21:42.788 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2023-11-06T01:21:42.963 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:43.068 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:43.184 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:43.227 INFO:teuthology.orchestra.run.smithi062.stdout:44 files removed 2023-11-06T01:21:43.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-06T01:21:43.324 INFO:teuthology.orchestra.run.smithi163.stdout:44 files removed 2023-11-06T01:21:43.371 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-06T01:21:43.444 INFO:teuthology.orchestra.run.smithi053.stdout:44 files removed 2023-11-06T01:21:43.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-06T01:21:43.625 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:43.706 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:43.841 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-06T01:21:45.054 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 97 kB/s | 80 kB 00:00 2023-11-06T01:21:45.667 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 50 kB/s | 80 kB 00:01 2023-11-06T01:21:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 48 kB/s | 80 kB 00:01 2023-11-06T01:21:45.821 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 13 kB/s | 9.4 kB 00:00 2023-11-06T01:21:46.420 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 14 kB/s | 9.4 kB 00:00 2023-11-06T01:21:46.468 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 12 kB/s | 9.4 kB 00:00 2023-11-06T01:21:46.539 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 2.1 kB/s | 1.5 kB 00:00 2023-11-06T01:21:46.664 INFO:teuthology.orchestra.run.smithi053.stdout:Copr repo for python3-asyncssh owned by ceph 34 kB/s | 3.5 kB 00:00 2023-11-06T01:21:47.213 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 1.9 kB/s | 1.5 kB 00:00 2023-11-06T01:21:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 1.9 kB/s | 1.5 kB 00:00 2023-11-06T01:21:47.339 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 34 kB/s | 3.5 kB 00:00 2023-11-06T01:21:47.402 INFO:teuthology.orchestra.run.smithi163.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-11-06T01:21:47.962 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-11-06T01:21:49.397 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 16 MB 00:01 2023-11-06T01:21:49.650 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 16 MB 00:02 2023-11-06T01:21:51.740 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 289 kB/s | 24 kB 00:00 2023-11-06T01:21:52.750 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 51 MB/s | 47 MB 00:00 2023-11-06T01:21:52.928 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 360 kB/s | 24 kB 00:00 2023-11-06T01:21:53.422 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 490 kB/s | 24 kB 00:00 2023-11-06T01:21:53.830 INFO:teuthology.orchestra.run.smithi163.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 MB/s | 47 MB 00:00 2023-11-06T01:21:54.474 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 54 MB/s | 53 MB 00:00 2023-11-06T01:22:01.268 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 MB/s | 53 MB 00:00 2023-11-06T01:22:01.764 INFO:teuthology.orchestra.run.smithi163.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2023-11-06T01:22:03.358 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 52 MB/s | 47 MB 00:00 2023-11-06T01:22:12.842 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-06T01:22:12.842 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-06T01:22:12.857 INFO:teuthology.orchestra.run.smithi163.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-06T01:22:12.858 INFO:teuthology.orchestra.run.smithi163.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-06T01:22:12.859 INFO:teuthology.orchestra.run.smithi163.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-06T01:22:12.860 INFO:teuthology.orchestra.run.smithi163.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-06T01:22:12.862 INFO:teuthology.orchestra.run.smithi163.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-06T01:22:12.865 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-06T01:22:12.867 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-06T01:22:12.999 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2023-11-06T01:22:13.012 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================================================== 2023-11-06T01:22:13.012 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2023-11-06T01:22:13.012 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================================================== 2023-11-06T01:22:13.013 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2023-11-06T01:22:13.013 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-06T01:22:13.013 INFO:teuthology.orchestra.run.smithi163.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-06T01:22:13.014 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 6.4 k 2023-11-06T01:22:13.014 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 5.4 M 2023-11-06T01:22:13.014 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 852 k 2023-11-06T01:22:13.014 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 165 k 2023-11-06T01:22:13.015 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 1.6 M 2023-11-06T01:22:13.015 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 132 k 2023-11-06T01:22:13.015 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 1.8 M 2023-11-06T01:22:13.015 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 7.4 M 2023-11-06T01:22:13.015 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 47 k 2023-11-06T01:22:13.016 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 13 M 2023-11-06T01:22:13.016 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 47 M 2023-11-06T01:22:13.016 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 261 k 2023-11-06T01:22:13.016 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 171 k 2023-11-06T01:22:13.017 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 258 k 2023-11-06T01:22:13.017 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 18 k 2023-11-06T01:22:13.017 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-06T01:22:13.017 INFO:teuthology.orchestra.run.smithi163.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-06T01:22:13.018 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-06T01:22:13.018 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 31 k 2023-11-06T01:22:13.018 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 774 k 2023-11-06T01:22:13.019 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 139 k 2023-11-06T01:22:13.019 INFO:teuthology.orchestra.run.smithi163.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-06T01:22:13.019 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-06T01:22:13.019 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-06T01:22:13.019 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-06T01:22:13.020 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-06T01:22:13.020 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-06T01:22:13.020 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 197 k 2023-11-06T01:22:13.020 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 371 k 2023-11-06T01:22:13.021 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 365 k 2023-11-06T01:22:13.021 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 107 k 2023-11-06T01:22:13.021 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 94 k 2023-11-06T01:22:13.021 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 3.4 M 2023-11-06T01:22:13.021 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 181 k 2023-11-06T01:22:13.022 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-06T01:22:13.022 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2023-11-06T01:22:13.022 INFO:teuthology.orchestra.run.smithi163.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-06T01:22:13.022 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 3.7 M 2023-11-06T01:22:13.022 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 3.6 M 2023-11-06T01:22:13.023 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2023-11-06T01:22:13.023 INFO:teuthology.orchestra.run.smithi163.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-06T01:22:13.023 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 22 M 2023-11-06T01:22:13.023 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 23 k 2023-11-06T01:22:13.024 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 2.3 M 2023-11-06T01:22:13.024 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 242 k 2023-11-06T01:22:13.024 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 4.6 M 2023-11-06T01:22:13.024 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 18 M 2023-11-06T01:22:13.024 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 15 k 2023-11-06T01:22:13.025 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 24 k 2023-11-06T01:22:13.025 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-06T01:22:13.025 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-06T01:22:13.025 INFO:teuthology.orchestra.run.smithi163.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-06T01:22:13.025 INFO:teuthology.orchestra.run.smithi163.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-06T01:22:13.026 INFO:teuthology.orchestra.run.smithi163.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-06T01:22:13.026 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 141 k 2023-11-06T01:22:13.026 INFO:teuthology.orchestra.run.smithi163.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-06T01:22:13.026 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-06T01:22:13.027 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-06T01:22:13.027 INFO:teuthology.orchestra.run.smithi163.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-06T01:22:13.027 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-06T01:22:13.027 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-06T01:22:13.027 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 476 k 2023-11-06T01:22:13.028 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-06T01:22:13.028 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 6.5 M 2023-11-06T01:22:13.028 INFO:teuthology.orchestra.run.smithi163.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-06T01:22:13.028 INFO:teuthology.orchestra.run.smithi163.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-06T01:22:13.028 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-06T01:22:13.029 INFO:teuthology.orchestra.run.smithi163.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-06T01:22:13.029 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-06T01:22:13.029 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-06T01:22:13.029 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-06T01:22:13.029 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-06T01:22:13.030 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-06T01:22:13.030 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-06T01:22:13.030 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-06T01:22:13.030 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-06T01:22:13.030 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-06T01:22:13.031 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-06T01:22:13.031 INFO:teuthology.orchestra.run.smithi163.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-06T01:22:13.031 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-06T01:22:13.031 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 45 k 2023-11-06T01:22:13.032 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 123 k 2023-11-06T01:22:13.032 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-06T01:22:13.032 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-06T01:22:13.033 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-06T01:22:13.033 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-06T01:22:13.033 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-06T01:22:13.033 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-06T01:22:13.034 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-06T01:22:13.034 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-06T01:22:13.034 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-06T01:22:13.034 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-06T01:22:13.034 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-06T01:22:13.035 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-06T01:22:13.035 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-06T01:22:13.035 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-06T01:22:13.035 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-06T01:22:13.035 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-06T01:22:13.036 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-06T01:22:13.036 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-06T01:22:13.036 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-06T01:22:13.036 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-06T01:22:13.036 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-06T01:22:13.037 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-06T01:22:13.037 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-06T01:22:13.037 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-06T01:22:13.037 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-06T01:22:13.037 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-06T01:22:13.038 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-06T01:22:13.038 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-06T01:22:13.038 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-06T01:22:13.038 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-06T01:22:13.038 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-06T01:22:13.039 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-06T01:22:13.039 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-06T01:22:13.039 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-06T01:22:13.039 INFO:teuthology.orchestra.run.smithi163.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-06T01:22:13.039 INFO:teuthology.orchestra.run.smithi163.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-06T01:22:13.040 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-06T01:22:13.040 INFO:teuthology.orchestra.run.smithi163.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-06T01:22:13.040 INFO:teuthology.orchestra.run.smithi163.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-06T01:22:13.040 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-06T01:22:13.040 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-06T01:22:13.041 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-06T01:22:13.041 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-06T01:22:13.041 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-06T01:22:13.041 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-06T01:22:13.042 INFO:teuthology.orchestra.run.smithi163.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-06T01:22:13.042 INFO:teuthology.orchestra.run.smithi163.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-06T01:22:13.042 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-06T01:22:13.042 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-06T01:22:13.042 INFO:teuthology.orchestra.run.smithi163.stdout:Enabling module streams: 2023-11-06T01:22:13.043 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql 10 2023-11-06T01:22:13.043 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:22:13.043 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2023-11-06T01:22:13.043 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================================================== 2023-11-06T01:22:13.043 INFO:teuthology.orchestra.run.smithi163.stdout:Install 128 Packages 2023-11-06T01:22:13.044 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 3 Packages 2023-11-06T01:22:13.044 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:22:13.045 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 203 M 2023-11-06T01:22:13.045 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2023-11-06T01:22:13.152 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-06T01:22:13.152 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-06T01:22:13.168 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-06T01:22:13.169 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-06T01:22:13.170 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-06T01:22:13.171 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-06T01:22:13.173 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-06T01:22:13.177 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-06T01:22:13.178 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-06T01:22:13.323 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2023-11-06T01:22:13.336 INFO:teuthology.orchestra.run.smithi053.stdout:=========================================================================================================================== 2023-11-06T01:22:13.337 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2023-11-06T01:22:13.337 INFO:teuthology.orchestra.run.smithi053.stdout:=========================================================================================================================== 2023-11-06T01:22:13.337 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2023-11-06T01:22:13.337 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-06T01:22:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-06T01:22:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 6.4 k 2023-11-06T01:22:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 5.4 M 2023-11-06T01:22:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 852 k 2023-11-06T01:22:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 165 k 2023-11-06T01:22:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 1.6 M 2023-11-06T01:22:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 132 k 2023-11-06T01:22:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 1.8 M 2023-11-06T01:22:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 7.4 M 2023-11-06T01:22:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 47 k 2023-11-06T01:22:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 13 M 2023-11-06T01:22:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 47 M 2023-11-06T01:22:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 261 k 2023-11-06T01:22:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 171 k 2023-11-06T01:22:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 258 k 2023-11-06T01:22:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 18 k 2023-11-06T01:22:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-06T01:22:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-06T01:22:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-06T01:22:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 31 k 2023-11-06T01:22:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 774 k 2023-11-06T01:22:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 139 k 2023-11-06T01:22:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-06T01:22:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-06T01:22:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-06T01:22:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-06T01:22:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-06T01:22:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-06T01:22:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 197 k 2023-11-06T01:22:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 371 k 2023-11-06T01:22:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 365 k 2023-11-06T01:22:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 107 k 2023-11-06T01:22:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 94 k 2023-11-06T01:22:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 3.4 M 2023-11-06T01:22:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 181 k 2023-11-06T01:22:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-06T01:22:13.345 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2023-11-06T01:22:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-06T01:22:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 3.7 M 2023-11-06T01:22:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 3.6 M 2023-11-06T01:22:13.346 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2023-11-06T01:22:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-06T01:22:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 22 M 2023-11-06T01:22:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 23 k 2023-11-06T01:22:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 2.3 M 2023-11-06T01:22:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 242 k 2023-11-06T01:22:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 4.6 M 2023-11-06T01:22:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 18 M 2023-11-06T01:22:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 15 k 2023-11-06T01:22:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 24 k 2023-11-06T01:22:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-06T01:22:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-06T01:22:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-06T01:22:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-06T01:22:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-06T01:22:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 141 k 2023-11-06T01:22:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-06T01:22:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-06T01:22:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-06T01:22:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-06T01:22:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-06T01:22:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-06T01:22:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 476 k 2023-11-06T01:22:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-06T01:22:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 6.5 M 2023-11-06T01:22:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-06T01:22:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-06T01:22:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-06T01:22:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-06T01:22:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-06T01:22:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-06T01:22:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-06T01:22:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-06T01:22:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-06T01:22:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-06T01:22:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-06T01:22:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-06T01:22:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-06T01:22:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-06T01:22:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-06T01:22:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-06T01:22:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 45 k 2023-11-06T01:22:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 123 k 2023-11-06T01:22:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-06T01:22:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-06T01:22:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-06T01:22:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-06T01:22:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-06T01:22:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-06T01:22:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-06T01:22:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-06T01:22:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-06T01:22:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-06T01:22:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-06T01:22:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-06T01:22:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-06T01:22:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-06T01:22:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-06T01:22:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-06T01:22:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-06T01:22:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-06T01:22:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-06T01:22:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-06T01:22:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-06T01:22:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-06T01:22:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-06T01:22:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-06T01:22:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-06T01:22:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-06T01:22:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-06T01:22:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-06T01:22:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-06T01:22:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-06T01:22:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-06T01:22:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-06T01:22:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-06T01:22:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-06T01:22:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-06T01:22:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-06T01:22:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-06T01:22:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-06T01:22:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-06T01:22:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-06T01:22:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-06T01:22:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-06T01:22:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-06T01:22:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-06T01:22:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-06T01:22:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-06T01:22:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-06T01:22:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-06T01:22:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-06T01:22:13.364 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling module streams: 2023-11-06T01:22:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql 10 2023-11-06T01:22:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:22:13.364 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2023-11-06T01:22:13.365 INFO:teuthology.orchestra.run.smithi053.stdout:=========================================================================================================================== 2023-11-06T01:22:13.365 INFO:teuthology.orchestra.run.smithi053.stdout:Install 128 Packages 2023-11-06T01:22:13.365 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 3 Packages 2023-11-06T01:22:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:22:13.366 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 203 M 2023-11-06T01:22:13.366 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2023-11-06T01:22:13.656 INFO:teuthology.orchestra.run.smithi163.stdout:(1/131): ceph-18.2.0-1181.gf7e9f9af.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2023-11-06T01:22:14.077 INFO:teuthology.orchestra.run.smithi053.stdout:(1/131): ceph-18.2.0-1181.gf7e9f9af.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-11-06T01:22:15.044 INFO:teuthology.orchestra.run.smithi053.stdout:(2/131): ceph-fuse-18.2.0-1181.gf7e9f9af.el8.x8 881 kB/s | 852 kB 00:00 2023-11-06T01:22:15.114 INFO:teuthology.orchestra.run.smithi163.stdout:(2/131): ceph-fuse-18.2.0-1181.gf7e9f9af.el8.x8 584 kB/s | 852 kB 00:01 2023-11-06T01:22:15.126 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-06T01:22:15.127 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-06T01:22:15.142 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-06T01:22:15.143 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-06T01:22:15.144 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-06T01:22:15.145 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-06T01:22:15.147 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-06T01:22:15.150 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-06T01:22:15.151 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-06T01:22:15.194 INFO:teuthology.orchestra.run.smithi053.stdout:(3/131): ceph-immutable-object-cache-18.2.0-118 1.1 MB/s | 165 kB 00:00 2023-11-06T01:22:15.295 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-11-06T01:22:15.308 INFO:teuthology.orchestra.run.smithi062.stdout:=========================================================================================================================== 2023-11-06T01:22:15.309 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-11-06T01:22:15.309 INFO:teuthology.orchestra.run.smithi062.stdout:=========================================================================================================================== 2023-11-06T01:22:15.309 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-11-06T01:22:15.309 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-06T01:22:15.309 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-06T01:22:15.310 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 6.4 k 2023-11-06T01:22:15.310 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 5.4 M 2023-11-06T01:22:15.310 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 852 k 2023-11-06T01:22:15.310 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 165 k 2023-11-06T01:22:15.310 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 1.6 M 2023-11-06T01:22:15.311 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 132 k 2023-11-06T01:22:15.311 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 1.8 M 2023-11-06T01:22:15.311 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 7.4 M 2023-11-06T01:22:15.311 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 47 k 2023-11-06T01:22:15.312 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 13 M 2023-11-06T01:22:15.312 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 47 M 2023-11-06T01:22:15.312 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 261 k 2023-11-06T01:22:15.312 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 171 k 2023-11-06T01:22:15.312 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 258 k 2023-11-06T01:22:15.313 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 18 k 2023-11-06T01:22:15.313 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-06T01:22:15.313 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-06T01:22:15.313 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-06T01:22:15.314 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 31 k 2023-11-06T01:22:15.314 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 774 k 2023-11-06T01:22:15.314 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 139 k 2023-11-06T01:22:15.314 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-06T01:22:15.314 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-06T01:22:15.315 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-06T01:22:15.315 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-06T01:22:15.315 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-06T01:22:15.315 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-06T01:22:15.315 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 197 k 2023-11-06T01:22:15.316 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 371 k 2023-11-06T01:22:15.316 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 365 k 2023-11-06T01:22:15.316 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 107 k 2023-11-06T01:22:15.316 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 94 k 2023-11-06T01:22:15.317 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 3.4 M 2023-11-06T01:22:15.317 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 181 k 2023-11-06T01:22:15.317 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-06T01:22:15.317 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2023-11-06T01:22:15.317 INFO:teuthology.orchestra.run.smithi062.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-06T01:22:15.318 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 3.7 M 2023-11-06T01:22:15.318 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 3.6 M 2023-11-06T01:22:15.318 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-11-06T01:22:15.318 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-06T01:22:15.318 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 22 M 2023-11-06T01:22:15.319 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 23 k 2023-11-06T01:22:15.319 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 2.3 M 2023-11-06T01:22:15.319 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 242 k 2023-11-06T01:22:15.319 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 4.6 M 2023-11-06T01:22:15.319 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 18 M 2023-11-06T01:22:15.320 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:18.2.0-1181.gf7e9f9af.el8 ceph-noarch 15 k 2023-11-06T01:22:15.320 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 24 k 2023-11-06T01:22:15.320 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-06T01:22:15.320 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-06T01:22:15.321 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-06T01:22:15.321 INFO:teuthology.orchestra.run.smithi062.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-06T01:22:15.321 INFO:teuthology.orchestra.run.smithi062.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-06T01:22:15.321 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 141 k 2023-11-06T01:22:15.322 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-06T01:22:15.322 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-06T01:22:15.322 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-06T01:22:15.322 INFO:teuthology.orchestra.run.smithi062.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-06T01:22:15.322 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-06T01:22:15.323 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-06T01:22:15.323 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 476 k 2023-11-06T01:22:15.323 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-06T01:22:15.324 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 6.5 M 2023-11-06T01:22:15.324 INFO:teuthology.orchestra.run.smithi062.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-06T01:22:15.324 INFO:teuthology.orchestra.run.smithi062.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-06T01:22:15.325 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-06T01:22:15.325 INFO:teuthology.orchestra.run.smithi062.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-06T01:22:15.325 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-06T01:22:15.326 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-06T01:22:15.326 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-06T01:22:15.326 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-06T01:22:15.327 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-06T01:22:15.327 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-06T01:22:15.327 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-06T01:22:15.327 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-06T01:22:15.327 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-06T01:22:15.328 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-06T01:22:15.328 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-06T01:22:15.328 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-06T01:22:15.328 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 45 k 2023-11-06T01:22:15.328 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:18.2.0-1181.gf7e9f9af.el8 ceph 123 k 2023-11-06T01:22:15.329 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-06T01:22:15.329 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-06T01:22:15.329 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-06T01:22:15.330 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-06T01:22:15.330 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-06T01:22:15.331 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-06T01:22:15.331 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-06T01:22:15.331 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-06T01:22:15.332 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-06T01:22:15.332 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-06T01:22:15.332 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-06T01:22:15.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-06T01:22:15.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-06T01:22:15.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-06T01:22:15.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-06T01:22:15.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-06T01:22:15.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-06T01:22:15.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-06T01:22:15.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-06T01:22:15.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-06T01:22:15.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-06T01:22:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-06T01:22:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-06T01:22:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-06T01:22:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-06T01:22:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-06T01:22:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-06T01:22:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-06T01:22:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-06T01:22:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-06T01:22:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-06T01:22:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-06T01:22:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-06T01:22:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-06T01:22:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-06T01:22:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-06T01:22:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-06T01:22:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-06T01:22:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-06T01:22:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-06T01:22:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-06T01:22:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-06T01:22:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-06T01:22:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-06T01:22:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-06T01:22:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-06T01:22:15.346 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-06T01:22:15.346 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-06T01:22:15.347 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-06T01:22:15.347 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling module streams: 2023-11-06T01:22:15.347 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql 10 2023-11-06T01:22:15.348 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:22:15.348 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-11-06T01:22:15.348 INFO:teuthology.orchestra.run.smithi062.stdout:=========================================================================================================================== 2023-11-06T01:22:15.349 INFO:teuthology.orchestra.run.smithi062.stdout:Install 128 Packages 2023-11-06T01:22:15.349 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 3 Packages 2023-11-06T01:22:15.349 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:22:15.351 INFO:teuthology.orchestra.run.smithi163.stdout:(3/131): ceph-immutable-object-cache-18.2.0-118 764 kB/s | 165 kB 00:00 2023-11-06T01:22:15.351 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 203 M 2023-11-06T01:22:15.352 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-11-06T01:22:15.942 INFO:teuthology.orchestra.run.smithi062.stdout:(1/131): ceph-18.2.0-1181.gf7e9f9af.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-11-06T01:22:16.003 INFO:teuthology.orchestra.run.smithi053.stdout:(4/131): ceph-base-18.2.0-1181.gf7e9f9af.el8.x8 2.4 MB/s | 5.4 MB 00:02 2023-11-06T01:22:16.669 INFO:teuthology.orchestra.run.smithi053.stdout:(5/131): ceph-mgr-18.2.0-1181.gf7e9f9af.el8.x86 2.3 MB/s | 1.6 MB 00:00 2023-11-06T01:22:16.809 INFO:teuthology.orchestra.run.smithi062.stdout:(2/131): ceph-fuse-18.2.0-1181.gf7e9f9af.el8.x8 983 kB/s | 852 kB 00:00 2023-11-06T01:22:17.494 INFO:teuthology.orchestra.run.smithi053.stdout:(6/131): ceph-mds-18.2.0-1181.gf7e9f9af.el8.x86 1.0 MB/s | 2.3 MB 00:02 2023-11-06T01:22:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:(3/131): ceph-immutable-object-cache-18.2.0-118 218 kB/s | 165 kB 00:00 2023-11-06T01:22:17.790 INFO:teuthology.orchestra.run.smithi163.stdout:(4/131): ceph-base-18.2.0-1181.gf7e9f9af.el8.x8 1.2 MB/s | 5.4 MB 00:04 2023-11-06T01:22:18.262 INFO:teuthology.orchestra.run.smithi053.stdout:(7/131): ceph-mon-18.2.0-1181.gf7e9f9af.el8.x86 2.9 MB/s | 4.6 MB 00:01 2023-11-06T01:22:18.515 INFO:teuthology.orchestra.run.smithi163.stdout:(5/131): ceph-mgr-18.2.0-1181.gf7e9f9af.el8.x86 2.1 MB/s | 1.6 MB 00:00 2023-11-06T01:22:18.910 INFO:teuthology.orchestra.run.smithi062.stdout:(4/131): ceph-mds-18.2.0-1181.gf7e9f9af.el8.x86 1.7 MB/s | 2.3 MB 00:01 2023-11-06T01:22:19.007 INFO:teuthology.orchestra.run.smithi163.stdout:(6/131): ceph-common-18.2.0-1181.gf7e9f9af.el8. 3.8 MB/s | 22 MB 00:05 2023-11-06T01:22:19.057 INFO:teuthology.orchestra.run.smithi163.stdout:(7/131): ceph-mds-18.2.0-1181.gf7e9f9af.el8.x86 629 kB/s | 2.3 MB 00:03 2023-11-06T01:22:19.843 INFO:teuthology.orchestra.run.smithi062.stdout:(5/131): ceph-mgr-18.2.0-1181.gf7e9f9af.el8.x86 1.7 MB/s | 1.6 MB 00:00 2023-11-06T01:22:21.224 INFO:teuthology.orchestra.run.smithi163.stdout:(8/131): ceph-mon-18.2.0-1181.gf7e9f9af.el8.x86 1.7 MB/s | 4.6 MB 00:02 2023-11-06T01:22:21.365 INFO:teuthology.orchestra.run.smithi163.stdout:(9/131): ceph-selinux-18.2.0-1181.gf7e9f9af.el8 172 kB/s | 24 kB 00:00 2023-11-06T01:22:21.918 INFO:teuthology.orchestra.run.smithi062.stdout:(6/131): ceph-mon-18.2.0-1181.gf7e9f9af.el8.x86 2.2 MB/s | 4.6 MB 00:02 2023-11-06T01:22:24.151 INFO:teuthology.orchestra.run.smithi053.stdout:(8/131): ceph-radosgw-18.2.0-1181.gf7e9f9af.el8 2.2 MB/s | 13 MB 00:05 2023-11-06T01:22:24.320 INFO:teuthology.orchestra.run.smithi053.stdout:(9/131): ceph-osd-18.2.0-1181.gf7e9f9af.el8.x86 2.6 MB/s | 18 MB 00:06 2023-11-06T01:22:24.346 INFO:teuthology.orchestra.run.smithi053.stdout:(10/131): ceph-selinux-18.2.0-1181.gf7e9f9af.el 125 kB/s | 24 kB 00:00 2023-11-06T01:22:24.596 INFO:teuthology.orchestra.run.smithi053.stdout:(11/131): cephfs-mirror-18.2.0-1181.gf7e9f9af.e 1.0 MB/s | 258 kB 00:00 2023-11-06T01:22:24.696 INFO:teuthology.orchestra.run.smithi053.stdout:(12/131): libcephfs-devel-18.2.0-1181.gf7e9f9af 309 kB/s | 31 kB 00:00 2023-11-06T01:22:25.230 INFO:teuthology.orchestra.run.smithi053.stdout:(13/131): libcephfs2-18.2.0-1181.gf7e9f9af.el8. 1.4 MB/s | 774 kB 00:00 2023-11-06T01:22:25.342 INFO:teuthology.orchestra.run.smithi163.stdout:(10/131): ceph-osd-18.2.0-1181.gf7e9f9af.el8.x8 2.8 MB/s | 18 MB 00:06 2023-11-06T01:22:25.413 INFO:teuthology.orchestra.run.smithi053.stdout:(14/131): libcephsqlite-18.2.0-1181.gf7e9f9af.e 770 kB/s | 141 kB 00:00 2023-11-06T01:22:25.484 INFO:teuthology.orchestra.run.smithi163.stdout:(11/131): cephfs-mirror-18.2.0-1181.gf7e9f9af.e 1.8 MB/s | 258 kB 00:00 2023-11-06T01:22:25.568 INFO:teuthology.orchestra.run.smithi163.stdout:(12/131): libcephfs-devel-18.2.0-1181.gf7e9f9af 370 kB/s | 31 kB 00:00 2023-11-06T01:22:25.622 INFO:teuthology.orchestra.run.smithi053.stdout:(15/131): librados-devel-18.2.0-1181.gf7e9f9af. 666 kB/s | 139 kB 00:00 2023-11-06T01:22:25.868 INFO:teuthology.orchestra.run.smithi163.stdout:(13/131): libcephfs2-18.2.0-1181.gf7e9f9af.el8. 2.5 MB/s | 774 kB 00:00 2023-11-06T01:22:25.968 INFO:teuthology.orchestra.run.smithi163.stdout:(14/131): libcephsqlite-18.2.0-1181.gf7e9f9af.e 1.4 MB/s | 141 kB 00:00 2023-11-06T01:22:25.998 INFO:teuthology.orchestra.run.smithi053.stdout:(16/131): libradosstriper1-18.2.0-1181.gf7e9f9a 1.2 MB/s | 476 kB 00:00 2023-11-06T01:22:26.052 INFO:teuthology.orchestra.run.smithi163.stdout:(15/131): librados-devel-18.2.0-1181.gf7e9f9af. 1.6 MB/s | 139 kB 00:00 2023-11-06T01:22:26.152 INFO:teuthology.orchestra.run.smithi062.stdout:(7/131): ceph-common-18.2.0-1181.gf7e9f9af.el8. 2.1 MB/s | 22 MB 00:10 2023-11-06T01:22:26.269 INFO:teuthology.orchestra.run.smithi163.stdout:(16/131): libradosstriper1-18.2.0-1181.gf7e9f9a 2.1 MB/s | 476 kB 00:00 2023-11-06T01:22:27.335 INFO:teuthology.orchestra.run.smithi062.stdout:(8/131): ceph-base-18.2.0-1181.gf7e9f9af.el8.x8 471 kB/s | 5.4 MB 00:11 2023-11-06T01:22:27.561 INFO:teuthology.orchestra.run.smithi062.stdout:(9/131): ceph-selinux-18.2.0-1181.gf7e9f9af.el8 108 kB/s | 24 kB 00:00 2023-11-06T01:22:28.269 INFO:teuthology.orchestra.run.smithi062.stdout:(10/131): ceph-osd-18.2.0-1181.gf7e9f9af.el8.x8 2.8 MB/s | 18 MB 00:06 2023-11-06T01:22:28.476 INFO:teuthology.orchestra.run.smithi062.stdout:(11/131): cephfs-mirror-18.2.0-1181.gf7e9f9af.e 1.2 MB/s | 258 kB 00:00 2023-11-06T01:22:28.494 INFO:teuthology.orchestra.run.smithi163.stdout:(17/131): librgw2-18.2.0-1181.gf7e9f9af.el8.x86 2.9 MB/s | 6.5 MB 00:02 2023-11-06T01:22:28.561 INFO:teuthology.orchestra.run.smithi062.stdout:(12/131): libcephfs-devel-18.2.0-1181.gf7e9f9af 365 kB/s | 31 kB 00:00 2023-11-06T01:22:28.586 INFO:teuthology.orchestra.run.smithi163.stdout:(18/131): python3-ceph-argparse-18.2.0-1181.gf7 488 kB/s | 45 kB 00:00 2023-11-06T01:22:28.686 INFO:teuthology.orchestra.run.smithi163.stdout:(19/131): python3-ceph-common-18.2.0-1181.gf7e9 1.2 MB/s | 123 kB 00:00 2023-11-06T01:22:28.953 INFO:teuthology.orchestra.run.smithi062.stdout:(13/131): libcephfs2-18.2.0-1181.gf7e9f9af.el8. 1.9 MB/s | 774 kB 00:00 2023-11-06T01:22:29.045 INFO:teuthology.orchestra.run.smithi062.stdout:(14/131): libcephsqlite-18.2.0-1181.gf7e9f9af.e 1.5 MB/s | 141 kB 00:00 2023-11-06T01:22:29.153 INFO:teuthology.orchestra.run.smithi062.stdout:(15/131): librados-devel-18.2.0-1181.gf7e9f9af. 1.3 MB/s | 139 kB 00:00 2023-11-06T01:22:29.311 INFO:teuthology.orchestra.run.smithi163.stdout:(20/131): python3-cephfs-18.2.0-1181.gf7e9f9af. 315 kB/s | 197 kB 00:00 2023-11-06T01:22:29.420 INFO:teuthology.orchestra.run.smithi062.stdout:(16/131): libradosstriper1-18.2.0-1181.gf7e9f9a 1.7 MB/s | 476 kB 00:00 2023-11-06T01:22:29.528 INFO:teuthology.orchestra.run.smithi163.stdout:(21/131): python3-rados-18.2.0-1181.gf7e9f9af.e 1.7 MB/s | 371 kB 00:00 2023-11-06T01:22:29.778 INFO:teuthology.orchestra.run.smithi163.stdout:(22/131): python3-rbd-18.2.0-1181.gf7e9f9af.el8 1.4 MB/s | 365 kB 00:00 2023-11-06T01:22:29.870 INFO:teuthology.orchestra.run.smithi163.stdout:(23/131): python3-rgw-18.2.0-1181.gf7e9f9af.el8 1.1 MB/s | 107 kB 00:00 2023-11-06T01:22:29.971 INFO:teuthology.orchestra.run.smithi163.stdout:(24/131): rbd-fuse-18.2.0-1181.gf7e9f9af.el8.x8 945 kB/s | 94 kB 00:00 2023-11-06T01:22:31.504 INFO:teuthology.orchestra.run.smithi163.stdout:(25/131): rbd-mirror-18.2.0-1181.gf7e9f9af.el8. 2.2 MB/s | 3.4 MB 00:01 2023-11-06T01:22:31.596 INFO:teuthology.orchestra.run.smithi163.stdout:(26/131): rbd-nbd-18.2.0-1181.gf7e9f9af.el8.x86 1.9 MB/s | 181 kB 00:00 2023-11-06T01:22:31.672 INFO:teuthology.orchestra.run.smithi163.stdout:(27/131): ceph-grafana-dashboards-18.2.0-1181.g 308 kB/s | 23 kB 00:00 2023-11-06T01:22:31.714 INFO:teuthology.orchestra.run.smithi053.stdout:(17/131): ceph-common-18.2.0-1181.gf7e9f9af.el8 1.2 MB/s | 22 MB 00:17 2023-11-06T01:22:31.755 INFO:teuthology.orchestra.run.smithi163.stdout:(28/131): ceph-mgr-cephadm-18.2.0-1181.gf7e9f9a 1.5 MB/s | 132 kB 00:00 2023-11-06T01:22:31.773 INFO:teuthology.orchestra.run.smithi053.stdout:(18/131): python3-ceph-argparse-18.2.0-1181.gf7 768 kB/s | 45 kB 00:00 2023-11-06T01:22:31.864 INFO:teuthology.orchestra.run.smithi053.stdout:(19/131): python3-ceph-common-18.2.0-1181.gf7e9 1.3 MB/s | 123 kB 00:00 2023-11-06T01:22:32.007 INFO:teuthology.orchestra.run.smithi053.stdout:(20/131): python3-cephfs-18.2.0-1181.gf7e9f9af. 1.4 MB/s | 197 kB 00:00 2023-11-06T01:22:32.037 INFO:teuthology.orchestra.run.smithi062.stdout:(17/131): ceph-radosgw-18.2.0-1181.gf7e9f9af.el 2.2 MB/s | 13 MB 00:05 2023-11-06T01:22:32.129 INFO:teuthology.orchestra.run.smithi062.stdout:(18/131): librgw2-18.2.0-1181.gf7e9f9af.el8.x86 2.4 MB/s | 6.5 MB 00:02 2023-11-06T01:22:32.154 INFO:teuthology.orchestra.run.smithi062.stdout:(19/131): python3-ceph-argparse-18.2.0-1181.gf7 383 kB/s | 45 kB 00:00 2023-11-06T01:22:32.156 INFO:teuthology.orchestra.run.smithi053.stdout:(21/131): python3-rados-18.2.0-1181.gf7e9f9af.e 2.4 MB/s | 371 kB 00:00 2023-11-06T01:22:32.196 INFO:teuthology.orchestra.run.smithi062.stdout:(20/131): python3-ceph-common-18.2.0-1181.gf7e9 1.8 MB/s | 123 kB 00:00 2023-11-06T01:22:32.213 INFO:teuthology.orchestra.run.smithi163.stdout:(29/131): ceph-mgr-dashboard-18.2.0-1181.gf7e9f 3.8 MB/s | 1.8 MB 00:00 2023-11-06T01:22:32.241 INFO:teuthology.orchestra.run.smithi053.stdout:(22/131): librgw2-18.2.0-1181.gf7e9f9af.el8.x86 1.0 MB/s | 6.5 MB 00:06 2023-11-06T01:22:32.280 INFO:teuthology.orchestra.run.smithi062.stdout:(21/131): python3-cephfs-18.2.0-1181.gf7e9f9af. 1.5 MB/s | 197 kB 00:00 2023-11-06T01:22:32.330 INFO:teuthology.orchestra.run.smithi062.stdout:(22/131): python3-rados-18.2.0-1181.gf7e9f9af.e 2.7 MB/s | 371 kB 00:00 2023-11-06T01:22:32.382 INFO:teuthology.orchestra.run.smithi053.stdout:(23/131): python3-rbd-18.2.0-1181.gf7e9f9af.el8 1.6 MB/s | 365 kB 00:00 2023-11-06T01:22:32.389 INFO:teuthology.orchestra.run.smithi062.stdout:(23/131): python3-rgw-18.2.0-1181.gf7e9f9af.el8 1.8 MB/s | 107 kB 00:00 2023-11-06T01:22:32.429 INFO:teuthology.orchestra.run.smithi053.stdout:(24/131): python3-rgw-18.2.0-1181.gf7e9f9af.el8 568 kB/s | 107 kB 00:00 2023-11-06T01:22:32.458 INFO:teuthology.orchestra.run.smithi053.stdout:(25/131): rbd-fuse-18.2.0-1181.gf7e9f9af.el8.x8 1.2 MB/s | 94 kB 00:00 2023-11-06T01:22:32.478 INFO:teuthology.orchestra.run.smithi062.stdout:(24/131): rbd-fuse-18.2.0-1181.gf7e9f9af.el8.x8 1.0 MB/s | 94 kB 00:00 2023-11-06T01:22:32.514 INFO:teuthology.orchestra.run.smithi062.stdout:(25/131): python3-rbd-18.2.0-1181.gf7e9f9af.el8 1.5 MB/s | 365 kB 00:00 2023-11-06T01:22:32.558 INFO:teuthology.orchestra.run.smithi053.stdout:(26/131): rbd-nbd-18.2.0-1181.gf7e9f9af.el8.x86 1.8 MB/s | 181 kB 00:00 2023-11-06T01:22:32.631 INFO:teuthology.orchestra.run.smithi062.stdout:(26/131): rbd-nbd-18.2.0-1181.gf7e9f9af.el8.x86 1.5 MB/s | 181 kB 00:00 2023-11-06T01:22:32.633 INFO:teuthology.orchestra.run.smithi053.stdout:(27/131): ceph-grafana-dashboards-18.2.0-1181.g 307 kB/s | 23 kB 00:00 2023-11-06T01:22:32.732 INFO:teuthology.orchestra.run.smithi062.stdout:(27/131): ceph-grafana-dashboards-18.2.0-1181.g 230 kB/s | 23 kB 00:00 2023-11-06T01:22:32.750 INFO:teuthology.orchestra.run.smithi053.stdout:(28/131): ceph-mgr-cephadm-18.2.0-1181.gf7e9f9a 1.1 MB/s | 132 kB 00:00 2023-11-06T01:22:32.799 INFO:teuthology.orchestra.run.smithi062.stdout:(28/131): ceph-mgr-cephadm-18.2.0-1181.gf7e9f9a 1.9 MB/s | 132 kB 00:00 2023-11-06T01:22:33.399 INFO:teuthology.orchestra.run.smithi062.stdout:(29/131): rbd-mirror-18.2.0-1181.gf7e9f9af.el8. 3.7 MB/s | 3.4 MB 00:00 2023-11-06T01:22:33.491 INFO:teuthology.orchestra.run.smithi062.stdout:(30/131): ceph-mgr-dashboard-18.2.0-1181.gf7e9f 2.5 MB/s | 1.8 MB 00:00 2023-11-06T01:22:33.601 INFO:teuthology.orchestra.run.smithi053.stdout:(29/131): ceph-mgr-dashboard-18.2.0-1181.gf7e9f 2.1 MB/s | 1.8 MB 00:00 2023-11-06T01:22:33.616 INFO:teuthology.orchestra.run.smithi062.stdout:(31/131): ceph-mgr-modules-core-18.2.0-1181.gf7 1.9 MB/s | 242 kB 00:00 2023-11-06T01:22:33.692 INFO:teuthology.orchestra.run.smithi062.stdout:(32/131): ceph-mgr-rook-18.2.0-1181.gf7e9f9af.e 633 kB/s | 47 kB 00:00 2023-11-06T01:22:33.787 INFO:teuthology.orchestra.run.smithi062.stdout:(33/131): ceph-prometheus-alerts-18.2.0-1181.gf 153 kB/s | 15 kB 00:00 2023-11-06T01:22:33.909 INFO:teuthology.orchestra.run.smithi062.stdout:(34/131): ceph-volume-18.2.0-1181.gf7e9f9af.el8 2.1 MB/s | 261 kB 00:00 2023-11-06T01:22:33.930 INFO:teuthology.orchestra.run.smithi163.stdout:(30/131): ceph-mgr-diskprediction-local-18.2.0- 4.3 MB/s | 7.4 MB 00:01 2023-11-06T01:22:34.026 INFO:teuthology.orchestra.run.smithi062.stdout:(35/131): cephadm-18.2.0-1181.gf7e9f9af.el8.noa 1.4 MB/s | 171 kB 00:00 2023-11-06T01:22:34.055 INFO:teuthology.orchestra.run.smithi163.stdout:(31/131): ceph-mgr-modules-core-18.2.0-1181.gf7 1.9 MB/s | 242 kB 00:00 2023-11-06T01:22:34.084 INFO:teuthology.orchestra.run.smithi062.stdout:(36/131): cephfs-top-18.2.0-1181.gf7e9f9af.el8. 308 kB/s | 18 kB 00:00 2023-11-06T01:22:34.131 INFO:teuthology.orchestra.run.smithi163.stdout:(32/131): ceph-mgr-rook-18.2.0-1181.gf7e9f9af.e 632 kB/s | 47 kB 00:00 2023-11-06T01:22:34.198 INFO:teuthology.orchestra.run.smithi163.stdout:(33/131): ceph-prometheus-alerts-18.2.0-1181.gf 219 kB/s | 15 kB 00:00 2023-11-06T01:22:34.315 INFO:teuthology.orchestra.run.smithi163.stdout:(34/131): ceph-volume-18.2.0-1181.gf7e9f9af.el8 2.2 MB/s | 261 kB 00:00 2023-11-06T01:22:34.382 INFO:teuthology.orchestra.run.smithi163.stdout:(35/131): cephadm-18.2.0-1181.gf7e9f9af.el8.noa 2.5 MB/s | 171 kB 00:00 2023-11-06T01:22:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2023-11-06T01:22:34.449 INFO:teuthology.orchestra.run.smithi163.stdout:(36/131): cephfs-top-18.2.0-1181.gf7e9f9af.el8. 270 kB/s | 18 kB 00:00 2023-11-06T01:22:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 785 kB/s | 59 kB 00:00 2023-11-06T01:22:34.519 INFO:teuthology.orchestra.run.smithi062.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-11-06T01:22:34.885 INFO:teuthology.orchestra.run.smithi062.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.3 MB/s | 476 kB 00:00 2023-11-06T01:22:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 748 kB/s | 44 kB 00:00 2023-11-06T01:22:35.028 INFO:teuthology.orchestra.run.smithi062.stdout:(42/131): ceph-mgr-diskprediction-local-18.2.0- 4.5 MB/s | 7.4 MB 00:01 2023-11-06T01:22:35.103 INFO:teuthology.orchestra.run.smithi062.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-11-06T01:22:35.170 INFO:teuthology.orchestra.run.smithi062.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 264 kB/s | 16 kB 00:00 2023-11-06T01:22:35.199 INFO:teuthology.orchestra.run.smithi163.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 409 kB/s | 306 kB 00:00 2023-11-06T01:22:35.245 INFO:teuthology.orchestra.run.smithi062.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.3 MB/s | 173 kB 00:00 2023-11-06T01:22:35.318 INFO:teuthology.orchestra.run.smithi053.stdout:(30/131): rbd-mirror-18.2.0-1181.gf7e9f9af.el8. 1.2 MB/s | 3.4 MB 00:02 2023-11-06T01:22:35.351 INFO:teuthology.orchestra.run.smithi163.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 391 kB/s | 59 kB 00:00 2023-11-06T01:22:35.379 INFO:teuthology.orchestra.run.smithi062.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2023-11-06T01:22:35.396 INFO:teuthology.orchestra.run.smithi062.stdout:(47/131): python3-cachetools-3.1.1-4.el8.noarch 90 kB/s | 33 kB 00:00 2023-11-06T01:22:35.454 INFO:teuthology.orchestra.run.smithi062.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-11-06T01:22:35.480 INFO:teuthology.orchestra.run.smithi062.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-11-06T01:22:35.492 INFO:teuthology.orchestra.run.smithi163.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 530 kB/s | 75 kB 00:00 2023-11-06T01:22:35.538 INFO:teuthology.orchestra.run.smithi062.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2023-11-06T01:22:35.576 INFO:teuthology.orchestra.run.smithi053.stdout:(31/131): ceph-mgr-modules-core-18.2.0-1181.gf7 938 kB/s | 242 kB 00:00 2023-11-06T01:22:35.614 INFO:teuthology.orchestra.run.smithi062.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 648 kB/s | 49 kB 00:00 2023-11-06T01:22:35.668 INFO:teuthology.orchestra.run.smithi053.stdout:(32/131): ceph-mgr-diskprediction-local-18.2.0- 3.6 MB/s | 7.4 MB 00:02 2023-11-06T01:22:35.689 INFO:teuthology.orchestra.run.smithi062.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 783 kB/s | 59 kB 00:00 2023-11-06T01:22:35.693 INFO:teuthology.orchestra.run.smithi053.stdout:(33/131): ceph-mgr-rook-18.2.0-1181.gf7e9f9af.e 406 kB/s | 47 kB 00:00 2023-11-06T01:22:35.719 INFO:teuthology.orchestra.run.smithi053.stdout:(34/131): ceph-prometheus-alerts-18.2.0-1181.gf 290 kB/s | 15 kB 00:00 2023-11-06T01:22:35.725 INFO:teuthology.orchestra.run.smithi163.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.0 MB/s | 476 kB 00:00 2023-11-06T01:22:35.772 INFO:teuthology.orchestra.run.smithi062.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 724 kB/s | 60 kB 00:00 2023-11-06T01:22:35.794 INFO:teuthology.orchestra.run.smithi053.stdout:(35/131): cephadm-18.2.0-1181.gf7e9f9af.el8.noa 2.2 MB/s | 171 kB 00:00 2023-11-06T01:22:35.823 INFO:teuthology.orchestra.run.smithi062.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 3.4 MB/s | 1.2 MB 00:00 2023-11-06T01:22:35.850 INFO:teuthology.orchestra.run.smithi163.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 349 kB/s | 44 kB 00:00 2023-11-06T01:22:35.852 INFO:teuthology.orchestra.run.smithi053.stdout:(36/131): cephfs-top-18.2.0-1181.gf7e9f9af.el8. 309 kB/s | 18 kB 00:00 2023-11-06T01:22:35.857 INFO:teuthology.orchestra.run.smithi062.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-11-06T01:22:35.899 INFO:teuthology.orchestra.run.smithi062.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 206 kB/s | 16 kB 00:00 2023-11-06T01:22:35.953 INFO:teuthology.orchestra.run.smithi053.stdout:(37/131): ceph-volume-18.2.0-1181.gf7e9f9af.el8 1.0 MB/s | 261 kB 00:00 2023-11-06T01:22:35.963 INFO:teuthology.orchestra.run.smithi062.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 319 kB/s | 34 kB 00:00 2023-11-06T01:22:35.984 INFO:teuthology.orchestra.run.smithi163.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-11-06T01:22:35.999 INFO:teuthology.orchestra.run.smithi062.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-11-06T01:22:36.033 INFO:teuthology.orchestra.run.smithi062.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 921 kB/s | 64 kB 00:00 2023-11-06T01:22:36.066 INFO:teuthology.orchestra.run.smithi062.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 280 kB/s | 19 kB 00:00 2023-11-06T01:22:36.109 INFO:teuthology.orchestra.run.smithi163.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 264 kB/s | 33 kB 00:00 2023-11-06T01:22:36.116 INFO:teuthology.orchestra.run.smithi062.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 287 kB/s | 24 kB 00:00 2023-11-06T01:22:36.142 INFO:teuthology.orchestra.run.smithi062.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 388 kB/s | 29 kB 00:00 2023-11-06T01:22:36.209 INFO:teuthology.orchestra.run.smithi062.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 299 kB/s | 27 kB 00:00 2023-11-06T01:22:36.234 INFO:teuthology.orchestra.run.smithi062.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-11-06T01:22:36.235 INFO:teuthology.orchestra.run.smithi163.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-11-06T01:22:36.245 INFO:teuthology.orchestra.run.smithi053.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 202 kB/s | 59 kB 00:00 2023-11-06T01:22:36.267 INFO:teuthology.orchestra.run.smithi062.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.2 MB/s | 251 kB 00:00 2023-11-06T01:22:36.293 INFO:teuthology.orchestra.run.smithi062.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-11-06T01:22:36.335 INFO:teuthology.orchestra.run.smithi062.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-11-06T01:22:36.354 INFO:teuthology.orchestra.run.smithi053.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 612 kB/s | 306 kB 00:00 2023-11-06T01:22:36.361 INFO:teuthology.orchestra.run.smithi062.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 343 kB/s | 23 kB 00:00 2023-11-06T01:22:36.376 INFO:teuthology.orchestra.run.smithi163.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.2 MB/s | 173 kB 00:00 2023-11-06T01:22:36.379 INFO:teuthology.orchestra.run.smithi053.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 562 kB/s | 75 kB 00:00 2023-11-06T01:22:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-11-06T01:22:36.463 INFO:teuthology.orchestra.run.smithi053.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 522 kB/s | 44 kB 00:00 2023-11-06T01:22:36.477 INFO:teuthology.orchestra.run.smithi062.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 804 kB/s | 47 kB 00:00 2023-11-06T01:22:36.519 INFO:teuthology.orchestra.run.smithi062.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 9.4 MB/s | 1.7 MB 00:00 2023-11-06T01:22:36.543 INFO:teuthology.orchestra.run.smithi163.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.3 MB/s | 384 kB 00:00 2023-11-06T01:22:36.544 INFO:teuthology.orchestra.run.smithi062.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.9 MB/s | 200 kB 00:00 2023-11-06T01:22:36.580 INFO:teuthology.orchestra.run.smithi053.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.6 MB/s | 185 kB 00:00 2023-11-06T01:22:36.617 INFO:teuthology.orchestra.run.smithi062.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 436 kB/s | 43 kB 00:00 2023-11-06T01:22:36.630 INFO:teuthology.orchestra.run.smithi053.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 662 kB/s | 33 kB 00:00 2023-11-06T01:22:36.653 INFO:teuthology.orchestra.run.smithi062.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.1 MB/s | 220 kB 00:00 2023-11-06T01:22:36.668 INFO:teuthology.orchestra.run.smithi163.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 651 kB/s | 81 kB 00:00 2023-11-06T01:22:36.679 INFO:teuthology.orchestra.run.smithi062.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 790 kB/s | 48 kB 00:00 2023-11-06T01:22:36.689 INFO:teuthology.orchestra.run.smithi053.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 279 kB/s | 16 kB 00:00 2023-11-06T01:22:36.704 INFO:teuthology.orchestra.run.smithi062.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-06T01:22:36.729 INFO:teuthology.orchestra.run.smithi062.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 849 kB/s | 43 kB 00:00 2023-11-06T01:22:36.755 INFO:teuthology.orchestra.run.smithi062.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-11-06T01:22:36.780 INFO:teuthology.orchestra.run.smithi062.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-11-06T01:22:36.780 INFO:teuthology.orchestra.run.smithi053.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.1 MB/s | 476 kB 00:00 2023-11-06T01:22:36.794 INFO:teuthology.orchestra.run.smithi163.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 89 kB/s | 11 kB 00:00 2023-11-06T01:22:36.805 INFO:teuthology.orchestra.run.smithi053.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2023-11-06T01:22:36.813 INFO:teuthology.orchestra.run.smithi062.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-11-06T01:22:36.856 INFO:teuthology.orchestra.run.smithi053.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-11-06T01:22:36.872 INFO:teuthology.orchestra.run.smithi062.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 203 kB/s | 19 kB 00:00 2023-11-06T01:22:36.897 INFO:teuthology.orchestra.run.smithi062.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.8 MB/s | 155 kB 00:00 2023-11-06T01:22:36.919 INFO:teuthology.orchestra.run.smithi163.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 141 kB/s | 18 kB 00:00 2023-11-06T01:22:36.923 INFO:teuthology.orchestra.run.smithi053.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2023-11-06T01:22:36.931 INFO:teuthology.orchestra.run.smithi062.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.4 MB/s | 559 kB 00:00 2023-11-06T01:22:36.956 INFO:teuthology.orchestra.run.smithi062.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-11-06T01:22:36.990 INFO:teuthology.orchestra.run.smithi053.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 264 kB/s | 18 kB 00:00 2023-11-06T01:22:36.991 INFO:teuthology.orchestra.run.smithi062.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-11-06T01:22:37.015 INFO:teuthology.orchestra.run.smithi053.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.6 MB/s | 384 kB 00:00 2023-11-06T01:22:37.016 INFO:teuthology.orchestra.run.smithi062.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.9 MB/s | 238 kB 00:00 2023-11-06T01:22:37.082 INFO:teuthology.orchestra.run.smithi053.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 731 kB/s | 49 kB 00:00 2023-11-06T01:22:37.091 INFO:teuthology.orchestra.run.smithi062.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 966 kB/s | 72 kB 00:00 2023-11-06T01:22:37.133 INFO:teuthology.orchestra.run.smithi062.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 2.0 MB/s | 83 kB 00:00 2023-11-06T01:22:37.152 INFO:teuthology.orchestra.run.smithi163.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2023-11-06T01:22:37.183 INFO:teuthology.orchestra.run.smithi062.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 12 MB/s | 2.3 MB 00:00 2023-11-06T01:22:37.191 INFO:teuthology.orchestra.run.smithi053.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 542 kB/s | 59 kB 00:00 2023-11-06T01:22:37.200 INFO:teuthology.orchestra.run.smithi062.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 895 kB/s | 60 kB 00:00 2023-11-06T01:22:37.249 INFO:teuthology.orchestra.run.smithi053.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-11-06T01:22:37.286 INFO:teuthology.orchestra.run.smithi163.stdout:(51/131): ceph-radosgw-18.2.0-1181.gf7e9f9af.el 734 kB/s | 13 MB 00:18 2023-11-06T01:22:37.311 INFO:teuthology.orchestra.run.smithi163.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 307 kB/s | 49 kB 00:00 2023-11-06T01:22:37.316 INFO:teuthology.orchestra.run.smithi053.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 3.6 MB/s | 1.2 MB 00:00 2023-11-06T01:22:37.367 INFO:teuthology.orchestra.run.smithi053.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 313 kB/s | 16 kB 00:00 2023-11-06T01:22:37.375 INFO:teuthology.orchestra.run.smithi062.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 512 kB/s | 98 kB 00:00 2023-11-06T01:22:37.392 INFO:teuthology.orchestra.run.smithi053.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2023-11-06T01:22:37.401 INFO:teuthology.orchestra.run.smithi062.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 305 kB/s | 58 kB 00:00 2023-11-06T01:22:37.418 INFO:teuthology.orchestra.run.smithi062.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 920 kB/s | 39 kB 00:00 2023-11-06T01:22:37.425 INFO:teuthology.orchestra.run.smithi053.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 574 kB/s | 34 kB 00:00 2023-11-06T01:22:37.445 INFO:teuthology.orchestra.run.smithi163.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 452 kB/s | 60 kB 00:00 2023-11-06T01:22:37.459 INFO:teuthology.orchestra.run.smithi062.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-11-06T01:22:37.484 INFO:teuthology.orchestra.run.smithi053.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-11-06T01:22:37.485 INFO:teuthology.orchestra.run.smithi062.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-11-06T01:22:37.510 INFO:teuthology.orchestra.run.smithi053.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2023-11-06T01:22:37.551 INFO:teuthology.orchestra.run.smithi053.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 280 kB/s | 19 kB 00:00 2023-11-06T01:22:37.552 INFO:teuthology.orchestra.run.smithi062.stdout:(96/131): NetworkManager-initscripts-updown-1.3 1.5 MB/s | 138 kB 00:00 2023-11-06T01:22:37.577 INFO:teuthology.orchestra.run.smithi053.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 357 kB/s | 24 kB 00:00 2023-11-06T01:22:37.602 INFO:teuthology.orchestra.run.smithi062.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.2 MB/s | 110 kB 00:00 2023-11-06T01:22:37.644 INFO:teuthology.orchestra.run.smithi053.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2023-11-06T01:22:37.652 INFO:teuthology.orchestra.run.smithi062.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-11-06T01:22:37.669 INFO:teuthology.orchestra.run.smithi053.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 300 kB/s | 27 kB 00:00 2023-11-06T01:22:37.695 INFO:teuthology.orchestra.run.smithi163.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2023-11-06T01:22:37.702 INFO:teuthology.orchestra.run.smithi062.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-11-06T01:22:37.770 INFO:teuthology.orchestra.run.smithi062.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-11-06T01:22:37.787 INFO:teuthology.orchestra.run.smithi163.stdout:(55/131): python3-more-itertools-7.2.0-3.el8.no 117 kB/s | 59 kB 00:00 2023-11-06T01:22:37.812 INFO:teuthology.orchestra.run.smithi163.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 134 kB/s | 16 kB 00:00 2023-11-06T01:22:37.820 INFO:teuthology.orchestra.run.smithi062.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 777 kB/s | 39 kB 00:00 2023-11-06T01:22:37.878 INFO:teuthology.orchestra.run.smithi062.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-11-06T01:22:37.929 INFO:teuthology.orchestra.run.smithi163.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 237 kB/s | 34 kB 00:00 2023-11-06T01:22:37.955 INFO:teuthology.orchestra.run.smithi163.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-11-06T01:22:38.054 INFO:teuthology.orchestra.run.smithi062.stdout:(103/131): python3-scipy-1.0.0-21.module+el8.5. 24 MB/s | 14 MB 00:00 2023-11-06T01:22:38.087 INFO:teuthology.orchestra.run.smithi062.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 2.5 MB/s | 538 kB 00:00 2023-11-06T01:22:38.088 INFO:teuthology.orchestra.run.smithi163.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2023-11-06T01:22:38.111 INFO:teuthology.orchestra.run.smithi053.stdout:(64/131): ceph-test-18.2.0-1181.gf7e9f9af.el8.x 3.4 MB/s | 47 MB 00:13 2023-11-06T01:22:38.112 INFO:teuthology.orchestra.run.smithi062.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-11-06T01:22:38.113 INFO:teuthology.orchestra.run.smithi163.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 349 kB/s | 64 kB 00:00 2023-11-06T01:22:38.136 INFO:teuthology.orchestra.run.smithi053.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 519 kB/s | 255 kB 00:00 2023-11-06T01:22:38.138 INFO:teuthology.orchestra.run.smithi062.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-11-06T01:22:38.161 INFO:teuthology.orchestra.run.smithi053.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 510 kB/s | 251 kB 00:00 2023-11-06T01:22:38.196 INFO:teuthology.orchestra.run.smithi062.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 8.3 MB/s | 713 kB 00:00 2023-11-06T01:22:38.228 INFO:teuthology.orchestra.run.smithi053.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 371 kB/s | 23 kB 00:00 2023-11-06T01:22:38.230 INFO:teuthology.orchestra.run.smithi163.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 168 kB/s | 24 kB 00:00 2023-11-06T01:22:38.247 INFO:teuthology.orchestra.run.smithi062.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 531 kB/s | 27 kB 00:00 2023-11-06T01:22:38.256 INFO:teuthology.orchestra.run.smithi163.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 206 kB/s | 29 kB 00:00 2023-11-06T01:22:38.279 INFO:teuthology.orchestra.run.smithi053.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 761 kB/s | 85 kB 00:00 2023-11-06T01:22:38.297 INFO:teuthology.orchestra.run.smithi062.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.2 MB/s | 320 kB 00:00 2023-11-06T01:22:38.304 INFO:teuthology.orchestra.run.smithi053.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-11-06T01:22:38.356 INFO:teuthology.orchestra.run.smithi163.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 220 kB/s | 27 kB 00:00 2023-11-06T01:22:38.364 INFO:teuthology.orchestra.run.smithi062.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-11-06T01:22:38.371 INFO:teuthology.orchestra.run.smithi053.stdout:(70/131): python3-websocket-client-0.56.0-5.el8 236 kB/s | 61 kB 00:00 2023-11-06T01:22:38.422 INFO:teuthology.orchestra.run.smithi062.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 351 kB/s | 44 kB 00:00 2023-11-06T01:22:38.429 INFO:teuthology.orchestra.run.smithi053.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-11-06T01:22:38.472 INFO:teuthology.orchestra.run.smithi053.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-11-06T01:22:38.481 INFO:teuthology.orchestra.run.smithi062.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-11-06T01:22:38.498 INFO:teuthology.orchestra.run.smithi163.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.7 MB/s | 251 kB 00:00 2023-11-06T01:22:38.539 INFO:teuthology.orchestra.run.smithi053.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2023-11-06T01:22:38.557 INFO:teuthology.orchestra.run.smithi163.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 850 kB/s | 255 kB 00:00 2023-11-06T01:22:38.565 INFO:teuthology.orchestra.run.smithi062.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 19 MB/s | 3.7 MB 00:00 2023-11-06T01:22:38.580 INFO:teuthology.orchestra.run.smithi053.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-11-06T01:22:38.616 INFO:teuthology.orchestra.run.smithi163.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 522 kB/s | 61 kB 00:00 2023-11-06T01:22:38.621 INFO:teuthology.orchestra.run.smithi062.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.6 MB/s | 225 kB 00:00 2023-11-06T01:22:38.622 INFO:teuthology.orchestra.run.smithi053.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 932 kB/s | 39 kB 00:00 2023-11-06T01:22:38.657 INFO:teuthology.orchestra.run.smithi062.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 695 kB/s | 64 kB 00:00 2023-11-06T01:22:38.673 INFO:teuthology.orchestra.run.smithi053.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-11-06T01:22:38.682 INFO:teuthology.orchestra.run.smithi062.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.0 MB/s | 126 kB 00:00 2023-11-06T01:22:38.699 INFO:teuthology.orchestra.run.smithi163.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 600 kB/s | 85 kB 00:00 2023-11-06T01:22:38.707 INFO:teuthology.orchestra.run.smithi053.stdout:(77/131): thrift-0.13.0-2.el8.x86_64.rpm 3.6 MB/s | 1.7 MB 00:00 2023-11-06T01:22:38.708 INFO:teuthology.orchestra.run.smithi062.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 399 kB/s | 20 kB 00:00 2023-11-06T01:22:38.725 INFO:teuthology.orchestra.run.smithi062.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-11-06T01:22:38.733 INFO:teuthology.orchestra.run.smithi163.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-11-06T01:22:38.749 INFO:teuthology.orchestra.run.smithi053.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2023-11-06T01:22:38.749 INFO:teuthology.orchestra.run.smithi163.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-11-06T01:22:38.774 INFO:teuthology.orchestra.run.smithi053.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-11-06T01:22:38.775 INFO:teuthology.orchestra.run.smithi062.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-11-06T01:22:38.790 INFO:teuthology.orchestra.run.smithi053.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-06T01:22:38.833 INFO:teuthology.orchestra.run.smithi053.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-11-06T01:22:38.858 INFO:teuthology.orchestra.run.smithi053.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 1.0 MB/s | 27 kB 00:00 2023-11-06T01:22:38.859 INFO:teuthology.orchestra.run.smithi062.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2023-11-06T01:22:38.900 INFO:teuthology.orchestra.run.smithi053.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-06T01:22:38.942 INFO:teuthology.orchestra.run.smithi062.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 22 MB/s | 4.9 MB 00:00 2023-11-06T01:22:38.992 INFO:teuthology.orchestra.run.smithi062.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-11-06T01:22:39.050 INFO:teuthology.orchestra.run.smithi163.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 46 MB/s | 14 MB 00:00 2023-11-06T01:22:39.058 INFO:teuthology.orchestra.run.smithi053.stdout:(84/131): python3-scipy-1.0.0-21.module+el8.5.0 18 MB/s | 14 MB 00:00 2023-11-06T01:22:39.076 INFO:teuthology.orchestra.run.smithi053.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 251 kB/s | 44 kB 00:00 2023-11-06T01:22:39.092 INFO:teuthology.orchestra.run.smithi163.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-11-06T01:22:39.101 INFO:teuthology.orchestra.run.smithi062.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-11-06T01:22:39.126 INFO:teuthology.orchestra.run.smithi053.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-11-06T01:22:39.127 INFO:teuthology.orchestra.run.smithi062.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-11-06T01:22:39.133 INFO:teuthology.orchestra.run.smithi163.stdout:(72/131): thrift-0.13.0-2.el8.x86_64.rpm 4.0 MB/s | 1.7 MB 00:00 2023-11-06T01:22:39.159 INFO:teuthology.orchestra.run.smithi163.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 809 kB/s | 54 kB 00:00 2023-11-06T01:22:39.202 INFO:teuthology.orchestra.run.smithi062.stdout:(125/131): postgresql-contrib-10.21-2.module+el 11 MB/s | 808 kB 00:00 2023-11-06T01:22:39.210 INFO:teuthology.orchestra.run.smithi163.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2023-11-06T01:22:39.210 INFO:teuthology.orchestra.run.smithi053.stdout:(87/131): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2023-11-06T01:22:39.235 INFO:teuthology.orchestra.run.smithi053.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.0 MB/s | 225 kB 00:00 2023-11-06T01:22:39.235 INFO:teuthology.orchestra.run.smithi163.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.5 MB/s | 323 kB 00:00 2023-11-06T01:22:39.260 INFO:teuthology.orchestra.run.smithi053.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2023-11-06T01:22:39.260 INFO:teuthology.orchestra.run.smithi163.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 775 kB/s | 39 kB 00:00 2023-11-06T01:22:39.285 INFO:teuthology.orchestra.run.smithi053.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-11-06T01:22:39.302 INFO:teuthology.orchestra.run.smithi163.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-11-06T01:22:39.310 INFO:teuthology.orchestra.run.smithi053.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 397 kB/s | 20 kB 00:00 2023-11-06T01:22:39.327 INFO:teuthology.orchestra.run.smithi062.stdout:(126/131): postgresql-server-10.21-2.module+el8 22 MB/s | 5.1 MB 00:00 2023-11-06T01:22:39.336 INFO:teuthology.orchestra.run.smithi053.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-11-06T01:22:39.344 INFO:teuthology.orchestra.run.smithi163.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 6.3 MB/s | 538 kB 00:00 2023-11-06T01:22:39.369 INFO:teuthology.orchestra.run.smithi163.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-11-06T01:22:39.385 INFO:teuthology.orchestra.run.smithi062.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 8.2 MB/s | 1.5 MB 00:00 2023-11-06T01:22:39.386 INFO:teuthology.orchestra.run.smithi163.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-06T01:22:39.411 INFO:teuthology.orchestra.run.smithi053.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 6.9 MB/s | 709 kB 00:00 2023-11-06T01:22:39.428 INFO:teuthology.orchestra.run.smithi163.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-11-06T01:22:39.462 INFO:teuthology.orchestra.run.smithi163.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 802 kB/s | 27 kB 00:00 2023-11-06T01:22:39.486 INFO:teuthology.orchestra.run.smithi053.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 8.7 MB/s | 3.7 MB 00:00 2023-11-06T01:22:39.504 INFO:teuthology.orchestra.run.smithi163.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-06T01:22:39.512 INFO:teuthology.orchestra.run.smithi053.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.9 MB/s | 198 kB 00:00 2023-11-06T01:22:39.537 INFO:teuthology.orchestra.run.smithi163.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-11-06T01:22:39.605 INFO:teuthology.orchestra.run.smithi163.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-11-06T01:22:39.646 INFO:teuthology.orchestra.run.smithi163.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-11-06T01:22:39.687 INFO:teuthology.orchestra.run.smithi053.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2023-11-06T01:22:39.688 INFO:teuthology.orchestra.run.smithi163.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-11-06T01:22:39.746 INFO:teuthology.orchestra.run.smithi163.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 18 MB/s | 3.7 MB 00:00 2023-11-06T01:22:39.794 INFO:teuthology.orchestra.run.smithi163.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 607 kB/s | 64 kB 00:00 2023-11-06T01:22:39.815 INFO:teuthology.orchestra.run.smithi053.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.7 MB/s | 354 kB 00:00 2023-11-06T01:22:39.822 INFO:teuthology.orchestra.run.smithi163.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.6 MB/s | 126 kB 00:00 2023-11-06T01:22:39.848 INFO:teuthology.orchestra.run.smithi163.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 373 kB/s | 20 kB 00:00 2023-11-06T01:22:39.873 INFO:teuthology.orchestra.run.smithi163.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-11-06T01:22:39.907 INFO:teuthology.orchestra.run.smithi163.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-11-06T01:22:39.938 INFO:teuthology.orchestra.run.smithi053.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 8.1 MB/s | 4.9 MB 00:00 2023-11-06T01:22:39.948 INFO:teuthology.orchestra.run.smithi163.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-11-06T01:22:40.013 INFO:teuthology.orchestra.run.smithi053.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 9.0 MB/s | 4.7 MB 00:00 2023-11-06T01:22:40.049 INFO:teuthology.orchestra.run.smithi163.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 28 MB/s | 4.9 MB 00:00 2023-11-06T01:22:40.055 INFO:teuthology.orchestra.run.smithi053.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-11-06T01:22:40.091 INFO:teuthology.orchestra.run.smithi163.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-11-06T01:22:40.094 INFO:teuthology.orchestra.run.smithi062.stdout:(128/131): librados2-18.2.0-1181.gf7e9f9af.el8. 4.8 MB/s | 3.7 MB 00:00 2023-11-06T01:22:40.097 INFO:teuthology.orchestra.run.smithi053.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-11-06T01:22:40.139 INFO:teuthology.orchestra.run.smithi053.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-11-06T01:22:40.161 INFO:teuthology.orchestra.run.smithi062.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-11-06T01:22:40.164 INFO:teuthology.orchestra.run.smithi053.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-11-06T01:22:40.166 INFO:teuthology.orchestra.run.smithi163.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-11-06T01:22:40.190 INFO:teuthology.orchestra.run.smithi053.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-11-06T01:22:40.191 INFO:teuthology.orchestra.run.smithi163.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2023-11-06T01:22:40.215 INFO:teuthology.orchestra.run.smithi053.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-06T01:22:40.242 INFO:teuthology.orchestra.run.smithi163.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 16 MB/s | 808 kB 00:00 2023-11-06T01:22:40.260 INFO:teuthology.orchestra.run.smithi053.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 684 kB/s | 48 kB 00:00 2023-11-06T01:22:40.290 INFO:teuthology.orchestra.run.smithi053.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2023-11-06T01:22:40.316 INFO:teuthology.orchestra.run.smithi053.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 776 kB/s | 43 kB 00:00 2023-11-06T01:22:40.341 INFO:teuthology.orchestra.run.smithi053.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-11-06T01:22:40.342 INFO:teuthology.orchestra.run.smithi163.stdout:(100/131): postgresql-server-10.21-2.module+el8 29 MB/s | 5.1 MB 00:00 2023-11-06T01:22:40.361 INFO:teuthology.orchestra.run.smithi062.stdout:(130/131): librbd1-18.2.0-1181.gf7e9f9af.el8.x8 3.7 MB/s | 3.6 MB 00:00 2023-11-06T01:22:40.375 INFO:teuthology.orchestra.run.smithi163.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-11-06T01:22:40.401 INFO:teuthology.orchestra.run.smithi163.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 800 kB/s | 47 kB 00:00 2023-11-06T01:22:40.408 INFO:teuthology.orchestra.run.smithi053.stdout:(110/131): postgresql-server-10.21-2.module+el8 8.5 MB/s | 5.1 MB 00:00 2023-11-06T01:22:40.426 INFO:teuthology.orchestra.run.smithi163.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-11-06T01:22:40.433 INFO:teuthology.orchestra.run.smithi053.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.9 MB/s | 229 kB 00:00 2023-11-06T01:22:40.452 INFO:teuthology.orchestra.run.smithi163.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 845 kB/s | 43 kB 00:00 2023-11-06T01:22:40.458 INFO:teuthology.orchestra.run.smithi053.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 373 kB/s | 19 kB 00:00 2023-11-06T01:22:40.477 INFO:teuthology.orchestra.run.smithi163.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-06T01:22:40.484 INFO:teuthology.orchestra.run.smithi053.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-11-06T01:22:40.502 INFO:teuthology.orchestra.run.smithi163.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 961 kB/s | 48 kB 00:00 2023-11-06T01:22:40.517 INFO:teuthology.orchestra.run.smithi053.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.1 MB/s | 560 kB 00:00 2023-11-06T01:22:40.527 INFO:teuthology.orchestra.run.smithi163.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-06T01:22:40.534 INFO:teuthology.orchestra.run.smithi053.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 357 kB/s | 18 kB 00:00 2023-11-06T01:22:40.553 INFO:teuthology.orchestra.run.smithi163.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 846 kB/s | 43 kB 00:00 2023-11-06T01:22:40.568 INFO:teuthology.orchestra.run.smithi053.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 5.0 MB/s | 559 kB 00:00 2023-11-06T01:22:40.569 INFO:teuthology.orchestra.run.smithi163.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.5 MB/s | 109 kB 00:00 2023-11-06T01:22:40.595 INFO:teuthology.orchestra.run.smithi163.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.3 MB/s | 229 kB 00:00 2023-11-06T01:22:40.601 INFO:teuthology.orchestra.run.smithi053.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2023-11-06T01:22:40.618 INFO:teuthology.orchestra.run.smithi053.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-11-06T01:22:40.628 INFO:teuthology.orchestra.run.smithi163.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-11-06T01:22:40.644 INFO:teuthology.orchestra.run.smithi053.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-11-06T01:22:40.653 INFO:teuthology.orchestra.run.smithi163.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 319 kB/s | 19 kB 00:00 2023-11-06T01:22:40.679 INFO:teuthology.orchestra.run.smithi163.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-11-06T01:22:40.702 INFO:teuthology.orchestra.run.smithi053.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 991 kB/s | 83 kB 00:00 2023-11-06T01:22:40.712 INFO:teuthology.orchestra.run.smithi163.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-11-06T01:22:40.727 INFO:teuthology.orchestra.run.smithi053.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 717 kB/s | 60 kB 00:00 2023-11-06T01:22:40.744 INFO:teuthology.orchestra.run.smithi053.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-11-06T01:22:40.754 INFO:teuthology.orchestra.run.smithi163.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 237 kB/s | 18 kB 00:00 2023-11-06T01:22:40.770 INFO:teuthology.orchestra.run.smithi053.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-11-06T01:22:40.788 INFO:teuthology.orchestra.run.smithi163.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.8 MB/s | 605 kB 00:00 2023-11-06T01:22:40.795 INFO:teuthology.orchestra.run.smithi053.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 767 kB/s | 39 kB 00:00 2023-11-06T01:22:40.813 INFO:teuthology.orchestra.run.smithi163.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-11-06T01:22:40.837 INFO:teuthology.orchestra.run.smithi053.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-11-06T01:22:40.849 INFO:teuthology.orchestra.run.smithi163.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 2.1 MB/s | 72 kB 00:00 2023-11-06T01:22:40.862 INFO:teuthology.orchestra.run.smithi053.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-11-06T01:22:40.887 INFO:teuthology.orchestra.run.smithi053.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-11-06T01:22:40.897 INFO:teuthology.orchestra.run.smithi163.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 21 MB/s | 2.3 MB 00:00 2023-11-06T01:22:40.923 INFO:teuthology.orchestra.run.smithi163.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-11-06T01:22:40.938 INFO:teuthology.orchestra.run.smithi053.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.3 MB/s | 2.3 MB 00:00 2023-11-06T01:22:40.948 INFO:teuthology.orchestra.run.smithi163.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-11-06T01:22:40.973 INFO:teuthology.orchestra.run.smithi163.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-11-06T01:22:40.988 INFO:teuthology.orchestra.run.smithi053.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-06T01:22:40.998 INFO:teuthology.orchestra.run.smithi163.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-11-06T01:22:41.024 INFO:teuthology.orchestra.run.smithi163.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 763 kB/s | 39 kB 00:00 2023-11-06T01:22:41.057 INFO:teuthology.orchestra.run.smithi163.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-11-06T01:22:41.083 INFO:teuthology.orchestra.run.smithi163.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-11-06T01:22:41.108 INFO:teuthology.orchestra.run.smithi163.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-11-06T01:22:41.714 INFO:teuthology.orchestra.run.smithi053.stdout:(130/131): librados2-18.2.0-1181.gf7e9f9af.el8. 4.3 MB/s | 3.7 MB 00:00 2023-11-06T01:22:42.008 INFO:teuthology.orchestra.run.smithi163.stdout:(128/131): librbd1-18.2.0-1181.gf7e9f9af.el8.x8 4.0 MB/s | 3.6 MB 00:00 2023-11-06T01:22:42.059 INFO:teuthology.orchestra.run.smithi163.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-06T01:22:43.422 INFO:teuthology.orchestra.run.smithi053.stdout:(131/131): librbd1-18.2.0-1181.gf7e9f9af.el8.x8 1.4 MB/s | 3.6 MB 00:02 2023-11-06T01:22:43.427 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2023-11-06T01:22:43.427 INFO:teuthology.orchestra.run.smithi053.stdout:Total 6.8 MB/s | 203 MB 00:30 2023-11-06T01:22:44.159 INFO:teuthology.orchestra.run.smithi163.stdout:(130/131): librados2-18.2.0-1181.gf7e9f9af.el8. 1.2 MB/s | 3.7 MB 00:03 2023-11-06T01:22:44.251 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2023-11-06T01:22:44.384 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2023-11-06T01:22:44.384 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2023-11-06T01:22:45.554 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2023-11-06T01:22:45.558 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2023-11-06T01:22:50.540 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2023-11-06T01:22:50.739 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-06T01:22:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-06T01:22:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-06T01:22:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-06T01:22:51.442 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-06T01:22:51.690 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-06T01:22:52.119 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-06T01:22:52.307 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 6/134 2023-11-06T01:22:52.711 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 6/134 2023-11-06T01:22:52.907 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 7/134 2023-11-06T01:22:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 7/134 2023-11-06T01:22:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 8/134 2023-11-06T01:22:53.349 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 8/134 2023-11-06T01:22:53.525 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:18.2.0-1181.gf7e9f9af.el8.x86_64 9/134 2023-11-06T01:22:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-06T01:22:53.953 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-06T01:22:54.142 INFO:teuthology.orchestra.run.smithi163.stdout:(131/131): ceph-test-18.2.0-1181.gf7e9f9af.el8. 1.4 MB/s | 47 MB 00:32 2023-11-06T01:22:54.143 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2023-11-06T01:22:54.143 INFO:teuthology.orchestra.run.smithi163.stdout:Total 4.9 MB/s | 203 MB 00:41 2023-11-06T01:22:54.354 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-06T01:22:54.631 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-06T01:22:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 14/134 2023-11-06T01:22:54.869 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 14/134 2023-11-06T01:22:54.964 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2023-11-06T01:22:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86 15/134 2023-11-06T01:22:55.084 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2023-11-06T01:22:55.084 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2023-11-06T01:22:55.129 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86 15/134 2023-11-06T01:22:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-06T01:22:55.417 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-06T01:22:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-06T01:22:56.181 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2023-11-06T01:22:56.185 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2023-11-06T01:22:56.254 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-06T01:22:56.481 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-06T01:22:57.117 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-06T01:22:57.323 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-06T01:22:58.086 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-06T01:22:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-06T01:22:58.708 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-06T01:22:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-06T01:22:59.131 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-06T01:22:59.353 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-06T01:22:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-06T01:22:59.763 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:18.2.0-1181.gf7e9f9af.el8. 26/134 2023-11-06T01:22:59.919 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:18.2.0-1181.gf7e9f9af.el 27/134 2023-11-06T01:23:00.026 INFO:teuthology.orchestra.run.smithi062.stdout:(131/131): ceph-test-18.2.0-1181.gf7e9f9af.el8. 1.4 MB/s | 47 MB 00:32 2023-11-06T01:23:00.030 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-11-06T01:23:00.030 INFO:teuthology.orchestra.run.smithi062.stdout:Total 4.5 MB/s | 203 MB 00:44 2023-11-06T01:23:00.863 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-11-06T01:23:00.887 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:18.2.0-1181.gf7e9f9af.el8.x86_6 28/134 2023-11-06T01:23:00.967 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2023-11-06T01:23:00.993 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-11-06T01:23:00.993 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-11-06T01:23:01.172 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-06T01:23:01.212 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-06T01:23:01.341 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-06T01:23:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-06T01:23:01.556 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-06T01:23:01.741 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-06T01:23:01.766 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-06T01:23:01.847 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-06T01:23:01.947 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-06T01:23:02.116 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-06T01:23:02.178 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-11-06T01:23:02.180 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-11-06T01:23:02.514 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-06T01:23:02.724 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 6/134 2023-11-06T01:23:03.121 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 6/134 2023-11-06T01:23:03.332 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 7/134 2023-11-06T01:23:03.458 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 7/134 2023-11-06T01:23:03.616 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 8/134 2023-11-06T01:23:03.712 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 8/134 2023-11-06T01:23:03.833 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-06T01:23:03.882 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:18.2.0-1181.gf7e9f9af.el8.x86_64 9/134 2023-11-06T01:23:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-06T01:23:04.162 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-06T01:23:04.377 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-06T01:23:04.396 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-06T01:23:04.594 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-06T01:23:04.768 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-06T01:23:04.803 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 36/134 2023-11-06T01:23:05.001 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_6 37/134 2023-11-06T01:23:05.047 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-06T01:23:05.198 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 14/134 2023-11-06T01:23:05.206 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-06T01:23:05.301 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 14/134 2023-11-06T01:23:05.457 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86 15/134 2023-11-06T01:23:05.504 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-06T01:23:05.540 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86 15/134 2023-11-06T01:23:05.673 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-06T01:23:05.804 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-06T01:23:05.965 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-06T01:23:06.008 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-06T01:23:06.169 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-06T01:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-06T01:23:06.581 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-06T01:23:06.608 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-06T01:23:06.807 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-06T01:23:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-06T01:23:07.406 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-06T01:23:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-06T01:23:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-06T01:23:07.623 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-06T01:23:07.661 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-11-06T01:23:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-06T01:23:07.863 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-06T01:23:08.097 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-06T01:23:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-06T01:23:08.298 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-06T01:23:08.352 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-06T01:23:08.410 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-06T01:23:08.479 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-06T01:23:08.630 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-06T01:23:08.656 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-06T01:23:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-06T01:23:08.850 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-06T01:23:08.868 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-06T01:23:09.068 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-06T01:23:09.105 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-06T01:23:09.111 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-06T01:23:09.270 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-06T01:23:09.274 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-06T01:23:09.381 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 5/134 2023-11-06T01:23:09.435 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-06T01:23:09.723 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-06T01:23:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-06T01:23:09.780 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 5/134 2023-11-06T01:23:09.956 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 6/134 2023-11-06T01:23:09.959 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-06T01:23:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-06T01:23:10.112 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 6/134 2023-11-06T01:23:10.120 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:18.2.0-1181.gf7e9f9af.el8. 26/134 2023-11-06T01:23:10.269 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:18.2.0-1181.gf7e9f9af.el 27/134 2023-11-06T01:23:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-06T01:23:10.331 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 7/134 2023-11-06T01:23:10.415 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 7/134 2023-11-06T01:23:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-06T01:23:10.610 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:18.2.0-1181.gf7e9f9af.el8.x86_64 8/134 2023-11-06T01:23:10.704 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-06T01:23:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-06T01:23:10.890 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-06T01:23:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-06T01:23:11.046 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-06T01:23:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-06T01:23:11.145 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:18.2.0-1181.gf7e9f9af.el8.x86_6 28/134 2023-11-06T01:23:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-06T01:23:11.406 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-06T01:23:11.496 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-06T01:23:11.601 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-06T01:23:11.751 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-06T01:23:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-06T01:23:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-06T01:23:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-06T01:23:12.003 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-06T01:23:12.014 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-06T01:23:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-06T01:23:12.164 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 14/134 2023-11-06T01:23:12.206 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-06T01:23:12.249 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-06T01:23:12.308 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 14/134 2023-11-06T01:23:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-06T01:23:12.497 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86 15/134 2023-11-06T01:23:12.612 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-06T01:23:12.653 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86 15/134 2023-11-06T01:23:12.893 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-06T01:23:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-06T01:23:13.096 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-06T01:23:13.212 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-06T01:23:13.316 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-06T01:23:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-06T01:23:13.472 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-06T01:23:13.550 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-06T01:23:13.630 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-06T01:23:13.751 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-06T01:23:13.772 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-06T01:23:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-06T01:23:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-06T01:23:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-06T01:23:14.309 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-06T01:23:14.315 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-06T01:23:14.458 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-06T01:23:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-06T01:23:14.525 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-06T01:23:14.622 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-06T01:23:14.691 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-06T01:23:14.707 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-06T01:23:14.902 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 36/134 2023-11-06T01:23:14.960 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-06T01:23:15.067 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_6 37/134 2023-11-06T01:23:15.127 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-06T01:23:15.254 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-06T01:23:15.281 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-06T01:23:15.409 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-06T01:23:15.431 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-06T01:23:15.496 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-06T01:23:15.612 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-06T01:23:15.616 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-06T01:23:15.767 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-06T01:23:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-06T01:23:15.993 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-06T01:23:16.017 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-06T01:23:16.226 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-06T01:23:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-06T01:23:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-06T01:23:16.582 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-06T01:23:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-06T01:23:16.858 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-06T01:23:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-06T01:23:17.082 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-06T01:23:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:18.2.0-1181.gf7e9f9af.el8. 26/134 2023-11-06T01:23:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-06T01:23:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:18.2.0-1181.gf7e9f9af.el 27/134 2023-11-06T01:23:17.285 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-06T01:23:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:18.2.0-1181.gf7e9f9af.el8.x86_6 28/134 2023-11-06T01:23:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-06T01:23:17.535 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-06T01:23:17.713 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-06T01:23:17.747 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-06T01:23:17.840 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-06T01:23:18.079 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-06T01:23:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-06T01:23:18.283 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-06T01:23:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-06T01:23:18.484 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-06T01:23:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-06T01:23:18.667 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-06T01:23:18.858 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-06T01:23:19.187 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-06T01:23:19.409 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-06T01:23:19.444 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-06T01:23:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-06T01:23:19.620 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 92/134 2023-11-06T01:23:19.680 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 92/134 2023-11-06T01:23:19.702 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-06T01:23:19.821 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 93/134 2023-11-06T01:23:19.978 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-06T01:23:20.153 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-06T01:23:20.305 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-06T01:23:20.471 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-06T01:23:20.547 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-06T01:23:20.644 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-06T01:23:20.793 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-06T01:23:21.011 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-06T01:23:21.015 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-06T01:23:21.163 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-06T01:23:21.242 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-06T01:23:21.312 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-06T01:23:21.450 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-06T01:23:21.552 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-06T01:23:21.622 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-06T01:23:21.770 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-06T01:23:21.794 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-06T01:23:21.965 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-06T01:23:22.025 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-06T01:23:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 36/134 2023-11-06T01:23:22.213 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-06T01:23:22.361 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_6 37/134 2023-11-06T01:23:22.436 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-06T01:23:22.554 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-06T01:23:22.575 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-06T01:23:22.731 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-06T01:23:22.739 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-06T01:23:22.946 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-06T01:23:22.946 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-06T01:23:23.022 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-06T01:23:23.157 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-06T01:23:23.306 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-06T01:23:23.322 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-06T01:23:23.490 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-06T01:23:23.526 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-06T01:23:23.642 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-06T01:23:23.687 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-06T01:23:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 94/134 2023-11-06T01:23:23.802 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-06T01:23:23.899 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-06T01:23:23.953 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 94/134 2023-11-06T01:23:24.003 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-06T01:23:24.124 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-06T01:23:24.166 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-06T01:23:24.309 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-06T01:23:24.363 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-06T01:23:24.543 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-06T01:23:24.557 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-06T01:23:24.568 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 94/134 2023-11-06T01:23:24.757 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-06T01:23:24.779 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-06T01:23:24.877 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 95/134 2023-11-06T01:23:24.909 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-06T01:23:24.937 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-06T01:23:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 95/134 2023-11-06T01:23:25.053 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 96/134 2023-11-06T01:23:25.123 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-06T01:23:25.163 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-06T01:23:25.340 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-06T01:23:25.733 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-06T01:23:26.099 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-06T01:23:26.273 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-06T01:23:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-06T01:23:26.591 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-06T01:23:26.624 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-06T01:23:26.862 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-06T01:23:26.886 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-06T01:23:27.173 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-06T01:23:27.262 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-06T01:23:27.468 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-06T01:23:27.496 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-06T01:23:27.688 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-06T01:23:27.734 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-06T01:23:27.978 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-06T01:23:28.185 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-06T01:23:28.383 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-06T01:23:28.442 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-06T01:23:28.628 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 92/134 2023-11-06T01:23:28.702 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 92/134 2023-11-06T01:23:28.755 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-06T01:23:28.870 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 93/134 2023-11-06T01:23:29.052 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-06T01:23:29.329 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-06T01:23:29.564 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-06T01:23:29.745 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-06T01:23:29.958 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-06T01:23:30.247 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-06T01:23:30.486 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-06T01:23:30.666 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-06T01:23:30.841 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-06T01:23:31.042 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-06T01:23:31.243 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-06T01:23:31.434 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-06T01:23:31.640 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-06T01:23:31.835 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-06T01:23:32.104 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-06T01:23:32.400 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-06T01:23:32.527 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 94/134 2023-11-06T01:23:32.609 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-06T01:23:32.753 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 94/134 2023-11-06T01:23:32.849 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-06T01:23:33.070 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-06T01:23:33.287 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-06T01:23:33.333 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 94/134 2023-11-06T01:23:33.541 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-06T01:23:33.604 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 95/134 2023-11-06T01:23:33.686 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 95/134 2023-11-06T01:23:33.852 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 96/134 2023-11-06T01:23:34.700 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-06T01:23:35.059 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-06T01:23:35.337 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-06T01:23:35.670 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-06T01:23:35.987 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-06T01:23:36.236 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-06T01:23:37.081 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-06T01:23:37.294 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 92/134 2023-11-06T01:23:37.355 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 92/134 2023-11-06T01:23:37.544 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 93/134 2023-11-06T01:23:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 94/134 2023-11-06T01:23:41.852 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 94/134 2023-11-06T01:23:42.479 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 94/134 2023-11-06T01:23:42.796 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 95/134 2023-11-06T01:23:42.865 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 95/134 2023-11-06T01:23:43.035 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 96/134 2023-11-06T01:23:44.480 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 96/134 2023-11-06T01:23:44.480 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2023-11-06T01:23:44.481 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2023-11-06T01:23:44.481 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2023-11-06T01:23:44.481 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /run 2023-11-06T01:23:44.481 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2023-11-06T01:23:44.481 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2023-11-06T01:23:44.482 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2023-11-06T01:23:44.482 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2023-11-06T01:23:44.482 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2023-11-06T01:23:44.482 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:23:44.842 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 97/134 2023-11-06T01:23:46.510 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 97/134 2023-11-06T01:23:46.674 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 98/134 2023-11-06T01:23:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 98/134 2023-11-06T01:23:47.116 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 99/134 2023-11-06T01:23:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 99/134 2023-11-06T01:23:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 100/134 2023-11-06T01:23:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 100/134 2023-11-06T01:23:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 101/134 2023-11-06T01:23:49.794 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 101/134 2023-11-06T01:23:49.923 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-1181.gf7e9f9af.e 102/134 2023-11-06T01:23:50.152 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-1181.gf7e9f9af. 103/134 2023-11-06T01:23:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noa 104/134 2023-11-06T01:23:50.544 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noa 104/134 2023-11-06T01:23:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.n 105/134 2023-11-06T01:23:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.n 105/134 2023-11-06T01:23:52.082 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e 106/134 2023-11-06T01:23:52.179 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e 106/134 2023-11-06T01:23:52.511 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-1181.gf7e9f9af.el 107/134 2023-11-06T01:23:52.528 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 96/134 2023-11-06T01:23:52.528 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2023-11-06T01:23:52.529 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2023-11-06T01:23:52.529 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /dev 2023-11-06T01:23:52.529 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /run 2023-11-06T01:23:52.529 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2023-11-06T01:23:52.529 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2023-11-06T01:23:52.530 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2023-11-06T01:23:52.530 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2023-11-06T01:23:52.530 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /dev 2023-11-06T01:23:52.530 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:23:52.732 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 108/134 2023-11-06T01:23:52.768 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 108/134 2023-11-06T01:23:52.866 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 97/134 2023-11-06T01:23:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 109/134 2023-11-06T01:23:52.911 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 109/134 2023-11-06T01:23:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:18.2.0-1181.gf7e9f9af.el8.x86_64 110/134 2023-11-06T01:23:54.431 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 97/134 2023-11-06T01:23:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 111/134 2023-11-06T01:23:54.487 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 111/134 2023-11-06T01:23:54.592 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 98/134 2023-11-06T01:23:54.632 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f 112/134 2023-11-06T01:23:54.676 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f 112/134 2023-11-06T01:23:54.819 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 98/134 2023-11-06T01:23:54.841 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 113/134 2023-11-06T01:23:54.999 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 99/134 2023-11-06T01:23:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 113/134 2023-11-06T01:23:55.382 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 114/134 2023-11-06T01:23:55.453 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 99/134 2023-11-06T01:23:55.599 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 100/134 2023-11-06T01:23:55.616 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 100/134 2023-11-06T01:23:57.291 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 101/134 2023-11-06T01:23:57.494 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 101/134 2023-11-06T01:23:57.613 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-1181.gf7e9f9af.e 102/134 2023-11-06T01:23:57.833 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-1181.gf7e9f9af. 103/134 2023-11-06T01:23:58.031 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noa 104/134 2023-11-06T01:23:58.254 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noa 104/134 2023-11-06T01:23:58.525 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.n 105/134 2023-11-06T01:23:59.486 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.n 105/134 2023-11-06T01:23:59.664 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e 106/134 2023-11-06T01:23:59.756 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e 106/134 2023-11-06T01:24:00.126 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 114/134 2023-11-06T01:24:00.136 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-1181.gf7e9f9af.el 107/134 2023-11-06T01:24:00.329 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 108/134 2023-11-06T01:24:00.363 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 108/134 2023-11-06T01:24:00.404 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:18.2.0-1181.gf7e9f9af.el8.x86_64 115/134 2023-11-06T01:24:00.531 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 109/134 2023-11-06T01:24:00.543 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 109/134 2023-11-06T01:24:00.573 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-06T01:24:00.780 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-06T01:24:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-06T01:24:01.815 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:18.2.0-1181.gf7e9f9af.el8.x86_64 110/134 2023-11-06T01:24:02.001 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 111/134 2023-11-06T01:24:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-06T01:24:02.043 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 111/134 2023-11-06T01:24:02.172 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f 112/134 2023-11-06T01:24:02.214 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f 112/134 2023-11-06T01:24:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 119/134 2023-11-06T01:24:02.347 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 113/134 2023-11-06T01:24:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-06T01:24:02.591 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 96/134 2023-11-06T01:24:02.592 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2023-11-06T01:24:02.592 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2023-11-06T01:24:02.592 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-11-06T01:24:02.592 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /run 2023-11-06T01:24:02.593 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2023-11-06T01:24:02.593 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2023-11-06T01:24:02.593 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2023-11-06T01:24:02.593 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2023-11-06T01:24:02.593 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-11-06T01:24:02.594 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:24:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_ 121/134 2023-11-06T01:24:02.697 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 113/134 2023-11-06T01:24:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-06T01:24:02.839 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 114/134 2023-11-06T01:24:02.957 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 97/134 2023-11-06T01:24:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 97/134 2023-11-06T01:24:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 98/134 2023-11-06T01:24:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 98/134 2023-11-06T01:24:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-06T01:24:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 99/134 2023-11-06T01:24:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-06T01:24:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-06T01:24:05.639 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:18.2.0-1181.gf7e9f9af.el8.noarch 124/134 2023-11-06T01:24:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 99/134 2023-11-06T01:24:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 100/134 2023-11-06T01:24:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 125/134 2023-11-06T01:24:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 100/134 2023-11-06T01:24:06.305 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 126/134 2023-11-06T01:24:06.591 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-06T01:24:06.722 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-06T01:24:06.731 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-06T01:24:06.912 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-06T01:24:07.138 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-06T01:24:07.233 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-06T01:24:07.295 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 114/134 2023-11-06T01:24:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-06T01:24:07.460 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-06T01:24:07.461 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-06T01:24:07.577 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:18.2.0-1181.gf7e9f9af.el8.x86_64 115/134 2023-11-06T01:24:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-06T01:24:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-06T01:24:07.730 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-06T01:24:07.857 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 101/134 2023-11-06T01:24:07.921 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-06T01:24:07.938 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-06T01:24:08.095 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 101/134 2023-11-06T01:24:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-06T01:24:08.296 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-1181.gf7e9f9af.e 102/134 2023-11-06T01:24:08.549 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-1181.gf7e9f9af. 103/134 2023-11-06T01:24:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-06T01:24:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:18.2.0-1181.gf7e9f9af.el8.x86_64 1/134 2023-11-06T01:24:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2/134 2023-11-06T01:24:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 3/134 2023-11-06T01:24:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 4/134 2023-11-06T01:24:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f 5/134 2023-11-06T01:24:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 6/134 2023-11-06T01:24:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 7/134 2023-11-06T01:24:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 8/134 2023-11-06T01:24:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 9/134 2023-11-06T01:24:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 10/134 2023-11-06T01:24:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 11/134 2023-11-06T01:24:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:18.2.0-1181.gf7e9f9af.el8.x86_64 12/134 2023-11-06T01:24:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 13/134 2023-11-06T01:24:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_ 14/134 2023-11-06T01:24:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 15/134 2023-11-06T01:24:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 16/134 2023-11-06T01:24:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_6 17/134 2023-11-06T01:24:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86 18/134 2023-11-06T01:24:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 19/134 2023-11-06T01:24:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:18.2.0-1181.gf7e9f9af.el 20/134 2023-11-06T01:24:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:18.2.0-1181.gf7e9f9af.el8. 21/134 2023-11-06T01:24:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:18.2.0-1181.gf7e9f9af.el8.x86_6 22/134 2023-11-06T01:24:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:18.2.0-1181.gf7e9f9af.el8.x86_64 23/134 2023-11-06T01:24:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 24/134 2023-11-06T01:24:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 25/134 2023-11-06T01:24:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 26/134 2023-11-06T01:24:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 27/134 2023-11-06T01:24:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 28/134 2023-11-06T01:24:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-1181.gf7e9f9af. 29/134 2023-11-06T01:24:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noa 30/134 2023-11-06T01:24:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.n 31/134 2023-11-06T01:24:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e 32/134 2023-11-06T01:24:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-1181.gf7e9f9af.el 33/134 2023-11-06T01:24:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 34/134 2023-11-06T01:24:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-1181.gf7e9f9af.e 35/134 2023-11-06T01:24:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 36/134 2023-11-06T01:24:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 37/134 2023-11-06T01:24:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:18.2.0-1181.gf7e9f9af.el8.noarch 38/134 2023-11-06T01:24:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-06T01:24:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-06T01:24:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-06T01:24:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-06T01:24:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-06T01:24:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-06T01:24:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-06T01:24:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-06T01:24:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-06T01:24:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-06T01:24:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-06T01:24:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-06T01:24:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-06T01:24:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-06T01:24:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-06T01:24:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-06T01:24:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-06T01:24:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-06T01:24:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-06T01:24:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-06T01:24:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-06T01:24:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-06T01:24:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-06T01:24:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-06T01:24:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-06T01:24:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-06T01:24:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-06T01:24:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-06T01:24:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-06T01:24:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-06T01:24:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-06T01:24:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-06T01:24:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-06T01:24:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-06T01:24:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-06T01:24:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-06T01:24:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-06T01:24:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-06T01:24:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-06T01:24:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-06T01:24:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-06T01:24:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-06T01:24:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-06T01:24:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-06T01:24:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-06T01:24:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-06T01:24:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-06T01:24:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-06T01:24:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-06T01:24:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-06T01:24:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-06T01:24:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-06T01:24:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-06T01:24:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-06T01:24:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-06T01:24:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-06T01:24:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-06T01:24:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-06T01:24:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-06T01:24:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-06T01:24:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-06T01:24:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-06T01:24:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-06T01:24:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-06T01:24:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-06T01:24:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-06T01:24:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-06T01:24:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-06T01:24:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-06T01:24:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-06T01:24:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-06T01:24:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-06T01:24:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-06T01:24:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-06T01:24:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-06T01:24:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-06T01:24:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-06T01:24:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-06T01:24:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-06T01:24:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-06T01:24:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-06T01:24:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-06T01:24:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-06T01:24:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-06T01:24:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-06T01:24:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-06T01:24:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-06T01:24:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-06T01:24:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-06T01:24:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-06T01:24:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 129/134 2023-11-06T01:24:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-06T01:24:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 131/134 2023-11-06T01:24:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-06T01:24:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-06T01:24:08.764 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noa 104/134 2023-11-06T01:24:08.828 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-06T01:24:08.991 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noa 104/134 2023-11-06T01:24:09.178 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-06T01:24:09.200 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.n 105/134 2023-11-06T01:24:09.341 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 119/134 2023-11-06T01:24:09.525 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-06T01:24:09.691 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_ 121/134 2023-11-06T01:24:09.872 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-06T01:24:10.221 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.n 105/134 2023-11-06T01:24:10.414 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e 106/134 2023-11-06T01:24:10.510 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e 106/134 2023-11-06T01:24:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-1181.gf7e9f9af.el 107/134 2023-11-06T01:24:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 108/134 2023-11-06T01:24:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 108/134 2023-11-06T01:24:11.160 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-06T01:24:11.161 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:02:11 ago on Mon 06 Nov 2023 01:22:00 AM UTC. 2023-11-06T01:24:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 109/134 2023-11-06T01:24:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 109/134 2023-11-06T01:24:12.045 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-06T01:24:12.279 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-06T01:24:12.296 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-06T01:24:12.454 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-top-2:18.2.0-1181.gf7e9f9af.el8.noarch 124/134 2023-11-06T01:24:12.692 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:18.2.0-1181.gf7e9f9af.el8.x86_64 110/134 2023-11-06T01:24:12.779 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 125/134 2023-11-06T01:24:12.902 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 111/134 2023-11-06T01:24:12.945 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 111/134 2023-11-06T01:24:13.104 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 126/134 2023-11-06T01:24:13.131 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f 112/134 2023-11-06T01:24:13.174 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f 112/134 2023-11-06T01:24:13.364 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 113/134 2023-11-06T01:24:13.414 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-06T01:24:13.595 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-06T01:24:13.604 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-06T01:24:13.743 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 113/134 2023-11-06T01:24:13.786 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-06T01:24:13.906 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 114/134 2023-11-06T01:24:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-06T01:24:14.055 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-06T01:24:14.237 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-06T01:24:14.249 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-06T01:24:14.249 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-06T01:24:14.440 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-06T01:24:14.440 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-06T01:24:14.675 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-06T01:24:14.789 INFO:teuthology.orchestra.run.smithi053.stdout:Installed products updated. 2023-11-06T01:24:14.869 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-06T01:24:15.025 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:24:15.025 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2023-11-06T01:24:15.025 INFO:teuthology.orchestra.run.smithi053.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-06T01:24:15.026 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.026 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.026 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2023-11-06T01:24:15.026 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-06T01:24:15.027 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.1-7.el8.noarch 2023-11-06T01:24:15.027 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.0.4-10.el8.x86_64 2023-11-06T01:24:15.027 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.027 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.027 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.028 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.028 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:15.028 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.028 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.028 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.028 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:15.029 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:15.029 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:15.029 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:15.029 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:15.029 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.030 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.030 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:15.030 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.030 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.030 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.031 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:15.031 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:15.031 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.031 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:15.031 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-06T01:24:15.032 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.1-9.el8.x86_64 2023-11-06T01:24:15.032 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-06T01:24:15.032 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-06T01:24:15.032 INFO:teuthology.orchestra.run.smithi053.stdout: jq-1.6-3.el8.x86_64 2023-11-06T01:24:15.032 INFO:teuthology.orchestra.run.smithi053.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-06T01:24:15.033 INFO:teuthology.orchestra.run.smithi053.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-06T01:24:15.033 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-06T01:24:15.033 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.033 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.034 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.034 INFO:teuthology.orchestra.run.smithi053.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-06T01:24:15.034 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-06T01:24:15.034 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-06T01:24:15.034 INFO:teuthology.orchestra.run.smithi053.stdout: libpq-13.5-1.el8.x86_64 2023-11-06T01:24:15.035 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-06T01:24:15.035 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-06T01:24:15.035 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.035 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.035 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-06T01:24:15.036 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.036 INFO:teuthology.orchestra.run.smithi053.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-06T01:24:15.036 INFO:teuthology.orchestra.run.smithi053.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-06T01:24:15.036 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-06T01:24:15.036 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-06T01:24:15.037 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-06T01:24:15.037 INFO:teuthology.orchestra.run.smithi053.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-06T01:24:15.037 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-06T01:24:15.037 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-06T01:24:15.037 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-06T01:24:15.037 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-06T01:24:15.038 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-06T01:24:15.038 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-06T01:24:15.038 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-06T01:24:15.038 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-06T01:24:15.038 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-06T01:24:15.039 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-06T01:24:15.039 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-06T01:24:15.039 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-06T01:24:15.039 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-06T01:24:15.039 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-06T01:24:15.040 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-06T01:24:15.040 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-06T01:24:15.040 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.040 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.040 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.040 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-06T01:24:15.041 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-06T01:24:15.041 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-06T01:24:15.041 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-06T01:24:15.041 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-06T01:24:15.041 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-06T01:24:15.042 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-06T01:24:15.042 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-06T01:24:15.042 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-06T01:24:15.042 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-06T01:24:15.042 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-06T01:24:15.043 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-06T01:24:15.043 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-06T01:24:15.043 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-06T01:24:15.043 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-06T01:24:15.043 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-06T01:24:15.043 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-06T01:24:15.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-06T01:24:15.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-06T01:24:15.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-06T01:24:15.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-06T01:24:15.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-06T01:24:15.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-2.6-1.el8.noarch 2023-11-06T01:24:15.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-06T01:24:15.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-06T01:24:15.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-06T01:24:15.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-06T01:24:15.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-06T01:24:15.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-06T01:24:15.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-06T01:24:15.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-06T01:24:15.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-06T01:24:15.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-06T01:24:15.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-06T01:24:15.048 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-06T01:24:15.048 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-06T01:24:15.048 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-06T01:24:15.048 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-06T01:24:15.048 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-06T01:24:15.049 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-06T01:24:15.049 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-06T01:24:15.049 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-06T01:24:15.049 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-06T01:24:15.049 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-06T01:24:15.049 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.050 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.050 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:15.050 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-06T01:24:15.050 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-06T01:24:15.051 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-06T01:24:15.051 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-06T01:24:15.051 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-06T01:24:15.052 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-06T01:24:15.052 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:24:15.052 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2023-11-06T01:24:15.258 DEBUG:teuthology.parallel:result is None 2023-11-06T01:24:15.387 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-06T01:24:15.387 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:18.2.0-1181.gf7e9f9af.el8.x86_64 1/134 2023-11-06T01:24:15.387 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2/134 2023-11-06T01:24:15.387 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 3/134 2023-11-06T01:24:15.388 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 4/134 2023-11-06T01:24:15.388 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f 5/134 2023-11-06T01:24:15.388 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 6/134 2023-11-06T01:24:15.388 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 7/134 2023-11-06T01:24:15.388 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 8/134 2023-11-06T01:24:15.389 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 9/134 2023-11-06T01:24:15.389 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 10/134 2023-11-06T01:24:15.389 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 11/134 2023-11-06T01:24:15.389 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:18.2.0-1181.gf7e9f9af.el8.x86_64 12/134 2023-11-06T01:24:15.389 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 13/134 2023-11-06T01:24:15.390 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_ 14/134 2023-11-06T01:24:15.390 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 15/134 2023-11-06T01:24:15.390 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 16/134 2023-11-06T01:24:15.390 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_6 17/134 2023-11-06T01:24:15.390 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86 18/134 2023-11-06T01:24:15.391 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 19/134 2023-11-06T01:24:15.391 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:18.2.0-1181.gf7e9f9af.el 20/134 2023-11-06T01:24:15.391 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:18.2.0-1181.gf7e9f9af.el8. 21/134 2023-11-06T01:24:15.391 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:18.2.0-1181.gf7e9f9af.el8.x86_6 22/134 2023-11-06T01:24:15.391 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:18.2.0-1181.gf7e9f9af.el8.x86_64 23/134 2023-11-06T01:24:15.391 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 24/134 2023-11-06T01:24:15.392 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 25/134 2023-11-06T01:24:15.392 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 26/134 2023-11-06T01:24:15.392 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 27/134 2023-11-06T01:24:15.401 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 28/134 2023-11-06T01:24:15.402 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-1181.gf7e9f9af. 29/134 2023-11-06T01:24:15.402 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noa 30/134 2023-11-06T01:24:15.402 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.n 31/134 2023-11-06T01:24:15.402 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e 32/134 2023-11-06T01:24:15.402 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-1181.gf7e9f9af.el 33/134 2023-11-06T01:24:15.403 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 34/134 2023-11-06T01:24:15.403 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-1181.gf7e9f9af.e 35/134 2023-11-06T01:24:15.403 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 36/134 2023-11-06T01:24:15.403 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 37/134 2023-11-06T01:24:15.403 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-top-2:18.2.0-1181.gf7e9f9af.el8.noarch 38/134 2023-11-06T01:24:15.403 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-06T01:24:15.404 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-06T01:24:15.404 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-06T01:24:15.404 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-06T01:24:15.404 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-06T01:24:15.404 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-06T01:24:15.405 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-06T01:24:15.405 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-06T01:24:15.405 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-06T01:24:15.405 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-06T01:24:15.405 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-06T01:24:15.406 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-06T01:24:15.406 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-06T01:24:15.406 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-06T01:24:15.406 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-06T01:24:15.406 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-06T01:24:15.406 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-06T01:24:15.407 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-06T01:24:15.407 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-06T01:24:15.407 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-06T01:24:15.407 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-06T01:24:15.407 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-06T01:24:15.408 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-06T01:24:15.408 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-06T01:24:15.408 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-06T01:24:15.408 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-06T01:24:15.408 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-06T01:24:15.409 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-06T01:24:15.409 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-06T01:24:15.409 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-06T01:24:15.409 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-06T01:24:15.409 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-06T01:24:15.409 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-06T01:24:15.410 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-06T01:24:15.410 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-06T01:24:15.410 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-06T01:24:15.410 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-06T01:24:15.410 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-06T01:24:15.411 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-06T01:24:15.411 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-06T01:24:15.411 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-06T01:24:15.411 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-06T01:24:15.411 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-06T01:24:15.412 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-06T01:24:15.412 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-06T01:24:15.412 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-06T01:24:15.412 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-06T01:24:15.412 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-06T01:24:15.413 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-06T01:24:15.413 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-06T01:24:15.413 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-06T01:24:15.413 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-06T01:24:15.413 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-06T01:24:15.413 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-06T01:24:15.414 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-06T01:24:15.414 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-06T01:24:15.414 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-06T01:24:15.414 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-06T01:24:15.414 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-06T01:24:15.415 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-06T01:24:15.415 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-06T01:24:15.415 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-06T01:24:15.415 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-06T01:24:15.415 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-06T01:24:15.415 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-06T01:24:15.416 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-06T01:24:15.416 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-06T01:24:15.416 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-06T01:24:15.416 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-06T01:24:15.416 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-06T01:24:15.417 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-06T01:24:15.417 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-06T01:24:15.417 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-06T01:24:15.417 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-06T01:24:15.417 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-06T01:24:15.418 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-06T01:24:15.418 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-06T01:24:15.418 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-06T01:24:15.418 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-06T01:24:15.418 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-06T01:24:15.419 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-06T01:24:15.419 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-06T01:24:15.419 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-06T01:24:15.419 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-06T01:24:15.419 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-06T01:24:15.419 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-06T01:24:15.420 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-06T01:24:15.420 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-06T01:24:15.420 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-06T01:24:15.420 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-06T01:24:15.420 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 129/134 2023-11-06T01:24:15.421 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-06T01:24:15.421 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 131/134 2023-11-06T01:24:15.421 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-06T01:24:15.421 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-06T01:24:17.902 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-06T01:24:17.902 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:02:16 ago on Mon 06 Nov 2023 01:22:01 AM UTC. 2023-11-06T01:24:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 114/134 2023-11-06T01:24:19.618 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:18.2.0-1181.gf7e9f9af.el8.x86_64 115/134 2023-11-06T01:24:19.966 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-06T01:24:20.167 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 117/134 2023-11-06T01:24:20.391 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-06T01:24:20.555 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-06T01:24:20.761 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-06T01:24:20.993 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-06T01:24:21.184 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_ 121/134 2023-11-06T01:24:21.373 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-06T01:24:21.636 INFO:teuthology.orchestra.run.smithi163.stdout:Installed products updated. 2023-11-06T01:24:21.848 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:24:21.849 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2023-11-06T01:24:21.849 INFO:teuthology.orchestra.run.smithi163.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-06T01:24:21.849 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.849 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.850 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2023-11-06T01:24:21.850 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-06T01:24:21.850 INFO:teuthology.orchestra.run.smithi163.stdout: automake-1.16.1-7.el8.noarch 2023-11-06T01:24:21.850 INFO:teuthology.orchestra.run.smithi163.stdout: bison-3.0.4-10.el8.x86_64 2023-11-06T01:24:21.850 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.851 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.851 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.851 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.851 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:21.851 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.851 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.852 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.852 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:21.852 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:21.852 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:21.852 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:21.853 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:21.853 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.853 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.853 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:21.853 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.853 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.854 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.854 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:21.854 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:21.854 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.854 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:21.855 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-06T01:24:21.855 INFO:teuthology.orchestra.run.smithi163.stdout: flex-2.6.1-9.el8.x86_64 2023-11-06T01:24:21.855 INFO:teuthology.orchestra.run.smithi163.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-06T01:24:21.855 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-06T01:24:21.855 INFO:teuthology.orchestra.run.smithi163.stdout: jq-1.6-3.el8.x86_64 2023-11-06T01:24:21.856 INFO:teuthology.orchestra.run.smithi163.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-06T01:24:21.856 INFO:teuthology.orchestra.run.smithi163.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-06T01:24:21.856 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-06T01:24:21.856 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.856 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.857 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.857 INFO:teuthology.orchestra.run.smithi163.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-06T01:24:21.857 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-06T01:24:21.857 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-06T01:24:21.857 INFO:teuthology.orchestra.run.smithi163.stdout: libpq-13.5-1.el8.x86_64 2023-11-06T01:24:21.857 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-06T01:24:21.858 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-06T01:24:21.858 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.858 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.858 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-06T01:24:21.858 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.859 INFO:teuthology.orchestra.run.smithi163.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-06T01:24:21.859 INFO:teuthology.orchestra.run.smithi163.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-06T01:24:21.859 INFO:teuthology.orchestra.run.smithi163.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-06T01:24:21.859 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-06T01:24:21.859 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-06T01:24:21.860 INFO:teuthology.orchestra.run.smithi163.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-06T01:24:21.860 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-06T01:24:21.860 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-06T01:24:21.860 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-06T01:24:21.860 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-06T01:24:21.861 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-06T01:24:21.861 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-06T01:24:21.861 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-06T01:24:21.861 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-06T01:24:21.861 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-06T01:24:21.862 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-06T01:24:21.862 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-06T01:24:21.862 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-06T01:24:21.862 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-06T01:24:21.862 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-06T01:24:21.863 INFO:teuthology.orchestra.run.smithi163.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-06T01:24:21.863 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-06T01:24:21.863 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.863 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.863 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.864 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-06T01:24:21.864 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-06T01:24:21.864 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-06T01:24:21.864 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-06T01:24:21.864 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-06T01:24:21.865 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-06T01:24:21.865 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-06T01:24:21.865 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-06T01:24:21.865 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-06T01:24:21.866 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-06T01:24:21.866 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-06T01:24:21.866 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-06T01:24:21.866 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-06T01:24:21.866 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-06T01:24:21.867 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-06T01:24:21.867 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-06T01:24:21.867 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-06T01:24:21.867 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-06T01:24:21.867 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-06T01:24:21.868 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-06T01:24:21.868 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-06T01:24:21.868 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-06T01:24:21.868 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-2.6-1.el8.noarch 2023-11-06T01:24:21.868 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-06T01:24:21.868 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-06T01:24:21.869 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-06T01:24:21.869 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-06T01:24:21.869 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-06T01:24:21.869 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-06T01:24:21.869 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.870 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.870 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-06T01:24:21.870 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-06T01:24:21.870 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.870 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-06T01:24:21.871 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-06T01:24:21.871 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-06T01:24:21.871 INFO:teuthology.orchestra.run.smithi163.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-06T01:24:21.871 INFO:teuthology.orchestra.run.smithi163.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-06T01:24:21.871 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-06T01:24:21.871 INFO:teuthology.orchestra.run.smithi163.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-06T01:24:21.872 INFO:teuthology.orchestra.run.smithi163.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-06T01:24:21.872 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-06T01:24:21.872 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-06T01:24:21.872 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-06T01:24:21.872 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-06T01:24:21.873 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-06T01:24:21.873 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.873 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.873 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:21.873 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-06T01:24:21.874 INFO:teuthology.orchestra.run.smithi163.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-06T01:24:21.874 INFO:teuthology.orchestra.run.smithi163.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-06T01:24:21.874 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-06T01:24:21.874 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-06T01:24:21.875 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-06T01:24:21.875 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:24:21.875 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2023-11-06T01:24:22.062 DEBUG:teuthology.parallel:result is None 2023-11-06T01:24:23.829 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-06T01:24:24.155 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-06T01:24:24.173 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-06T01:24:24.439 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:18.2.0-1181.gf7e9f9af.el8.noarch 124/134 2023-11-06T01:24:24.664 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 125/134 2023-11-06T01:24:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 126/134 2023-11-06T01:24:25.417 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-06T01:24:25.637 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-06T01:24:25.733 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-06T01:24:25.963 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-06T01:24:25.976 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-06T01:24:26.121 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-06T01:24:26.129 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-06T01:24:26.298 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-06T01:24:26.299 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-06T01:24:26.524 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-06T01:24:26.525 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-06T01:24:26.778 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-06T01:24:26.979 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-06T01:24:27.582 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-06T01:24:27.582 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:18.2.0-1181.gf7e9f9af.el8.x86_64 1/134 2023-11-06T01:24:27.582 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2/134 2023-11-06T01:24:27.583 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 3/134 2023-11-06T01:24:27.583 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 4/134 2023-11-06T01:24:27.583 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f 5/134 2023-11-06T01:24:27.583 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 6/134 2023-11-06T01:24:27.583 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 7/134 2023-11-06T01:24:27.584 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 8/134 2023-11-06T01:24:27.584 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 9/134 2023-11-06T01:24:27.584 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 10/134 2023-11-06T01:24:27.584 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 11/134 2023-11-06T01:24:27.584 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:18.2.0-1181.gf7e9f9af.el8.x86_64 12/134 2023-11-06T01:24:27.585 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 13/134 2023-11-06T01:24:27.585 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_ 14/134 2023-11-06T01:24:27.585 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 15/134 2023-11-06T01:24:27.585 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 16/134 2023-11-06T01:24:27.585 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_6 17/134 2023-11-06T01:24:27.586 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86 18/134 2023-11-06T01:24:27.586 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 19/134 2023-11-06T01:24:27.586 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:18.2.0-1181.gf7e9f9af.el 20/134 2023-11-06T01:24:27.586 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:18.2.0-1181.gf7e9f9af.el8. 21/134 2023-11-06T01:24:27.586 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:18.2.0-1181.gf7e9f9af.el8.x86_6 22/134 2023-11-06T01:24:27.587 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:18.2.0-1181.gf7e9f9af.el8.x86_64 23/134 2023-11-06T01:24:27.587 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 24/134 2023-11-06T01:24:27.587 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 25/134 2023-11-06T01:24:27.587 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 26/134 2023-11-06T01:24:27.587 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 27/134 2023-11-06T01:24:27.588 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 28/134 2023-11-06T01:24:27.588 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-1181.gf7e9f9af. 29/134 2023-11-06T01:24:27.588 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noa 30/134 2023-11-06T01:24:27.588 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.n 31/134 2023-11-06T01:24:27.588 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e 32/134 2023-11-06T01:24:27.588 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-1181.gf7e9f9af.el 33/134 2023-11-06T01:24:27.589 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 34/134 2023-11-06T01:24:27.589 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-1181.gf7e9f9af.e 35/134 2023-11-06T01:24:27.589 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 36/134 2023-11-06T01:24:27.589 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 37/134 2023-11-06T01:24:27.589 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:18.2.0-1181.gf7e9f9af.el8.noarch 38/134 2023-11-06T01:24:27.590 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-06T01:24:27.590 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-06T01:24:27.590 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-06T01:24:27.590 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-06T01:24:27.590 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-06T01:24:27.591 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-06T01:24:27.591 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-06T01:24:27.591 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-06T01:24:27.591 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-06T01:24:27.592 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-06T01:24:27.592 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-06T01:24:27.592 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-06T01:24:27.592 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-06T01:24:27.592 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-06T01:24:27.592 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-06T01:24:27.593 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-06T01:24:27.593 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-06T01:24:27.593 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-06T01:24:27.593 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-06T01:24:27.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-06T01:24:27.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-06T01:24:27.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-06T01:24:27.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-06T01:24:27.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-06T01:24:27.595 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-06T01:24:27.595 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-06T01:24:27.595 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-06T01:24:27.595 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-06T01:24:27.595 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-06T01:24:27.595 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-06T01:24:27.596 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-06T01:24:27.596 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-06T01:24:27.596 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-06T01:24:27.596 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-06T01:24:27.596 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-06T01:24:27.597 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-06T01:24:27.597 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-06T01:24:27.597 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-06T01:24:27.597 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-06T01:24:27.597 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-06T01:24:27.598 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-06T01:24:27.598 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-06T01:24:27.598 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-06T01:24:27.598 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-06T01:24:27.598 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-06T01:24:27.599 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-06T01:24:27.599 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-06T01:24:27.599 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-06T01:24:27.599 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-06T01:24:27.599 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-06T01:24:27.600 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-06T01:24:27.600 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-06T01:24:27.600 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-06T01:24:27.600 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-06T01:24:27.600 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-06T01:24:27.600 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-06T01:24:27.601 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-06T01:24:27.601 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-06T01:24:27.601 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-06T01:24:27.601 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-06T01:24:27.601 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-06T01:24:27.602 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-06T01:24:27.602 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-06T01:24:27.602 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-06T01:24:27.602 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-06T01:24:27.603 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-06T01:24:27.603 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-06T01:24:27.603 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-06T01:24:27.603 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-06T01:24:27.603 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-06T01:24:27.604 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-06T01:24:27.604 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-06T01:24:27.604 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-06T01:24:27.604 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-06T01:24:27.604 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-06T01:24:27.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-06T01:24:27.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-06T01:24:27.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-06T01:24:27.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-06T01:24:27.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-06T01:24:27.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-06T01:24:27.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-06T01:24:27.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-06T01:24:27.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-06T01:24:27.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-06T01:24:27.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-06T01:24:27.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-06T01:24:27.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-06T01:24:27.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-06T01:24:27.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-06T01:24:27.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 129/134 2023-11-06T01:24:27.608 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-06T01:24:27.608 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 131/134 2023-11-06T01:24:27.608 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-06T01:24:27.608 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-06T01:24:30.269 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-06T01:24:30.270 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:28 ago on Mon 06 Nov 2023 01:22:02 AM UTC. 2023-11-06T01:24:33.518 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-11-06T01:24:33.751 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:24:33.752 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2023-11-06T01:24:33.752 INFO:teuthology.orchestra.run.smithi062.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-06T01:24:33.752 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.752 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.752 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-11-06T01:24:33.753 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-06T01:24:33.753 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.1-7.el8.noarch 2023-11-06T01:24:33.753 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.0.4-10.el8.x86_64 2023-11-06T01:24:33.753 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.753 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.754 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.754 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.754 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:33.754 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.754 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.755 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.755 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:33.755 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:33.755 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:33.755 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:33.756 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:33.756 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.756 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.756 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:33.756 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.757 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.757 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.757 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:33.757 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:33.757 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.758 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:18.2.0-1181.gf7e9f9af.el8.noarch 2023-11-06T01:24:33.758 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-06T01:24:33.758 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.1-9.el8.x86_64 2023-11-06T01:24:33.758 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-06T01:24:33.759 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-06T01:24:33.759 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.6-3.el8.x86_64 2023-11-06T01:24:33.759 INFO:teuthology.orchestra.run.smithi062.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-06T01:24:33.759 INFO:teuthology.orchestra.run.smithi062.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-06T01:24:33.759 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-06T01:24:33.759 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.760 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.760 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.760 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-06T01:24:33.760 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-06T01:24:33.760 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-06T01:24:33.761 INFO:teuthology.orchestra.run.smithi062.stdout: libpq-13.5-1.el8.x86_64 2023-11-06T01:24:33.761 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-06T01:24:33.761 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-06T01:24:33.761 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.761 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.762 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-06T01:24:33.762 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.762 INFO:teuthology.orchestra.run.smithi062.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-06T01:24:33.762 INFO:teuthology.orchestra.run.smithi062.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-06T01:24:33.762 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-06T01:24:33.763 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-06T01:24:33.763 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-06T01:24:33.763 INFO:teuthology.orchestra.run.smithi062.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-06T01:24:33.763 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-06T01:24:33.763 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-06T01:24:33.764 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-06T01:24:33.764 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-06T01:24:33.764 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-06T01:24:33.764 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-06T01:24:33.764 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-06T01:24:33.765 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-06T01:24:33.765 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-06T01:24:33.765 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-06T01:24:33.765 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-06T01:24:33.765 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-06T01:24:33.766 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-06T01:24:33.766 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-06T01:24:33.766 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-06T01:24:33.766 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-06T01:24:33.766 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-06T01:24:33.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-06T01:24:33.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-06T01:24:33.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-06T01:24:33.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-06T01:24:33.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-06T01:24:33.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-06T01:24:33.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-06T01:24:33.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-06T01:24:33.769 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-06T01:24:33.769 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-06T01:24:33.769 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-06T01:24:33.769 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-06T01:24:33.769 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-06T01:24:33.770 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-06T01:24:33.770 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-06T01:24:33.770 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-06T01:24:33.770 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-06T01:24:33.770 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-06T01:24:33.771 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-06T01:24:33.771 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-06T01:24:33.771 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-06T01:24:33.771 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-2.6-1.el8.noarch 2023-11-06T01:24:33.771 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-06T01:24:33.771 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-06T01:24:33.772 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-06T01:24:33.772 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-06T01:24:33.772 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-06T01:24:33.772 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-06T01:24:33.772 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.772 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.773 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-06T01:24:33.773 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-06T01:24:33.773 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.773 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-06T01:24:33.773 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-06T01:24:33.773 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-06T01:24:33.774 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-06T01:24:33.774 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-06T01:24:33.774 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-06T01:24:33.774 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-06T01:24:33.774 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-06T01:24:33.775 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-06T01:24:33.775 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-06T01:24:33.775 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-06T01:24:33.775 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-06T01:24:33.775 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-06T01:24:33.776 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.776 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.776 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:18.2.0-1181.gf7e9f9af.el8.x86_64 2023-11-06T01:24:33.776 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-06T01:24:33.776 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-06T01:24:33.777 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-06T01:24:33.777 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-06T01:24:33.777 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-06T01:24:33.778 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-06T01:24:33.778 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-11-06T01:24:33.778 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-11-06T01:24:33.966 DEBUG:teuthology.parallel:result is None 2023-11-06T01:24:33.967 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T01:24:34.189 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-06T01:24:34.218 INFO:teuthology.orchestra.run.smithi053.stdout:18.2.0-1181.gf7e9f9af.el8 2023-11-06T01:24:34.219 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1181.gf7e9f9af.el8 2023-11-06T01:24:34.220 INFO:teuthology.task.install:The correct ceph version 18.2.0-1181.gf7e9f9af is installed. 2023-11-06T01:24:34.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T01:24:34.366 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-06T01:24:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:18.2.0-1181.gf7e9f9af.el8 2023-11-06T01:24:34.392 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1181.gf7e9f9af.el8 2023-11-06T01:24:34.393 INFO:teuthology.task.install:The correct ceph version 18.2.0-1181.gf7e9f9af is installed. 2023-11-06T01:24:34.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T01:24:34.616 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-06T01:24:34.644 INFO:teuthology.orchestra.run.smithi163.stdout:18.2.0-1181.gf7e9f9af.el8 2023-11-06T01:24:34.645 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1181.gf7e9f9af.el8 2023-11-06T01:24:34.645 INFO:teuthology.task.install:The correct ceph version 18.2.0-1181.gf7e9f9af is installed. 2023-11-06T01:24:34.646 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-06T01:24:34.646 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:24:34.646 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-06T01:24:34.698 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-11-06T01:24:34.698 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-06T01:24:34.745 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-11-06T01:24:34.746 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-06T01:24:34.794 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-06T01:24:34.795 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:24:34.795 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2023-11-06T01:24:34.828 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-06T01:24:34.899 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-11-06T01:24:34.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2023-11-06T01:24:34.930 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-06T01:24:34.999 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-11-06T01:24:34.999 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2023-11-06T01:24:35.032 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-06T01:24:35.102 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-06T01:24:35.102 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:24:35.103 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-06T01:24:35.135 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-06T01:24:35.206 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-11-06T01:24:35.206 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-06T01:24:35.237 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-06T01:24:35.306 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-11-06T01:24:35.306 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-06T01:24:35.338 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-06T01:24:35.410 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-06T01:24:35.411 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:24:35.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2023-11-06T01:24:35.444 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-06T01:24:35.516 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-11-06T01:24:35.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2023-11-06T01:24:35.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-06T01:24:35.616 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-11-06T01:24:35.616 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2023-11-06T01:24:35.648 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-06T01:24:35.720 INFO:teuthology.run_tasks:Running task cephadm... 2023-11-06T01:24:35.797 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-11-06T01:24:35.797 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T01:24:35.798 INFO:tasks.cephadm:Cluster fsid is 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:24:35.798 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-11-06T01:24:35.799 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.53', 'mon.b': '172.21.15.62', 'mon.c': '172.21.15.163'} 2023-11-06T01:24:35.799 INFO:tasks.cephadm:First mon is mon.a on smithi053 2023-11-06T01:24:35.799 INFO:tasks.cephadm:First mgr is x 2023-11-06T01:24:35.800 INFO:tasks.cephadm:Normalizing hostnames... 2023-11-06T01:24:35.800 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2023-11-06T01:24:35.834 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2023-11-06T01:24:35.865 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname $(hostname -s) 2023-11-06T01:24:35.898 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-11-06T01:24:35.898 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T01:24:36.040 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/53727/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.0-1181-gf7e9f9af', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.0-1181.gf7e9f9af'}, 'url': 'https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-11-04 16:18:52.573630', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73868/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.0-1181-gf7e9f9af', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.0-1181.gf7e9f9af'}, 'url': 'https://4.chacra.ceph.com/r/ceph/reef-release/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-11-04 14:46:33.998179', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef-release', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/reef-release/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-11-06T01:24:36.189 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef, sha1 f7e9f9af51acf98e0ccc3037487018e4ae40caa5 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T01:24:36.189 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/x86_64/flavors/default/cephadm 2023-11-06T01:24:36.190 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-06T01:24:36.899 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-rw-r--. 1 ubuntu ubuntu 690689 Nov 6 01:24 /home/ubuntu/cephtest/cephadm 2023-11-06T01:24:36.900 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-06T01:24:37.452 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-rw-r--. 1 ubuntu ubuntu 690689 Nov 6 01:24 /home/ubuntu/cephtest/cephadm 2023-11-06T01:24:37.452 DEBUG:teuthology.orchestra.run.smithi163:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-06T01:24:37.850 INFO:teuthology.orchestra.run.smithi163.stdout:-rw-rw-r--. 1 ubuntu ubuntu 690689 Nov 6 01:24 /home/ubuntu/cephtest/cephadm 2023-11-06T01:24:37.851 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-06T01:24:37.873 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-06T01:24:37.896 DEBUG:teuthology.orchestra.run.smithi163:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-06T01:24:37.935 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 on all hosts... 2023-11-06T01:24:37.935 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 pull 2023-11-06T01:24:37.938 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 pull 2023-11-06T01:24:37.941 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 pull 2023-11-06T01:24:40.075 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5... 2023-11-06T01:24:41.034 INFO:teuthology.orchestra.run.smithi163.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5... 2023-11-06T01:24:43.415 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5... 2023-11-06T01:25:05.652 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:25:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T01:25:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "fe94629dd8412671eebe0af3189cc84c3a5d41ab807bae18a1e13cf671063216", 2023-11-06T01:25:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2023-11-06T01:25:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:801c764ebcb3d4c201ae63c6f6c4dd97da44ffffe122eed1091041bcab805777" 2023-11-06T01:25:05.654 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-06T01:25:05.654 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:25:09.720 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-11-06T01:25:09.721 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T01:25:09.721 INFO:teuthology.orchestra.run.smithi163.stdout: "image_id": "fe94629dd8412671eebe0af3189cc84c3a5d41ab807bae18a1e13cf671063216", 2023-11-06T01:25:09.721 INFO:teuthology.orchestra.run.smithi163.stdout: "repo_digests": [ 2023-11-06T01:25:09.721 INFO:teuthology.orchestra.run.smithi163.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:801c764ebcb3d4c201ae63c6f6c4dd97da44ffffe122eed1091041bcab805777" 2023-11-06T01:25:09.721 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-11-06T01:25:09.722 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-11-06T01:25:17.319 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-11-06T01:25:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T01:25:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "fe94629dd8412671eebe0af3189cc84c3a5d41ab807bae18a1e13cf671063216", 2023-11-06T01:25:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2023-11-06T01:25:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:801c764ebcb3d4c201ae63c6f6c4dd97da44ffffe122eed1091041bcab805777" 2023-11-06T01:25:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2023-11-06T01:25:17.321 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-11-06T01:25:17.340 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2023-11-06T01:25:17.378 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2023-11-06T01:25:17.410 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph 2023-11-06T01:25:17.448 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2023-11-06T01:25:17.483 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2023-11-06T01:25:17.511 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /etc/ceph 2023-11-06T01:25:17.544 INFO:tasks.cephadm:Writing seed config... 2023-11-06T01:25:17.545 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-11-06T01:25:17.545 INFO:tasks.cephadm: override: [client] debug client = 20 2023-11-06T01:25:17.546 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-11-06T01:25:17.546 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-11-06T01:25:17.546 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-11-06T01:25:17.547 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-11-06T01:25:17.547 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-11-06T01:25:17.547 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-11-06T01:25:17.547 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-11-06T01:25:17.548 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-11-06T01:25:17.548 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-11-06T01:25:17.548 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-11-06T01:25:17.549 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-11-06T01:25:17.549 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-11-06T01:25:17.549 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-11-06T01:25:17.549 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-11-06T01:25:17.550 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-11-06T01:25:17.550 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-11-06T01:25:17.550 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-11-06T01:25:17.550 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-11-06T01:25:17.551 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-11-06T01:25:17.551 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-11-06T01:25:17.551 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-11-06T01:25:17.551 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-11-06T01:25:17.552 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-11-06T01:25:17.552 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-11-06T01:25:17.552 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-11-06T01:25:17.552 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-11-06T01:25:17.553 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-11-06T01:25:17.553 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-11-06T01:25:17.553 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-11-06T01:25:17.554 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-11-06T01:25:17.554 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-11-06T01:25:17.554 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-11-06T01:25:17.554 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-11-06T01:25:17.555 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-11-06T01:25:17.555 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-11-06T01:25:17.555 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-11-06T01:25:17.556 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-11-06T01:25:17.556 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-11-06T01:25:17.556 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-11-06T01:25:17.557 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-11-06T01:25:17.557 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-11-06T01:25:17.558 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:25:17.558 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-11-06T01:25:17.579 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-11-06T01:25:17.580 DEBUG:teuthology.orchestra.run.smithi053:mon.a> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.a.service 2023-11-06T01:25:17.622 DEBUG:teuthology.orchestra.run.smithi053:mgr.x> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.x.service 2023-11-06T01:25:17.653 INFO:journalctl@ceph.mon.a.smithi053.stdout:-- Logs begin at Mon 2023-11-06 01:09:30 UTC. -- 2023-11-06T01:25:17.654 INFO:tasks.cephadm:Bootstrapping... 2023-11-06T01:25:17.655 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 -v bootstrap --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.53 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-11-06T01:25:17.683 INFO:journalctl@ceph.mgr.x.smithi053.stdout:-- Logs begin at Mon 2023-11-06 01:09:30 UTC. -- 2023-11-06T01:25:17.826 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2023-11-06T01:25:17.826 INFO:teuthology.orchestra.run.smithi053.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5', '-v', 'bootstrap', '--fsid', '3f13dbd6-7c43-11ee-8db9-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.53', '--skip-admin-label'] 2023-11-06T01:25:17.970 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stdout 4.1.1 2023-11-06T01:25:17.971 INFO:teuthology.orchestra.run.smithi053.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-11-06T01:25:17.971 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying podman|docker is present... 2023-11-06T01:25:18.122 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stdout 4.1.1 2023-11-06T01:25:18.123 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying lvm2 is present... 2023-11-06T01:25:18.123 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying time synchronization is in place... 2023-11-06T01:25:18.130 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-06T01:25:18.130 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-06T01:25:18.136 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-06T01:25:18.136 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2023-11-06T01:25:18.141 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout enabled 2023-11-06T01:25:18.147 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout active 2023-11-06T01:25:18.148 INFO:teuthology.orchestra.run.smithi053.stdout:Unit chronyd.service is enabled and running 2023-11-06T01:25:18.148 INFO:teuthology.orchestra.run.smithi053.stdout:Repeating the final host check... 2023-11-06T01:25:18.290 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stdout 4.1.1 2023-11-06T01:25:18.290 INFO:teuthology.orchestra.run.smithi053.stdout:podman (/bin/podman) version 4.1.1 is present 2023-11-06T01:25:18.290 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl is present 2023-11-06T01:25:18.290 INFO:teuthology.orchestra.run.smithi053.stdout:lvcreate is present 2023-11-06T01:25:18.296 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-06T01:25:18.296 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-06T01:25:18.302 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-06T01:25:18.302 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2023-11-06T01:25:18.307 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout enabled 2023-11-06T01:25:18.314 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout active 2023-11-06T01:25:18.314 INFO:teuthology.orchestra.run.smithi053.stdout:Unit chronyd.service is enabled and running 2023-11-06T01:25:18.314 INFO:teuthology.orchestra.run.smithi053.stdout:Host looks OK 2023-11-06T01:25:18.314 INFO:teuthology.orchestra.run.smithi053.stdout:Cluster fsid: 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:25:18.315 INFO:teuthology.orchestra.run.smithi053.stdout:Acquiring lock 140561848211944 on /run/cephadm/3f13dbd6-7c43-11ee-8db9-212e2dc638e7.lock 2023-11-06T01:25:18.315 INFO:teuthology.orchestra.run.smithi053.stdout:Lock 140561848211944 acquired on /run/cephadm/3f13dbd6-7c43-11ee-8db9-212e2dc638e7.lock 2023-11-06T01:25:18.315 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying IP 172.21.15.53 port 3300 ... 2023-11-06T01:25:18.316 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying IP 172.21.15.53 port 6789 ... 2023-11-06T01:25:18.316 INFO:teuthology.orchestra.run.smithi053.stdout:Base mon IP(s) is [172.21.15.53:3300, 172.21.15.53:6789], mon addrv is [v2:172.21.15.53:3300,v1:172.21.15.53:6789] 2023-11-06T01:25:18.318 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-11-06T01:25:18.319 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.53 metric 100 2023-11-06T01:25:18.321 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-11-06T01:25:18.321 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-11-06T01:25:18.322 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2023-11-06T01:25:18.324 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-11-06T01:25:18.324 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-11-06T01:25:18.324 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-06T01:25:18.324 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout 3: ens1f0: mtu 1500 state UP qlen 1000 2023-11-06T01:25:18.325 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:3718/64 scope link 2023-11-06T01:25:18.325 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-06T01:25:18.326 INFO:teuthology.orchestra.run.smithi053.stdout:Mon IP `172.21.15.53` is in CIDR network `172.21.0.0/20` 2023-11-06T01:25:18.326 INFO:teuthology.orchestra.run.smithi053.stdout:Mon IP `172.21.15.53` is in CIDR network `172.21.0.0/20` 2023-11-06T01:25:18.326 INFO:teuthology.orchestra.run.smithi053.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-11-06T01:25:18.326 INFO:teuthology.orchestra.run.smithi053.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-11-06T01:25:18.327 INFO:teuthology.orchestra.run.smithi053.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5... 2023-11-06T01:25:18.874 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stdout fe94629dd8412671eebe0af3189cc84c3a5d41ab807bae18a1e13cf671063216 2023-11-06T01:25:18.875 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5... 2023-11-06T01:25:18.875 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Getting image source signatures 2023-11-06T01:25:18.875 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Copying blob sha256:9691f923f02fced31a9eef90e1490516f06755f808906d6a9c7c994691fa6bf0 2023-11-06T01:25:18.876 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Copying blob sha256:1f2d6592634967e9b1df5882d96d46932a2df60410f4a263d4c393afb79ac5ba 2023-11-06T01:25:18.876 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Copying config sha256:fe94629dd8412671eebe0af3189cc84c3a5d41ab807bae18a1e13cf671063216 2023-11-06T01:25:18.876 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Writing manifest to image destination 2023-11-06T01:25:18.876 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Storing signatures 2023-11-06T01:25:19.727 INFO:teuthology.orchestra.run.smithi053.stdout:ceph: stdout ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable) 2023-11-06T01:25:19.727 INFO:teuthology.orchestra.run.smithi053.stdout:Ceph version: ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable) 2023-11-06T01:25:19.727 INFO:teuthology.orchestra.run.smithi053.stdout:Extracting ceph user uid/gid from container image... 2023-11-06T01:25:20.478 INFO:teuthology.orchestra.run.smithi053.stdout:stat: stdout 167 167 2023-11-06T01:25:20.478 INFO:teuthology.orchestra.run.smithi053.stdout:Creating initial keys... 2023-11-06T01:25:21.229 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph-authtool: stdout AQCAQEhlixwENRAADhdw7X2yfXfoKXqh45ZsHQ== 2023-11-06T01:25:21.876 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph-authtool: stdout AQCBQEhlFPOwIxAATinwVGXZc9gTHo8wKEZpRw== 2023-11-06T01:25:22.628 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph-authtool: stdout AQCCQEhlI/B6DxAA5373sTqMmmAB8eOLLjB3BQ== 2023-11-06T01:25:22.630 INFO:teuthology.orchestra.run.smithi053.stdout:Creating initial monmap... 2023-11-06T01:25:23.481 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-06T01:25:23.481 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-11-06T01:25:23.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:25:23.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-06T01:25:23.482 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool for a [v2:172.21.15.53:3300,v1:172.21.15.53:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-06T01:25:23.482 INFO:teuthology.orchestra.run.smithi053.stdout:setting min_mon_release = pacific 2023-11-06T01:25:23.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: set fsid to 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:25:23.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-06T01:25:23.483 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:25:23.483 INFO:teuthology.orchestra.run.smithi053.stdout:Creating mon... 2023-11-06T01:25:24.340 INFO:teuthology.orchestra.run.smithi053.stdout:create mon.a on 2023-11-06T01:25:24.470 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-11-06T01:25:24.562 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-11-06T01:25:24.710 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7.target → /etc/systemd/system/ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7.target. 2023-11-06T01:25:24.710 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7.target → /etc/systemd/system/ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7.target. 2023-11-06T01:25:24.894 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.a 2023-11-06T01:25:24.895 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.a.service: Unit ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.a.service not loaded. 2023-11-06T01:25:24.983 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7.target.wants/ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@.service. 2023-11-06T01:25:25.266 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:24 smithi053 systemd[1]: Starting Ceph mon.a for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:25:25.490 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-06T01:25:25.490 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2023-11-06T01:25:25.497 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-06T01:25:25.498 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2023-11-06T01:25:25.498 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2023-11-06T01:25:25.498 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to enable service . firewalld.service is not available 2023-11-06T01:25:25.499 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mon to start... 2023-11-06T01:25:25.499 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mon... 2023-11-06T01:25:25.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:25 smithi053 podman[98931]: 2023-11-06T01:25:25.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:25 smithi053 bash[98931]: a8735ea97f840f1592d41233089fcde30069eeb328d7fee2cf9513972e01cb24 2023-11-06T01:25:25.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:25 smithi053 systemd[1]: Started Ceph mon.a for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:25:25.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:25 smithi053 ceph-mon[98971]: mkfs 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:25:25.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:25 smithi053 ceph-mon[98971]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-06T01:25:26.457 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout cluster: 2023-11-06T01:25:26.457 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout id: 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:25:26.458 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-11-06T01:25:26.458 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:26.458 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout services: 2023-11-06T01:25:26.459 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.62615s) 2023-11-06T01:25:26.459 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-11-06T01:25:26.459 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-11-06T01:25:26.460 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:26.460 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout data: 2023-11-06T01:25:26.461 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-11-06T01:25:26.461 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-11-06T01:25:26.461 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-11-06T01:25:26.462 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout pgs: 2023-11-06T01:25:26.462 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:26.463 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.981+0000 7f5c89d6a700 1 Processor -- start 2023-11-06T01:25:26.463 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.981+0000 7f5c89d6a700 1 -- start start 2023-11-06T01:25:26.463 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.982+0000 7f5c89d6a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c84104c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:26.463 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.982+0000 7f5c89d6a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c841051d0 con 0x7f5c841047e0 2023-11-06T01:25:26.463 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.982+0000 7f5c837fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c84104c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:26.464 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.982+0000 7f5c837fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c84104c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36620/0 (socket says 172.21.15.53:36620) 2023-11-06T01:25:26.464 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.982+0000 7f5c837fe700 1 -- 172.21.15.53:0/2296453520 learned_addr learned my addr 172.21.15.53:0/2296453520 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:26.464 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.982+0000 7f5c837fe700 1 -- 172.21.15.53:0/2296453520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c84105310 con 0x7f5c841047e0 2023-11-06T01:25:26.465 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.983+0000 7f5c837fe700 1 --2- 172.21.15.53:0/2296453520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c84104c00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5c740098a0 tx=0x7f5c74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2bb57c7884ff43 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:26.465 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.983+0000 7f5c827fc700 1 -- 172.21.15.53:0/2296453520 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c74004160 con 0x7f5c841047e0 2023-11-06T01:25:26.465 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.983+0000 7f5c827fc700 1 -- 172.21.15.53:0/2296453520 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c740042c0 con 0x7f5c841047e0 2023-11-06T01:25:26.466 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.983+0000 7f5c827fc700 1 -- 172.21.15.53:0/2296453520 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c740045a0 con 0x7f5c841047e0 2023-11-06T01:25:26.466 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.983+0000 7f5c89d6a700 1 -- 172.21.15.53:0/2296453520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 msgr2=0x7f5c84104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:26.466 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.983+0000 7f5c89d6a700 1 --2- 172.21.15.53:0/2296453520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c84104c00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5c740098a0 tx=0x7f5c74009bb0 comp rx=0 tx=0).stop 2023-11-06T01:25:26.466 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.983+0000 7f5c89d6a700 1 -- 172.21.15.53:0/2296453520 shutdown_connections 2023-11-06T01:25:26.467 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.983+0000 7f5c89d6a700 1 --2- 172.21.15.53:0/2296453520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c84104c00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:26.467 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.983+0000 7f5c89d6a700 1 -- 172.21.15.53:0/2296453520 >> 172.21.15.53:0/2296453520 conn(0x7f5c84100140 msgr2=0x7f5c84102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:26.467 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.983+0000 7f5c89d6a700 1 -- 172.21.15.53:0/2296453520 shutdown_connections 2023-11-06T01:25:26.467 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.983+0000 7f5c89d6a700 1 -- 172.21.15.53:0/2296453520 wait complete. 2023-11-06T01:25:26.468 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.984+0000 7f5c89d6a700 1 Processor -- start 2023-11-06T01:25:26.468 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.984+0000 7f5c89d6a700 1 -- start start 2023-11-06T01:25:26.468 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.984+0000 7f5c89d6a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c841071c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:26.468 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.984+0000 7f5c89d6a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c84107700 con 0x7f5c841047e0 2023-11-06T01:25:26.468 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.984+0000 7f5c837fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c841071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.985+0000 7f5c837fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c841071c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36622/0 (socket says 172.21.15.53:36622) 2023-11-06T01:25:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.985+0000 7f5c837fe700 1 -- 172.21.15.53:0/2903153616 learned_addr learned my addr 172.21.15.53:0/2903153616 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.985+0000 7f5c837fe700 1 -- 172.21.15.53:0/2903153616 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c74009550 con 0x7f5c841047e0 2023-11-06T01:25:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.985+0000 7f5c837fe700 1 --2- 172.21.15.53:0/2903153616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c841071c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5c74004000 tx=0x7f5c74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.985+0000 7f5c80ff9700 1 -- 172.21.15.53:0/2903153616 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c74011630 con 0x7f5c841047e0 2023-11-06T01:25:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.985+0000 7f5c80ff9700 1 -- 172.21.15.53:0/2903153616 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c74011790 con 0x7f5c841047e0 2023-11-06T01:25:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.985+0000 7f5c89d6a700 1 -- 172.21.15.53:0/2903153616 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c84105870 con 0x7f5c841047e0 2023-11-06T01:25:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.985+0000 7f5c80ff9700 1 -- 172.21.15.53:0/2903153616 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c74011a30 con 0x7f5c841047e0 2023-11-06T01:25:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.985+0000 7f5c89d6a700 1 -- 172.21.15.53:0/2903153616 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c84105d70 con 0x7f5c841047e0 2023-11-06T01:25:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.985+0000 7f5c80ff9700 1 -- 172.21.15.53:0/2903153616 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5c74022070 con 0x7f5c841047e0 2023-11-06T01:25:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.985+0000 7f5c80ff9700 1 -- 172.21.15.53:0/2903153616 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5c7401c790 con 0x7f5c841047e0 2023-11-06T01:25:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.986+0000 7f5c6a7fc700 1 -- 172.21.15.53:0/2903153616 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c640052f0 con 0x7f5c841047e0 2023-11-06T01:25:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:25.989+0000 7f5c80ff9700 1 -- 172.21.15.53:0/2903153616 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f5c7401bca0 con 0x7f5c841047e0 2023-11-06T01:25:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.123+0000 7f5c6a7fc700 1 -- 172.21.15.53:0/2903153616 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5c64005c90 con 0x7f5c841047e0 2023-11-06T01:25:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.123+0000 7f5c80ff9700 1 -- 172.21.15.53:0/2903153616 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7f5c7401be40 con 0x7f5c841047e0 2023-11-06T01:25:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.124+0000 7f5c6a7fc700 1 -- 172.21.15.53:0/2903153616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 msgr2=0x7f5c841071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.124+0000 7f5c6a7fc700 1 --2- 172.21.15.53:0/2903153616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c841071c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5c74004000 tx=0x7f5c74004750 comp rx=0 tx=0).stop 2023-11-06T01:25:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.124+0000 7f5c6a7fc700 1 -- 172.21.15.53:0/2903153616 shutdown_connections 2023-11-06T01:25:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.124+0000 7f5c6a7fc700 1 --2- 172.21.15.53:0/2903153616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c841047e0 0x7f5c841071c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.124+0000 7f5c6a7fc700 1 -- 172.21.15.53:0/2903153616 >> 172.21.15.53:0/2903153616 conn(0x7f5c84100140 msgr2=0x7f5c8418e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.124+0000 7f5c6a7fc700 1 -- 172.21.15.53:0/2903153616 shutdown_connections 2023-11-06T01:25:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.124+0000 7f5c6a7fc700 1 -- 172.21.15.53:0/2903153616 wait complete. 2023-11-06T01:25:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:mon is available 2023-11-06T01:25:26.474 INFO:teuthology.orchestra.run.smithi053.stdout:Assimilating anything we can from ceph.conf... 2023-11-06T01:25:26.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:26 smithi053 ceph-mon[98971]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-06T01:25:26.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:26 smithi053 ceph-mon[98971]: monmap e1: 1 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-11-06T01:25:26.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:26 smithi053 ceph-mon[98971]: fsmap 2023-11-06T01:25:26.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:26 smithi053 ceph-mon[98971]: osdmap e1: 0 total, 0 up, 0 in 2023-11-06T01:25:26.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:26 smithi053 ceph-mon[98971]: mgrmap e1: no daemons active 2023-11-06T01:25:26.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:26 smithi053 ceph-mon[98971]: from='client.? 172.21.15.53:0/2903153616' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-11-06T01:25:27.404 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:27.404 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [global] 2023-11-06T01:25:27.405 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout fsid = 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:25:27.405 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.53:3300,v1:172.21.15.53:6789] 2023-11-06T01:25:27.405 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-06T01:25:27.405 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-06T01:25:27.405 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-06T01:25:27.405 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-06T01:25:27.406 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:27.406 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-06T01:25:27.406 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-06T01:25:27.406 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:27.407 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [osd] 2023-11-06T01:25:27.407 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-06T01:25:27.407 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-06T01:25:27.407 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-06T01:25:27.407 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-06T01:25:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-06T01:25:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.880+0000 7f21ca485700 1 Processor -- start 2023-11-06T01:25:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.880+0000 7f21ca485700 1 -- start start 2023-11-06T01:25:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.880+0000 7f21ca485700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c4107180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:27.408 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.880+0000 7f21ca485700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c4079e90 con 0x7f21c4104d50 2023-11-06T01:25:27.409 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.881+0000 7f21c3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c4107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:27.409 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.881+0000 7f21c3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c4107180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36624/0 (socket says 172.21.15.53:36624) 2023-11-06T01:25:27.410 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.881+0000 7f21c3fff700 1 -- 172.21.15.53:0/2339709319 learned_addr learned my addr 172.21.15.53:0/2339709319 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:27.410 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.881+0000 7f21c3fff700 1 -- 172.21.15.53:0/2339709319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c41076c0 con 0x7f21c4104d50 2023-11-06T01:25:27.410 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.881+0000 7f21c3fff700 1 --2- 172.21.15.53:0/2339709319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c4107180 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0098a0 tx=0x7f21ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210b03e4302d7119 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:27.410 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.881+0000 7f21c2ffd700 1 -- 172.21.15.53:0/2339709319 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21ac004160 con 0x7f21c4104d50 2023-11-06T01:25:27.410 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.881+0000 7f21c2ffd700 1 -- 172.21.15.53:0/2339709319 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ac0042c0 con 0x7f21c4104d50 2023-11-06T01:25:27.411 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.881+0000 7f21c2ffd700 1 -- 172.21.15.53:0/2339709319 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21ac0045a0 con 0x7f21c4104d50 2023-11-06T01:25:27.411 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.882+0000 7f21ca485700 1 -- 172.21.15.53:0/2339709319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 msgr2=0x7f21c4107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:27.411 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.882+0000 7f21ca485700 1 --2- 172.21.15.53:0/2339709319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c4107180 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0098a0 tx=0x7f21ac009bb0 comp rx=0 tx=0).stop 2023-11-06T01:25:27.411 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.882+0000 7f21ca485700 1 -- 172.21.15.53:0/2339709319 shutdown_connections 2023-11-06T01:25:27.411 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.882+0000 7f21ca485700 1 --2- 172.21.15.53:0/2339709319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c4107180 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:27.412 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.882+0000 7f21ca485700 1 -- 172.21.15.53:0/2339709319 >> 172.21.15.53:0/2339709319 conn(0x7f21c41009b0 msgr2=0x7f21c4102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:27.412 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.882+0000 7f21ca485700 1 -- 172.21.15.53:0/2339709319 shutdown_connections 2023-11-06T01:25:27.412 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.882+0000 7f21ca485700 1 -- 172.21.15.53:0/2339709319 wait complete. 2023-11-06T01:25:27.412 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.883+0000 7f21ca485700 1 Processor -- start 2023-11-06T01:25:27.412 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.883+0000 7f21ca485700 1 -- start start 2023-11-06T01:25:27.413 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.883+0000 7f21ca485700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c419ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:27.413 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.883+0000 7f21ca485700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c41a0490 con 0x7f21c4104d50 2023-11-06T01:25:27.413 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.883+0000 7f21c3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c419ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:27.413 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.883+0000 7f21c3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c419ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36626/0 (socket says 172.21.15.53:36626) 2023-11-06T01:25:27.413 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.883+0000 7f21c3fff700 1 -- 172.21.15.53:0/2774731087 learned_addr learned my addr 172.21.15.53:0/2774731087 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:27.414 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.883+0000 7f21c3fff700 1 -- 172.21.15.53:0/2774731087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac009550 con 0x7f21c4104d50 2023-11-06T01:25:27.414 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.884+0000 7f21c3fff700 1 --2- 172.21.15.53:0/2774731087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c419ff50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f21c40792d0 tx=0x7f21ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:27.414 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.884+0000 7f21c17fa700 1 -- 172.21.15.53:0/2774731087 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21ac011630 con 0x7f21c4104d50 2023-11-06T01:25:27.414 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.884+0000 7f21c17fa700 1 -- 172.21.15.53:0/2774731087 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ac011790 con 0x7f21c4104d50 2023-11-06T01:25:27.414 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.884+0000 7f21ca485700 1 -- 172.21.15.53:0/2774731087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c41a0690 con 0x7f21c4104d50 2023-11-06T01:25:27.415 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.884+0000 7f21c17fa700 1 -- 172.21.15.53:0/2774731087 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21ac011a30 con 0x7f21c4104d50 2023-11-06T01:25:27.415 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.884+0000 7f21ca485700 1 -- 172.21.15.53:0/2774731087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c41a0b30 con 0x7f21c4104d50 2023-11-06T01:25:27.415 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.884+0000 7f21c17fa700 1 -- 172.21.15.53:0/2774731087 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f21ac022070 con 0x7f21c4104d50 2023-11-06T01:25:27.415 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.884+0000 7f21c17fa700 1 -- 172.21.15.53:0/2774731087 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f21ac01c450 con 0x7f21c4104d50 2023-11-06T01:25:27.415 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.885+0000 7f21ca485700 1 -- 172.21.15.53:0/2774731087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c419a030 con 0x7f21c4104d50 2023-11-06T01:25:27.416 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:26.887+0000 7f21c17fa700 1 -- 172.21.15.53:0/2774731087 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f21ac01b450 con 0x7f21c4104d50 2023-11-06T01:25:27.416 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.014+0000 7f21ca485700 1 -- 172.21.15.53:0/2774731087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f21c40623c0 con 0x7f21c4104d50 2023-11-06T01:25:27.416 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.018+0000 7f21c17fa700 1 -- 172.21.15.53:0/2774731087 <== mon.0 v2:172.21.15.53:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f21ac01b5f0 con 0x7f21c4104d50 2023-11-06T01:25:27.416 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.018+0000 7f21c17fa700 1 -- 172.21.15.53:0/2774731087 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f21ac024630 con 0x7f21c4104d50 2023-11-06T01:25:27.416 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.019+0000 7f21ca485700 1 -- 172.21.15.53:0/2774731087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 msgr2=0x7f21c419ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:27.417 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.019+0000 7f21ca485700 1 --2- 172.21.15.53:0/2774731087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c419ff50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f21c40792d0 tx=0x7f21ac004750 comp rx=0 tx=0).stop 2023-11-06T01:25:27.417 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.019+0000 7f21ca485700 1 -- 172.21.15.53:0/2774731087 shutdown_connections 2023-11-06T01:25:27.417 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.019+0000 7f21ca485700 1 --2- 172.21.15.53:0/2774731087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21c4104d50 0x7f21c419ff50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:27.417 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.019+0000 7f21ca485700 1 -- 172.21.15.53:0/2774731087 >> 172.21.15.53:0/2774731087 conn(0x7f21c41009b0 msgr2=0x7f21c41021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:27.418 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.019+0000 7f21ca485700 1 -- 172.21.15.53:0/2774731087 shutdown_connections 2023-11-06T01:25:27.418 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.019+0000 7f21ca485700 1 -- 172.21.15.53:0/2774731087 wait complete. 2023-11-06T01:25:27.418 INFO:teuthology.orchestra.run.smithi053.stdout:Generating new minimal ceph.conf... 2023-11-06T01:25:27.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:27 smithi053 ceph-mon[98971]: from='client.? 172.21.15.53:0/2774731087' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-06T01:25:27.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:27 smithi053 ceph-mon[98971]: from='client.? 172.21.15.53:0/2774731087' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-11-06T01:25:28.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:28 smithi053 ceph-mon[98971]: from='client.? 172.21.15.53:0/2966215125' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:25:30.452 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.829+0000 7ff27ce8a700 1 Processor -- start 2023-11-06T01:25:30.453 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.829+0000 7ff27ce8a700 1 -- start start 2023-11-06T01:25:30.453 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.829+0000 7ff27ce8a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff2781051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:30.453 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.830+0000 7ff27ce8a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278079e10 con 0x7ff278104dd0 2023-11-06T01:25:30.453 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.830+0000 7ff27659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff2781051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:30.454 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.830+0000 7ff27659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff2781051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36628/0 (socket says 172.21.15.53:36628) 2023-11-06T01:25:30.454 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.830+0000 7ff27659c700 1 -- 172.21.15.53:0/1424772459 learned_addr learned my addr 172.21.15.53:0/1424772459 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:30.454 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.830+0000 7ff27659c700 1 -- 172.21.15.53:0/1424772459 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278105730 con 0x7ff278104dd0 2023-11-06T01:25:30.454 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.830+0000 7ff27659c700 1 --2- 172.21.15.53:0/1424772459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff2781051f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff2680098a0 tx=0x7ff268009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26296d05bac3fe88 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:30.454 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.830+0000 7ff27559a700 1 -- 172.21.15.53:0/1424772459 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff268004160 con 0x7ff278104dd0 2023-11-06T01:25:30.455 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.830+0000 7ff27559a700 1 -- 172.21.15.53:0/1424772459 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2680042c0 con 0x7ff278104dd0 2023-11-06T01:25:30.455 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.830+0000 7ff27559a700 1 -- 172.21.15.53:0/1424772459 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff268011a00 con 0x7ff278104dd0 2023-11-06T01:25:30.455 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.831+0000 7ff27ce8a700 1 -- 172.21.15.53:0/1424772459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 msgr2=0x7ff2781051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:30.455 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.831+0000 7ff27ce8a700 1 --2- 172.21.15.53:0/1424772459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff2781051f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff2680098a0 tx=0x7ff268009bb0 comp rx=0 tx=0).stop 2023-11-06T01:25:30.459 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.831+0000 7ff27ce8a700 1 -- 172.21.15.53:0/1424772459 shutdown_connections 2023-11-06T01:25:30.460 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.831+0000 7ff27ce8a700 1 --2- 172.21.15.53:0/1424772459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff2781051f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:30.460 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.831+0000 7ff27ce8a700 1 -- 172.21.15.53:0/1424772459 >> 172.21.15.53:0/1424772459 conn(0x7ff2781009b0 msgr2=0x7ff278102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:30.460 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.831+0000 7ff27ce8a700 1 -- 172.21.15.53:0/1424772459 shutdown_connections 2023-11-06T01:25:30.460 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.831+0000 7ff27ce8a700 1 -- 172.21.15.53:0/1424772459 wait complete. 2023-11-06T01:25:30.461 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.832+0000 7ff27ce8a700 1 Processor -- start 2023-11-06T01:25:30.461 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.832+0000 7ff27ce8a700 1 -- start start 2023-11-06T01:25:30.461 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.832+0000 7ff27ce8a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff27819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:30.461 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.832+0000 7ff27ce8a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27819c880 con 0x7ff278104dd0 2023-11-06T01:25:30.461 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.832+0000 7ff27659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff27819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:30.462 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.832+0000 7ff27659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff27819c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36630/0 (socket says 172.21.15.53:36630) 2023-11-06T01:25:30.462 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.832+0000 7ff27659c700 1 -- 172.21.15.53:0/2966215125 learned_addr learned my addr 172.21.15.53:0/2966215125 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:30.462 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.833+0000 7ff27659c700 1 -- 172.21.15.53:0/2966215125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff268009550 con 0x7ff278104dd0 2023-11-06T01:25:30.462 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.833+0000 7ff27659c700 1 --2- 172.21.15.53:0/2966215125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff27819c340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff268009fb0 tx=0x7ff268011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:30.462 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.833+0000 7ff26f7fe700 1 -- 172.21.15.53:0/2966215125 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff268003930 con 0x7ff278104dd0 2023-11-06T01:25:30.463 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.833+0000 7ff26f7fe700 1 -- 172.21.15.53:0/2966215125 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff268003a90 con 0x7ff278104dd0 2023-11-06T01:25:30.463 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.833+0000 7ff26f7fe700 1 -- 172.21.15.53:0/2966215125 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff268019610 con 0x7ff278104dd0 2023-11-06T01:25:30.463 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.833+0000 7ff27ce8a700 1 -- 172.21.15.53:0/2966215125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27819ca80 con 0x7ff278104dd0 2023-11-06T01:25:30.463 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.833+0000 7ff27ce8a700 1 -- 172.21.15.53:0/2966215125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27819cf20 con 0x7ff278104dd0 2023-11-06T01:25:30.463 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.833+0000 7ff26f7fe700 1 -- 172.21.15.53:0/2966215125 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff268019b80 con 0x7ff278104dd0 2023-11-06T01:25:30.464 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.833+0000 7ff26f7fe700 1 -- 172.21.15.53:0/2966215125 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff2680243f0 con 0x7ff278104dd0 2023-11-06T01:25:30.464 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.834+0000 7ff27ce8a700 1 -- 172.21.15.53:0/2966215125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff278195c20 con 0x7ff278104dd0 2023-11-06T01:25:30.464 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.836+0000 7ff26f7fe700 1 -- 172.21.15.53:0/2966215125 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7ff26802de40 con 0x7ff278104dd0 2023-11-06T01:25:30.464 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.964+0000 7ff27ce8a700 1 -- 172.21.15.53:0/2966215125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff2780623c0 con 0x7ff278104dd0 2023-11-06T01:25:30.464 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.964+0000 7ff26f7fe700 1 -- 172.21.15.53:0/2966215125 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7ff268003cb0 con 0x7ff278104dd0 2023-11-06T01:25:30.465 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.965+0000 7ff27ce8a700 1 -- 172.21.15.53:0/2966215125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 msgr2=0x7ff27819c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:30.465 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.965+0000 7ff27ce8a700 1 --2- 172.21.15.53:0/2966215125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff27819c340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff268009fb0 tx=0x7ff268011ad0 comp rx=0 tx=0).stop 2023-11-06T01:25:30.465 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.965+0000 7ff27ce8a700 1 -- 172.21.15.53:0/2966215125 shutdown_connections 2023-11-06T01:25:30.465 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.965+0000 7ff27ce8a700 1 --2- 172.21.15.53:0/2966215125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff278104dd0 0x7ff27819c340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:30.465 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.965+0000 7ff27ce8a700 1 -- 172.21.15.53:0/2966215125 >> 172.21.15.53:0/2966215125 conn(0x7ff2781009b0 msgr2=0x7ff278190990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:30.465 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.965+0000 7ff27ce8a700 1 -- 172.21.15.53:0/2966215125 shutdown_connections 2023-11-06T01:25:30.466 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:27.965+0000 7ff27ce8a700 1 -- 172.21.15.53:0/2966215125 wait complete. 2023-11-06T01:25:30.466 INFO:teuthology.orchestra.run.smithi053.stdout:Restarting the monitor... 2023-11-06T01:25:30.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:30 smithi053 systemd[1]: Stopping Ceph mon.a for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:25:31.326 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-a[98947]: 2023-11-06T01:25:31.052+0000 7f269cae8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:25:31.326 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-a[98947]: 2023-11-06T01:25:31.052+0000 7f269cae8700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-11-06T01:25:33.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:33 smithi053 bash[99308]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-a 2023-11-06T01:25:33.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:33 smithi053 systemd[1]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.a.service: Succeeded. 2023-11-06T01:25:33.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:33 smithi053 systemd[1]: Stopped Ceph mon.a for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:25:33.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:33 smithi053 systemd[1]: Starting Ceph mon.a for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:25:35.039 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:34 smithi053 podman[99432]: 2023-11-06T01:25:35.783 INFO:teuthology.orchestra.run.smithi053.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-11-06T01:25:35.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: set uid:gid to 167:167 (ceph:ceph) 2023-11-06T01:25:35.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable), process ceph-mon, pid 7 2023-11-06T01:25:35.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: pidfile_write: ignore empty --pid-file 2023-11-06T01:25:35.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: load: jerasure load: lrc 2023-11-06T01:25:35.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: RocksDB version: 7.9.2 2023-11-06T01:25:35.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Git sha 0 2023-11-06T01:25:35.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Compile date 2023-11-04 15:03:31 2023-11-06T01:25:35.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: DB SUMMARY 2023-11-06T01:25:35.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: DB Session ID: VIPMXQBGDCW2MN2U3YAR 2023-11-06T01:25:35.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: CURRENT file: CURRENT 2023-11-06T01:25:35.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: IDENTITY file: IDENTITY 2023-11-06T01:25:35.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-11-06T01:25:35.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-11-06T01:25:35.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98870 ; 2023-11-06T01:25:35.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.error_if_exists: 0 2023-11-06T01:25:35.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.create_if_missing: 0 2023-11-06T01:25:35.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.paranoid_checks: 1 2023-11-06T01:25:35.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-06T01:25:35.850 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-06T01:25:35.850 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-06T01:25:35.850 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.env: 0x55bc22951720 2023-11-06T01:25:35.850 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.fs: PosixFileSystem 2023-11-06T01:25:35.851 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.info_log: 0x55bc245b9440 2023-11-06T01:25:35.851 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_file_opening_threads: 16 2023-11-06T01:25:35.851 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.statistics: (nil) 2023-11-06T01:25:35.851 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.use_fsync: 0 2023-11-06T01:25:35.851 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_log_file_size: 0 2023-11-06T01:25:35.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-06T01:25:35.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-06T01:25:35.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.keep_log_file_num: 1000 2023-11-06T01:25:35.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.recycle_log_file_num: 0 2023-11-06T01:25:35.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.allow_fallocate: 1 2023-11-06T01:25:35.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.allow_mmap_reads: 0 2023-11-06T01:25:35.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.allow_mmap_writes: 0 2023-11-06T01:25:35.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.use_direct_reads: 0 2023-11-06T01:25:35.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-06T01:25:35.854 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.create_missing_column_families: 0 2023-11-06T01:25:35.854 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.db_log_dir: 2023-11-06T01:25:35.854 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.wal_dir: 2023-11-06T01:25:35.854 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-06T01:25:35.854 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-06T01:25:35.855 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-06T01:25:35.855 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-06T01:25:35.855 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-06T01:25:35.855 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-06T01:25:35.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.advise_random_on_open: 1 2023-11-06T01:25:35.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.db_write_buffer_size: 0 2023-11-06T01:25:35.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.write_buffer_manager: 0x55bc2384a320 2023-11-06T01:25:35.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-06T01:25:35.857 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-06T01:25:35.857 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-06T01:25:35.857 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.rate_limiter: (nil) 2023-11-06T01:25:35.857 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-06T01:25:35.857 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.wal_recovery_mode: 2 2023-11-06T01:25:35.858 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.enable_thread_tracking: 0 2023-11-06T01:25:35.858 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.enable_pipelined_write: 0 2023-11-06T01:25:35.858 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.unordered_write: 0 2023-11-06T01:25:35.858 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-06T01:25:35.859 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-06T01:25:35.859 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-06T01:25:35.859 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-06T01:25:35.859 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.row_cache: None 2023-11-06T01:25:35.859 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.wal_filter: None 2023-11-06T01:25:35.860 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-06T01:25:35.860 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.allow_ingest_behind: 0 2023-11-06T01:25:35.860 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.two_write_queues: 0 2023-11-06T01:25:35.861 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.manual_wal_flush: 0 2023-11-06T01:25:35.861 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.wal_compression: 0 2023-11-06T01:25:35.861 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.atomic_flush: 0 2023-11-06T01:25:35.862 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-06T01:25:35.862 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-06T01:25:35.862 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-06T01:25:35.862 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.log_readahead_size: 0 2023-11-06T01:25:35.862 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-06T01:25:35.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.best_efforts_recovery: 0 2023-11-06T01:25:35.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-06T01:25:35.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-06T01:25:35.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.allow_data_in_errors: 0 2023-11-06T01:25:35.863 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.db_host_id: __hostname__ 2023-11-06T01:25:35.864 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-06T01:25:35.864 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_background_jobs: 2 2023-11-06T01:25:35.864 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_background_compactions: -1 2023-11-06T01:25:35.864 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_subcompactions: 1 2023-11-06T01:25:35.864 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-06T01:25:35.864 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-06T01:25:35.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-06T01:25:35.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_total_wal_size: 0 2023-11-06T01:25:35.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-06T01:25:35.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-06T01:25:35.865 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-06T01:25:35.866 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-06T01:25:35.866 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_open_files: -1 2023-11-06T01:25:35.866 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bytes_per_sync: 0 2023-11-06T01:25:35.866 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-06T01:25:35.866 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-06T01:25:35.867 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_readahead_size: 0 2023-11-06T01:25:35.867 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_background_flushes: -1 2023-11-06T01:25:35.867 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Compression algorithms supported: 2023-11-06T01:25:35.867 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-06T01:25:35.868 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: kZSTD supported: 0 2023-11-06T01:25:35.868 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: kXpressCompression supported: 0 2023-11-06T01:25:35.868 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: kLZ4HCCompression supported: 1 2023-11-06T01:25:35.868 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: kZlibCompression supported: 1 2023-11-06T01:25:35.868 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: kSnappyCompression supported: 1 2023-11-06T01:25:35.869 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: kLZ4Compression supported: 1 2023-11-06T01:25:35.869 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: kBZip2Compression supported: 0 2023-11-06T01:25:35.869 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-06T01:25:35.869 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-06T01:25:35.869 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-06T01:25:35.870 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-06T01:25:35.870 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-06T01:25:35.870 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.merge_operator: 2023-11-06T01:25:35.870 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_filter: None 2023-11-06T01:25:35.870 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_filter_factory: None 2023-11-06T01:25:35.871 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.sst_partitioner_factory: None 2023-11-06T01:25:35.871 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-06T01:25:35.871 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-06T01:25:35.871 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bc245b9540) 2023-11-06T01:25:35.871 INFO:journalctl@ceph.mon.a.smithi053.stdout: cache_index_and_filter_blocks: 1 2023-11-06T01:25:35.872 INFO:journalctl@ceph.mon.a.smithi053.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-06T01:25:35.872 INFO:journalctl@ceph.mon.a.smithi053.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-06T01:25:35.872 INFO:journalctl@ceph.mon.a.smithi053.stdout: pin_top_level_index_and_filter: 1 2023-11-06T01:25:35.872 INFO:journalctl@ceph.mon.a.smithi053.stdout: index_type: 0 2023-11-06T01:25:35.872 INFO:journalctl@ceph.mon.a.smithi053.stdout: data_block_index_type: 0 2023-11-06T01:25:35.872 INFO:journalctl@ceph.mon.a.smithi053.stdout: index_shortening: 1 2023-11-06T01:25:35.873 INFO:journalctl@ceph.mon.a.smithi053.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-06T01:25:35.873 INFO:journalctl@ceph.mon.a.smithi053.stdout: checksum: 4 2023-11-06T01:25:35.873 INFO:journalctl@ceph.mon.a.smithi053.stdout: no_block_cache: 0 2023-11-06T01:25:35.873 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_cache: 0x55bc238cb1f0 2023-11-06T01:25:35.873 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_cache_name: BinnedLRUCache 2023-11-06T01:25:35.874 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_cache_options: 2023-11-06T01:25:35.874 INFO:journalctl@ceph.mon.a.smithi053.stdout: capacity : 536870912 2023-11-06T01:25:35.874 INFO:journalctl@ceph.mon.a.smithi053.stdout: num_shard_bits : 4 2023-11-06T01:25:35.874 INFO:journalctl@ceph.mon.a.smithi053.stdout: strict_capacity_limit : 0 2023-11-06T01:25:35.874 INFO:journalctl@ceph.mon.a.smithi053.stdout: high_pri_pool_ratio: 0.000 2023-11-06T01:25:35.875 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_cache_compressed: (nil) 2023-11-06T01:25:35.875 INFO:journalctl@ceph.mon.a.smithi053.stdout: persistent_cache: (nil) 2023-11-06T01:25:35.875 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_size: 4096 2023-11-06T01:25:35.875 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_size_deviation: 10 2023-11-06T01:25:35.875 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_restart_interval: 16 2023-11-06T01:25:35.875 INFO:journalctl@ceph.mon.a.smithi053.stdout: index_block_restart_interval: 1 2023-11-06T01:25:35.876 INFO:journalctl@ceph.mon.a.smithi053.stdout: metadata_block_size: 4096 2023-11-06T01:25:35.876 INFO:journalctl@ceph.mon.a.smithi053.stdout: partition_filters: 0 2023-11-06T01:25:35.876 INFO:journalctl@ceph.mon.a.smithi053.stdout: use_delta_encoding: 1 2023-11-06T01:25:35.876 INFO:journalctl@ceph.mon.a.smithi053.stdout: filter_policy: bloomfilter 2023-11-06T01:25:35.876 INFO:journalctl@ceph.mon.a.smithi053.stdout: whole_key_filtering: 1 2023-11-06T01:25:35.877 INFO:journalctl@ceph.mon.a.smithi053.stdout: verify_compression: 0 2023-11-06T01:25:35.877 INFO:journalctl@ceph.mon.a.smithi053.stdout: read_amp_bytes_per_bit: 0 2023-11-06T01:25:35.877 INFO:journalctl@ceph.mon.a.smithi053.stdout: format_version: 5 2023-11-06T01:25:35.877 INFO:journalctl@ceph.mon.a.smithi053.stdout: enable_index_compression: 1 2023-11-06T01:25:35.877 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_align: 0 2023-11-06T01:25:35.878 INFO:journalctl@ceph.mon.a.smithi053.stdout: max_auto_readahead_size: 262144 2023-11-06T01:25:35.878 INFO:journalctl@ceph.mon.a.smithi053.stdout: prepopulate_block_cache: 0 2023-11-06T01:25:35.878 INFO:journalctl@ceph.mon.a.smithi053.stdout: initial_auto_readahead_size: 8192 2023-11-06T01:25:35.878 INFO:journalctl@ceph.mon.a.smithi053.stdout: num_file_reads_for_auto_readahead: 2 2023-11-06T01:25:35.878 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.write_buffer_size: 33554432 2023-11-06T01:25:35.879 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_write_buffer_number: 2 2023-11-06T01:25:35.879 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compression: NoCompression 2023-11-06T01:25:35.879 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bottommost_compression: Disabled 2023-11-06T01:25:35.879 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.prefix_extractor: nullptr 2023-11-06T01:25:35.879 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-06T01:25:35.880 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.num_levels: 7 2023-11-06T01:25:35.880 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-06T01:25:35.880 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-06T01:25:35.880 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-06T01:25:35.880 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-06T01:25:35.880 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-06T01:25:35.881 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-06T01:25:35.881 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-06T01:25:35.881 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-06T01:25:35.881 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-06T01:25:35.881 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-06T01:25:35.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-06T01:25:35.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-06T01:25:35.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-06T01:25:35.882 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compression_opts.level: 32767 2023-11-06T01:25:35.883 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compression_opts.strategy: 0 2023-11-06T01:25:35.883 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-06T01:25:35.883 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-06T01:25:35.883 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-06T01:25:35.883 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-06T01:25:35.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compression_opts.enabled: false 2023-11-06T01:25:35.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-06T01:25:35.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-06T01:25:35.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-06T01:25:35.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-06T01:25:35.884 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.target_file_size_base: 67108864 2023-11-06T01:25:35.885 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-06T01:25:35.885 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-06T01:25:35.885 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-06T01:25:35.885 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-06T01:25:35.885 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-06T01:25:35.886 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-06T01:25:35.886 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-06T01:25:35.886 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-06T01:25:35.886 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-06T01:25:35.886 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-06T01:25:35.887 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-06T01:25:35.887 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-06T01:25:35.887 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-06T01:25:35.887 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-06T01:25:35.887 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.arena_block_size: 1048576 2023-11-06T01:25:35.888 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-06T01:25:35.888 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-06T01:25:35.889 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.disable_auto_compactions: 0 2023-11-06T01:25:35.889 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-06T01:25:35.889 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-06T01:25:35.889 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-06T01:25:35.889 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-06T01:25:35.890 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-06T01:25:35.890 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-06T01:25:35.890 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-06T01:25:35.890 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-06T01:25:35.890 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-06T01:25:35.891 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-06T01:25:35.891 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.table_properties_collectors: 2023-11-06T01:25:35.891 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.inplace_update_support: 0 2023-11-06T01:25:35.891 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-06T01:25:35.891 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-06T01:25:35.892 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-06T01:25:35.892 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-06T01:25:35.892 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.bloom_locality: 0 2023-11-06T01:25:35.892 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.max_successive_merges: 0 2023-11-06T01:25:35.892 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-06T01:25:35.893 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.paranoid_file_checks: 0 2023-11-06T01:25:35.893 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.force_consistency_checks: 1 2023-11-06T01:25:35.893 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.report_bg_io_stats: 0 2023-11-06T01:25:35.893 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.ttl: 2592000 2023-11-06T01:25:35.893 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-06T01:25:35.893 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-06T01:25:35.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-06T01:25:35.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.enable_blob_files: false 2023-11-06T01:25:35.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.min_blob_size: 0 2023-11-06T01:25:35.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.blob_file_size: 268435456 2023-11-06T01:25:35.894 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-06T01:25:35.895 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-06T01:25:35.895 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-06T01:25:35.895 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-06T01:25:35.895 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-06T01:25:35.895 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.blob_file_starting_level: 0 2023-11-06T01:25:35.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-06T01:25:35.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-11-06T01:25:35.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-11-06T01:25:35.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 660846a2-c746-4175-9f54-e2db7a2ca234 2023-11-06T01:25:35.896 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: EVENT_LOG_v1 {"time_micros": 1699233935571021, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-11-06T01:25:35.897 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-11-06T01:25:35.897 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: EVENT_LOG_v1 {"time_micros": 1699233935572205, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93956, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 321, "table_properties": {"data_size": 91951, "index_size": 287, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13545, "raw_average_key_size": 49, "raw_value_size": 84525, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 272, "num_filter_entries": 272, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1699233935, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "660846a2-c746-4175-9f54-e2db7a2ca234", "db_session_id": "VIPMXQBGDCW2MN2U3YAR", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-11-06T01:25:35.897 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: EVENT_LOG_v1 {"time_micros": 1699233935572309, "job": 1, "event": "recovery_finished"} 2023-11-06T01:25:35.897 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-11-06T01:25:35.897 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-06T01:25:35.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bc23968000 2023-11-06T01:25:35.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: DB pointer 0x55bc23954000 2023-11-06T01:25:35.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-06T01:25:35.898 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-06T01:25:35.898 INFO:journalctl@ceph.mon.a.smithi053.stdout: ** DB Stats ** 2023-11-06T01:25:35.899 INFO:journalctl@ceph.mon.a.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-06T01:25:35.899 INFO:journalctl@ceph.mon.a.smithi053.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-06T01:25:35.899 INFO:journalctl@ceph.mon.a.smithi053.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-06T01:25:35.899 INFO:journalctl@ceph.mon.a.smithi053.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-06T01:25:35.899 INFO:journalctl@ceph.mon.a.smithi053.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-06T01:25:35.900 INFO:journalctl@ceph.mon.a.smithi053.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-06T01:25:35.900 INFO:journalctl@ceph.mon.a.smithi053.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-06T01:25:35.900 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2023-11-06T01:25:35.900 INFO:journalctl@ceph.mon.a.smithi053.stdout: ** Compaction Stats [default] ** 2023-11-06T01:25:35.900 INFO:journalctl@ceph.mon.a.smithi053.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-06T01:25:35.901 INFO:journalctl@ceph.mon.a.smithi053.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-06T01:25:35.901 INFO:journalctl@ceph.mon.a.smithi053.stdout: L0 2/0 93.59 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-06T01:25:35.901 INFO:journalctl@ceph.mon.a.smithi053.stdout: Sum 2/0 93.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-06T01:25:35.901 INFO:journalctl@ceph.mon.a.smithi053.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-06T01:25:35.901 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2023-11-06T01:25:35.902 INFO:journalctl@ceph.mon.a.smithi053.stdout: ** Compaction Stats [default] ** 2023-11-06T01:25:35.902 INFO:journalctl@ceph.mon.a.smithi053.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-06T01:25:35.902 INFO:journalctl@ceph.mon.a.smithi053.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-06T01:25:35.902 INFO:journalctl@ceph.mon.a.smithi053.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-06T01:25:35.902 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2023-11-06T01:25:35.903 INFO:journalctl@ceph.mon.a.smithi053.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-06T01:25:35.903 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2023-11-06T01:25:35.903 INFO:journalctl@ceph.mon.a.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-06T01:25:35.903 INFO:journalctl@ceph.mon.a.smithi053.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-06T01:25:35.903 INFO:journalctl@ceph.mon.a.smithi053.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-06T01:25:35.904 INFO:journalctl@ceph.mon.a.smithi053.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-06T01:25:35.904 INFO:journalctl@ceph.mon.a.smithi053.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-06T01:25:35.904 INFO:journalctl@ceph.mon.a.smithi053.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-06T01:25:35.904 INFO:journalctl@ceph.mon.a.smithi053.stdout: Cumulative compaction: 0.00 GB write, 24.92 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-06T01:25:35.904 INFO:journalctl@ceph.mon.a.smithi053.stdout: Interval compaction: 0.00 GB write, 24.92 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-06T01:25:35.905 INFO:journalctl@ceph.mon.a.smithi053.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-06T01:25:35.905 INFO:journalctl@ceph.mon.a.smithi053.stdout: Block cache BinnedLRUCache@0x55bc238cb1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-11-06T01:25:35.905 INFO:journalctl@ceph.mon.a.smithi053.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-11-06T01:25:35.905 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2023-11-06T01:25:35.905 INFO:journalctl@ceph.mon.a.smithi053.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-06T01:25:35.906 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: starting mon.a rank 0 at public addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] at bind addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:25:35.906 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: mon.a@-1(???) e1 preinit fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:25:35.906 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: mon.a@-1(???).mds e1 new map 2023-11-06T01:25:35.906 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: mon.a@-1(???).mds e1 print_map 2023-11-06T01:25:35.906 INFO:journalctl@ceph.mon.a.smithi053.stdout: e1 2023-11-06T01:25:35.907 INFO:journalctl@ceph.mon.a.smithi053.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-06T01:25:35.907 INFO:journalctl@ceph.mon.a.smithi053.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-06T01:25:35.907 INFO:journalctl@ceph.mon.a.smithi053.stdout: legacy client fscid: -1 2023-11-06T01:25:35.907 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2023-11-06T01:25:35.907 INFO:journalctl@ceph.mon.a.smithi053.stdout: No filesystems configured 2023-11-06T01:25:35.908 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-06T01:25:35.908 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-06T01:25:35.908 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-06T01:25:35.908 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-06T01:25:35.908 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-11-06T01:25:35.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta expand map: {default=false} 2023-11-06T01:25:35.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta from 'false' to 'false' 2023-11-06T01:25:35.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta expanded map: {default=false} 2023-11-06T01:25:35.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta expand map: {default=info} 2023-11-06T01:25:35.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta from 'info' to 'info' 2023-11-06T01:25:35.909 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta expanded map: {default=info} 2023-11-06T01:25:35.910 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta expand map: {default=daemon} 2023-11-06T01:25:35.910 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-06T01:25:35.910 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta expanded map: {default=daemon} 2023-11-06T01:25:35.910 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta expand map: {default=debug} 2023-11-06T01:25:35.910 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta from 'debug' to 'debug' 2023-11-06T01:25:35.911 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: expand_channel_meta expanded map: {default=debug} 2023-11-06T01:25:35.911 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-06T01:25:35.911 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: monmap e1: 1 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-11-06T01:25:35.911 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: fsmap 2023-11-06T01:25:35.911 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: osdmap e1: 0 total, 0 up, 0 in 2023-11-06T01:25:35.912 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 ceph-mon[99472]: mgrmap e1: no daemons active 2023-11-06T01:25:35.912 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 bash[99432]: 9fac24cd4d2a9cc81f2cf67e5e7fd8e9305a4f27acfbcc88d727e76836ab4c48 2023-11-06T01:25:35.912 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:35 smithi053 systemd[1]: Started Ceph mon.a for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:25:37.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.300+0000 7f89fc376700 1 Processor -- start 2023-11-06T01:25:37.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.300+0000 7f89fc376700 1 -- start start 2023-11-06T01:25:37.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.300+0000 7f89fc376700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f4104c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:37.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.300+0000 7f89fc376700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f41051e0 con 0x7f89f41047f0 2023-11-06T01:25:37.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.301+0000 7f89fa112700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f4104c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:37.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.301+0000 7f89fa112700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f4104c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36632/0 (socket says 172.21.15.53:36632) 2023-11-06T01:25:37.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.301+0000 7f89fa112700 1 -- 172.21.15.53:0/525193047 learned_addr learned my addr 172.21.15.53:0/525193047 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:37.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.301+0000 7f89fa112700 1 -- 172.21.15.53:0/525193047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f4107780 con 0x7f89f41047f0 2023-11-06T01:25:37.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.301+0000 7f89fa112700 1 --2- 172.21.15.53:0/525193047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f4104c10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f89f00098a0 tx=0x7f89f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a53dca38627db9b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:37.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.301+0000 7f89f9110700 1 -- 172.21.15.53:0/525193047 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89f0004160 con 0x7f89f41047f0 2023-11-06T01:25:37.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.301+0000 7f89f9110700 1 -- 172.21.15.53:0/525193047 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f89f00042c0 con 0x7f89f41047f0 2023-11-06T01:25:37.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.301+0000 7f89f9110700 1 -- 172.21.15.53:0/525193047 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89f0011a00 con 0x7f89f41047f0 2023-11-06T01:25:37.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.302+0000 7f89fc376700 1 -- 172.21.15.53:0/525193047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 msgr2=0x7f89f4104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:37.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.302+0000 7f89fc376700 1 --2- 172.21.15.53:0/525193047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f4104c10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f89f00098a0 tx=0x7f89f0009bb0 comp rx=0 tx=0).stop 2023-11-06T01:25:37.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.302+0000 7f89fc376700 1 -- 172.21.15.53:0/525193047 shutdown_connections 2023-11-06T01:25:37.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.302+0000 7f89fc376700 1 --2- 172.21.15.53:0/525193047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f4104c10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:37.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.302+0000 7f89fc376700 1 -- 172.21.15.53:0/525193047 >> 172.21.15.53:0/525193047 conn(0x7f89f4100140 msgr2=0x7f89f4102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:37.748 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.302+0000 7f89fc376700 1 -- 172.21.15.53:0/525193047 shutdown_connections 2023-11-06T01:25:37.748 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.302+0000 7f89fc376700 1 -- 172.21.15.53:0/525193047 wait complete. 2023-11-06T01:25:37.748 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.303+0000 7f89fc376700 1 Processor -- start 2023-11-06T01:25:37.748 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.303+0000 7f89fc376700 1 -- start start 2023-11-06T01:25:37.749 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.303+0000 7f89fc376700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:37.749 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.303+0000 7f89fc376700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f419a6d0 con 0x7f89f41047f0 2023-11-06T01:25:37.749 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.303+0000 7f89fa112700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:37.749 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.303+0000 7f89fa112700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36634/0 (socket says 172.21.15.53:36634) 2023-11-06T01:25:37.749 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.303+0000 7f89fa112700 1 -- 172.21.15.53:0/3731083000 learned_addr learned my addr 172.21.15.53:0/3731083000 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:37.750 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.304+0000 7f89fa112700 1 -- 172.21.15.53:0/3731083000 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0009550 con 0x7f89f41047f0 2023-11-06T01:25:37.750 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.304+0000 7f89fa112700 1 --2- 172.21.15.53:0/3731083000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f419a190 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009fb0 tx=0x7f89f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:37.750 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.304+0000 7f89eb7fe700 1 -- 172.21.15.53:0/3731083000 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89f0003930 con 0x7f89f41047f0 2023-11-06T01:25:37.750 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.304+0000 7f89eb7fe700 1 -- 172.21.15.53:0/3731083000 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f89f0003a90 con 0x7f89f41047f0 2023-11-06T01:25:37.751 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.304+0000 7f89fc376700 1 -- 172.21.15.53:0/3731083000 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f419a8d0 con 0x7f89f41047f0 2023-11-06T01:25:37.751 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.304+0000 7f89eb7fe700 1 -- 172.21.15.53:0/3731083000 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89f00195e0 con 0x7f89f41047f0 2023-11-06T01:25:37.751 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.304+0000 7f89fc376700 1 -- 172.21.15.53:0/3731083000 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f419ad70 con 0x7f89f41047f0 2023-11-06T01:25:37.751 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.304+0000 7f89eb7fe700 1 -- 172.21.15.53:0/3731083000 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f89f0019ba0 con 0x7f89f41047f0 2023-11-06T01:25:37.751 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.304+0000 7f89eb7fe700 1 -- 172.21.15.53:0/3731083000 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f89f00243f0 con 0x7f89f41047f0 2023-11-06T01:25:37.751 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.305+0000 7f89fc376700 1 -- 172.21.15.53:0/3731083000 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f40623c0 con 0x7f89f41047f0 2023-11-06T01:25:37.752 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.308+0000 7f89eb7fe700 1 -- 172.21.15.53:0/3731083000 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f89f0003c00 con 0x7f89f41047f0 2023-11-06T01:25:37.752 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.434+0000 7f89fc376700 1 -- 172.21.15.53:0/3731083000 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f89f410aa30 con 0x7f89f41047f0 2023-11-06T01:25:37.752 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.436+0000 7f89eb7fe700 1 -- 172.21.15.53:0/3731083000 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f89f001a9d0 con 0x7f89f41047f0 2023-11-06T01:25:37.752 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.437+0000 7f89fc376700 1 -- 172.21.15.53:0/3731083000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 msgr2=0x7f89f419a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:37.752 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.437+0000 7f89fc376700 1 --2- 172.21.15.53:0/3731083000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f419a190 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009fb0 tx=0x7f89f0011ad0 comp rx=0 tx=0).stop 2023-11-06T01:25:37.753 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.437+0000 7f89fc376700 1 -- 172.21.15.53:0/3731083000 shutdown_connections 2023-11-06T01:25:37.753 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.437+0000 7f89fc376700 1 --2- 172.21.15.53:0/3731083000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f41047f0 0x7f89f419a190 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:37.753 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.437+0000 7f89fc376700 1 -- 172.21.15.53:0/3731083000 >> 172.21.15.53:0/3731083000 conn(0x7f89f4100140 msgr2=0x7f89f4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:37.753 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.437+0000 7f89fc376700 1 -- 172.21.15.53:0/3731083000 shutdown_connections 2023-11-06T01:25:37.753 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:37.437+0000 7f89fc376700 1 -- 172.21.15.53:0/3731083000 wait complete. 2023-11-06T01:25:37.754 INFO:teuthology.orchestra.run.smithi053.stdout:Wrote config to /etc/ceph/ceph.conf 2023-11-06T01:25:37.765 INFO:teuthology.orchestra.run.smithi053.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-11-06T01:25:37.765 INFO:teuthology.orchestra.run.smithi053.stdout:Creating mgr... 2023-11-06T01:25:37.766 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying port 0.0.0.0:9283 ... 2023-11-06T01:25:37.766 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying port 0.0.0.0:8765 ... 2023-11-06T01:25:38.002 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.x 2023-11-06T01:25:38.002 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.x.service: Unit ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.x.service not loaded. 2023-11-06T01:25:38.095 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7.target.wants/ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@.service. 2023-11-06T01:25:38.419 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:38 smithi053 systemd[1]: Starting Ceph mgr.x for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:25:38.691 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:38 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3731083000' entity='client.admin' 2023-11-06T01:25:38.692 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:38 smithi053 podman[99711]: 2023-11-06T01:25:38.692 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:38 smithi053 bash[99711]: 48add67455ec8a7b2299083b0e9ec041986d76afa7fa79727f0450d034b79007 2023-11-06T01:25:38.705 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-06T01:25:38.705 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2023-11-06T01:25:38.712 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-06T01:25:38.712 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2023-11-06T01:25:38.713 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2023-11-06T01:25:38.713 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to enable service . firewalld.service is not available 2023-11-06T01:25:38.721 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-06T01:25:38.721 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2023-11-06T01:25:38.729 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-06T01:25:38.729 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2023-11-06T01:25:38.729 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2023-11-06T01:25:38.729 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-11-06T01:25:38.729 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr to start... 2023-11-06T01:25:38.730 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr... 2023-11-06T01:25:38.966 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:38 smithi053 systemd[1]: Started Ceph mgr.x for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:25:38.967 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:38 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:38.785+0000 7fc7dc112200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-06T01:25:38.967 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:38 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:38.881+0000 7fc7dc112200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T01:25:39.323 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:38 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:38.996+0000 7fc7dc112200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T01:25:39.323 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:39 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:39.108+0000 7fc7dc112200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T01:25:39.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:39 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1056808849' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-06T01:25:39.594 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:39 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:39.321+0000 7fc7dc112200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T01:25:39.778 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:39.778 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2023-11-06T01:25:39.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:25:39.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "health": { 2023-11-06T01:25:39.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-06T01:25:39.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-06T01:25:39.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-06T01:25:39.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:39.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-06T01:25:39.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-06T01:25:39.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 0 2023-11-06T01:25:39.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:39.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-06T01:25:39.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "a" 2023-11-06T01:25:39.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:39.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-11-06T01:25:39.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-06T01:25:39.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:39.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-06T01:25:39.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-06T01:25:39.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:39.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-06T01:25:39.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:39.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-06T01:25:39.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-06T01:25:39.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-06T01:25:39.791 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-06T01:25:39.791 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-06T01:25:39.791 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-06T01:25:39.791 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:39.792 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-06T01:25:39.792 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-06T01:25:39.792 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-06T01:25:39.792 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-06T01:25:39.792 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-06T01:25:39.793 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-06T01:25:39.793 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-06T01:25:39.793 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-06T01:25:39.793 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-06T01:25:39.793 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:39.794 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-06T01:25:39.794 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:39.794 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-06T01:25:39.794 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-06T01:25:39.794 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:39.795 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-06T01:25:39.795 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-06T01:25:39.795 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-06T01:25:39.796 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-06T01:25:39.796 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-06T01:25:39.796 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-06T01:25:39.796 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "restful" 2023-11-06T01:25:39.796 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:39.796 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-06T01:25:39.797 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:39.797 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-06T01:25:39.797 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:39.797 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modified": "2023-11-06T01:25:25.500097+0000", 2023-11-06T01:25:39.797 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-06T01:25:39.798 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:39.798 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-06T01:25:39.798 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2023-11-06T01:25:39.798 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.226+0000 7fb779184700 1 Processor -- start 2023-11-06T01:25:39.798 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.226+0000 7fb779184700 1 -- start start 2023-11-06T01:25:39.799 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.227+0000 7fb779184700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb7741093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:39.799 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.227+0000 7fb779184700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb774079e30 con 0x7fb774106fc0 2023-11-06T01:25:39.799 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.227+0000 7fb772d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb7741093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:39.799 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.227+0000 7fb772d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb7741093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36640/0 (socket says 172.21.15.53:36640) 2023-11-06T01:25:39.799 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.227+0000 7fb772d9d700 1 -- 172.21.15.53:0/3791508286 learned_addr learned my addr 172.21.15.53:0/3791508286 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:39.800 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.227+0000 7fb772d9d700 1 -- 172.21.15.53:0/3791508286 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7741098f0 con 0x7fb774106fc0 2023-11-06T01:25:39.800 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.227+0000 7fb772d9d700 1 --2- 172.21.15.53:0/3791508286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb7741093b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7640098a0 tx=0x7fb764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de03898afc6b5766 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:39.800 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.228+0000 7fb771d9b700 1 -- 172.21.15.53:0/3791508286 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb764004160 con 0x7fb774106fc0 2023-11-06T01:25:39.800 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.228+0000 7fb771d9b700 1 -- 172.21.15.53:0/3791508286 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb7640042c0 con 0x7fb774106fc0 2023-11-06T01:25:39.801 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.228+0000 7fb771d9b700 1 -- 172.21.15.53:0/3791508286 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb764011a00 con 0x7fb774106fc0 2023-11-06T01:25:39.801 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.228+0000 7fb779184700 1 -- 172.21.15.53:0/3791508286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 msgr2=0x7fb7741093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:39.801 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.228+0000 7fb779184700 1 --2- 172.21.15.53:0/3791508286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb7741093b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7640098a0 tx=0x7fb764009bb0 comp rx=0 tx=0).stop 2023-11-06T01:25:39.801 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.228+0000 7fb779184700 1 -- 172.21.15.53:0/3791508286 shutdown_connections 2023-11-06T01:25:39.802 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.228+0000 7fb779184700 1 --2- 172.21.15.53:0/3791508286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb7741093b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:39.802 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.228+0000 7fb779184700 1 -- 172.21.15.53:0/3791508286 >> 172.21.15.53:0/3791508286 conn(0x7fb7741009b0 msgr2=0x7fb774102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:39.802 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.229+0000 7fb779184700 1 -- 172.21.15.53:0/3791508286 shutdown_connections 2023-11-06T01:25:39.802 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.229+0000 7fb779184700 1 -- 172.21.15.53:0/3791508286 wait complete. 2023-11-06T01:25:39.802 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.229+0000 7fb779184700 1 Processor -- start 2023-11-06T01:25:39.803 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.230+0000 7fb779184700 1 -- start start 2023-11-06T01:25:39.803 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.230+0000 7fb779184700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb774106920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:39.803 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.230+0000 7fb779184700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb774106e60 con 0x7fb774106fc0 2023-11-06T01:25:39.803 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.230+0000 7fb772d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb774106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:39.803 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.230+0000 7fb772d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb774106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36642/0 (socket says 172.21.15.53:36642) 2023-11-06T01:25:39.803 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.230+0000 7fb772d9d700 1 -- 172.21.15.53:0/1056808849 learned_addr learned my addr 172.21.15.53:0/1056808849 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.230+0000 7fb772d9d700 1 -- 172.21.15.53:0/1056808849 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb764009550 con 0x7fb774106fc0 2023-11-06T01:25:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.231+0000 7fb772d9d700 1 --2- 172.21.15.53:0/1056808849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb774106920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb764009fb0 tx=0x7fb764011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.231+0000 7fb75bfff700 1 -- 172.21.15.53:0/1056808849 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb764003930 con 0x7fb774106fc0 2023-11-06T01:25:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.231+0000 7fb75bfff700 1 -- 172.21.15.53:0/1056808849 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb764003a90 con 0x7fb774106fc0 2023-11-06T01:25:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.231+0000 7fb779184700 1 -- 172.21.15.53:0/1056808849 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb774104f70 con 0x7fb774106fc0 2023-11-06T01:25:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.231+0000 7fb75bfff700 1 -- 172.21.15.53:0/1056808849 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb764019610 con 0x7fb774106fc0 2023-11-06T01:25:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.231+0000 7fb779184700 1 -- 172.21.15.53:0/1056808849 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb774105410 con 0x7fb774106fc0 2023-11-06T01:25:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.232+0000 7fb75bfff700 1 -- 172.21.15.53:0/1056808849 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb764019910 con 0x7fb774106fc0 2023-11-06T01:25:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.232+0000 7fb75bfff700 1 -- 172.21.15.53:0/1056808849 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb76402dab0 con 0x7fb774106fc0 2023-11-06T01:25:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.232+0000 7fb779184700 1 -- 172.21.15.53:0/1056808849 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77410a690 con 0x7fb774106fc0 2023-11-06T01:25:39.806 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.234+0000 7fb75bfff700 1 -- 172.21.15.53:0/1056808849 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fb764003c00 con 0x7fb774106fc0 2023-11-06T01:25:39.806 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.369+0000 7fb779184700 1 -- 172.21.15.53:0/1056808849 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb77410a8b0 con 0x7fb774106fc0 2023-11-06T01:25:39.806 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.369+0000 7fb75bfff700 1 -- 172.21.15.53:0/1056808849 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fb764019d50 con 0x7fb774106fc0 2023-11-06T01:25:39.806 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.370+0000 7fb779184700 1 -- 172.21.15.53:0/1056808849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 msgr2=0x7fb774106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.370+0000 7fb779184700 1 --2- 172.21.15.53:0/1056808849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb774106920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb764009fb0 tx=0x7fb764011ad0 comp rx=0 tx=0).stop 2023-11-06T01:25:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.370+0000 7fb779184700 1 -- 172.21.15.53:0/1056808849 shutdown_connections 2023-11-06T01:25:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.370+0000 7fb779184700 1 --2- 172.21.15.53:0/1056808849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb774106fc0 0x7fb774106920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.370+0000 7fb779184700 1 -- 172.21.15.53:0/1056808849 >> 172.21.15.53:0/1056808849 conn(0x7fb7741009b0 msgr2=0x7fb774101700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.371+0000 7fb779184700 1 -- 172.21.15.53:0/1056808849 shutdown_connections 2023-11-06T01:25:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:39.371+0000 7fb779184700 1 -- 172.21.15.53:0/1056808849 wait complete. 2023-11-06T01:25:39.808 INFO:teuthology.orchestra.run.smithi053.stdout:mgr not available, waiting (1/15)... 2023-11-06T01:25:40.085 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:39 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:39.818+0000 7fc7dc112200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T01:25:40.086 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:39 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:39.976+0000 7fc7dc112200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T01:25:40.344 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:40 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:40.084+0000 7fc7dc112200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T01:25:40.344 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:40 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:40.198+0000 7fc7dc112200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T01:25:41.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:40 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:40.709+0000 7fc7dc112200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T01:25:41.844 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:41 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:41.506+0000 7fc7dc112200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T01:25:41.844 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:41 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:41.604+0000 7fc7dc112200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T01:25:42.833 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2023-11-06T01:25:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:25:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "health": { 2023-11-06T01:25:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-06T01:25:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-06T01:25:42.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-06T01:25:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:42.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-06T01:25:42.837 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:42 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:42.597+0000 7fc7dc112200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T01:25:42.837 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:42 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:42.709+0000 7fc7dc112200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-06T01:25:42.838 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:42 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1230254472' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-06T01:25:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-06T01:25:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 0 2023-11-06T01:25:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:42.847 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-06T01:25:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "a" 2023-11-06T01:25:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-11-06T01:25:42.848 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-06T01:25:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-06T01:25:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-06T01:25:42.849 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-06T01:25:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-06T01:25:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-06T01:25:42.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-06T01:25:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-06T01:25:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-06T01:25:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-06T01:25:42.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-06T01:25:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-06T01:25:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-06T01:25:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-06T01:25:42.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-06T01:25:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-06T01:25:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-06T01:25:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-06T01:25:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-06T01:25:42.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-06T01:25:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-06T01:25:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-06T01:25:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-06T01:25:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-06T01:25:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-06T01:25:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-06T01:25:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-06T01:25:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-06T01:25:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "restful" 2023-11-06T01:25:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:42.857 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-06T01:25:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-06T01:25:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modified": "2023-11-06T01:25:25.500097+0000", 2023-11-06T01:25:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-06T01:25:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:42.859 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-06T01:25:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2023-11-06T01:25:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.317+0000 7f2c06212700 1 Processor -- start 2023-11-06T01:25:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.317+0000 7f2c06212700 1 -- start start 2023-11-06T01:25:42.860 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.317+0000 7f2c06212700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c00105810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.317+0000 7f2c06212700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c00105de0 con 0x7f2c001053f0 2023-11-06T01:25:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.317+0000 7f2bff7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c00105810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.317+0000 7f2bff7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c00105810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36644/0 (socket says 172.21.15.53:36644) 2023-11-06T01:25:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.317+0000 7f2bff7fe700 1 -- 172.21.15.53:0/1394185541 learned_addr learned my addr 172.21.15.53:0/1394185541 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.318+0000 7f2bff7fe700 1 -- 172.21.15.53:0/1394185541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c001065f0 con 0x7f2c001053f0 2023-11-06T01:25:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.318+0000 7f2bff7fe700 1 --2- 172.21.15.53:0/1394185541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c00105810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00098a0 tx=0x7f2bf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b1b75d7e3694c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.318+0000 7f2bfe7fc700 1 -- 172.21.15.53:0/1394185541 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bf0004160 con 0x7f2c001053f0 2023-11-06T01:25:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.318+0000 7f2bfe7fc700 1 -- 172.21.15.53:0/1394185541 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2bf00042c0 con 0x7f2c001053f0 2023-11-06T01:25:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.318+0000 7f2bfe7fc700 1 -- 172.21.15.53:0/1394185541 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bf0011a00 con 0x7f2c001053f0 2023-11-06T01:25:42.863 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.319+0000 7f2c06212700 1 -- 172.21.15.53:0/1394185541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 msgr2=0x7f2c00105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.319+0000 7f2c06212700 1 --2- 172.21.15.53:0/1394185541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c00105810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00098a0 tx=0x7f2bf0009bb0 comp rx=0 tx=0).stop 2023-11-06T01:25:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.319+0000 7f2c06212700 1 -- 172.21.15.53:0/1394185541 shutdown_connections 2023-11-06T01:25:42.864 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.319+0000 7f2c06212700 1 --2- 172.21.15.53:0/1394185541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c00105810 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.319+0000 7f2c06212700 1 -- 172.21.15.53:0/1394185541 >> 172.21.15.53:0/1394185541 conn(0x7f2c001009b0 msgr2=0x7f2c00102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.319+0000 7f2c06212700 1 -- 172.21.15.53:0/1394185541 shutdown_connections 2023-11-06T01:25:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.319+0000 7f2c06212700 1 -- 172.21.15.53:0/1394185541 wait complete. 2023-11-06T01:25:42.865 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.320+0000 7f2c06212700 1 Processor -- start 2023-11-06T01:25:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.320+0000 7f2c06212700 1 -- start start 2023-11-06T01:25:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.320+0000 7f2c06212700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c0019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.320+0000 7f2c06212700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0019a6e0 con 0x7f2c001053f0 2023-11-06T01:25:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.320+0000 7f2bff7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c0019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.320+0000 7f2bff7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c0019a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36646/0 (socket says 172.21.15.53:36646) 2023-11-06T01:25:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.320+0000 7f2bff7fe700 1 -- 172.21.15.53:0/1230254472 learned_addr learned my addr 172.21.15.53:0/1230254472 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.321+0000 7f2bff7fe700 1 -- 172.21.15.53:0/1230254472 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf0009550 con 0x7f2c001053f0 2023-11-06T01:25:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.321+0000 7f2bff7fe700 1 --2- 172.21.15.53:0/1230254472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c0019a1a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009fb0 tx=0x7f2bf0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.321+0000 7f2bfcff9700 1 -- 172.21.15.53:0/1230254472 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bf0003930 con 0x7f2c001053f0 2023-11-06T01:25:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.321+0000 7f2c06212700 1 -- 172.21.15.53:0/1230254472 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0019a8e0 con 0x7f2c001053f0 2023-11-06T01:25:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.321+0000 7f2bfcff9700 1 -- 172.21.15.53:0/1230254472 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2bf0003a90 con 0x7f2c001053f0 2023-11-06T01:25:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.321+0000 7f2bfcff9700 1 -- 172.21.15.53:0/1230254472 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bf00195e0 con 0x7f2c001053f0 2023-11-06T01:25:42.868 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.321+0000 7f2c06212700 1 -- 172.21.15.53:0/1230254472 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0019ad80 con 0x7f2c001053f0 2023-11-06T01:25:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.321+0000 7f2bfcff9700 1 -- 172.21.15.53:0/1230254472 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2bf0019bd0 con 0x7f2c001053f0 2023-11-06T01:25:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.321+0000 7f2bfcff9700 1 -- 172.21.15.53:0/1230254472 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2bf00243f0 con 0x7f2c001053f0 2023-11-06T01:25:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.322+0000 7f2c06212700 1 -- 172.21.15.53:0/1230254472 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c000623c0 con 0x7f2c001053f0 2023-11-06T01:25:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.324+0000 7f2bfcff9700 1 -- 172.21.15.53:0/1230254472 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f2bf002de40 con 0x7f2c001053f0 2023-11-06T01:25:42.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.458+0000 7f2c06212700 1 -- 172.21.15.53:0/1230254472 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2c00109850 con 0x7f2c001053f0 2023-11-06T01:25:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.458+0000 7f2bfcff9700 1 -- 172.21.15.53:0/1230254472 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f2bf0003c00 con 0x7f2c001053f0 2023-11-06T01:25:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.459+0000 7f2c06212700 1 -- 172.21.15.53:0/1230254472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 msgr2=0x7f2c0019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.459+0000 7f2c06212700 1 --2- 172.21.15.53:0/1230254472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c0019a1a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009fb0 tx=0x7f2bf0011ad0 comp rx=0 tx=0).stop 2023-11-06T01:25:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.459+0000 7f2c06212700 1 -- 172.21.15.53:0/1230254472 shutdown_connections 2023-11-06T01:25:42.870 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.459+0000 7f2c06212700 1 --2- 172.21.15.53:0/1230254472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c001053f0 0x7f2c0019a1a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.459+0000 7f2c06212700 1 -- 172.21.15.53:0/1230254472 >> 172.21.15.53:0/1230254472 conn(0x7f2c001009b0 msgr2=0x7f2c000798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.459+0000 7f2c06212700 1 -- 172.21.15.53:0/1230254472 shutdown_connections 2023-11-06T01:25:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:42.459+0000 7f2c06212700 1 -- 172.21.15.53:0/1230254472 wait complete. 2023-11-06T01:25:42.871 INFO:teuthology.orchestra.run.smithi053.stdout:mgr not available, waiting (2/15)... 2023-11-06T01:25:43.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:42 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:42.929+0000 7fc7dc112200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T01:25:43.531 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:43 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:43.278+0000 7fc7dc112200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-06T01:25:43.844 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:43 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:43.530+0000 7fc7dc112200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T01:25:43.844 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:43 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:43.712+0000 7fc7dc112200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T01:25:44.344 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:44 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:44.041+0000 7fc7dc112200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T01:25:45.301 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:44 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:44.882+0000 7fc7dc112200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T01:25:45.301 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:45 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:45.045+0000 7fc7dc112200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T01:25:45.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:45 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2549214278' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-06T01:25:45.594 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:45 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:45.300+0000 7fc7dc112200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T01:25:45.594 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:45 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:45.439+0000 7fc7dc112200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T01:25:45.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:45.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2023-11-06T01:25:45.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:25:45.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "health": { 2023-11-06T01:25:45.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-06T01:25:45.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-06T01:25:45.893 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-06T01:25:45.893 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:45.893 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-06T01:25:45.893 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-06T01:25:45.893 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 0 2023-11-06T01:25:45.894 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:45.894 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-06T01:25:45.894 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "a" 2023-11-06T01:25:45.894 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:45.894 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-11-06T01:25:45.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-06T01:25:45.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:45.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-06T01:25:45.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-06T01:25:45.895 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:45.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-06T01:25:45.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:45.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-06T01:25:45.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-06T01:25:45.896 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-06T01:25:45.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-06T01:25:45.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-06T01:25:45.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-06T01:25:45.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:45.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-06T01:25:45.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-06T01:25:45.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-06T01:25:45.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-06T01:25:45.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-06T01:25:45.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-06T01:25:45.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-06T01:25:45.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-06T01:25:45.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-06T01:25:45.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:45.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-06T01:25:45.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:45.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-06T01:25:45.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-06T01:25:45.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:45.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-06T01:25:45.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-06T01:25:45.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-06T01:25:45.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-06T01:25:45.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-06T01:25:45.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-06T01:25:45.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "restful" 2023-11-06T01:25:45.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:45.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-06T01:25:45.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:45.903 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-06T01:25:45.903 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:45.903 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modified": "2023-11-06T01:25:25.500097+0000", 2023-11-06T01:25:45.903 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-06T01:25:45.904 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:45.904 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-06T01:25:45.904 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2023-11-06T01:25:45.904 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.352+0000 7f3080a07700 1 Processor -- start 2023-11-06T01:25:45.905 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.352+0000 7f3080a07700 1 -- start start 2023-11-06T01:25:45.906 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.352+0000 7f3080a07700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:45.906 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.352+0000 7f3080a07700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c079e50 con 0x7f307c106fa0 2023-11-06T01:25:45.906 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.352+0000 7f307a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:45.907 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.353+0000 7f307a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36648/0 (socket says 172.21.15.53:36648) 2023-11-06T01:25:45.907 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.353+0000 7f307a59c700 1 -- 172.21.15.53:0/2428444928 learned_addr learned my addr 172.21.15.53:0/2428444928 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:45.907 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.353+0000 7f307a59c700 1 -- 172.21.15.53:0/2428444928 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c1098d0 con 0x7f307c106fa0 2023-11-06T01:25:45.907 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.353+0000 7f307a59c700 1 --2- 172.21.15.53:0/2428444928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c109390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f306c0098a0 tx=0x7f306c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f9aa370a47a8bd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:45.907 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.353+0000 7f307959a700 1 -- 172.21.15.53:0/2428444928 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f306c004160 con 0x7f307c106fa0 2023-11-06T01:25:45.908 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.353+0000 7f307959a700 1 -- 172.21.15.53:0/2428444928 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f306c0042c0 con 0x7f307c106fa0 2023-11-06T01:25:45.908 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.353+0000 7f307959a700 1 -- 172.21.15.53:0/2428444928 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f306c011a00 con 0x7f307c106fa0 2023-11-06T01:25:45.908 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.354+0000 7f3080a07700 1 -- 172.21.15.53:0/2428444928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 msgr2=0x7f307c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:45.908 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.354+0000 7f3080a07700 1 --2- 172.21.15.53:0/2428444928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c109390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f306c0098a0 tx=0x7f306c009bb0 comp rx=0 tx=0).stop 2023-11-06T01:25:45.908 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.354+0000 7f3080a07700 1 -- 172.21.15.53:0/2428444928 shutdown_connections 2023-11-06T01:25:45.909 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.354+0000 7f3080a07700 1 --2- 172.21.15.53:0/2428444928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c109390 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:45.909 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.354+0000 7f3080a07700 1 -- 172.21.15.53:0/2428444928 >> 172.21.15.53:0/2428444928 conn(0x7f307c1009b0 msgr2=0x7f307c102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:45.909 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.354+0000 7f3080a07700 1 -- 172.21.15.53:0/2428444928 shutdown_connections 2023-11-06T01:25:45.909 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.354+0000 7f3080a07700 1 -- 172.21.15.53:0/2428444928 wait complete. 2023-11-06T01:25:45.910 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.355+0000 7f3080a07700 1 Processor -- start 2023-11-06T01:25:45.910 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.355+0000 7f3080a07700 1 -- start start 2023-11-06T01:25:45.910 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.355+0000 7f3080a07700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c106900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:45.910 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.355+0000 7f3080a07700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c106e40 con 0x7f307c106fa0 2023-11-06T01:25:45.910 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.355+0000 7f307a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c106900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:45.911 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.355+0000 7f307a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c106900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36650/0 (socket says 172.21.15.53:36650) 2023-11-06T01:25:45.911 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.355+0000 7f307a59c700 1 -- 172.21.15.53:0/2549214278 learned_addr learned my addr 172.21.15.53:0/2549214278 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:45.911 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.355+0000 7f307a59c700 1 -- 172.21.15.53:0/2549214278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c009550 con 0x7f307c106fa0 2023-11-06T01:25:45.911 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.356+0000 7f307a59c700 1 --2- 172.21.15.53:0/2549214278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c106900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f306c009fb0 tx=0x7f306c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:45.911 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.356+0000 7f306b7fe700 1 -- 172.21.15.53:0/2549214278 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f306c003930 con 0x7f307c106fa0 2023-11-06T01:25:45.912 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.356+0000 7f3080a07700 1 -- 172.21.15.53:0/2549214278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c104f50 con 0x7f307c106fa0 2023-11-06T01:25:45.912 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.356+0000 7f306b7fe700 1 -- 172.21.15.53:0/2549214278 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f306c003a90 con 0x7f307c106fa0 2023-11-06T01:25:45.912 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.356+0000 7f306b7fe700 1 -- 172.21.15.53:0/2549214278 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f306c019610 con 0x7f307c106fa0 2023-11-06T01:25:45.912 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.356+0000 7f3080a07700 1 -- 172.21.15.53:0/2549214278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c1053f0 con 0x7f307c106fa0 2023-11-06T01:25:45.912 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.357+0000 7f306b7fe700 1 -- 172.21.15.53:0/2549214278 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f306c0198f0 con 0x7f307c106fa0 2023-11-06T01:25:45.913 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.357+0000 7f3080a07700 1 -- 172.21.15.53:0/2549214278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c10a670 con 0x7f307c106fa0 2023-11-06T01:25:45.913 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.357+0000 7f306b7fe700 1 -- 172.21.15.53:0/2549214278 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f306c02da60 con 0x7f307c106fa0 2023-11-06T01:25:45.913 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.359+0000 7f306b7fe700 1 -- 172.21.15.53:0/2549214278 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f306c003c00 con 0x7f307c106fa0 2023-11-06T01:25:45.913 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.492+0000 7f3080a07700 1 -- 172.21.15.53:0/2549214278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f307c105a50 con 0x7f307c106fa0 2023-11-06T01:25:45.913 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.493+0000 7f306b7fe700 1 -- 172.21.15.53:0/2549214278 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f306c019d30 con 0x7f307c106fa0 2023-11-06T01:25:45.914 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.493+0000 7f3080a07700 1 -- 172.21.15.53:0/2549214278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 msgr2=0x7f307c106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:45.914 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.493+0000 7f3080a07700 1 --2- 172.21.15.53:0/2549214278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c106900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f306c009fb0 tx=0x7f306c011ad0 comp rx=0 tx=0).stop 2023-11-06T01:25:45.914 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.494+0000 7f3080a07700 1 -- 172.21.15.53:0/2549214278 shutdown_connections 2023-11-06T01:25:45.914 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.494+0000 7f3080a07700 1 --2- 172.21.15.53:0/2549214278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f307c106fa0 0x7f307c106900 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:45.914 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.494+0000 7f3080a07700 1 -- 172.21.15.53:0/2549214278 >> 172.21.15.53:0/2549214278 conn(0x7f307c1009b0 msgr2=0x7f307c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:45.915 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.494+0000 7f3080a07700 1 -- 172.21.15.53:0/2549214278 shutdown_connections 2023-11-06T01:25:45.915 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:45.494+0000 7f3080a07700 1 -- 172.21.15.53:0/2549214278 wait complete. 2023-11-06T01:25:45.915 INFO:teuthology.orchestra.run.smithi053.stdout:mgr not available, waiting (3/15)... 2023-11-06T01:25:46.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:45 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:45.724+0000 7fc7dc112200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T01:25:47.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:46.770+0000 7fc7dc112200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T01:25:47.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: Activating manager daemon x 2023-11-06T01:25:47.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: mgrmap e2: x(active, starting, since 0.00305023s) 2023-11-06T01:25:47.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: from='mgr.14100 172.21.15.53:0/3571217528' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-06T01:25:47.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: from='mgr.14100 172.21.15.53:0/3571217528' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-06T01:25:47.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: from='mgr.14100 172.21.15.53:0/3571217528' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-06T01:25:47.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: from='mgr.14100 172.21.15.53:0/3571217528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:25:47.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: from='mgr.14100 172.21.15.53:0/3571217528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-06T01:25:47.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: Manager daemon x is now available 2023-11-06T01:25:47.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: from='mgr.14100 172.21.15.53:0/3571217528' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-06T01:25:47.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: from='mgr.14100 172.21.15.53:0/3571217528' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-06T01:25:47.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: from='mgr.14100 172.21.15.53:0/3571217528' entity='mgr.x' 2023-11-06T01:25:47.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: from='mgr.14100 172.21.15.53:0/3571217528' entity='mgr.x' 2023-11-06T01:25:47.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:46 smithi053 ceph-mon[99472]: from='mgr.14100 172.21.15.53:0/3571217528' entity='mgr.x' 2023-11-06T01:25:49.058 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:48 smithi053 ceph-mon[99472]: mgrmap e3: x(active, since 1.00765s) 2023-11-06T01:25:49.357 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:49.358 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2023-11-06T01:25:49.358 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:25:49.359 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "health": { 2023-11-06T01:25:49.359 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-06T01:25:49.359 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-06T01:25:49.360 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-06T01:25:49.360 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:49.360 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-06T01:25:49.360 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-06T01:25:49.360 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 0 2023-11-06T01:25:49.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:49.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-06T01:25:49.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "a" 2023-11-06T01:25:49.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:49.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2023-11-06T01:25:49.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-06T01:25:49.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:49.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-06T01:25:49.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-06T01:25:49.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:49.363 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-06T01:25:49.363 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:49.363 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-06T01:25:49.363 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-06T01:25:49.363 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-06T01:25:49.363 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-06T01:25:49.364 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-06T01:25:49.364 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-06T01:25:49.364 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:49.364 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-06T01:25:49.364 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-06T01:25:49.365 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-06T01:25:49.365 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-06T01:25:49.365 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-06T01:25:49.365 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-06T01:25:49.365 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-06T01:25:49.366 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-06T01:25:49.366 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-06T01:25:49.366 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:49.366 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-06T01:25:49.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:49.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-06T01:25:49.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-06T01:25:49.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:49.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-06T01:25:49.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-06T01:25:49.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-06T01:25:49.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-06T01:25:49.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-06T01:25:49.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-06T01:25:49.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "restful" 2023-11-06T01:25:49.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2023-11-06T01:25:49.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-06T01:25:49.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:49.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-06T01:25:49.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-06T01:25:49.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modified": "2023-11-06T01:25:25.500097+0000", 2023-11-06T01:25:49.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-06T01:25:49.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2023-11-06T01:25:49.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-06T01:25:49.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2023-11-06T01:25:49.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.395+0000 7f7274d76700 1 Processor -- start 2023-11-06T01:25:49.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.395+0000 7f7274d76700 1 -- start start 2023-11-06T01:25:49.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.395+0000 7f7274d76700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f7270105810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:49.374 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.395+0000 7f7274d76700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7270105de0 con 0x7f72701053f0 2023-11-06T01:25:49.374 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.396+0000 7f726e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f7270105810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:49.375 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.396+0000 7f726e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f7270105810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36676/0 (socket says 172.21.15.53:36676) 2023-11-06T01:25:49.375 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.396+0000 7f726e59c700 1 -- 172.21.15.53:0/449929844 learned_addr learned my addr 172.21.15.53:0/449929844 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:49.375 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.396+0000 7f726e59c700 1 -- 172.21.15.53:0/449929844 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72701065f0 con 0x7f72701053f0 2023-11-06T01:25:49.376 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.396+0000 7f726e59c700 1 --2- 172.21.15.53:0/449929844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f7270105810 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f72580098a0 tx=0x7f7258009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f0dc06443ab9bd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:49.376 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.396+0000 7f726d59a700 1 -- 172.21.15.53:0/449929844 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7258004160 con 0x7f72701053f0 2023-11-06T01:25:49.377 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.396+0000 7f726d59a700 1 -- 172.21.15.53:0/449929844 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72580042c0 con 0x7f72701053f0 2023-11-06T01:25:49.377 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.396+0000 7f726d59a700 1 -- 172.21.15.53:0/449929844 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7258011a00 con 0x7f72701053f0 2023-11-06T01:25:49.377 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.397+0000 7f7274d76700 1 -- 172.21.15.53:0/449929844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 msgr2=0x7f7270105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:49.378 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.397+0000 7f7274d76700 1 --2- 172.21.15.53:0/449929844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f7270105810 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f72580098a0 tx=0x7f7258009bb0 comp rx=0 tx=0).stop 2023-11-06T01:25:49.378 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.397+0000 7f7274d76700 1 -- 172.21.15.53:0/449929844 shutdown_connections 2023-11-06T01:25:49.378 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.397+0000 7f7274d76700 1 --2- 172.21.15.53:0/449929844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f7270105810 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:49.378 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.397+0000 7f7274d76700 1 -- 172.21.15.53:0/449929844 >> 172.21.15.53:0/449929844 conn(0x7f72701009f0 msgr2=0x7f7270102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:49.378 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.397+0000 7f7274d76700 1 -- 172.21.15.53:0/449929844 shutdown_connections 2023-11-06T01:25:49.379 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.397+0000 7f7274d76700 1 -- 172.21.15.53:0/449929844 wait complete. 2023-11-06T01:25:49.379 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.398+0000 7f7274d76700 1 Processor -- start 2023-11-06T01:25:49.379 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.398+0000 7f7274d76700 1 -- start start 2023-11-06T01:25:49.379 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.398+0000 7f7274d76700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f727019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:49.380 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.398+0000 7f7274d76700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727019a6a0 con 0x7f72701053f0 2023-11-06T01:25:49.380 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.398+0000 7f726e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f727019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:49.380 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.398+0000 7f726e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f727019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36678/0 (socket says 172.21.15.53:36678) 2023-11-06T01:25:49.380 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.398+0000 7f726e59c700 1 -- 172.21.15.53:0/2544183613 learned_addr learned my addr 172.21.15.53:0/2544183613 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:49.380 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.398+0000 7f726e59c700 1 -- 172.21.15.53:0/2544183613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7258009550 con 0x7f72701053f0 2023-11-06T01:25:49.381 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.399+0000 7f726e59c700 1 --2- 172.21.15.53:0/2544183613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f727019a160 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7258009fb0 tx=0x7f7258011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:49.381 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.399+0000 7f72677fe700 1 -- 172.21.15.53:0/2544183613 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7258003930 con 0x7f72701053f0 2023-11-06T01:25:49.381 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.399+0000 7f72677fe700 1 -- 172.21.15.53:0/2544183613 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7258003a90 con 0x7f72701053f0 2023-11-06T01:25:49.381 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.399+0000 7f7274d76700 1 -- 172.21.15.53:0/2544183613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727019a8a0 con 0x7f72701053f0 2023-11-06T01:25:49.381 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.399+0000 7f72677fe700 1 -- 172.21.15.53:0/2544183613 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72580195e0 con 0x7f72701053f0 2023-11-06T01:25:49.382 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.399+0000 7f7274d76700 1 -- 172.21.15.53:0/2544183613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727019ad40 con 0x7f72701053f0 2023-11-06T01:25:49.382 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.400+0000 7f72677fe700 1 -- 172.21.15.53:0/2544183613 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 3) v1 ==== 45151+0+0 (secure 0 0 0) 0x7f725801a430 con 0x7f72701053f0 2023-11-06T01:25:49.382 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.400+0000 7f72677fe700 1 --2- 172.21.15.53:0/2544183613 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f725c038190 0x7f725c03a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:49.382 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.400+0000 7f72677fe700 1 -- 172.21.15.53:0/2544183613 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f725804bcc0 con 0x7f72701053f0 2023-11-06T01:25:49.382 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.400+0000 7f7274d76700 1 -- 172.21.15.53:0/2544183613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72700623c0 con 0x7f72701053f0 2023-11-06T01:25:49.383 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.400+0000 7f726dd9b700 1 --2- 172.21.15.53:0/2544183613 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f725c038190 0x7f725c03a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:49.383 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.401+0000 7f726dd9b700 1 --2- 172.21.15.53:0/2544183613 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f725c038190 0x7f725c03a650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7260006fd0 tx=0x7f7260006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:49.383 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.405+0000 7f72677fe700 1 -- 172.21.15.53:0/2544183613 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f72580237b0 con 0x7f72701053f0 2023-11-06T01:25:49.383 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.927+0000 7f7274d76700 1 -- 172.21.15.53:0/2544183613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f727019dd30 con 0x7f72701053f0 2023-11-06T01:25:49.383 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.928+0000 7f72677fe700 1 -- 172.21.15.53:0/2544183613 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f725802ce40 con 0x7f72701053f0 2023-11-06T01:25:49.384 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.930+0000 7f7274d76700 1 -- 172.21.15.53:0/2544183613 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f725c038190 msgr2=0x7f725c03a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:49.384 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.930+0000 7f7274d76700 1 --2- 172.21.15.53:0/2544183613 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f725c038190 0x7f725c03a650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7260006fd0 tx=0x7f7260006e40 comp rx=0 tx=0).stop 2023-11-06T01:25:49.384 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.930+0000 7f7274d76700 1 -- 172.21.15.53:0/2544183613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 msgr2=0x7f727019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:49.384 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.930+0000 7f7274d76700 1 --2- 172.21.15.53:0/2544183613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f727019a160 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7258009fb0 tx=0x7f7258011ad0 comp rx=0 tx=0).stop 2023-11-06T01:25:49.384 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.930+0000 7f7274d76700 1 -- 172.21.15.53:0/2544183613 shutdown_connections 2023-11-06T01:25:49.385 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.930+0000 7f7274d76700 1 --2- 172.21.15.53:0/2544183613 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f725c038190 0x7f725c03a650 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:49.385 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.930+0000 7f7274d76700 1 --2- 172.21.15.53:0/2544183613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72701053f0 0x7f727019a160 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:49.385 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.930+0000 7f7274d76700 1 -- 172.21.15.53:0/2544183613 >> 172.21.15.53:0/2544183613 conn(0x7f72701009f0 msgr2=0x7f72700798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:49.385 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.930+0000 7f7274d76700 1 -- 172.21.15.53:0/2544183613 shutdown_connections 2023-11-06T01:25:49.385 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:48.930+0000 7f7274d76700 1 -- 172.21.15.53:0/2544183613 wait complete. 2023-11-06T01:25:49.386 INFO:teuthology.orchestra.run.smithi053.stdout:mgr is available 2023-11-06T01:25:49.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:49 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2544183613' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-06T01:25:50.590 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:50.590 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [global] 2023-11-06T01:25:50.590 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout fsid = 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:25:50.590 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-06T01:25:50.591 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-06T01:25:50.591 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-06T01:25:50.591 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-06T01:25:50.591 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:50.592 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-06T01:25:50.592 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-06T01:25:50.592 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2023-11-06T01:25:50.592 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [osd] 2023-11-06T01:25:50.592 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-06T01:25:50.593 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-06T01:25:50.593 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-06T01:25:50.593 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-06T01:25:50.593 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-06T01:25:50.593 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.824+0000 7f33bb38a700 1 Processor -- start 2023-11-06T01:25:50.594 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.824+0000 7f33bb38a700 1 -- start start 2023-11-06T01:25:50.594 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.825+0000 7f33bb38a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b41093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:50.594 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.825+0000 7f33bb38a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4079e10 con 0x7f33b4106fe0 2023-11-06T01:25:50.594 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.825+0000 7f33b9126700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b41093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:50.594 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.825+0000 7f33b9126700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b41093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36682/0 (socket says 172.21.15.53:36682) 2023-11-06T01:25:50.595 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.825+0000 7f33b9126700 1 -- 172.21.15.53:0/3200210569 learned_addr learned my addr 172.21.15.53:0/3200210569 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:50.595 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.825+0000 7f33b9126700 1 -- 172.21.15.53:0/3200210569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4109910 con 0x7f33b4106fe0 2023-11-06T01:25:50.595 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.825+0000 7f33b9126700 1 --2- 172.21.15.53:0/3200210569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b41093d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f33b00098a0 tx=0x7f33b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b0bd6fb6076a47 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:50.595 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.825+0000 7f33a7fff700 1 -- 172.21.15.53:0/3200210569 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33b0004160 con 0x7f33b4106fe0 2023-11-06T01:25:50.595 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.825+0000 7f33a7fff700 1 -- 172.21.15.53:0/3200210569 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f33b00042c0 con 0x7f33b4106fe0 2023-11-06T01:25:50.596 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.825+0000 7f33a7fff700 1 -- 172.21.15.53:0/3200210569 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33b0011a00 con 0x7f33b4106fe0 2023-11-06T01:25:50.596 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.826+0000 7f33bb38a700 1 -- 172.21.15.53:0/3200210569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 msgr2=0x7f33b41093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:50.596 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.826+0000 7f33bb38a700 1 --2- 172.21.15.53:0/3200210569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b41093d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f33b00098a0 tx=0x7f33b0009bb0 comp rx=0 tx=0).stop 2023-11-06T01:25:50.596 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.826+0000 7f33bb38a700 1 -- 172.21.15.53:0/3200210569 shutdown_connections 2023-11-06T01:25:50.597 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.826+0000 7f33bb38a700 1 --2- 172.21.15.53:0/3200210569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b41093d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:50.597 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.826+0000 7f33bb38a700 1 -- 172.21.15.53:0/3200210569 >> 172.21.15.53:0/3200210569 conn(0x7f33b41009b0 msgr2=0x7f33b4102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:50.597 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.826+0000 7f33bb38a700 1 -- 172.21.15.53:0/3200210569 shutdown_connections 2023-11-06T01:25:50.597 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.826+0000 7f33bb38a700 1 -- 172.21.15.53:0/3200210569 wait complete. 2023-11-06T01:25:50.597 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.827+0000 7f33bb38a700 1 Processor -- start 2023-11-06T01:25:50.598 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.827+0000 7f33bb38a700 1 -- start start 2023-11-06T01:25:50.598 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.827+0000 7f33bb38a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b4106940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:50.598 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.827+0000 7f33bb38a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4106e80 con 0x7f33b4106fe0 2023-11-06T01:25:50.598 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.827+0000 7f33b9126700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b4106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:50.598 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.827+0000 7f33b9126700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b4106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36684/0 (socket says 172.21.15.53:36684) 2023-11-06T01:25:50.599 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.827+0000 7f33b9126700 1 -- 172.21.15.53:0/3336547739 learned_addr learned my addr 172.21.15.53:0/3336547739 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:50.599 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.827+0000 7f33b9126700 1 -- 172.21.15.53:0/3336547739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b0009550 con 0x7f33b4106fe0 2023-11-06T01:25:50.599 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.828+0000 7f33b9126700 1 --2- 172.21.15.53:0/3336547739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b4106940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009fb0 tx=0x7f33b0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:50.599 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.828+0000 7f33a67fc700 1 -- 172.21.15.53:0/3336547739 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33b0003930 con 0x7f33b4106fe0 2023-11-06T01:25:50.599 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.828+0000 7f33a67fc700 1 -- 172.21.15.53:0/3336547739 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f33b0003a90 con 0x7f33b4106fe0 2023-11-06T01:25:50.599 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.828+0000 7f33bb38a700 1 -- 172.21.15.53:0/3336547739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b4104f90 con 0x7f33b4106fe0 2023-11-06T01:25:50.600 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.828+0000 7f33a67fc700 1 -- 172.21.15.53:0/3336547739 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33b00195c0 con 0x7f33b4106fe0 2023-11-06T01:25:50.600 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.828+0000 7f33bb38a700 1 -- 172.21.15.53:0/3336547739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b4105430 con 0x7f33b4106fe0 2023-11-06T01:25:50.600 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.829+0000 7f33a67fc700 1 -- 172.21.15.53:0/3336547739 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 45257+0+0 (secure 0 0 0) 0x7f33b001a9d0 con 0x7f33b4106fe0 2023-11-06T01:25:50.600 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.829+0000 7f33bb38a700 1 -- 172.21.15.53:0/3336547739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b4062430 con 0x7f33b4106fe0 2023-11-06T01:25:50.600 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.829+0000 7f33a67fc700 1 --2- 172.21.15.53:0/3336547739 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f33a0038300 0x7f33a003a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:50.601 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.829+0000 7f33b8925700 1 --2- 172.21.15.53:0/3336547739 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f33a0038300 0x7f33a003a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:50.601 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.829+0000 7f33a67fc700 1 -- 172.21.15.53:0/3336547739 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f33b004bf10 con 0x7f33b4106fe0 2023-11-06T01:25:50.601 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.830+0000 7f33b8925700 1 --2- 172.21.15.53:0/3336547739 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f33a0038300 0x7f33a003a7c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f33a8006fd0 tx=0x7f33a8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:50.601 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:49.834+0000 7f33a67fc700 1 -- 172.21.15.53:0/3336547739 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f33b4062430 con 0x7f33b4106fe0 2023-11-06T01:25:50.601 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.197+0000 7f33bb38a700 1 -- 172.21.15.53:0/3336547739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f33b4105c30 con 0x7f33b4106fe0 2023-11-06T01:25:50.602 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.198+0000 7f33a67fc700 1 -- 172.21.15.53:0/3336547739 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f33b004e020 con 0x7f33b4106fe0 2023-11-06T01:25:50.602 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.201+0000 7f33bb38a700 1 -- 172.21.15.53:0/3336547739 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f33a0038300 msgr2=0x7f33a003a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:50.602 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.201+0000 7f33bb38a700 1 --2- 172.21.15.53:0/3336547739 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f33a0038300 0x7f33a003a7c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f33a8006fd0 tx=0x7f33a8006e40 comp rx=0 tx=0).stop 2023-11-06T01:25:50.602 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.201+0000 7f33bb38a700 1 -- 172.21.15.53:0/3336547739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 msgr2=0x7f33b4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:50.602 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.201+0000 7f33bb38a700 1 --2- 172.21.15.53:0/3336547739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b4106940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009fb0 tx=0x7f33b0011ad0 comp rx=0 tx=0).stop 2023-11-06T01:25:50.603 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.201+0000 7f33bb38a700 1 -- 172.21.15.53:0/3336547739 shutdown_connections 2023-11-06T01:25:50.603 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.201+0000 7f33bb38a700 1 --2- 172.21.15.53:0/3336547739 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f33a0038300 0x7f33a003a7c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:50.603 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.201+0000 7f33bb38a700 1 --2- 172.21.15.53:0/3336547739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b4106fe0 0x7f33b4106940 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:50.603 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.201+0000 7f33bb38a700 1 -- 172.21.15.53:0/3336547739 >> 172.21.15.53:0/3336547739 conn(0x7f33b41009b0 msgr2=0x7f33b4101720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:50.603 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.201+0000 7f33bb38a700 1 -- 172.21.15.53:0/3336547739 shutdown_connections 2023-11-06T01:25:50.604 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:50.201+0000 7f33bb38a700 1 -- 172.21.15.53:0/3336547739 wait complete. 2023-11-06T01:25:50.604 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling cephadm module... 2023-11-06T01:25:51.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:50 smithi053 ceph-mon[99472]: mgrmap e4: x(active, since 3s) 2023-11-06T01:25:51.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:50 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3336547739' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-06T01:25:52.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:51 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1546374102' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-11-06T01:25:52.082 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:51 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: ignoring --setuser ceph since I am not root 2023-11-06T01:25:52.082 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:51 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: ignoring --setgroup ceph since I am not root 2023-11-06T01:25:52.082 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:51 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:51.984+0000 7f04976f6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-06T01:25:52.082 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:52 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:52.080+0000 7f04976f6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T01:25:52.136 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.130+0000 7f142f14b700 1 Processor -- start 2023-11-06T01:25:52.136 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.130+0000 7f142f14b700 1 -- start start 2023-11-06T01:25:52.136 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.130+0000 7f142f14b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f1428105810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:52.137 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.130+0000 7f142f14b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1428105de0 con 0x7f14281053f0 2023-11-06T01:25:52.137 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.130+0000 7f142cee7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f1428105810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:52.137 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.130+0000 7f142cee7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f1428105810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36688/0 (socket says 172.21.15.53:36688) 2023-11-06T01:25:52.137 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.130+0000 7f142cee7700 1 -- 172.21.15.53:0/2973816722 learned_addr learned my addr 172.21.15.53:0/2973816722 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:52.138 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.131+0000 7f142cee7700 1 -- 172.21.15.53:0/2973816722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14281065f0 con 0x7f14281053f0 2023-11-06T01:25:52.138 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.131+0000 7f142cee7700 1 --2- 172.21.15.53:0/2973816722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f1428105810 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f141c00bb40 tx=0x7f141c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e01c0bb724a3b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:52.138 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.131+0000 7f14277fe700 1 -- 172.21.15.53:0/2973816722 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f141c00dbe0 con 0x7f14281053f0 2023-11-06T01:25:52.138 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.131+0000 7f14277fe700 1 -- 172.21.15.53:0/2973816722 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f141c0044d0 con 0x7f14281053f0 2023-11-06T01:25:52.138 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.131+0000 7f14277fe700 1 -- 172.21.15.53:0/2973816722 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f141c010470 con 0x7f14281053f0 2023-11-06T01:25:52.139 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.131+0000 7f142f14b700 1 -- 172.21.15.53:0/2973816722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 msgr2=0x7f1428105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:52.139 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.132+0000 7f142f14b700 1 --2- 172.21.15.53:0/2973816722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f1428105810 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f141c00bb40 tx=0x7f141c00be50 comp rx=0 tx=0).stop 2023-11-06T01:25:52.139 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.132+0000 7f142f14b700 1 -- 172.21.15.53:0/2973816722 shutdown_connections 2023-11-06T01:25:52.139 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.132+0000 7f142f14b700 1 --2- 172.21.15.53:0/2973816722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f1428105810 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:52.139 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.132+0000 7f142f14b700 1 -- 172.21.15.53:0/2973816722 >> 172.21.15.53:0/2973816722 conn(0x7f14281009b0 msgr2=0x7f1428102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:52.140 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.132+0000 7f142f14b700 1 -- 172.21.15.53:0/2973816722 shutdown_connections 2023-11-06T01:25:52.140 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.132+0000 7f142f14b700 1 -- 172.21.15.53:0/2973816722 wait complete. 2023-11-06T01:25:52.140 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.132+0000 7f142f14b700 1 Processor -- start 2023-11-06T01:25:52.140 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.132+0000 7f142f14b700 1 -- start start 2023-11-06T01:25:52.140 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.132+0000 7f142f14b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f142819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:52.141 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.132+0000 7f142f14b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142819c880 con 0x7f14281053f0 2023-11-06T01:25:52.141 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.133+0000 7f142cee7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f142819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:52.141 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.133+0000 7f142cee7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f142819c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36690/0 (socket says 172.21.15.53:36690) 2023-11-06T01:25:52.141 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.133+0000 7f142cee7700 1 -- 172.21.15.53:0/1546374102 learned_addr learned my addr 172.21.15.53:0/1546374102 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:52.141 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.133+0000 7f142cee7700 1 -- 172.21.15.53:0/1546374102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c00b7f0 con 0x7f14281053f0 2023-11-06T01:25:52.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.133+0000 7f142cee7700 1 --2- 172.21.15.53:0/1546374102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f142819c340 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b340 tx=0x7f141c003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:52.142 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.133+0000 7f1425ffb700 1 -- 172.21.15.53:0/1546374102 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f141c004310 con 0x7f14281053f0 2023-11-06T01:25:52.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.133+0000 7f1425ffb700 1 -- 172.21.15.53:0/1546374102 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f141c010bd0 con 0x7f14281053f0 2023-11-06T01:25:52.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.133+0000 7f142f14b700 1 -- 172.21.15.53:0/1546374102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142819ca80 con 0x7f14281053f0 2023-11-06T01:25:52.143 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.133+0000 7f1425ffb700 1 -- 172.21.15.53:0/1546374102 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f141c021c90 con 0x7f14281053f0 2023-11-06T01:25:52.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.133+0000 7f142f14b700 1 -- 172.21.15.53:0/1546374102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142819cec0 con 0x7f14281053f0 2023-11-06T01:25:52.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.134+0000 7f1425ffb700 1 -- 172.21.15.53:0/1546374102 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 45257+0+0 (secure 0 0 0) 0x7f141c010730 con 0x7f14281053f0 2023-11-06T01:25:52.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.135+0000 7f1425ffb700 1 --2- 172.21.15.53:0/1546374102 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f14180382b0 0x7f141803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:52.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.135+0000 7f1425ffb700 1 -- 172.21.15.53:0/1546374102 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f141c04c310 con 0x7f14281053f0 2023-11-06T01:25:52.144 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.135+0000 7f142f14b700 1 -- 172.21.15.53:0/1546374102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1428191c80 con 0x7f14281053f0 2023-11-06T01:25:52.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.135+0000 7f1427fff700 1 --2- 172.21.15.53:0/1546374102 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f14180382b0 0x7f141803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:52.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.135+0000 7f1427fff700 1 --2- 172.21.15.53:0/1546374102 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f14180382b0 0x7f141803a770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:52.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.140+0000 7f1425ffb700 1 -- 172.21.15.53:0/1546374102 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f141c021410 con 0x7f14281053f0 2023-11-06T01:25:52.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.616+0000 7f142f14b700 1 -- 172.21.15.53:0/1546374102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f142802d090 con 0x7f14281053f0 2023-11-06T01:25:52.145 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.793+0000 7f1425ffb700 1 -- 172.21.15.53:0/1546374102 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 5) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f141c0193f0 con 0x7f14281053f0 2023-11-06T01:25:52.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.795+0000 7f1425ffb700 1 -- 172.21.15.53:0/1546374102 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f141c029c10 con 0x7f14281053f0 2023-11-06T01:25:52.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.797+0000 7f142f14b700 1 -- 172.21.15.53:0/1546374102 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f14180382b0 msgr2=0x7f141803a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:52.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.797+0000 7f142f14b700 1 --2- 172.21.15.53:0/1546374102 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f14180382b0 0x7f141803a770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414006e40 comp rx=0 tx=0).stop 2023-11-06T01:25:52.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.797+0000 7f142f14b700 1 -- 172.21.15.53:0/1546374102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 msgr2=0x7f142819c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:52.146 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.797+0000 7f142f14b700 1 --2- 172.21.15.53:0/1546374102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f142819c340 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b340 tx=0x7f141c003df0 comp rx=0 tx=0).stop 2023-11-06T01:25:52.147 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.797+0000 7f142f14b700 1 -- 172.21.15.53:0/1546374102 shutdown_connections 2023-11-06T01:25:52.147 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.797+0000 7f142f14b700 1 --2- 172.21.15.53:0/1546374102 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f14180382b0 0x7f141803a770 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:52.147 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.797+0000 7f142f14b700 1 --2- 172.21.15.53:0/1546374102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14281053f0 0x7f142819c340 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b340 tx=0x7f141c003df0 comp rx=0 tx=0).stop 2023-11-06T01:25:52.147 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.797+0000 7f142f14b700 1 -- 172.21.15.53:0/1546374102 >> 172.21.15.53:0/1546374102 conn(0x7f14281009b0 msgr2=0x7f142818edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:52.147 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.797+0000 7f142f14b700 1 -- 172.21.15.53:0/1546374102 shutdown_connections 2023-11-06T01:25:52.148 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:51.797+0000 7f142f14b700 1 -- 172.21.15.53:0/1546374102 wait complete. 2023-11-06T01:25:52.344 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:52 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:52.196+0000 7f04976f6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T01:25:52.344 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:52 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:52.308+0000 7f04976f6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T01:25:52.793 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:52 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:52.525+0000 7f04976f6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T01:25:53.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:52 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1546374102' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-11-06T01:25:53.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:52 smithi053 ceph-mon[99472]: mgrmap e5: x(active, since 5s) 2023-11-06T01:25:53.095 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:53 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:53.004+0000 7f04976f6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T01:25:53.459 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:53 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:53.159+0000 7f04976f6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T01:25:53.459 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:53 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:53.275+0000 7f04976f6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T01:25:53.459 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:53 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:53.392+0000 7f04976f6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T01:25:53.487 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2023-11-06T01:25:53.487 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-11-06T01:25:53.488 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-06T01:25:53.488 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-06T01:25:53.488 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-06T01:25:53.488 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2023-11-06T01:25:53.489 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.630+0000 7f49756bd700 1 Processor -- start 2023-11-06T01:25:53.489 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.630+0000 7f49756bd700 1 -- start start 2023-11-06T01:25:53.489 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.630+0000 7f49756bd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f4970105810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:53.489 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.630+0000 7f49756bd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970105de0 con 0x7f49701053f0 2023-11-06T01:25:53.490 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.630+0000 7f496effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f4970105810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:53.490 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.630+0000 7f496effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f4970105810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36704/0 (socket says 172.21.15.53:36704) 2023-11-06T01:25:53.490 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.630+0000 7f496effd700 1 -- 172.21.15.53:0/294098613 learned_addr learned my addr 172.21.15.53:0/294098613 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:53.490 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.630+0000 7f496effd700 1 -- 172.21.15.53:0/294098613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49701065f0 con 0x7f49701053f0 2023-11-06T01:25:53.491 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.631+0000 7f496effd700 1 --2- 172.21.15.53:0/294098613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f4970105810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f496000bb40 tx=0x7f496000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71ac0523e28ea6c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:53.491 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.631+0000 7f496dffb700 1 -- 172.21.15.53:0/294098613 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f496000dbe0 con 0x7f49701053f0 2023-11-06T01:25:53.491 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.631+0000 7f496dffb700 1 -- 172.21.15.53:0/294098613 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49600044d0 con 0x7f49701053f0 2023-11-06T01:25:53.491 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.631+0000 7f496dffb700 1 -- 172.21.15.53:0/294098613 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4960010470 con 0x7f49701053f0 2023-11-06T01:25:53.491 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.631+0000 7f49756bd700 1 -- 172.21.15.53:0/294098613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 msgr2=0x7f4970105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:53.492 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.631+0000 7f49756bd700 1 --2- 172.21.15.53:0/294098613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f4970105810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f496000bb40 tx=0x7f496000be50 comp rx=0 tx=0).stop 2023-11-06T01:25:53.492 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.631+0000 7f49756bd700 1 -- 172.21.15.53:0/294098613 shutdown_connections 2023-11-06T01:25:53.492 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.631+0000 7f49756bd700 1 --2- 172.21.15.53:0/294098613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f4970105810 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:53.492 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.631+0000 7f49756bd700 1 -- 172.21.15.53:0/294098613 >> 172.21.15.53:0/294098613 conn(0x7f49701009b0 msgr2=0x7f4970102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:53.492 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.631+0000 7f49756bd700 1 -- 172.21.15.53:0/294098613 shutdown_connections 2023-11-06T01:25:53.493 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.631+0000 7f49756bd700 1 -- 172.21.15.53:0/294098613 wait complete. 2023-11-06T01:25:53.493 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.632+0000 7f49756bd700 1 Processor -- start 2023-11-06T01:25:53.493 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.632+0000 7f49756bd700 1 -- start start 2023-11-06T01:25:53.493 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.632+0000 7f49756bd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f497019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:53.493 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.632+0000 7f49756bd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497019c880 con 0x7f49701053f0 2023-11-06T01:25:53.493 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.633+0000 7f496effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f497019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:25:53.494 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.633+0000 7f496effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f497019c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36706/0 (socket says 172.21.15.53:36706) 2023-11-06T01:25:53.494 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.633+0000 7f496effd700 1 -- 172.21.15.53:0/3697465937 learned_addr learned my addr 172.21.15.53:0/3697465937 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:25:53.494 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.633+0000 7f496effd700 1 -- 172.21.15.53:0/3697465937 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496000b7f0 con 0x7f49701053f0 2023-11-06T01:25:53.494 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.633+0000 7f496effd700 1 --2- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f497019c340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4960003a80 tx=0x7f4960003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:25:53.494 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.633+0000 7f4957fff700 1 -- 172.21.15.53:0/3697465937 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49600106e0 con 0x7f49701053f0 2023-11-06T01:25:53.495 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.633+0000 7f49756bd700 1 -- 172.21.15.53:0/3697465937 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497019ca80 con 0x7f49701053f0 2023-11-06T01:25:53.495 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.633+0000 7f4957fff700 1 -- 172.21.15.53:0/3697465937 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4960010ce0 con 0x7f49701053f0 2023-11-06T01:25:53.496 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.633+0000 7f4957fff700 1 -- 172.21.15.53:0/3697465937 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4960019c40 con 0x7f49701053f0 2023-11-06T01:25:53.496 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.633+0000 7f49756bd700 1 -- 172.21.15.53:0/3697465937 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497019cec0 con 0x7f49701053f0 2023-11-06T01:25:53.496 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.634+0000 7f4957fff700 1 -- 172.21.15.53:0/3697465937 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f496002a450 con 0x7f49701053f0 2023-11-06T01:25:53.496 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.635+0000 7f4957fff700 1 --2- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f4958038300 0x7f495803a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:25:53.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.635+0000 7f4957fff700 1 -- 172.21.15.53:0/3697465937 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f496004c540 con 0x7f49701053f0 2023-11-06T01:25:53.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.635+0000 7f496e7fc700 1 -- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f4958038300 msgr2=0x7f495803a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3877634851 2023-11-06T01:25:53.497 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.635+0000 7f496e7fc700 1 --2- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f4958038300 0x7f495803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T01:25:53.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.635+0000 7f49756bd700 1 -- 172.21.15.53:0/3697465937 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4970191c80 con 0x7f49701053f0 2023-11-06T01:25:53.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.640+0000 7f4957fff700 1 -- 172.21.15.53:0/3697465937 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f496002a700 con 0x7f49701053f0 2023-11-06T01:25:53.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.836+0000 7f496e7fc700 1 -- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f4958038300 msgr2=0x7f495803a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3877634851 2023-11-06T01:25:53.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:52.836+0000 7f496e7fc700 1 --2- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f4958038300 0x7f495803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T01:25:53.498 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.115+0000 7f49756bd700 1 -- 172.21.15.53:0/3697465937 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f49700623c0 con 0x7f49701053f0 2023-11-06T01:25:53.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.116+0000 7f4957fff700 1 -- 172.21.15.53:0/3697465937 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f4960028350 con 0x7f49701053f0 2023-11-06T01:25:53.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.118+0000 7f49756bd700 1 -- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f4958038300 msgr2=0x7f495803a7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:25:53.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.118+0000 7f49756bd700 1 --2- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f4958038300 0x7f495803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:53.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.118+0000 7f49756bd700 1 -- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 msgr2=0x7f497019c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:25:53.499 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.118+0000 7f49756bd700 1 --2- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f497019c340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4960003a80 tx=0x7f4960003df0 comp rx=0 tx=0).stop 2023-11-06T01:25:53.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.118+0000 7f49756bd700 1 -- 172.21.15.53:0/3697465937 shutdown_connections 2023-11-06T01:25:53.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.118+0000 7f49756bd700 1 --2- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f4958038300 0x7f495803a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:53.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.118+0000 7f49756bd700 1 --2- 172.21.15.53:0/3697465937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49701053f0 0x7f497019c340 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:25:53.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.118+0000 7f49756bd700 1 -- 172.21.15.53:0/3697465937 >> 172.21.15.53:0/3697465937 conn(0x7f49701009b0 msgr2=0x7f497018edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:25:53.500 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.119+0000 7f49756bd700 1 -- 172.21.15.53:0/3697465937 shutdown_connections 2023-11-06T01:25:53.501 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.119+0000 7f49756bd700 1 -- 172.21.15.53:0/3697465937 wait complete. 2023-11-06T01:25:53.501 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for the mgr to restart... 2023-11-06T01:25:53.501 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr epoch 5... 2023-11-06T01:25:54.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:53 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3697465937' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-06T01:25:54.095 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:53 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:53.916+0000 7f04976f6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T01:25:55.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:54 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:54.698+0000 7f04976f6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T01:25:55.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:54 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:54.796+0000 7f04976f6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T01:25:56.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:55 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:55.766+0000 7f04976f6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T01:25:56.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:55 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:55.876+0000 7f04976f6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-06T01:25:56.446 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:56 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:56.096+0000 7f04976f6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T01:25:56.697 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:56 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:56.445+0000 7f04976f6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-06T01:25:57.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:56 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:56.696+0000 7f04976f6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T01:25:57.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:56 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:56.878+0000 7f04976f6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T01:25:57.594 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:57 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:57.207+0000 7f04976f6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T01:25:58.344 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:58 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:58.030+0000 7f04976f6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T01:25:58.344 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:58 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:58.189+0000 7f04976f6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T01:25:58.843 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:58 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:58.433+0000 7f04976f6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T01:25:58.843 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:58 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:58.570+0000 7f04976f6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T01:25:59.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:58 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:58.841+0000 7f04976f6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T01:26:00.344 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:25:59.883+0000 7f04976f6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T01:26:00.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: Active manager daemon x restarted 2023-11-06T01:26:00.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: Activating manager daemon x 2023-11-06T01:26:00.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: osdmap e2: 0 total, 0 up, 0 in 2023-11-06T01:26:00.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: mgrmap e6: x(active, starting, since 0.00397524s) 2023-11-06T01:26:00.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:26:00.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-06T01:26:00.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-06T01:26:00.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-06T01:26:00.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-06T01:26:00.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: Manager daemon x is now available 2023-11-06T01:26:00.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:00.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:00.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:00.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:25:59 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:01.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2023-11-06T01:26:01.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-11-06T01:26:01.345 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-06T01:26:01.345 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2023-11-06T01:26:01.345 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.946+0000 7f81a5771700 1 Processor -- start 2023-11-06T01:26:01.345 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.946+0000 7f81a5771700 1 -- start start 2023-11-06T01:26:01.345 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.946+0000 7f81a5771700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a0105210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:01.346 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.946+0000 7f81a5771700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0079df0 con 0x7f81a0104df0 2023-11-06T01:26:01.346 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.946+0000 7f819effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a0105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:01.346 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.946+0000 7f819effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a0105210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36714/0 (socket says 172.21.15.53:36714) 2023-11-06T01:26:01.346 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.946+0000 7f819effd700 1 -- 172.21.15.53:0/3874943069 learned_addr learned my addr 172.21.15.53:0/3874943069 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:01.346 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.947+0000 7f819effd700 1 -- 172.21.15.53:0/3874943069 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a0079f30 con 0x7f81a0104df0 2023-11-06T01:26:01.347 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.947+0000 7f819effd700 1 --2- 172.21.15.53:0/3874943069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a0105210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f81900098a0 tx=0x7f8190009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7646fa0b8279eb64 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:01.347 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.947+0000 7f819dffb700 1 -- 172.21.15.53:0/3874943069 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8190004160 con 0x7f81a0104df0 2023-11-06T01:26:01.347 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.947+0000 7f819dffb700 1 -- 172.21.15.53:0/3874943069 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f81900042c0 con 0x7f81a0104df0 2023-11-06T01:26:01.347 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.947+0000 7f819dffb700 1 -- 172.21.15.53:0/3874943069 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8190011a00 con 0x7f81a0104df0 2023-11-06T01:26:01.347 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.948+0000 7f81a5771700 1 -- 172.21.15.53:0/3874943069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 msgr2=0x7f81a0105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:01.348 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.948+0000 7f81a5771700 1 --2- 172.21.15.53:0/3874943069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a0105210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f81900098a0 tx=0x7f8190009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:01.348 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.948+0000 7f81a5771700 1 -- 172.21.15.53:0/3874943069 shutdown_connections 2023-11-06T01:26:01.348 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.948+0000 7f81a5771700 1 --2- 172.21.15.53:0/3874943069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a0105210 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:01.348 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.948+0000 7f81a5771700 1 -- 172.21.15.53:0/3874943069 >> 172.21.15.53:0/3874943069 conn(0x7f81a01009b0 msgr2=0x7f81a0102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:01.348 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.948+0000 7f81a5771700 1 -- 172.21.15.53:0/3874943069 shutdown_connections 2023-11-06T01:26:01.349 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.948+0000 7f81a5771700 1 -- 172.21.15.53:0/3874943069 wait complete. 2023-11-06T01:26:01.349 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.949+0000 7f81a5771700 1 Processor -- start 2023-11-06T01:26:01.349 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.949+0000 7f81a5771700 1 -- start start 2023-11-06T01:26:01.349 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.949+0000 7f81a5771700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:01.349 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.949+0000 7f81a5771700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a019c880 con 0x7f81a0104df0 2023-11-06T01:26:01.350 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.949+0000 7f819effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:01.350 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.949+0000 7f819effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a019c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36716/0 (socket says 172.21.15.53:36716) 2023-11-06T01:26:01.350 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.949+0000 7f819effd700 1 -- 172.21.15.53:0/338724971 learned_addr learned my addr 172.21.15.53:0/338724971 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:01.350 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.950+0000 7f819effd700 1 -- 172.21.15.53:0/338724971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8190009550 con 0x7f81a0104df0 2023-11-06T01:26:01.351 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.950+0000 7f819effd700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a019c340 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8190009fb0 tx=0x7f8190011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:01.351 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.950+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8190003930 con 0x7f81a0104df0 2023-11-06T01:26:01.351 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.950+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8190003a90 con 0x7f81a0104df0 2023-11-06T01:26:01.351 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.950+0000 7f81a5771700 1 -- 172.21.15.53:0/338724971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a019ca80 con 0x7f81a0104df0 2023-11-06T01:26:01.351 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.950+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8190019610 con 0x7f81a0104df0 2023-11-06T01:26:01.352 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.950+0000 7f81a5771700 1 -- 172.21.15.53:0/338724971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a019cf20 con 0x7f81a0104df0 2023-11-06T01:26:01.352 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.951+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f8190019880 con 0x7f81a0104df0 2023-11-06T01:26:01.352 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.951+0000 7f8187fff700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 0x7f818803a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:01.352 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.951+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 --> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f818803aed0 con 0x7f8188038300 2023-11-06T01:26:01.352 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.951+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f819004bfa0 con 0x7f81a0104df0 2023-11-06T01:26:01.353 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.951+0000 7f819e7fc700 1 -- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 msgr2=0x7f818803a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3877634851 2023-11-06T01:26:01.353 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:53.951+0000 7f819e7fc700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 0x7f818803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T01:26:01.353 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:54.152+0000 7f819e7fc700 1 -- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 msgr2=0x7f818803a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3877634851 2023-11-06T01:26:01.353 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:54.152+0000 7f819e7fc700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 0x7f818803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T01:26:01.353 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:54.553+0000 7f819e7fc700 1 -- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 msgr2=0x7f818803a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3877634851 2023-11-06T01:26:01.354 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:54.553+0000 7f819e7fc700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 0x7f818803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-06T01:26:01.354 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: Found migration_current of "None". Setting to last migration. 2023-11-06T01:26:01.354 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:01.354 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-06T01:26:01.355 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-06T01:26:01.355 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:01.355 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:01.355 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: [06/Nov/2023:01:26:00] ENGINE Bus STARTING 2023-11-06T01:26:01.355 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: [06/Nov/2023:01:26:00] ENGINE Serving on https://172.21.15.53:7150 2023-11-06T01:26:01.356 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: [06/Nov/2023:01:26:00] ENGINE Serving on http://172.21.15.53:8765 2023-11-06T01:26:01.356 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: [06/Nov/2023:01:26:00] ENGINE Bus STARTED 2023-11-06T01:26:01.356 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:01.356 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:00 smithi053 ceph-mon[99472]: mgrmap e7: x(active, since 1.00613s) 2023-11-06T01:26:01.359 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:55.354+0000 7f819e7fc700 1 -- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 msgr2=0x7f818803a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3877634851 2023-11-06T01:26:01.359 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:55.354+0000 7f819e7fc700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 0x7f818803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-06T01:26:01.359 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:56.955+0000 7f819e7fc700 1 -- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 msgr2=0x7f818803a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3877634851 2023-11-06T01:26:01.359 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:56.955+0000 7f819e7fc700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 0x7f818803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-06T01:26:01.360 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:59.899+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mgrmap(e 6) v1 ==== 45035+0+0 (secure 0 0 0) 0x7f819002d960 con 0x7f81a0104df0 2023-11-06T01:26:01.360 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:59.899+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 msgr2=0x7f818803a7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:26:01.360 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:25:59.899+0000 7f8187fff700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 0x7f818803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:01.360 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.901+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 7) v1 ==== 45162+0+0 (secure 0 0 0) 0x7f8190018350 con 0x7f81a0104df0 2023-11-06T01:26:01.360 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.901+0000 7f8187fff700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f818803bbe0 0x7f818803dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:01.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.901+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 --> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f818803aed0 con 0x7f818803bbe0 2023-11-06T01:26:01.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.901+0000 7f819e7fc700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f818803bbe0 0x7f818803dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:01.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.902+0000 7f819e7fc700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f818803bbe0 0x7f818803dfd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8194003a10 tx=0x7f81940092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:01.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.902+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 <== mgr.14120 v2:172.21.15.53:6800/2231847638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f818803aed0 con 0x7f818803bbe0 2023-11-06T01:26:01.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.913+0000 7f81a5771700 1 -- 172.21.15.53:0/338724971 --> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f81a019d680 con 0x7f818803bbe0 2023-11-06T01:26:01.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.913+0000 7f8187fff700 1 -- 172.21.15.53:0/338724971 <== mgr.14120 v2:172.21.15.53:6800/2231847638 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f81a019d680 con 0x7f818803bbe0 2023-11-06T01:26:01.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.914+0000 7f81a5771700 1 -- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f818803bbe0 msgr2=0x7f818803dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:01.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.914+0000 7f81a5771700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f818803bbe0 0x7f818803dfd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8194003a10 tx=0x7f81940092b0 comp rx=0 tx=0).stop 2023-11-06T01:26:01.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.914+0000 7f81a5771700 1 -- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 msgr2=0x7f81a019c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:01.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.914+0000 7f81a5771700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a019c340 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8190009fb0 tx=0x7f8190011ad0 comp rx=0 tx=0).stop 2023-11-06T01:26:01.363 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.914+0000 7f81a5771700 1 -- 172.21.15.53:0/338724971 shutdown_connections 2023-11-06T01:26:01.363 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.914+0000 7f81a5771700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f818803bbe0 0x7f818803dfd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:01.363 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.914+0000 7f81a5771700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81a0104df0 0x7f81a019c340 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:01.363 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.914+0000 7f81a5771700 1 --2- 172.21.15.53:0/338724971 >> [v2:172.21.15.53:6800/3877634851,v1:172.21.15.53:6801/3877634851] conn(0x7f8188038300 0x7f818803a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:01.363 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.914+0000 7f81a5771700 1 -- 172.21.15.53:0/338724971 >> 172.21.15.53:0/338724971 conn(0x7f81a01009b0 msgr2=0x7f81a01909b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:01.364 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.914+0000 7f81a5771700 1 -- 172.21.15.53:0/338724971 shutdown_connections 2023-11-06T01:26:01.364 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:00.914+0000 7f81a5771700 1 -- 172.21.15.53:0/338724971 wait complete. 2023-11-06T01:26:01.364 INFO:teuthology.orchestra.run.smithi053.stdout:mgr epoch 5 is available 2023-11-06T01:26:01.364 INFO:teuthology.orchestra.run.smithi053.stdout:Setting orchestrator backend to cephadm... 2023-11-06T01:26:02.267 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:01 smithi053 ceph-mon[99472]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-11-06T01:26:02.268 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:01 smithi053 ceph-mon[99472]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-11-06T01:26:02.596 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.819+0000 7f08b14d3700 1 Processor -- start 2023-11-06T01:26:02.596 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.819+0000 7f08b14d3700 1 -- start start 2023-11-06T01:26:02.596 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.819+0000 7f08b14d3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:02.597 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.819+0000 7f08b14d3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac079e50 con 0x7f08ac106fa0 2023-11-06T01:26:02.597 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.819+0000 7f08aaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:02.597 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.819+0000 7f08aaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36768/0 (socket says 172.21.15.53:36768) 2023-11-06T01:26:02.597 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.819+0000 7f08aaffd700 1 -- 172.21.15.53:0/2819035201 learned_addr learned my addr 172.21.15.53:0/2819035201 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:02.597 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.819+0000 7f08aaffd700 1 -- 172.21.15.53:0/2819035201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac1098d0 con 0x7f08ac106fa0 2023-11-06T01:26:02.598 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.820+0000 7f08aaffd700 1 --2- 172.21.15.53:0/2819035201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac109390 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f089c0098a0 tx=0x7f089c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53db2ceee55aa6b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:02.598 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.820+0000 7f08a9ffb700 1 -- 172.21.15.53:0/2819035201 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f089c004160 con 0x7f08ac106fa0 2023-11-06T01:26:02.598 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.820+0000 7f08a9ffb700 1 -- 172.21.15.53:0/2819035201 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f089c0042c0 con 0x7f08ac106fa0 2023-11-06T01:26:02.598 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.820+0000 7f08a9ffb700 1 -- 172.21.15.53:0/2819035201 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f089c011a00 con 0x7f08ac106fa0 2023-11-06T01:26:02.599 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.820+0000 7f08b14d3700 1 -- 172.21.15.53:0/2819035201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 msgr2=0x7f08ac109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:02.599 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.820+0000 7f08b14d3700 1 --2- 172.21.15.53:0/2819035201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac109390 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f089c0098a0 tx=0x7f089c009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:02.599 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.820+0000 7f08b14d3700 1 -- 172.21.15.53:0/2819035201 shutdown_connections 2023-11-06T01:26:02.599 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.820+0000 7f08b14d3700 1 --2- 172.21.15.53:0/2819035201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac109390 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:02.600 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.820+0000 7f08b14d3700 1 -- 172.21.15.53:0/2819035201 >> 172.21.15.53:0/2819035201 conn(0x7f08ac1009f0 msgr2=0x7f08ac102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:02.600 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.821+0000 7f08b14d3700 1 -- 172.21.15.53:0/2819035201 shutdown_connections 2023-11-06T01:26:02.600 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.821+0000 7f08b14d3700 1 -- 172.21.15.53:0/2819035201 wait complete. 2023-11-06T01:26:02.600 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.821+0000 7f08b14d3700 1 Processor -- start 2023-11-06T01:26:02.601 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.821+0000 7f08b14d3700 1 -- start start 2023-11-06T01:26:02.601 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.821+0000 7f08b14d3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:02.601 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.822+0000 7f08b14d3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac19a690 con 0x7f08ac106fa0 2023-11-06T01:26:02.601 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.822+0000 7f08aaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:02.601 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.822+0000 7f08aaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36770/0 (socket says 172.21.15.53:36770) 2023-11-06T01:26:02.602 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.822+0000 7f08aaffd700 1 -- 172.21.15.53:0/2809604325 learned_addr learned my addr 172.21.15.53:0/2809604325 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:02.602 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.822+0000 7f08aaffd700 1 -- 172.21.15.53:0/2809604325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c009550 con 0x7f08ac106fa0 2023-11-06T01:26:02.602 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.822+0000 7f08aaffd700 1 --2- 172.21.15.53:0/2809604325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac19a150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f089c006af0 tx=0x7f089c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:02.602 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.822+0000 7f0893fff700 1 -- 172.21.15.53:0/2809604325 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f089c003a40 con 0x7f08ac106fa0 2023-11-06T01:26:02.603 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.822+0000 7f0893fff700 1 -- 172.21.15.53:0/2809604325 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f089c003ba0 con 0x7f08ac106fa0 2023-11-06T01:26:02.603 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.822+0000 7f08b14d3700 1 -- 172.21.15.53:0/2809604325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac19a890 con 0x7f08ac106fa0 2023-11-06T01:26:02.603 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.822+0000 7f0893fff700 1 -- 172.21.15.53:0/2809604325 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f089c01a6e0 con 0x7f08ac106fa0 2023-11-06T01:26:02.603 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.822+0000 7f08b14d3700 1 -- 172.21.15.53:0/2809604325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac19ad30 con 0x7f08ac106fa0 2023-11-06T01:26:02.603 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.823+0000 7f0893fff700 1 -- 172.21.15.53:0/2809604325 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 7) v1 ==== 45162+0+0 (secure 0 0 0) 0x7f089c024430 con 0x7f08ac106fa0 2023-11-06T01:26:02.604 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.823+0000 7f0893fff700 1 --2- 172.21.15.53:0/2809604325 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f0894038270 0x7f089403a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:02.604 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.823+0000 7f08b14d3700 1 -- 172.21.15.53:0/2809604325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac10a670 con 0x7f08ac106fa0 2023-11-06T01:26:02.604 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.823+0000 7f0893fff700 1 -- 172.21.15.53:0/2809604325 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f089c04c010 con 0x7f08ac106fa0 2023-11-06T01:26:02.605 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.824+0000 7f08aa7fc700 1 --2- 172.21.15.53:0/2809604325 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f0894038270 0x7f089403a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:02.605 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.824+0000 7f08aa7fc700 1 --2- 172.21.15.53:0/2809604325 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f0894038270 0x7f089403a730 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f08a0006fd0 tx=0x7f08a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:02.605 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.829+0000 7f0893fff700 1 -- 172.21.15.53:0/2809604325 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f089c0246e0 con 0x7f08ac106fa0 2023-11-06T01:26:02.605 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:01.952+0000 7f0893fff700 1 -- 172.21.15.53:0/2809604325 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f089c02d910 con 0x7f08ac106fa0 2023-11-06T01:26:02.606 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.224+0000 7f08b14d3700 1 -- 172.21.15.53:0/2809604325 --> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f08ac101ec0 con 0x7f0894038270 2023-11-06T01:26:02.606 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.231+0000 7f0893fff700 1 -- 172.21.15.53:0/2809604325 <== mgr.14120 v2:172.21.15.53:6800/2231847638 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f08ac101ec0 con 0x7f0894038270 2023-11-06T01:26:02.606 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.233+0000 7f08b14d3700 1 -- 172.21.15.53:0/2809604325 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f0894038270 msgr2=0x7f089403a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:02.607 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.233+0000 7f08b14d3700 1 --2- 172.21.15.53:0/2809604325 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f0894038270 0x7f089403a730 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f08a0006fd0 tx=0x7f08a0006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:02.607 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.233+0000 7f08b14d3700 1 -- 172.21.15.53:0/2809604325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 msgr2=0x7f08ac19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:02.607 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.233+0000 7f08b14d3700 1 --2- 172.21.15.53:0/2809604325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac19a150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f089c006af0 tx=0x7f089c011ad0 comp rx=0 tx=0).stop 2023-11-06T01:26:02.607 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.233+0000 7f08b14d3700 1 -- 172.21.15.53:0/2809604325 shutdown_connections 2023-11-06T01:26:02.607 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.233+0000 7f08b14d3700 1 --2- 172.21.15.53:0/2809604325 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f0894038270 0x7f089403a730 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:02.608 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.233+0000 7f08b14d3700 1 --2- 172.21.15.53:0/2809604325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ac106fa0 0x7f08ac19a150 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:02.608 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.233+0000 7f08b14d3700 1 -- 172.21.15.53:0/2809604325 >> 172.21.15.53:0/2809604325 conn(0x7f08ac1009f0 msgr2=0x7f08ac1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:02.608 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.233+0000 7f08b14d3700 1 -- 172.21.15.53:0/2809604325 shutdown_connections 2023-11-06T01:26:02.608 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:02.234+0000 7f08b14d3700 1 -- 172.21.15.53:0/2809604325 wait complete. 2023-11-06T01:26:03.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:02 smithi053 ceph-mon[99472]: mgrmap e8: x(active, since 2s) 2023-11-06T01:26:03.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:02 smithi053 ceph-mon[99472]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:03.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:02 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:03.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:02 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:03.741 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout value unchanged 2023-11-06T01:26:03.741 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.052+0000 7f61e6c11700 1 Processor -- start 2023-11-06T01:26:03.741 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.052+0000 7f61e6c11700 1 -- start start 2023-11-06T01:26:03.741 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.052+0000 7f61e6c11700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e0106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:03.742 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.052+0000 7f61e6c11700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0106ee0 con 0x7f61e01064f0 2023-11-06T01:26:03.742 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.052+0000 7f61e49ad700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e0106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:03.742 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.052+0000 7f61e49ad700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e0106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36774/0 (socket says 172.21.15.53:36774) 2023-11-06T01:26:03.743 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.052+0000 7f61e49ad700 1 -- 172.21.15.53:0/3188607771 learned_addr learned my addr 172.21.15.53:0/3188607771 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:03.743 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.052+0000 7f61e49ad700 1 -- 172.21.15.53:0/3188607771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e01076f0 con 0x7f61e01064f0 2023-11-06T01:26:03.743 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.053+0000 7f61e49ad700 1 --2- 172.21.15.53:0/3188607771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e0106910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61d40098a0 tx=0x7f61d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a99d36cc7db392 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:03.743 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.053+0000 7f61df7fe700 1 -- 172.21.15.53:0/3188607771 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61d4004160 con 0x7f61e01064f0 2023-11-06T01:26:03.743 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.053+0000 7f61df7fe700 1 -- 172.21.15.53:0/3188607771 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61d40042c0 con 0x7f61e01064f0 2023-11-06T01:26:03.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.053+0000 7f61df7fe700 1 -- 172.21.15.53:0/3188607771 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61d4011a00 con 0x7f61e01064f0 2023-11-06T01:26:03.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.053+0000 7f61e6c11700 1 -- 172.21.15.53:0/3188607771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 msgr2=0x7f61e0106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:03.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.053+0000 7f61e6c11700 1 --2- 172.21.15.53:0/3188607771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e0106910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61d40098a0 tx=0x7f61d4009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:03.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.053+0000 7f61e6c11700 1 -- 172.21.15.53:0/3188607771 shutdown_connections 2023-11-06T01:26:03.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.053+0000 7f61e6c11700 1 --2- 172.21.15.53:0/3188607771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e0106910 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:03.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.053+0000 7f61e6c11700 1 -- 172.21.15.53:0/3188607771 >> 172.21.15.53:0/3188607771 conn(0x7f61e0101ad0 msgr2=0x7f61e0103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:03.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.054+0000 7f61e6c11700 1 -- 172.21.15.53:0/3188607771 shutdown_connections 2023-11-06T01:26:03.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.054+0000 7f61e6c11700 1 -- 172.21.15.53:0/3188607771 wait complete. 2023-11-06T01:26:03.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.054+0000 7f61e6c11700 1 Processor -- start 2023-11-06T01:26:03.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.054+0000 7f61e6c11700 1 -- start start 2023-11-06T01:26:03.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.055+0000 7f61e6c11700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:03.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.055+0000 7f61e6c11700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e019a6b0 con 0x7f61e01064f0 2023-11-06T01:26:03.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.055+0000 7f61e49ad700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:03.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.055+0000 7f61e49ad700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36776/0 (socket says 172.21.15.53:36776) 2023-11-06T01:26:03.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.055+0000 7f61e49ad700 1 -- 172.21.15.53:0/1248906089 learned_addr learned my addr 172.21.15.53:0/1248906089 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:03.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.055+0000 7f61e49ad700 1 -- 172.21.15.53:0/1248906089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4009550 con 0x7f61e01064f0 2023-11-06T01:26:03.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.055+0000 7f61e49ad700 1 --2- 172.21.15.53:0/1248906089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e019a170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009fb0 tx=0x7f61d4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:03.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.055+0000 7f61ddffb700 1 -- 172.21.15.53:0/1248906089 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61d4003930 con 0x7f61e01064f0 2023-11-06T01:26:03.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.055+0000 7f61e6c11700 1 -- 172.21.15.53:0/1248906089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e019a8b0 con 0x7f61e01064f0 2023-11-06T01:26:03.748 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.055+0000 7f61ddffb700 1 -- 172.21.15.53:0/1248906089 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61d4003a90 con 0x7f61e01064f0 2023-11-06T01:26:03.748 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.055+0000 7f61ddffb700 1 -- 172.21.15.53:0/1248906089 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61d40195e0 con 0x7f61e01064f0 2023-11-06T01:26:03.748 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.056+0000 7f61e6c11700 1 -- 172.21.15.53:0/1248906089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e019ad50 con 0x7f61e01064f0 2023-11-06T01:26:03.748 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.057+0000 7f61ddffb700 1 -- 172.21.15.53:0/1248906089 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f61d401a430 con 0x7f61e01064f0 2023-11-06T01:26:03.748 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.057+0000 7f61e6c11700 1 -- 172.21.15.53:0/1248906089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e00623c0 con 0x7f61e01064f0 2023-11-06T01:26:03.749 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.057+0000 7f61ddffb700 1 --2- 172.21.15.53:0/1248906089 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f61d00382b0 0x7f61d003a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:03.749 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.057+0000 7f61ddffb700 1 -- 172.21.15.53:0/1248906089 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f61d404c2c0 con 0x7f61e01064f0 2023-11-06T01:26:03.749 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.057+0000 7f61dffff700 1 --2- 172.21.15.53:0/1248906089 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f61d00382b0 0x7f61d003a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:03.749 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.058+0000 7f61dffff700 1 --2- 172.21.15.53:0/1248906089 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f61d00382b0 0x7f61d003a770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61cc006fd0 tx=0x7f61cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:03.750 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.062+0000 7f61ddffb700 1 -- 172.21.15.53:0/1248906089 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f61d4023b80 con 0x7f61e01064f0 2023-11-06T01:26:03.750 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.426+0000 7f61e6c11700 1 -- 172.21.15.53:0/1248906089 --> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f61e0102fc0 con 0x7f61d00382b0 2023-11-06T01:26:03.750 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.427+0000 7f61ddffb700 1 -- 172.21.15.53:0/1248906089 <== mgr.14120 v2:172.21.15.53:6800/2231847638 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f61e0102fc0 con 0x7f61d00382b0 2023-11-06T01:26:03.751 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.429+0000 7f61e6c11700 1 -- 172.21.15.53:0/1248906089 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f61d00382b0 msgr2=0x7f61d003a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:03.751 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.429+0000 7f61e6c11700 1 --2- 172.21.15.53:0/1248906089 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f61d00382b0 0x7f61d003a770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61cc006fd0 tx=0x7f61cc006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:03.751 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.429+0000 7f61e6c11700 1 -- 172.21.15.53:0/1248906089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 msgr2=0x7f61e019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:03.751 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.429+0000 7f61e6c11700 1 --2- 172.21.15.53:0/1248906089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e019a170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009fb0 tx=0x7f61d4011ad0 comp rx=0 tx=0).stop 2023-11-06T01:26:03.751 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.429+0000 7f61e6c11700 1 -- 172.21.15.53:0/1248906089 shutdown_connections 2023-11-06T01:26:03.752 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.429+0000 7f61e6c11700 1 --2- 172.21.15.53:0/1248906089 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f61d00382b0 0x7f61d003a770 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:03.752 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.429+0000 7f61e6c11700 1 --2- 172.21.15.53:0/1248906089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e01064f0 0x7f61e019a170 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:03.752 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.429+0000 7f61e6c11700 1 -- 172.21.15.53:0/1248906089 >> 172.21.15.53:0/1248906089 conn(0x7f61e0101ad0 msgr2=0x7f61e01028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:03.753 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.429+0000 7f61e6c11700 1 -- 172.21.15.53:0/1248906089 shutdown_connections 2023-11-06T01:26:03.753 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:03.429+0000 7f61e6c11700 1 -- 172.21.15.53:0/1248906089 wait complete. 2023-11-06T01:26:03.753 INFO:teuthology.orchestra.run.smithi053.stdout:Generating ssh key... 2023-11-06T01:26:04.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:03 smithi053 ceph-mon[99472]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:04.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.213+0000 7f2f4c1a8700 1 Processor -- start 2023-11-06T01:26:04.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.213+0000 7f2f4c1a8700 1 -- start start 2023-11-06T01:26:04.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.213+0000 7f2f4c1a8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f44104bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:04.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.213+0000 7f2f4c1a8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f441051c0 con 0x7f2f441047d0 2023-11-06T01:26:04.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.213+0000 7f2f49f44700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f44104bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:04.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.213+0000 7f2f49f44700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f44104bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36780/0 (socket says 172.21.15.53:36780) 2023-11-06T01:26:04.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.213+0000 7f2f49f44700 1 -- 172.21.15.53:0/2825463626 learned_addr learned my addr 172.21.15.53:0/2825463626 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:04.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.214+0000 7f2f49f44700 1 -- 172.21.15.53:0/2825463626 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f44105300 con 0x7f2f441047d0 2023-11-06T01:26:04.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.214+0000 7f2f49f44700 1 --2- 172.21.15.53:0/2825463626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f44104bf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2f400098a0 tx=0x7f2f40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a65ba5befc6b863 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:04.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.214+0000 7f2f48f42700 1 -- 172.21.15.53:0/2825463626 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f40004160 con 0x7f2f441047d0 2023-11-06T01:26:04.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.214+0000 7f2f48f42700 1 -- 172.21.15.53:0/2825463626 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2f400042c0 con 0x7f2f441047d0 2023-11-06T01:26:04.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.214+0000 7f2f48f42700 1 -- 172.21.15.53:0/2825463626 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f40011a00 con 0x7f2f441047d0 2023-11-06T01:26:04.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.214+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/2825463626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 msgr2=0x7f2f44104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:04.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.214+0000 7f2f4c1a8700 1 --2- 172.21.15.53:0/2825463626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f44104bf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2f400098a0 tx=0x7f2f40009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:04.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.214+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/2825463626 shutdown_connections 2023-11-06T01:26:04.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.214+0000 7f2f4c1a8700 1 --2- 172.21.15.53:0/2825463626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f44104bf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:04.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.214+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/2825463626 >> 172.21.15.53:0/2825463626 conn(0x7f2f44100170 msgr2=0x7f2f44102590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:04.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.215+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/2825463626 shutdown_connections 2023-11-06T01:26:04.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.215+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/2825463626 wait complete. 2023-11-06T01:26:04.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.215+0000 7f2f4c1a8700 1 Processor -- start 2023-11-06T01:26:04.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.215+0000 7f2f4c1a8700 1 -- start start 2023-11-06T01:26:04.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.216+0000 7f2f4c1a8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f441071b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:04.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.216+0000 7f2f4c1a8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f441076f0 con 0x7f2f441047d0 2023-11-06T01:26:04.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.216+0000 7f2f49f44700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f441071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:04.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.216+0000 7f2f49f44700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f441071b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36782/0 (socket says 172.21.15.53:36782) 2023-11-06T01:26:04.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.216+0000 7f2f49f44700 1 -- 172.21.15.53:0/4238808840 learned_addr learned my addr 172.21.15.53:0/4238808840 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:04.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.216+0000 7f2f49f44700 1 -- 172.21.15.53:0/4238808840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40009550 con 0x7f2f441047d0 2023-11-06T01:26:04.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.216+0000 7f2f49f44700 1 --2- 172.21.15.53:0/4238808840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f441071b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009fb0 tx=0x7f2f40011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:04.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.216+0000 7f2f3affd700 1 -- 172.21.15.53:0/4238808840 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f40003930 con 0x7f2f441047d0 2023-11-06T01:26:04.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.216+0000 7f2f3affd700 1 -- 172.21.15.53:0/4238808840 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2f40003a90 con 0x7f2f441047d0 2023-11-06T01:26:04.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.216+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/4238808840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f44105800 con 0x7f2f441047d0 2023-11-06T01:26:04.993 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.217+0000 7f2f3affd700 1 -- 172.21.15.53:0/4238808840 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f400195e0 con 0x7f2f441047d0 2023-11-06T01:26:04.993 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.217+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/4238808840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f44105ca0 con 0x7f2f441047d0 2023-11-06T01:26:04.993 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.217+0000 7f2f3affd700 1 -- 172.21.15.53:0/4238808840 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f2f4001a430 con 0x7f2f441047d0 2023-11-06T01:26:04.993 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.218+0000 7f2f3affd700 1 --2- 172.21.15.53:0/4238808840 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2f30038300 0x7f2f3003a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:04.993 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.218+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/4238808840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f440623c0 con 0x7f2f441047d0 2023-11-06T01:26:04.994 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.218+0000 7f2f3affd700 1 -- 172.21.15.53:0/4238808840 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2f4004c290 con 0x7f2f441047d0 2023-11-06T01:26:04.994 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.218+0000 7f2f49743700 1 --2- 172.21.15.53:0/4238808840 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2f30038300 0x7f2f3003a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:04.994 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.218+0000 7f2f49743700 1 --2- 172.21.15.53:0/4238808840 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2f30038300 0x7f2f3003a7c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2f34006fd0 tx=0x7f2f34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:04.994 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.223+0000 7f2f3affd700 1 -- 172.21.15.53:0/4238808840 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2f4002ce00 con 0x7f2f441047d0 2023-11-06T01:26:04.994 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.588+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/4238808840 --> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f44101670 con 0x7f2f30038300 2023-11-06T01:26:04.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.676+0000 7f2f3affd700 1 -- 172.21.15.53:0/4238808840 <== mgr.14120 v2:172.21.15.53:6800/2231847638 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2f44101670 con 0x7f2f30038300 2023-11-06T01:26:04.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.678+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/4238808840 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2f30038300 msgr2=0x7f2f3003a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:04.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.678+0000 7f2f4c1a8700 1 --2- 172.21.15.53:0/4238808840 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2f30038300 0x7f2f3003a7c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2f34006fd0 tx=0x7f2f34006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:04.995 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.678+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/4238808840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 msgr2=0x7f2f441071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:04.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.678+0000 7f2f4c1a8700 1 --2- 172.21.15.53:0/4238808840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f441071b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009fb0 tx=0x7f2f40011ad0 comp rx=0 tx=0).stop 2023-11-06T01:26:04.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.678+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/4238808840 shutdown_connections 2023-11-06T01:26:04.996 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.678+0000 7f2f4c1a8700 1 --2- 172.21.15.53:0/4238808840 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2f30038300 0x7f2f3003a7c0 secure :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2f34006fd0 tx=0x7f2f34006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:04.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.678+0000 7f2f4c1a8700 1 --2- 172.21.15.53:0/4238808840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f441047d0 0x7f2f441071b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:04.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.678+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/4238808840 >> 172.21.15.53:0/4238808840 conn(0x7f2f44100170 msgr2=0x7f2f44100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:04.997 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.678+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/4238808840 shutdown_connections 2023-11-06T01:26:04.998 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:04.678+0000 7f2f4c1a8700 1 -- 172.21.15.53:0/4238808840 wait complete. 2023-11-06T01:26:05.016 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: Generating public/private rsa key pair. 2023-11-06T01:26:05.017 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: Your identification has been saved in /tmp/tmpdupg037d/key. 2023-11-06T01:26:05.017 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: Your public key has been saved in /tmp/tmpdupg037d/key.pub. 2023-11-06T01:26:05.017 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: The key fingerprint is: 2023-11-06T01:26:05.017 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: SHA256:SmzeJOTd4moQhRCgdalGpF3hAMzJOjpclWTJoO/JalY ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:26:05.017 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: The key's randomart image is: 2023-11-06T01:26:05.018 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: +---[RSA 3072]----+ 2023-11-06T01:26:05.018 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: |*+B+B*o | 2023-11-06T01:26:05.018 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: |.Oo=+=. | 2023-11-06T01:26:05.018 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: |+o..o.. | 2023-11-06T01:26:05.019 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: |o +..+ . . | 2023-11-06T01:26:05.019 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: |oo.. .* S . | 2023-11-06T01:26:05.019 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: |o.oEo+ * . | 2023-11-06T01:26:05.019 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: | ..+ .o o | 2023-11-06T01:26:05.019 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: | o. .. | 2023-11-06T01:26:05.019 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: |o. .. | 2023-11-06T01:26:05.020 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: +----[SHA256]-----+ 2023-11-06T01:26:06.005 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:05 smithi053 ceph-mon[99472]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:06.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:05 smithi053 ceph-mon[99472]: Generating ssh key... 2023-11-06T01:26:06.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:05 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:06.006 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:05 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:06.231 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgjBbf6Kp8aICaklGQQMvixLLKgGoExmEJdJwu19q6vSKgZRfS2FRiGwTMeHaGZTFiw7QnLLRBGAm+wYyVTWbV/H1E6aZMhCvBxPXpz99LMvenUaW7hO+E4LO8ipR6BAN9VWuYFspN/42KyGxR7dldOpfLJ66hk/Tv8UJ15TN3YRhmalZXajGR2p1GWRvPLTztiINVyN/tgwQnqCCr0/P0b96SQPOgt/0QQ2qg+H2sZW1vS2AmnMETTRXg5Rm+zvO5Sx73duya5w9JX8LIZGJEgD5adEzjwSccK1TKRWeRVzVB8804G9e9P+U+IpSUt1uHd26gOkpXyDSHbMvVNIWxYRxQtu4LNjAbkuJfWXadUV7qtfYqd8cvzANLPrvG8lQdgaYNCmjpFbHQToVH9OcTzCKV5QmAxshfvAL0VRZYrfndSdh3gQ3LScJezXMfGBImpzaJtf5mCeiIcohyD5mLpTocyHWXDmagUX15qO+zPv/yFZtC8BzPNvtUQoTl/u0= ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:26:06.231 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.466+0000 7fdd7cc48700 1 Processor -- start 2023-11-06T01:26:06.231 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.466+0000 7fdd7cc48700 1 -- start start 2023-11-06T01:26:06.231 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.467+0000 7fdd7cc48700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd78104c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:06.232 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.467+0000 7fdd7cc48700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd781051e0 con 0x7fdd781047f0 2023-11-06T01:26:06.232 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.467+0000 7fdd7659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd78104c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:06.232 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.467+0000 7fdd7659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd78104c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36786/0 (socket says 172.21.15.53:36786) 2023-11-06T01:26:06.232 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.467+0000 7fdd7659c700 1 -- 172.21.15.53:0/2565104739 learned_addr learned my addr 172.21.15.53:0/2565104739 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:06.232 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.467+0000 7fdd7659c700 1 -- 172.21.15.53:0/2565104739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd78107780 con 0x7fdd781047f0 2023-11-06T01:26:06.233 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.467+0000 7fdd7659c700 1 --2- 172.21.15.53:0/2565104739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd78104c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdd600098a0 tx=0x7fdd60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7514ab976d1326 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:06.233 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.468+0000 7fdd7559a700 1 -- 172.21.15.53:0/2565104739 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd60004160 con 0x7fdd781047f0 2023-11-06T01:26:06.233 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.468+0000 7fdd7559a700 1 -- 172.21.15.53:0/2565104739 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd600042c0 con 0x7fdd781047f0 2023-11-06T01:26:06.233 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.468+0000 7fdd7559a700 1 -- 172.21.15.53:0/2565104739 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd60011a00 con 0x7fdd781047f0 2023-11-06T01:26:06.233 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.468+0000 7fdd7cc48700 1 -- 172.21.15.53:0/2565104739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 msgr2=0x7fdd78104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:06.234 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.468+0000 7fdd7cc48700 1 --2- 172.21.15.53:0/2565104739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd78104c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdd600098a0 tx=0x7fdd60009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:06.234 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.468+0000 7fdd7cc48700 1 -- 172.21.15.53:0/2565104739 shutdown_connections 2023-11-06T01:26:06.234 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.468+0000 7fdd7cc48700 1 --2- 172.21.15.53:0/2565104739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd78104c10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:06.234 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.468+0000 7fdd7cc48700 1 -- 172.21.15.53:0/2565104739 >> 172.21.15.53:0/2565104739 conn(0x7fdd78100140 msgr2=0x7fdd78102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:06.234 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.468+0000 7fdd7cc48700 1 -- 172.21.15.53:0/2565104739 shutdown_connections 2023-11-06T01:26:06.235 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.468+0000 7fdd7cc48700 1 -- 172.21.15.53:0/2565104739 wait complete. 2023-11-06T01:26:06.235 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.469+0000 7fdd7cc48700 1 Processor -- start 2023-11-06T01:26:06.236 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.469+0000 7fdd7cc48700 1 -- start start 2023-11-06T01:26:06.236 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.469+0000 7fdd7cc48700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd781070e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:06.236 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.469+0000 7fdd7cc48700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78107620 con 0x7fdd781047f0 2023-11-06T01:26:06.237 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.469+0000 7fdd7659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd781070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:06.237 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.469+0000 7fdd7659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd781070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36788/0 (socket says 172.21.15.53:36788) 2023-11-06T01:26:06.237 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.469+0000 7fdd7659c700 1 -- 172.21.15.53:0/520699683 learned_addr learned my addr 172.21.15.53:0/520699683 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:06.237 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.470+0000 7fdd7659c700 1 -- 172.21.15.53:0/520699683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd60009550 con 0x7fdd781047f0 2023-11-06T01:26:06.237 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.470+0000 7fdd7659c700 1 --2- 172.21.15.53:0/520699683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd781070e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd60006af0 tx=0x7fdd60011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:06.238 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.470+0000 7fdd6f7fe700 1 -- 172.21.15.53:0/520699683 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd60003a40 con 0x7fdd781047f0 2023-11-06T01:26:06.238 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.470+0000 7fdd6f7fe700 1 -- 172.21.15.53:0/520699683 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd60003ba0 con 0x7fdd781047f0 2023-11-06T01:26:06.238 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.470+0000 7fdd7cc48700 1 -- 172.21.15.53:0/520699683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd78105730 con 0x7fdd781047f0 2023-11-06T01:26:06.238 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.470+0000 7fdd7cc48700 1 -- 172.21.15.53:0/520699683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd78105bd0 con 0x7fdd781047f0 2023-11-06T01:26:06.239 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.470+0000 7fdd6f7fe700 1 -- 172.21.15.53:0/520699683 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd6001a700 con 0x7fdd781047f0 2023-11-06T01:26:06.239 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.471+0000 7fdd6f7fe700 1 -- 172.21.15.53:0/520699683 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7fdd6001a980 con 0x7fdd781047f0 2023-11-06T01:26:06.239 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.471+0000 7fdd6f7fe700 1 --2- 172.21.15.53:0/520699683 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fdd64038300 0x7fdd6403a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:06.239 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.471+0000 7fdd6f7fe700 1 -- 172.21.15.53:0/520699683 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdd6004c180 con 0x7fdd781047f0 2023-11-06T01:26:06.240 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.471+0000 7fdd7cc48700 1 -- 172.21.15.53:0/520699683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd58005320 con 0x7fdd781047f0 2023-11-06T01:26:06.240 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.471+0000 7fdd75d9b700 1 --2- 172.21.15.53:0/520699683 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fdd64038300 0x7fdd6403a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:06.240 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.472+0000 7fdd75d9b700 1 --2- 172.21.15.53:0/520699683 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fdd64038300 0x7fdd6403a7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006fd0 tx=0x7fdd68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:06.240 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.476+0000 7fdd6f7fe700 1 -- 172.21.15.53:0/520699683 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdd60018d80 con 0x7fdd781047f0 2023-11-06T01:26:06.240 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.842+0000 7fdd7cc48700 1 -- 172.21.15.53:0/520699683 --> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd58000bf0 con 0x7fdd64038300 2023-11-06T01:26:06.241 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.843+0000 7fdd6f7fe700 1 -- 172.21.15.53:0/520699683 <== mgr.14120 v2:172.21.15.53:6800/2231847638 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fdd58000bf0 con 0x7fdd64038300 2023-11-06T01:26:06.241 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.845+0000 7fdd7cc48700 1 -- 172.21.15.53:0/520699683 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fdd64038300 msgr2=0x7fdd6403a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:06.241 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.845+0000 7fdd7cc48700 1 --2- 172.21.15.53:0/520699683 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fdd64038300 0x7fdd6403a7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006fd0 tx=0x7fdd68006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:06.241 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.845+0000 7fdd7cc48700 1 -- 172.21.15.53:0/520699683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 msgr2=0x7fdd781070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:06.241 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.845+0000 7fdd7cc48700 1 --2- 172.21.15.53:0/520699683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd781070e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd60006af0 tx=0x7fdd60011ad0 comp rx=0 tx=0).stop 2023-11-06T01:26:06.242 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.845+0000 7fdd7cc48700 1 -- 172.21.15.53:0/520699683 shutdown_connections 2023-11-06T01:26:06.242 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.845+0000 7fdd7cc48700 1 --2- 172.21.15.53:0/520699683 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fdd64038300 0x7fdd6403a7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:06.242 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.845+0000 7fdd7cc48700 1 --2- 172.21.15.53:0/520699683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd781047f0 0x7fdd781070e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:06.242 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.845+0000 7fdd7cc48700 1 -- 172.21.15.53:0/520699683 >> 172.21.15.53:0/520699683 conn(0x7fdd78100140 msgr2=0x7fdd78100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:06.243 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.845+0000 7fdd7cc48700 1 -- 172.21.15.53:0/520699683 shutdown_connections 2023-11-06T01:26:06.243 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:05.845+0000 7fdd7cc48700 1 -- 172.21.15.53:0/520699683 wait complete. 2023-11-06T01:26:06.243 INFO:teuthology.orchestra.run.smithi053.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-11-06T01:26:06.243 INFO:teuthology.orchestra.run.smithi053.stdout:Adding key to root@localhost authorized_keys... 2023-11-06T01:26:06.243 INFO:teuthology.orchestra.run.smithi053.stdout:Adding host smithi053... 2023-11-06T01:26:07.162 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:06 smithi053 ceph-mon[99472]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:07.984 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:07 smithi053 ceph-mon[99472]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "addr": "172.21.15.53", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:07.985 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:07 smithi053 ceph-mon[99472]: Deploying cephadm binary to smithi053 2023-11-06T01:26:10.285 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Added host 'smithi053' with addr '172.21.15.53' 2023-11-06T01:26:10.286 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.753+0000 7f9dbf06a700 1 Processor -- start 2023-11-06T01:26:10.286 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.753+0000 7f9dbf06a700 1 -- start start 2023-11-06T01:26:10.286 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.753+0000 7f9dbf06a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db81093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:10.286 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.753+0000 7f9dbf06a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db8079e10 con 0x7f9db8106fe0 2023-11-06T01:26:10.286 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.753+0000 7f9dbce06700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db81093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:10.287 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.753+0000 7f9dbce06700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db81093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36792/0 (socket says 172.21.15.53:36792) 2023-11-06T01:26:10.287 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.753+0000 7f9dbce06700 1 -- 172.21.15.53:0/4213713761 learned_addr learned my addr 172.21.15.53:0/4213713761 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:10.287 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.754+0000 7f9dbce06700 1 -- 172.21.15.53:0/4213713761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db8109910 con 0x7f9db8106fe0 2023-11-06T01:26:10.287 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.754+0000 7f9dbce06700 1 --2- 172.21.15.53:0/4213713761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db81093d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0098a0 tx=0x7f9dac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234eb3237a6d65e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:10.288 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.754+0000 7f9db77fe700 1 -- 172.21.15.53:0/4213713761 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dac004160 con 0x7f9db8106fe0 2023-11-06T01:26:10.288 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.754+0000 7f9db77fe700 1 -- 172.21.15.53:0/4213713761 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9dac0042c0 con 0x7f9db8106fe0 2023-11-06T01:26:10.288 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.754+0000 7f9db77fe700 1 -- 172.21.15.53:0/4213713761 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dac011a00 con 0x7f9db8106fe0 2023-11-06T01:26:10.288 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.755+0000 7f9dbf06a700 1 -- 172.21.15.53:0/4213713761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 msgr2=0x7f9db81093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:10.288 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.755+0000 7f9dbf06a700 1 --2- 172.21.15.53:0/4213713761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db81093d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0098a0 tx=0x7f9dac009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:10.289 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.755+0000 7f9dbf06a700 1 -- 172.21.15.53:0/4213713761 shutdown_connections 2023-11-06T01:26:10.289 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.755+0000 7f9dbf06a700 1 --2- 172.21.15.53:0/4213713761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db81093d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:10.289 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.755+0000 7f9dbf06a700 1 -- 172.21.15.53:0/4213713761 >> 172.21.15.53:0/4213713761 conn(0x7f9db81009b0 msgr2=0x7f9db8102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:10.289 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.755+0000 7f9dbf06a700 1 -- 172.21.15.53:0/4213713761 shutdown_connections 2023-11-06T01:26:10.289 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.755+0000 7f9dbf06a700 1 -- 172.21.15.53:0/4213713761 wait complete. 2023-11-06T01:26:10.290 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.755+0000 7f9dbf06a700 1 Processor -- start 2023-11-06T01:26:10.290 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.755+0000 7f9dbf06a700 1 -- start start 2023-11-06T01:26:10.290 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.756+0000 7f9dbf06a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:10.291 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.756+0000 7f9dbf06a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db819c860 con 0x7f9db8106fe0 2023-11-06T01:26:10.291 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.756+0000 7f9dbce06700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:10.291 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.756+0000 7f9dbce06700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db819c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36794/0 (socket says 172.21.15.53:36794) 2023-11-06T01:26:10.291 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.756+0000 7f9dbce06700 1 -- 172.21.15.53:0/3074066103 learned_addr learned my addr 172.21.15.53:0/3074066103 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:10.291 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.756+0000 7f9dbce06700 1 -- 172.21.15.53:0/3074066103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac009550 con 0x7f9db8106fe0 2023-11-06T01:26:10.292 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.756+0000 7f9dbce06700 1 --2- 172.21.15.53:0/3074066103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db819c320 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9dac006af0 tx=0x7f9dac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:10.292 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.756+0000 7f9db5ffb700 1 -- 172.21.15.53:0/3074066103 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dac003a40 con 0x7f9db8106fe0 2023-11-06T01:26:10.292 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.756+0000 7f9db5ffb700 1 -- 172.21.15.53:0/3074066103 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9dac003ba0 con 0x7f9db8106fe0 2023-11-06T01:26:10.292 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.756+0000 7f9dbf06a700 1 -- 172.21.15.53:0/3074066103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db819ca60 con 0x7f9db8106fe0 2023-11-06T01:26:10.292 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.756+0000 7f9db5ffb700 1 -- 172.21.15.53:0/3074066103 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dac01a6e0 con 0x7f9db8106fe0 2023-11-06T01:26:10.293 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.757+0000 7f9dbf06a700 1 -- 172.21.15.53:0/3074066103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db819cf00 con 0x7f9db8106fe0 2023-11-06T01:26:10.293 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.757+0000 7f9db5ffb700 1 -- 172.21.15.53:0/3074066103 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f9dac024a60 con 0x7f9db8106fe0 2023-11-06T01:26:10.293 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.758+0000 7f9db5ffb700 1 --2- 172.21.15.53:0/3074066103 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f9da8038300 0x7f9da803a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:10.293 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.758+0000 7f9dbf06a700 1 -- 172.21.15.53:0/3074066103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db810a660 con 0x7f9db8106fe0 2023-11-06T01:26:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.758+0000 7f9db5ffb700 1 -- 172.21.15.53:0/3074066103 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9dac04c1f0 con 0x7f9db8106fe0 2023-11-06T01:26:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.758+0000 7f9db7fff700 1 --2- 172.21.15.53:0/3074066103 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f9da8038300 0x7f9da803a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.758+0000 7f9db7fff700 1 --2- 172.21.15.53:0/3074066103 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f9da8038300 0x7f9da803a7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9da4006fd0 tx=0x7f9da4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:06.763+0000 7f9db5ffb700 1 -- 172.21.15.53:0/3074066103 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9dac024430 con 0x7f9db8106fe0 2023-11-06T01:26:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:07.136+0000 7f9dbf06a700 1 -- 172.21.15.53:0/3074066103 --> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi053", "addr": "172.21.15.53", "target": ["mon-mgr", ""]}) v1 -- 0x7f9db8101e30 con 0x7f9da8038300 2023-11-06T01:26:10.295 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:09.954+0000 7f9db5ffb700 1 -- 172.21.15.53:0/3074066103 <== mgr.14120 v2:172.21.15.53:6800/2231847638 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f9db8101e30 con 0x7f9da8038300 2023-11-06T01:26:10.295 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:09.957+0000 7f9dbf06a700 1 -- 172.21.15.53:0/3074066103 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f9da8038300 msgr2=0x7f9da803a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:10.295 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:09.957+0000 7f9dbf06a700 1 --2- 172.21.15.53:0/3074066103 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f9da8038300 0x7f9da803a7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9da4006fd0 tx=0x7f9da4006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:10.295 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:09.957+0000 7f9dbf06a700 1 -- 172.21.15.53:0/3074066103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 msgr2=0x7f9db819c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:10.295 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:09.957+0000 7f9dbf06a700 1 --2- 172.21.15.53:0/3074066103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db819c320 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9dac006af0 tx=0x7f9dac011ad0 comp rx=0 tx=0).stop 2023-11-06T01:26:10.296 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:09.957+0000 7f9dbf06a700 1 -- 172.21.15.53:0/3074066103 shutdown_connections 2023-11-06T01:26:10.296 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:09.957+0000 7f9dbf06a700 1 --2- 172.21.15.53:0/3074066103 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f9da8038300 0x7f9da803a7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:10.296 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:09.957+0000 7f9dbf06a700 1 --2- 172.21.15.53:0/3074066103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9db8106fe0 0x7f9db819c320 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:10.296 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:09.957+0000 7f9dbf06a700 1 -- 172.21.15.53:0/3074066103 >> 172.21.15.53:0/3074066103 conn(0x7f9db81009b0 msgr2=0x7f9db8101720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:10.296 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:09.958+0000 7f9dbf06a700 1 -- 172.21.15.53:0/3074066103 shutdown_connections 2023-11-06T01:26:10.297 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:09.958+0000 7f9dbf06a700 1 -- 172.21.15.53:0/3074066103 wait complete. 2023-11-06T01:26:10.297 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying unmanaged mon service... 2023-11-06T01:26:11.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:10 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:11.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:10 smithi053 ceph-mon[99472]: Added host smithi053 2023-11-06T01:26:11.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:10 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:11.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-11-06T01:26:11.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.796+0000 7f570add3700 1 Processor -- start 2023-11-06T01:26:11.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.796+0000 7f570add3700 1 -- start start 2023-11-06T01:26:11.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.796+0000 7f570add3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f5704106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:11.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.796+0000 7f570add3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5704106ef0 con 0x7f5704106500 2023-11-06T01:26:11.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.796+0000 7f5708b6f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f5704106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:11.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.796+0000 7f5708b6f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f5704106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36818/0 (socket says 172.21.15.53:36818) 2023-11-06T01:26:11.838 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.796+0000 7f5708b6f700 1 -- 172.21.15.53:0/3711443493 learned_addr learned my addr 172.21.15.53:0/3711443493 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:11.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.797+0000 7f5708b6f700 1 -- 172.21.15.53:0/3711443493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5704107700 con 0x7f5704106500 2023-11-06T01:26:11.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.797+0000 7f5708b6f700 1 --2- 172.21.15.53:0/3711443493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f5704106920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f56f80098a0 tx=0x7f56f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d3dfdabccccfce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:11.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.797+0000 7f57037fe700 1 -- 172.21.15.53:0/3711443493 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56f8004160 con 0x7f5704106500 2023-11-06T01:26:11.839 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.797+0000 7f57037fe700 1 -- 172.21.15.53:0/3711443493 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f56f80042c0 con 0x7f5704106500 2023-11-06T01:26:11.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.797+0000 7f57037fe700 1 -- 172.21.15.53:0/3711443493 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56f8011a00 con 0x7f5704106500 2023-11-06T01:26:11.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.797+0000 7f570add3700 1 -- 172.21.15.53:0/3711443493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 msgr2=0x7f5704106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:11.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.797+0000 7f570add3700 1 --2- 172.21.15.53:0/3711443493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f5704106920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f56f80098a0 tx=0x7f56f8009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:11.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.798+0000 7f570add3700 1 -- 172.21.15.53:0/3711443493 shutdown_connections 2023-11-06T01:26:11.840 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.798+0000 7f570add3700 1 --2- 172.21.15.53:0/3711443493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f5704106920 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:11.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.798+0000 7f570add3700 1 -- 172.21.15.53:0/3711443493 >> 172.21.15.53:0/3711443493 conn(0x7f5704101b00 msgr2=0x7f5704103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:11.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.798+0000 7f570add3700 1 -- 172.21.15.53:0/3711443493 shutdown_connections 2023-11-06T01:26:11.841 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.798+0000 7f570add3700 1 -- 172.21.15.53:0/3711443493 wait complete. 2023-11-06T01:26:11.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.798+0000 7f570add3700 1 Processor -- start 2023-11-06T01:26:11.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.798+0000 7f570add3700 1 -- start start 2023-11-06T01:26:11.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.798+0000 7f570add3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f570407bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:11.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.798+0000 7f570add3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570407c170 con 0x7f5704106500 2023-11-06T01:26:11.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.799+0000 7f5708b6f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f570407bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:11.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.799+0000 7f5708b6f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f570407bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36820/0 (socket says 172.21.15.53:36820) 2023-11-06T01:26:11.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.799+0000 7f5708b6f700 1 -- 172.21.15.53:0/150557046 learned_addr learned my addr 172.21.15.53:0/150557046 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:11.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.799+0000 7f5708b6f700 1 -- 172.21.15.53:0/150557046 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8009550 con 0x7f5704106500 2023-11-06T01:26:11.845 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.799+0000 7f5708b6f700 1 --2- 172.21.15.53:0/150557046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f570407bc30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009fb0 tx=0x7f56f8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:11.845 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.799+0000 7f5701ffb700 1 -- 172.21.15.53:0/150557046 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56f8003930 con 0x7f5704106500 2023-11-06T01:26:11.845 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.799+0000 7f5701ffb700 1 -- 172.21.15.53:0/150557046 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f56f8003a90 con 0x7f5704106500 2023-11-06T01:26:11.846 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.799+0000 7f570add3700 1 -- 172.21.15.53:0/150557046 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570407a280 con 0x7f5704106500 2023-11-06T01:26:11.846 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.799+0000 7f5701ffb700 1 -- 172.21.15.53:0/150557046 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56f80195e0 con 0x7f5704106500 2023-11-06T01:26:11.846 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.799+0000 7f570add3700 1 -- 172.21.15.53:0/150557046 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570407a720 con 0x7f5704106500 2023-11-06T01:26:11.847 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.801+0000 7f5701ffb700 1 -- 172.21.15.53:0/150557046 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f56f801a9d0 con 0x7f5704106500 2023-11-06T01:26:11.847 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.801+0000 7f570add3700 1 -- 172.21.15.53:0/150557046 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57040623c0 con 0x7f5704106500 2023-11-06T01:26:11.848 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.801+0000 7f5701ffb700 1 --2- 172.21.15.53:0/150557046 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f56f40382b0 0x7f56f403a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:11.849 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.801+0000 7f5701ffb700 1 -- 172.21.15.53:0/150557046 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f56f804c2e0 con 0x7f5704106500 2023-11-06T01:26:11.849 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.801+0000 7f5703fff700 1 --2- 172.21.15.53:0/150557046 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f56f40382b0 0x7f56f403a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:11.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.801+0000 7f5703fff700 1 --2- 172.21.15.53:0/150557046 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f56f40382b0 0x7f56f403a770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f56f0006fd0 tx=0x7f56f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:11.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:10.806+0000 7f5701ffb700 1 -- 172.21.15.53:0/150557046 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f56f8003c00 con 0x7f5704106500 2023-11-06T01:26:11.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.199+0000 7f570add3700 1 -- 172.21.15.53:0/150557046 --> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5704103000 con 0x7f56f40382b0 2023-11-06T01:26:11.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.205+0000 7f5701ffb700 1 -- 172.21.15.53:0/150557046 <== mgr.14120 v2:172.21.15.53:6800/2231847638 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5704103000 con 0x7f56f40382b0 2023-11-06T01:26:11.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.207+0000 7f570add3700 1 -- 172.21.15.53:0/150557046 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f56f40382b0 msgr2=0x7f56f403a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:11.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.207+0000 7f570add3700 1 --2- 172.21.15.53:0/150557046 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f56f40382b0 0x7f56f403a770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f56f0006fd0 tx=0x7f56f0006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:11.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.207+0000 7f570add3700 1 -- 172.21.15.53:0/150557046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 msgr2=0x7f570407bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:11.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.207+0000 7f570add3700 1 --2- 172.21.15.53:0/150557046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f570407bc30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009fb0 tx=0x7f56f8011ad0 comp rx=0 tx=0).stop 2023-11-06T01:26:11.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.208+0000 7f570add3700 1 -- 172.21.15.53:0/150557046 shutdown_connections 2023-11-06T01:26:11.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.208+0000 7f570add3700 1 --2- 172.21.15.53:0/150557046 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f56f40382b0 0x7f56f403a770 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:11.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.208+0000 7f570add3700 1 --2- 172.21.15.53:0/150557046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704106500 0x7f570407bc30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:11.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.208+0000 7f570add3700 1 -- 172.21.15.53:0/150557046 >> 172.21.15.53:0/150557046 conn(0x7f5704101b00 msgr2=0x7f57041028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:11.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.208+0000 7f570add3700 1 -- 172.21.15.53:0/150557046 shutdown_connections 2023-11-06T01:26:11.854 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:11.208+0000 7f570add3700 1 -- 172.21.15.53:0/150557046 wait complete. 2023-11-06T01:26:11.854 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying unmanaged mgr service... 2023-11-06T01:26:12.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:12 smithi053 ceph-mon[99472]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:12.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:12 smithi053 ceph-mon[99472]: Saving service mon spec with placement count:5 2023-11-06T01:26:12.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:12 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:12.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:12 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:12.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:12 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:13.076 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-11-06T01:26:13.076 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.320+0000 7f6ab82c2700 1 Processor -- start 2023-11-06T01:26:13.076 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.320+0000 7f6ab82c2700 1 -- start start 2023-11-06T01:26:13.077 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.320+0000 7f6ab82c2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab0104fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:13.077 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.320+0000 7f6ab82c2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab0105570 con 0x7f6ab0104b80 2023-11-06T01:26:13.077 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.320+0000 7f6ab605e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab0104fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:13.077 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.320+0000 7f6ab605e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab0104fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36824/0 (socket says 172.21.15.53:36824) 2023-11-06T01:26:13.077 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.320+0000 7f6ab605e700 1 -- 172.21.15.53:0/1715387387 learned_addr learned my addr 172.21.15.53:0/1715387387 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:13.078 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.321+0000 7f6ab605e700 1 -- 172.21.15.53:0/1715387387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab0105d80 con 0x7f6ab0104b80 2023-11-06T01:26:13.078 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.321+0000 7f6ab605e700 1 --2- 172.21.15.53:0/1715387387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab0104fa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0098a0 tx=0x7f6aac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a3b1f34363e5f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:13.078 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.321+0000 7f6ab505c700 1 -- 172.21.15.53:0/1715387387 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6aac004160 con 0x7f6ab0104b80 2023-11-06T01:26:13.078 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.321+0000 7f6ab505c700 1 -- 172.21.15.53:0/1715387387 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aac0042c0 con 0x7f6ab0104b80 2023-11-06T01:26:13.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.321+0000 7f6ab505c700 1 -- 172.21.15.53:0/1715387387 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6aac0119b0 con 0x7f6ab0104b80 2023-11-06T01:26:13.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.322+0000 7f6ab82c2700 1 -- 172.21.15.53:0/1715387387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 msgr2=0x7f6ab0104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:13.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.322+0000 7f6ab82c2700 1 --2- 172.21.15.53:0/1715387387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab0104fa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0098a0 tx=0x7f6aac009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:13.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.322+0000 7f6ab82c2700 1 -- 172.21.15.53:0/1715387387 shutdown_connections 2023-11-06T01:26:13.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.322+0000 7f6ab82c2700 1 --2- 172.21.15.53:0/1715387387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab0104fa0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:13.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.322+0000 7f6ab82c2700 1 -- 172.21.15.53:0/1715387387 >> 172.21.15.53:0/1715387387 conn(0x7f6ab0100140 msgr2=0x7f6ab0102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:13.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.322+0000 7f6ab82c2700 1 -- 172.21.15.53:0/1715387387 shutdown_connections 2023-11-06T01:26:13.081 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.322+0000 7f6ab82c2700 1 -- 172.21.15.53:0/1715387387 wait complete. 2023-11-06T01:26:13.081 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.322+0000 7f6ab82c2700 1 Processor -- start 2023-11-06T01:26:13.081 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.323+0000 7f6ab82c2700 1 -- start start 2023-11-06T01:26:13.081 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.323+0000 7f6ab82c2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:13.081 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.323+0000 7f6ab82c2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab019a710 con 0x7f6ab0104b80 2023-11-06T01:26:13.082 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.323+0000 7f6ab605e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:13.082 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.323+0000 7f6ab605e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36826/0 (socket says 172.21.15.53:36826) 2023-11-06T01:26:13.082 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.323+0000 7f6ab605e700 1 -- 172.21.15.53:0/4157522390 learned_addr learned my addr 172.21.15.53:0/4157522390 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:13.082 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.323+0000 7f6ab605e700 1 -- 172.21.15.53:0/4157522390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac009550 con 0x7f6ab0104b80 2023-11-06T01:26:13.083 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.323+0000 7f6ab605e700 1 --2- 172.21.15.53:0/4157522390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab019a1d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6aac003e60 tx=0x7f6aac011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:13.083 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.323+0000 7f6aa77fe700 1 -- 172.21.15.53:0/4157522390 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6aac003970 con 0x7f6ab0104b80 2023-11-06T01:26:13.083 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.323+0000 7f6aa77fe700 1 -- 172.21.15.53:0/4157522390 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aac003ad0 con 0x7f6ab0104b80 2023-11-06T01:26:13.083 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.323+0000 7f6ab82c2700 1 -- 172.21.15.53:0/4157522390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab019a910 con 0x7f6ab0104b80 2023-11-06T01:26:13.083 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.324+0000 7f6aa77fe700 1 -- 172.21.15.53:0/4157522390 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6aac0195e0 con 0x7f6ab0104b80 2023-11-06T01:26:13.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.324+0000 7f6ab82c2700 1 -- 172.21.15.53:0/4157522390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab019adb0 con 0x7f6ab0104b80 2023-11-06T01:26:13.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.324+0000 7f6aa77fe700 1 -- 172.21.15.53:0/4157522390 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f6aac0198c0 con 0x7f6ab0104b80 2023-11-06T01:26:13.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.325+0000 7f6aa77fe700 1 --2- 172.21.15.53:0/4157522390 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f6a9c0382b0 0x7f6a9c03a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:13.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.325+0000 7f6aa77fe700 1 -- 172.21.15.53:0/4157522390 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6aac051580 con 0x7f6ab0104b80 2023-11-06T01:26:13.084 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.325+0000 7f6ab585d700 1 --2- 172.21.15.53:0/4157522390 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f6a9c0382b0 0x7f6a9c03a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:13.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.325+0000 7f6ab585d700 1 --2- 172.21.15.53:0/4157522390 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f6a9c0382b0 0x7f6a9c03a770 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0006fd0 tx=0x7f6aa0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:13.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.325+0000 7f6ab82c2700 1 -- 172.21.15.53:0/4157522390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab0106330 con 0x7f6ab0104b80 2023-11-06T01:26:13.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.330+0000 7f6aa77fe700 1 -- 172.21.15.53:0/4157522390 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6aac052060 con 0x7f6ab0104b80 2023-11-06T01:26:13.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.740+0000 7f6ab82c2700 1 -- 172.21.15.53:0/4157522390 --> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6ab0101f70 con 0x7f6a9c0382b0 2023-11-06T01:26:13.085 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.745+0000 7f6aa77fe700 1 -- 172.21.15.53:0/4157522390 <== mgr.14120 v2:172.21.15.53:6800/2231847638 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6ab0101f70 con 0x7f6a9c0382b0 2023-11-06T01:26:13.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.748+0000 7f6ab82c2700 1 -- 172.21.15.53:0/4157522390 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f6a9c0382b0 msgr2=0x7f6a9c03a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:13.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.748+0000 7f6ab82c2700 1 --2- 172.21.15.53:0/4157522390 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f6a9c0382b0 0x7f6a9c03a770 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0006fd0 tx=0x7f6aa0006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:13.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.748+0000 7f6ab82c2700 1 -- 172.21.15.53:0/4157522390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 msgr2=0x7f6ab019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:13.086 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.748+0000 7f6ab82c2700 1 --2- 172.21.15.53:0/4157522390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab019a1d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6aac003e60 tx=0x7f6aac011aa0 comp rx=0 tx=0).stop 2023-11-06T01:26:13.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.748+0000 7f6ab82c2700 1 -- 172.21.15.53:0/4157522390 shutdown_connections 2023-11-06T01:26:13.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.748+0000 7f6ab82c2700 1 --2- 172.21.15.53:0/4157522390 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f6a9c0382b0 0x7f6a9c03a770 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:13.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.748+0000 7f6ab82c2700 1 --2- 172.21.15.53:0/4157522390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ab0104b80 0x7f6ab019a1d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:13.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.748+0000 7f6ab82c2700 1 -- 172.21.15.53:0/4157522390 >> 172.21.15.53:0/4157522390 conn(0x7f6ab0100140 msgr2=0x7f6ab01098c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:13.087 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.748+0000 7f6ab82c2700 1 -- 172.21.15.53:0/4157522390 shutdown_connections 2023-11-06T01:26:13.088 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:12.748+0000 7f6ab82c2700 1 -- 172.21.15.53:0/4157522390 wait complete. 2023-11-06T01:26:14.088 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:13 smithi053 ceph-mon[99472]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:14.088 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:13 smithi053 ceph-mon[99472]: Saving service mgr spec with placement count:2 2023-11-06T01:26:14.088 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:13 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:14.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.559+0000 7f3c1cd3f700 1 Processor -- start 2023-11-06T01:26:14.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.559+0000 7f3c1cd3f700 1 -- start start 2023-11-06T01:26:14.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.559+0000 7f3c1cd3f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c181051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:14.319 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.559+0000 7f3c1cd3f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c18079e50 con 0x7f3c18104d90 2023-11-06T01:26:14.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.560+0000 7f3c1659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c181051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:14.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.560+0000 7f3c1659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c181051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36830/0 (socket says 172.21.15.53:36830) 2023-11-06T01:26:14.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.560+0000 7f3c1659c700 1 -- 172.21.15.53:0/1573331614 learned_addr learned my addr 172.21.15.53:0/1573331614 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:14.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.560+0000 7f3c1659c700 1 -- 172.21.15.53:0/1573331614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c181056f0 con 0x7f3c18104d90 2023-11-06T01:26:14.320 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.560+0000 7f3c1659c700 1 --2- 172.21.15.53:0/1573331614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c181051b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3c080098a0 tx=0x7f3c08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a18ef614db4731b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:14.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.560+0000 7f3c1559a700 1 -- 172.21.15.53:0/1573331614 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c08004160 con 0x7f3c18104d90 2023-11-06T01:26:14.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.560+0000 7f3c1559a700 1 -- 172.21.15.53:0/1573331614 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c080042c0 con 0x7f3c18104d90 2023-11-06T01:26:14.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.560+0000 7f3c1559a700 1 -- 172.21.15.53:0/1573331614 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c080119b0 con 0x7f3c18104d90 2023-11-06T01:26:14.321 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.561+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1573331614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 msgr2=0x7f3c181051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:14.322 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.561+0000 7f3c1cd3f700 1 --2- 172.21.15.53:0/1573331614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c181051b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3c080098a0 tx=0x7f3c08009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:14.322 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.561+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1573331614 shutdown_connections 2023-11-06T01:26:14.322 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.561+0000 7f3c1cd3f700 1 --2- 172.21.15.53:0/1573331614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c181051b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:14.322 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.561+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1573331614 >> 172.21.15.53:0/1573331614 conn(0x7f3c181009b0 msgr2=0x7f3c18102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:14.322 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.561+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1573331614 shutdown_connections 2023-11-06T01:26:14.323 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.561+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1573331614 wait complete. 2023-11-06T01:26:14.323 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.562+0000 7f3c1cd3f700 1 Processor -- start 2023-11-06T01:26:14.323 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.562+0000 7f3c1cd3f700 1 -- start start 2023-11-06T01:26:14.323 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.562+0000 7f3c1cd3f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c1819c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:14.324 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.562+0000 7f3c1cd3f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1819c8d0 con 0x7f3c18104d90 2023-11-06T01:26:14.324 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.562+0000 7f3c1659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c1819c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:14.324 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.562+0000 7f3c1659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c1819c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36832/0 (socket says 172.21.15.53:36832) 2023-11-06T01:26:14.324 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.562+0000 7f3c1659c700 1 -- 172.21.15.53:0/1706473666 learned_addr learned my addr 172.21.15.53:0/1706473666 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:14.324 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.563+0000 7f3c1659c700 1 -- 172.21.15.53:0/1706473666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c08009550 con 0x7f3c18104d90 2023-11-06T01:26:14.325 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.563+0000 7f3c1659c700 1 --2- 172.21.15.53:0/1706473666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c1819c390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3c08011980 tx=0x7f3c08011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:14.325 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.563+0000 7f3c077fe700 1 -- 172.21.15.53:0/1706473666 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c08003a60 con 0x7f3c18104d90 2023-11-06T01:26:14.325 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.563+0000 7f3c077fe700 1 -- 172.21.15.53:0/1706473666 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c08003bc0 con 0x7f3c18104d90 2023-11-06T01:26:14.325 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.563+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1706473666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1819cad0 con 0x7f3c18104d90 2023-11-06T01:26:14.325 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.563+0000 7f3c077fe700 1 -- 172.21.15.53:0/1706473666 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c0801a670 con 0x7f3c18104d90 2023-11-06T01:26:14.326 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.563+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1706473666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1819cf70 con 0x7f3c18104d90 2023-11-06T01:26:14.326 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.564+0000 7f3c077fe700 1 -- 172.21.15.53:0/1706473666 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f3c08024a00 con 0x7f3c18104d90 2023-11-06T01:26:14.326 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.564+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1706473666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c18195c10 con 0x7f3c18104d90 2023-11-06T01:26:14.326 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.565+0000 7f3c077fe700 1 --2- 172.21.15.53:0/1706473666 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f3c00038300 0x7f3c0003a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:14.326 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.565+0000 7f3c077fe700 1 -- 172.21.15.53:0/1706473666 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3c0804c2a0 con 0x7f3c18104d90 2023-11-06T01:26:14.327 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.565+0000 7f3c15d9b700 1 --2- 172.21.15.53:0/1706473666 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f3c00038300 0x7f3c0003a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:14.327 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.565+0000 7f3c15d9b700 1 --2- 172.21.15.53:0/1706473666 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f3c00038300 0x7f3c0003a7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c006fd0 tx=0x7f3c0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:14.327 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.569+0000 7f3c077fe700 1 -- 172.21.15.53:0/1706473666 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3c08024440 con 0x7f3c18104d90 2023-11-06T01:26:14.327 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.949+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1706473666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f3c1802cc70 con 0x7f3c18104d90 2023-11-06T01:26:14.327 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.952+0000 7f3c077fe700 1 -- 172.21.15.53:0/1706473666 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f3c08018350 con 0x7f3c18104d90 2023-11-06T01:26:14.328 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.954+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1706473666 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f3c00038300 msgr2=0x7f3c0003a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:14.328 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.954+0000 7f3c1cd3f700 1 --2- 172.21.15.53:0/1706473666 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f3c00038300 0x7f3c0003a7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c006fd0 tx=0x7f3c0c006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:14.328 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.954+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1706473666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 msgr2=0x7f3c1819c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:14.328 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.954+0000 7f3c1cd3f700 1 --2- 172.21.15.53:0/1706473666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c1819c390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3c08011980 tx=0x7f3c08011a60 comp rx=0 tx=0).stop 2023-11-06T01:26:14.328 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.955+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1706473666 shutdown_connections 2023-11-06T01:26:14.329 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.955+0000 7f3c1cd3f700 1 --2- 172.21.15.53:0/1706473666 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f3c00038300 0x7f3c0003a7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:14.329 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.955+0000 7f3c1cd3f700 1 --2- 172.21.15.53:0/1706473666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c18104d90 0x7f3c1819c390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:14.329 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.955+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1706473666 >> 172.21.15.53:0/1706473666 conn(0x7f3c181009b0 msgr2=0x7f3c18190950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:14.329 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.955+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1706473666 shutdown_connections 2023-11-06T01:26:14.329 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:13.955+0000 7f3c1cd3f700 1 -- 172.21.15.53:0/1706473666 wait complete. 2023-11-06T01:26:15.237 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:14 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:15.238 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:14 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1706473666' entity='client.admin' 2023-11-06T01:26:15.238 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:14 smithi053 ceph-mon[99472]: from='mgr.14120 172.21.15.53:0/1693620539' entity='mgr.x' 2023-11-06T01:26:15.566 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.828+0000 7f2b69066700 1 Processor -- start 2023-11-06T01:26:15.571 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.828+0000 7f2b69066700 1 -- start start 2023-11-06T01:26:15.572 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.828+0000 7f2b69066700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64105810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:15.572 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.828+0000 7f2b69066700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64105de0 con 0x7f2b641053f0 2023-11-06T01:26:15.572 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.829+0000 7f2b62d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64105810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:15.572 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.829+0000 7f2b62d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64105810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36836/0 (socket says 172.21.15.53:36836) 2023-11-06T01:26:15.572 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.829+0000 7f2b62d9d700 1 -- 172.21.15.53:0/428983308 learned_addr learned my addr 172.21.15.53:0/428983308 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:15.573 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.829+0000 7f2b62d9d700 1 -- 172.21.15.53:0/428983308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b641065f0 con 0x7f2b641053f0 2023-11-06T01:26:15.573 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.829+0000 7f2b62d9d700 1 --2- 172.21.15.53:0/428983308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64105810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b540098a0 tx=0x7f2b54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1567683f1a3cb6d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:15.573 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.829+0000 7f2b61d9b700 1 -- 172.21.15.53:0/428983308 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b54004160 con 0x7f2b641053f0 2023-11-06T01:26:15.573 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.829+0000 7f2b61d9b700 1 -- 172.21.15.53:0/428983308 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b540042c0 con 0x7f2b641053f0 2023-11-06T01:26:15.573 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.829+0000 7f2b61d9b700 1 -- 172.21.15.53:0/428983308 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b540119b0 con 0x7f2b641053f0 2023-11-06T01:26:15.574 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.830+0000 7f2b69066700 1 -- 172.21.15.53:0/428983308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 msgr2=0x7f2b64105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:15.574 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.830+0000 7f2b69066700 1 --2- 172.21.15.53:0/428983308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64105810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b540098a0 tx=0x7f2b54009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:15.574 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.830+0000 7f2b69066700 1 -- 172.21.15.53:0/428983308 shutdown_connections 2023-11-06T01:26:15.574 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.830+0000 7f2b69066700 1 --2- 172.21.15.53:0/428983308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64105810 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:15.574 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.830+0000 7f2b69066700 1 -- 172.21.15.53:0/428983308 >> 172.21.15.53:0/428983308 conn(0x7f2b641009b0 msgr2=0x7f2b64102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:15.575 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.830+0000 7f2b69066700 1 -- 172.21.15.53:0/428983308 shutdown_connections 2023-11-06T01:26:15.575 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.830+0000 7f2b69066700 1 -- 172.21.15.53:0/428983308 wait complete. 2023-11-06T01:26:15.575 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.831+0000 7f2b69066700 1 Processor -- start 2023-11-06T01:26:15.575 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.831+0000 7f2b69066700 1 -- start start 2023-11-06T01:26:15.575 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.831+0000 7f2b69066700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:15.576 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.831+0000 7f2b69066700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b641984b0 con 0x7f2b641053f0 2023-11-06T01:26:15.576 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.832+0000 7f2b62d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:15.576 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.832+0000 7f2b62d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36838/0 (socket says 172.21.15.53:36838) 2023-11-06T01:26:15.576 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.832+0000 7f2b62d9d700 1 -- 172.21.15.53:0/1996531853 learned_addr learned my addr 172.21.15.53:0/1996531853 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:15.576 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.832+0000 7f2b62d9d700 1 -- 172.21.15.53:0/1996531853 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b54009550 con 0x7f2b641053f0 2023-11-06T01:26:15.577 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.832+0000 7f2b62d9d700 1 --2- 172.21.15.53:0/1996531853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64197f70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2b54003e60 tx=0x7f2b54011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:15.577 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.832+0000 7f2b4bfff700 1 -- 172.21.15.53:0/1996531853 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b54003970 con 0x7f2b641053f0 2023-11-06T01:26:15.577 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.832+0000 7f2b4bfff700 1 -- 172.21.15.53:0/1996531853 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b54003ad0 con 0x7f2b641053f0 2023-11-06T01:26:15.577 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.832+0000 7f2b69066700 1 -- 172.21.15.53:0/1996531853 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b641986b0 con 0x7f2b641053f0 2023-11-06T01:26:15.578 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.832+0000 7f2b4bfff700 1 -- 172.21.15.53:0/1996531853 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b540195e0 con 0x7f2b641053f0 2023-11-06T01:26:15.578 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.832+0000 7f2b69066700 1 -- 172.21.15.53:0/1996531853 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b64198b50 con 0x7f2b641053f0 2023-11-06T01:26:15.578 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.833+0000 7f2b4bfff700 1 -- 172.21.15.53:0/1996531853 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f2b54019890 con 0x7f2b641053f0 2023-11-06T01:26:15.578 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.834+0000 7f2b4bfff700 1 --2- 172.21.15.53:0/1996531853 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2b4c03a500 0x7f2b4c03c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:15.578 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.834+0000 7f2b4bfff700 1 -- 172.21.15.53:0/1996531853 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2b5404d340 con 0x7f2b641053f0 2023-11-06T01:26:15.579 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.834+0000 7f2b69066700 1 -- 172.21.15.53:0/1996531853 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b64191c80 con 0x7f2b641053f0 2023-11-06T01:26:15.579 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.834+0000 7f2b6259c700 1 --2- 172.21.15.53:0/1996531853 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2b4c03a500 0x7f2b4c03c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:15.579 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.834+0000 7f2b6259c700 1 --2- 172.21.15.53:0/1996531853 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2b4c03a500 0x7f2b4c03c9c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2b58006fd0 tx=0x7f2b58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:15.579 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:14.839+0000 7f2b4bfff700 1 -- 172.21.15.53:0/1996531853 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2b54019b90 con 0x7f2b641053f0 2023-11-06T01:26:15.579 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.206+0000 7f2b69066700 1 -- 172.21.15.53:0/1996531853 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f2b6402d090 con 0x7f2b641053f0 2023-11-06T01:26:15.580 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.210+0000 7f2b4bfff700 1 -- 172.21.15.53:0/1996531853 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f2b54018350 con 0x7f2b641053f0 2023-11-06T01:26:15.580 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.212+0000 7f2b69066700 1 -- 172.21.15.53:0/1996531853 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2b4c03a500 msgr2=0x7f2b4c03c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:15.580 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.212+0000 7f2b69066700 1 --2- 172.21.15.53:0/1996531853 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2b4c03a500 0x7f2b4c03c9c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2b58006fd0 tx=0x7f2b58006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:15.580 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.212+0000 7f2b69066700 1 -- 172.21.15.53:0/1996531853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 msgr2=0x7f2b64197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:15.580 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.212+0000 7f2b69066700 1 --2- 172.21.15.53:0/1996531853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64197f70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2b54003e60 tx=0x7f2b54011aa0 comp rx=0 tx=0).stop 2023-11-06T01:26:15.581 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.212+0000 7f2b69066700 1 -- 172.21.15.53:0/1996531853 shutdown_connections 2023-11-06T01:26:15.581 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.212+0000 7f2b69066700 1 --2- 172.21.15.53:0/1996531853 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f2b4c03a500 0x7f2b4c03c9c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:15.581 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.212+0000 7f2b69066700 1 --2- 172.21.15.53:0/1996531853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b641053f0 0x7f2b64197f70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:15.581 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.212+0000 7f2b69066700 1 -- 172.21.15.53:0/1996531853 >> 172.21.15.53:0/1996531853 conn(0x7f2b641009b0 msgr2=0x7f2b6418edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:15.581 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.212+0000 7f2b69066700 1 -- 172.21.15.53:0/1996531853 shutdown_connections 2023-11-06T01:26:15.581 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:15.212+0000 7f2b69066700 1 -- 172.21.15.53:0/1996531853 wait complete. 2023-11-06T01:26:15.582 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling the dashboard module... 2023-11-06T01:26:16.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:16 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1996531853' entity='client.admin' 2023-11-06T01:26:17.526 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:17 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2497478104' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-11-06T01:26:17.527 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: ignoring --setuser ceph since I am not root 2023-11-06T01:26:17.527 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: ignoring --setgroup ceph since I am not root 2023-11-06T01:26:17.527 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:17.428+0000 7fb0ba2d4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-06T01:26:17.620 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.058+0000 7f549017b700 1 Processor -- start 2023-11-06T01:26:17.620 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.058+0000 7f549017b700 1 -- start start 2023-11-06T01:26:17.621 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.059+0000 7f549017b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488105810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:17.621 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.059+0000 7f549017b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488105de0 con 0x7f54881053f0 2023-11-06T01:26:17.621 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.059+0000 7f548df17700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488105810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:17.621 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.059+0000 7f548df17700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488105810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36842/0 (socket says 172.21.15.53:36842) 2023-11-06T01:26:17.622 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.059+0000 7f548df17700 1 -- 172.21.15.53:0/1773517542 learned_addr learned my addr 172.21.15.53:0/1773517542 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:17.622 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.059+0000 7f548df17700 1 -- 172.21.15.53:0/1773517542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54881065f0 con 0x7f54881053f0 2023-11-06T01:26:17.622 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.059+0000 7f548df17700 1 --2- 172.21.15.53:0/1773517542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488105810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f54840098a0 tx=0x7f5484009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5550bbe2fa4c8e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:17.622 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.060+0000 7f548cf15700 1 -- 172.21.15.53:0/1773517542 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5484004160 con 0x7f54881053f0 2023-11-06T01:26:17.622 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.060+0000 7f548cf15700 1 -- 172.21.15.53:0/1773517542 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54840042c0 con 0x7f54881053f0 2023-11-06T01:26:17.623 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.060+0000 7f548cf15700 1 -- 172.21.15.53:0/1773517542 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54840119b0 con 0x7f54881053f0 2023-11-06T01:26:17.623 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.060+0000 7f549017b700 1 -- 172.21.15.53:0/1773517542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 msgr2=0x7f5488105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:17.623 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.060+0000 7f549017b700 1 --2- 172.21.15.53:0/1773517542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488105810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f54840098a0 tx=0x7f5484009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:17.623 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.060+0000 7f549017b700 1 -- 172.21.15.53:0/1773517542 shutdown_connections 2023-11-06T01:26:17.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.060+0000 7f549017b700 1 --2- 172.21.15.53:0/1773517542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488105810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:17.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.060+0000 7f549017b700 1 -- 172.21.15.53:0/1773517542 >> 172.21.15.53:0/1773517542 conn(0x7f54881009b0 msgr2=0x7f5488102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:17.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.060+0000 7f549017b700 1 -- 172.21.15.53:0/1773517542 shutdown_connections 2023-11-06T01:26:17.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.061+0000 7f549017b700 1 -- 172.21.15.53:0/1773517542 wait complete. 2023-11-06T01:26:17.624 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.061+0000 7f549017b700 1 Processor -- start 2023-11-06T01:26:17.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.061+0000 7f549017b700 1 -- start start 2023-11-06T01:26:17.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.062+0000 7f549017b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:17.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.062+0000 7f549017b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54881995d0 con 0x7f54881053f0 2023-11-06T01:26:17.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.062+0000 7f548df17700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:17.625 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.062+0000 7f548df17700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36844/0 (socket says 172.21.15.53:36844) 2023-11-06T01:26:17.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.062+0000 7f548df17700 1 -- 172.21.15.53:0/2497478104 learned_addr learned my addr 172.21.15.53:0/2497478104 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:17.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.062+0000 7f548df17700 1 -- 172.21.15.53:0/2497478104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484009550 con 0x7f54881053f0 2023-11-06T01:26:17.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.062+0000 7f548df17700 1 --2- 172.21.15.53:0/2497478104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488199090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5484003e60 tx=0x7f5484011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:17.626 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.063+0000 7f547effd700 1 -- 172.21.15.53:0/2497478104 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5484003970 con 0x7f54881053f0 2023-11-06T01:26:17.627 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.063+0000 7f547effd700 1 -- 172.21.15.53:0/2497478104 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5484003ad0 con 0x7f54881053f0 2023-11-06T01:26:17.627 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.063+0000 7f549017b700 1 -- 172.21.15.53:0/2497478104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54881997d0 con 0x7f54881053f0 2023-11-06T01:26:17.627 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.063+0000 7f547effd700 1 -- 172.21.15.53:0/2497478104 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54840195e0 con 0x7f54881053f0 2023-11-06T01:26:17.627 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.063+0000 7f549017b700 1 -- 172.21.15.53:0/2497478104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5488199c70 con 0x7f54881053f0 2023-11-06T01:26:17.627 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.064+0000 7f547effd700 1 -- 172.21.15.53:0/2497478104 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f5484019850 con 0x7f54881053f0 2023-11-06T01:26:17.628 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.064+0000 7f549017b700 1 -- 172.21.15.53:0/2497478104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5488109820 con 0x7f54881053f0 2023-11-06T01:26:17.628 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.064+0000 7f547effd700 1 --2- 172.21.15.53:0/2497478104 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f547403a510 0x7f547403c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:17.628 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.064+0000 7f547effd700 1 -- 172.21.15.53:0/2497478104 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f548404c390 con 0x7f54881053f0 2023-11-06T01:26:17.628 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.064+0000 7f548d716700 1 --2- 172.21.15.53:0/2497478104 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f547403a510 0x7f547403c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:17.629 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.065+0000 7f548d716700 1 --2- 172.21.15.53:0/2497478104 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f547403a510 0x7f547403c9d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5478006fd0 tx=0x7f5478006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:17.629 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.070+0000 7f547effd700 1 -- 172.21.15.53:0/2497478104 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f548402b340 con 0x7f54881053f0 2023-11-06T01:26:17.629 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:16.551+0000 7f549017b700 1 -- 172.21.15.53:0/2497478104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f5488109a40 con 0x7f54881053f0 2023-11-06T01:26:17.629 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.213+0000 7f547effd700 1 -- 172.21.15.53:0/2497478104 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 9) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f548402d960 con 0x7f54881053f0 2023-11-06T01:26:17.629 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.215+0000 7f547effd700 1 -- 172.21.15.53:0/2497478104 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f5484003c40 con 0x7f54881053f0 2023-11-06T01:26:17.630 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.218+0000 7f549017b700 1 -- 172.21.15.53:0/2497478104 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f547403a510 msgr2=0x7f547403c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:17.630 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.218+0000 7f549017b700 1 --2- 172.21.15.53:0/2497478104 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f547403a510 0x7f547403c9d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5478006fd0 tx=0x7f5478006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:17.630 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.218+0000 7f549017b700 1 -- 172.21.15.53:0/2497478104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 msgr2=0x7f5488199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:17.630 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.218+0000 7f549017b700 1 --2- 172.21.15.53:0/2497478104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488199090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5484003e60 tx=0x7f5484011aa0 comp rx=0 tx=0).stop 2023-11-06T01:26:17.631 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.218+0000 7f549017b700 1 -- 172.21.15.53:0/2497478104 shutdown_connections 2023-11-06T01:26:17.631 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.218+0000 7f549017b700 1 --2- 172.21.15.53:0/2497478104 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7f547403a510 0x7f547403c9d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:17.631 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.218+0000 7f549017b700 1 --2- 172.21.15.53:0/2497478104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54881053f0 0x7f5488199090 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:17.631 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.218+0000 7f549017b700 1 -- 172.21.15.53:0/2497478104 >> 172.21.15.53:0/2497478104 conn(0x7f54881009b0 msgr2=0x7f5488101760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:17.631 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.219+0000 7f549017b700 1 -- 172.21.15.53:0/2497478104 shutdown_connections 2023-11-06T01:26:17.632 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:17.219+0000 7f549017b700 1 -- 172.21.15.53:0/2497478104 wait complete. 2023-11-06T01:26:17.844 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:17.524+0000 7fb0ba2d4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T01:26:17.845 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:17.636+0000 7fb0ba2d4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T01:26:17.845 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:17.756+0000 7fb0ba2d4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T01:26:18.214 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:17.972+0000 7fb0ba2d4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T01:26:18.594 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:18 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:18.451+0000 7fb0ba2d4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T01:26:18.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:18 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2497478104' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-11-06T01:26:18.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:18 smithi053 ceph-mon[99472]: mgrmap e9: x(active, since 17s) 2023-11-06T01:26:18.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2023-11-06T01:26:18.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-11-06T01:26:18.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-06T01:26:18.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-06T01:26:18.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-06T01:26:18.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2023-11-06T01:26:18.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.151+0000 7fc6dd1b2700 1 Processor -- start 2023-11-06T01:26:18.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.151+0000 7fc6dd1b2700 1 -- start start 2023-11-06T01:26:18.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.151+0000 7fc6dd1b2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:18.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.151+0000 7fc6dd1b2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d807a8d0 con 0x7fc6d807bea0 2023-11-06T01:26:18.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.151+0000 7fc6d6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:18.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.151+0000 7fc6d6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36858/0 (socket says 172.21.15.53:36858) 2023-11-06T01:26:18.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.151+0000 7fc6d6d9d700 1 -- 172.21.15.53:0/201495260 learned_addr learned my addr 172.21.15.53:0/201495260 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:18.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.151+0000 7fc6d6d9d700 1 -- 172.21.15.53:0/201495260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d807aa10 con 0x7fc6d807bea0 2023-11-06T01:26:18.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.152+0000 7fc6d6d9d700 1 --2- 172.21.15.53:0/201495260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d807a300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80098a0 tx=0x7fc6c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfcf0c99055ba40 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:18.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.152+0000 7fc6d5d9b700 1 -- 172.21.15.53:0/201495260 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6c8004160 con 0x7fc6d807bea0 2023-11-06T01:26:18.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.152+0000 7fc6d5d9b700 1 -- 172.21.15.53:0/201495260 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6c80042c0 con 0x7fc6d807bea0 2023-11-06T01:26:18.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.152+0000 7fc6d5d9b700 1 -- 172.21.15.53:0/201495260 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6c80119b0 con 0x7fc6d807bea0 2023-11-06T01:26:18.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.152+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/201495260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 msgr2=0x7fc6d807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:18.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.152+0000 7fc6dd1b2700 1 --2- 172.21.15.53:0/201495260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d807a300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80098a0 tx=0x7fc6c8009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:18.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.152+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/201495260 shutdown_connections 2023-11-06T01:26:18.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.152+0000 7fc6dd1b2700 1 --2- 172.21.15.53:0/201495260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d807a300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:18.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.152+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/201495260 >> 172.21.15.53:0/201495260 conn(0x7fc6d8101ad0 msgr2=0x7fc6d8103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:18.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.153+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/201495260 shutdown_connections 2023-11-06T01:26:18.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.153+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/201495260 wait complete. 2023-11-06T01:26:18.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.153+0000 7fc6dd1b2700 1 Processor -- start 2023-11-06T01:26:18.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.153+0000 7fc6dd1b2700 1 -- start start 2023-11-06T01:26:18.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.154+0000 7fc6dd1b2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d819c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:18.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.154+0000 7fc6dd1b2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d819c8a0 con 0x7fc6d807bea0 2023-11-06T01:26:18.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.154+0000 7fc6d6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:18.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.154+0000 7fc6d6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d819c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36860/0 (socket says 172.21.15.53:36860) 2023-11-06T01:26:18.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.154+0000 7fc6d6d9d700 1 -- 172.21.15.53:0/1110492733 learned_addr learned my addr 172.21.15.53:0/1110492733 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:18.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.154+0000 7fc6d6d9d700 1 -- 172.21.15.53:0/1110492733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c8009550 con 0x7fc6d807bea0 2023-11-06T01:26:18.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.154+0000 7fc6d6d9d700 1 --2- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d819c360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8000c00 tx=0x7fc6c8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:18.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.155+0000 7fc6bffff700 1 -- 172.21.15.53:0/1110492733 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6c80039f0 con 0x7fc6d807bea0 2023-11-06T01:26:18.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.155+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/1110492733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d819caa0 con 0x7fc6d807bea0 2023-11-06T01:26:18.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.155+0000 7fc6bffff700 1 -- 172.21.15.53:0/1110492733 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6c8003b50 con 0x7fc6d807bea0 2023-11-06T01:26:18.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.155+0000 7fc6bffff700 1 -- 172.21.15.53:0/1110492733 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6c801a780 con 0x7fc6d807bea0 2023-11-06T01:26:18.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.155+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/1110492733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d819cf40 con 0x7fc6d807bea0 2023-11-06T01:26:18.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.156+0000 7fc6bffff700 1 -- 172.21.15.53:0/1110492733 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 9) v1 ==== 45281+0+0 (secure 0 0 0) 0x7fc6c8024440 con 0x7fc6d807bea0 2023-11-06T01:26:18.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.156+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/1110492733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d810c8f0 con 0x7fc6d807bea0 2023-11-06T01:26:18.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.156+0000 7fc6bffff700 1 --2- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fc6c0038350 0x7fc6c003a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:18.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.156+0000 7fc6bffff700 1 -- 172.21.15.53:0/1110492733 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc6c804c2e0 con 0x7fc6d807bea0 2023-11-06T01:26:18.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.156+0000 7fc6d659c700 1 -- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fc6c0038350 msgr2=0x7fc6c003a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/2231847638 2023-11-06T01:26:18.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.156+0000 7fc6d659c700 1 --2- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fc6c0038350 0x7fc6c003a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T01:26:18.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.161+0000 7fc6bffff700 1 -- 172.21.15.53:0/1110492733 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc6c80189a0 con 0x7fc6d807bea0 2023-11-06T01:26:18.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.357+0000 7fc6d659c700 1 -- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fc6c0038350 msgr2=0x7fc6c003a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/2231847638 2023-11-06T01:26:18.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.357+0000 7fc6d659c700 1 --2- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fc6c0038350 0x7fc6c003a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T01:26:18.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.633+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/1110492733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc6d810cb10 con 0x7fc6d807bea0 2023-11-06T01:26:18.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.633+0000 7fc6bffff700 1 -- 172.21.15.53:0/1110492733 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fc6c8018350 con 0x7fc6d807bea0 2023-11-06T01:26:18.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.636+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fc6c0038350 msgr2=0x7fc6c003a810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:26:18.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.636+0000 7fc6dd1b2700 1 --2- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fc6c0038350 0x7fc6c003a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:18.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.636+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 msgr2=0x7fc6d819c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:18.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.636+0000 7fc6dd1b2700 1 --2- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d819c360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8000c00 tx=0x7fc6c8011a30 comp rx=0 tx=0).stop 2023-11-06T01:26:18.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.636+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/1110492733 shutdown_connections 2023-11-06T01:26:18.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.636+0000 7fc6dd1b2700 1 --2- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fc6c0038350 0x7fc6c003a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:18.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.636+0000 7fc6dd1b2700 1 --2- 172.21.15.53:0/1110492733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d807bea0 0x7fc6d819c360 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:18.993 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.636+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/1110492733 >> 172.21.15.53:0/1110492733 conn(0x7fc6d8101ad0 msgr2=0x7fc6d8102820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:18.993 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.636+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/1110492733 shutdown_connections 2023-11-06T01:26:18.993 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:18.636+0000 7fc6dd1b2700 1 -- 172.21.15.53:0/1110492733 wait complete. 2023-11-06T01:26:18.993 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for the mgr to restart... 2023-11-06T01:26:18.993 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr epoch 9... 2023-11-06T01:26:18.994 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:18 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:18.604+0000 7fb0ba2d4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T01:26:18.994 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:18 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:18.714+0000 7fb0ba2d4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T01:26:18.994 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:18 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:18.834+0000 7fb0ba2d4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T01:26:19.238 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:19 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1110492733' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-06T01:26:19.594 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:19 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:19.358+0000 7fb0ba2d4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T01:26:20.594 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:20 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:20.141+0000 7fb0ba2d4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T01:26:20.594 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:20 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:20.240+0000 7fb0ba2d4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T01:26:21.543 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:21 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:21.208+0000 7fb0ba2d4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T01:26:21.543 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:21 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:21.320+0000 7fb0ba2d4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-06T01:26:22.144 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:21 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:21.541+0000 7fb0ba2d4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T01:26:22.145 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:21 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:21.890+0000 7fb0ba2d4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-06T01:26:22.594 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:22 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:22.142+0000 7fb0ba2d4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T01:26:22.594 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:22 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:22.324+0000 7fb0ba2d4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T01:26:23.094 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:22 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:22.657+0000 7fb0ba2d4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T01:26:23.844 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:23 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:23.515+0000 7fb0ba2d4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T01:26:23.844 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:23 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:23.674+0000 7fb0ba2d4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T01:26:24.342 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:23 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:23.924+0000 7fb0ba2d4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T01:26:24.342 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:24 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:24.062+0000 7fb0ba2d4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T01:26:24.594 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:24 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:24.340+0000 7fb0ba2d4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T01:26:25.844 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:26:25.383+0000 7fb0ba2d4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T01:26:25.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: Active manager daemon x restarted 2023-11-06T01:26:25.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: Activating manager daemon x 2023-11-06T01:26:25.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: osdmap e3: 0 total, 0 up, 0 in 2023-11-06T01:26:25.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: mgrmap e10: x(active, starting, since 0.00324903s) 2023-11-06T01:26:25.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:26:25.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-06T01:26:25.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-06T01:26:25.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-06T01:26:25.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-06T01:26:25.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: Manager daemon x is now available 2023-11-06T01:26:25.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:25.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:26.771 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-06T01:26:26.771 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-06T01:26:26.771 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:26 smithi053 ceph-mon[99472]: mgrmap e11: x(active, since 1.00569s) 2023-11-06T01:26:26.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2023-11-06T01:26:26.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-11-06T01:26:26.842 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-06T01:26:26.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2023-11-06T01:26:26.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.449+0000 7fb01e6fd700 1 Processor -- start 2023-11-06T01:26:26.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.449+0000 7fb01e6fd700 1 -- start start 2023-11-06T01:26:26.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.449+0000 7fb01e6fd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb0181051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:26.843 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.449+0000 7fb01e6fd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018079e30 con 0x7fb018104db0 2023-11-06T01:26:26.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.450+0000 7fb017fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb0181051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:26.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.450+0000 7fb017fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb0181051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36868/0 (socket says 172.21.15.53:36868) 2023-11-06T01:26:26.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.450+0000 7fb017fff700 1 -- 172.21.15.53:0/2970408460 learned_addr learned my addr 172.21.15.53:0/2970408460 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:26.844 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.450+0000 7fb017fff700 1 -- 172.21.15.53:0/2970408460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb018105710 con 0x7fb018104db0 2023-11-06T01:26:26.845 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.450+0000 7fb017fff700 1 --2- 172.21.15.53:0/2970408460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb0181051d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb0080098a0 tx=0x7fb008009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1453184a65737b89 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:26.845 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.450+0000 7fb016ffd700 1 -- 172.21.15.53:0/2970408460 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb008004160 con 0x7fb018104db0 2023-11-06T01:26:26.845 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.450+0000 7fb016ffd700 1 -- 172.21.15.53:0/2970408460 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0080042c0 con 0x7fb018104db0 2023-11-06T01:26:26.845 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.450+0000 7fb016ffd700 1 -- 172.21.15.53:0/2970408460 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0080119b0 con 0x7fb018104db0 2023-11-06T01:26:26.845 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.451+0000 7fb01e6fd700 1 -- 172.21.15.53:0/2970408460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 msgr2=0x7fb0181051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:26.846 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.451+0000 7fb01e6fd700 1 --2- 172.21.15.53:0/2970408460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb0181051d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb0080098a0 tx=0x7fb008009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:26.846 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.451+0000 7fb01e6fd700 1 -- 172.21.15.53:0/2970408460 shutdown_connections 2023-11-06T01:26:26.846 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.451+0000 7fb01e6fd700 1 --2- 172.21.15.53:0/2970408460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb0181051d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:26.846 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.451+0000 7fb01e6fd700 1 -- 172.21.15.53:0/2970408460 >> 172.21.15.53:0/2970408460 conn(0x7fb0181009b0 msgr2=0x7fb018102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:26.846 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.451+0000 7fb01e6fd700 1 -- 172.21.15.53:0/2970408460 shutdown_connections 2023-11-06T01:26:26.847 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.451+0000 7fb01e6fd700 1 -- 172.21.15.53:0/2970408460 wait complete. 2023-11-06T01:26:26.847 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.452+0000 7fb01e6fd700 1 Processor -- start 2023-11-06T01:26:26.847 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.452+0000 7fb01e6fd700 1 -- start start 2023-11-06T01:26:26.847 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.452+0000 7fb01e6fd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb01819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:26.847 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.452+0000 7fb01e6fd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01819c890 con 0x7fb018104db0 2023-11-06T01:26:26.848 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.453+0000 7fb017fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb01819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:26.848 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.453+0000 7fb017fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb01819c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36870/0 (socket says 172.21.15.53:36870) 2023-11-06T01:26:26.848 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.453+0000 7fb017fff700 1 -- 172.21.15.53:0/2919496179 learned_addr learned my addr 172.21.15.53:0/2919496179 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:26.848 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.453+0000 7fb017fff700 1 -- 172.21.15.53:0/2919496179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb008009550 con 0x7fb018104db0 2023-11-06T01:26:26.848 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.453+0000 7fb017fff700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb01819c350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb008003e60 tx=0x7fb008011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:26.849 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.453+0000 7fb0157fa700 1 -- 172.21.15.53:0/2919496179 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb008003970 con 0x7fb018104db0 2023-11-06T01:26:26.849 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.453+0000 7fb0157fa700 1 -- 172.21.15.53:0/2919496179 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb008003ad0 con 0x7fb018104db0 2023-11-06T01:26:26.849 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.453+0000 7fb01e6fd700 1 -- 172.21.15.53:0/2919496179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01819ca90 con 0x7fb018104db0 2023-11-06T01:26:26.849 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.453+0000 7fb0157fa700 1 -- 172.21.15.53:0/2919496179 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0080195e0 con 0x7fb018104db0 2023-11-06T01:26:26.849 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.453+0000 7fb01e6fd700 1 -- 172.21.15.53:0/2919496179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01819cf30 con 0x7fb018104db0 2023-11-06T01:26:26.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.454+0000 7fb0157fa700 1 -- 172.21.15.53:0/2919496179 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 9) v1 ==== 45281+0+0 (secure 0 0 0) 0x7fb0080198e0 con 0x7fb018104db0 2023-11-06T01:26:26.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.454+0000 7fb0157fa700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 0x7fb00003a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:26.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.454+0000 7faffeffd700 1 -- 172.21.15.53:0/2919496179 --> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7faff8000d10 con 0x7fb000038300 2023-11-06T01:26:26.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.454+0000 7fb0157fa700 1 -- 172.21.15.53:0/2919496179 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb00804c680 con 0x7fb018104db0 2023-11-06T01:26:26.850 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.454+0000 7fb0177fe700 1 -- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 msgr2=0x7fb00003a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/2231847638 2023-11-06T01:26:26.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.455+0000 7fb0177fe700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 0x7fb00003a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T01:26:26.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.656+0000 7fb0177fe700 1 -- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 msgr2=0x7fb00003a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/2231847638 2023-11-06T01:26:26.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:19.656+0000 7fb0177fe700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 0x7fb00003a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T01:26:26.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:20.057+0000 7fb0177fe700 1 -- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 msgr2=0x7fb00003a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/2231847638 2023-11-06T01:26:26.851 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:20.057+0000 7fb0177fe700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 0x7fb00003a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-06T01:26:26.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:20.858+0000 7fb0177fe700 1 -- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 msgr2=0x7fb00003a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/2231847638 2023-11-06T01:26:26.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:20.858+0000 7fb0177fe700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 0x7fb00003a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-06T01:26:26.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:22.459+0000 7fb0177fe700 1 -- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 msgr2=0x7fb00003a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/2231847638 2023-11-06T01:26:26.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:22.459+0000 7fb0177fe700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 0x7fb00003a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-06T01:26:26.852 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:25.399+0000 7fb0157fa700 1 -- 172.21.15.53:0/2919496179 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mgrmap(e 10) v1 ==== 45048+0+0 (secure 0 0 0) 0x7fb00802daa0 con 0x7fb018104db0 2023-11-06T01:26:26.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:25.399+0000 7fb0157fa700 1 -- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 msgr2=0x7fb00003a7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:26:26.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:25.399+0000 7fb0157fa700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 0x7fb00003a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:26.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.400+0000 7fb0157fa700 1 -- 172.21.15.53:0/2919496179 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 11) v1 ==== 45175+0+0 (secure 0 0 0) 0x7fb00801aaf0 con 0x7fb018104db0 2023-11-06T01:26:26.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.400+0000 7fb0157fa700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb00003bbc0 0x7fb00003dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:26.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.401+0000 7fb0157fa700 1 -- 172.21.15.53:0/2919496179 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7faff8000d10 con 0x7fb00003bbc0 2023-11-06T01:26:26.853 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.401+0000 7fb0177fe700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb00003bbc0 0x7fb00003dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:26.854 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.402+0000 7fb0177fe700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb00003bbc0 0x7fb00003dfb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb00c003a10 tx=0x7fb00c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:26.854 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.402+0000 7fb0157fa700 1 -- 172.21.15.53:0/2919496179 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7faff8000d10 con 0x7fb00003bbc0 2023-11-06T01:26:26.854 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.413+0000 7faffeffd700 1 -- 172.21.15.53:0/2919496179 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7faff80027d0 con 0x7fb00003bbc0 2023-11-06T01:26:26.854 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.413+0000 7fb0157fa700 1 -- 172.21.15.53:0/2919496179 <== mgr.14152 v2:172.21.15.53:6800/1491433716 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7faff80027d0 con 0x7fb00003bbc0 2023-11-06T01:26:26.854 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.414+0000 7faffeffd700 1 -- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb00003bbc0 msgr2=0x7fb00003dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:26.855 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.414+0000 7faffeffd700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb00003bbc0 0x7fb00003dfb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb00c003a10 tx=0x7fb00c0092b0 comp rx=0 tx=0).stop 2023-11-06T01:26:26.855 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.414+0000 7faffeffd700 1 -- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 msgr2=0x7fb01819c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:26.855 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.414+0000 7faffeffd700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb01819c350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb008003e60 tx=0x7fb008011aa0 comp rx=0 tx=0).stop 2023-11-06T01:26:26.855 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.414+0000 7faffeffd700 1 -- 172.21.15.53:0/2919496179 shutdown_connections 2023-11-06T01:26:26.855 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.414+0000 7faffeffd700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb00003bbc0 0x7fb00003dfb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:26.856 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.414+0000 7faffeffd700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018104db0 0x7fb01819c350 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:26.856 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.414+0000 7faffeffd700 1 --2- 172.21.15.53:0/2919496179 >> [v2:172.21.15.53:6800/2231847638,v1:172.21.15.53:6801/2231847638] conn(0x7fb000038300 0x7fb00003a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:26.856 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.414+0000 7faffeffd700 1 -- 172.21.15.53:0/2919496179 >> 172.21.15.53:0/2919496179 conn(0x7fb0181009b0 msgr2=0x7fb018190970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:26.856 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.414+0000 7faffeffd700 1 -- 172.21.15.53:0/2919496179 shutdown_connections 2023-11-06T01:26:26.856 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:26.414+0000 7faffeffd700 1 -- 172.21.15.53:0/2919496179 wait complete. 2023-11-06T01:26:26.857 INFO:teuthology.orchestra.run.smithi053.stdout:mgr epoch 9 is available 2023-11-06T01:26:26.857 INFO:teuthology.orchestra.run.smithi053.stdout:Generating a dashboard self-signed certificate... 2023-11-06T01:26:28.289 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:28 smithi053 ceph-mon[99472]: [06/Nov/2023:01:26:27] ENGINE Bus STARTING 2023-11-06T01:26:28.290 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:28 smithi053 ceph-mon[99472]: [06/Nov/2023:01:26:27] ENGINE Serving on https://172.21.15.53:7150 2023-11-06T01:26:28.290 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:28.291 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:28 smithi053 ceph-mon[99472]: [06/Nov/2023:01:26:27] ENGINE Serving on http://172.21.15.53:8765 2023-11-06T01:26:28.291 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:28 smithi053 ceph-mon[99472]: [06/Nov/2023:01:26:27] ENGINE Bus STARTED 2023-11-06T01:26:28.291 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:28.292 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:28.292 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:28.338 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-11-06T01:26:28.339 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.362+0000 7fb52a717700 1 Processor -- start 2023-11-06T01:26:28.339 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.362+0000 7fb52a717700 1 -- start start 2023-11-06T01:26:28.339 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.362+0000 7fb52a717700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb524105810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:28.340 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.362+0000 7fb52a717700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524105de0 con 0x7fb5241053f0 2023-11-06T01:26:28.340 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.363+0000 7fb523fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb524105810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:28.340 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.363+0000 7fb523fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb524105810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36924/0 (socket says 172.21.15.53:36924) 2023-11-06T01:26:28.340 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.363+0000 7fb523fff700 1 -- 172.21.15.53:0/2495179845 learned_addr learned my addr 172.21.15.53:0/2495179845 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:28.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.363+0000 7fb523fff700 1 -- 172.21.15.53:0/2495179845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5241065f0 con 0x7fb5241053f0 2023-11-06T01:26:28.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.363+0000 7fb523fff700 1 --2- 172.21.15.53:0/2495179845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb524105810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5140098a0 tx=0x7fb514009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38010bafb9492d9f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:28.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.363+0000 7fb522ffd700 1 -- 172.21.15.53:0/2495179845 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb514004160 con 0x7fb5241053f0 2023-11-06T01:26:28.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.363+0000 7fb522ffd700 1 -- 172.21.15.53:0/2495179845 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5140042c0 con 0x7fb5241053f0 2023-11-06T01:26:28.341 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.363+0000 7fb522ffd700 1 -- 172.21.15.53:0/2495179845 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5140119b0 con 0x7fb5241053f0 2023-11-06T01:26:28.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.364+0000 7fb52a717700 1 -- 172.21.15.53:0/2495179845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 msgr2=0x7fb524105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:28.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.364+0000 7fb52a717700 1 --2- 172.21.15.53:0/2495179845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb524105810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5140098a0 tx=0x7fb514009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:28.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.364+0000 7fb52a717700 1 -- 172.21.15.53:0/2495179845 shutdown_connections 2023-11-06T01:26:28.342 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.364+0000 7fb52a717700 1 --2- 172.21.15.53:0/2495179845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb524105810 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:28.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.364+0000 7fb52a717700 1 -- 172.21.15.53:0/2495179845 >> 172.21.15.53:0/2495179845 conn(0x7fb5241009f0 msgr2=0x7fb524102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:28.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.364+0000 7fb52a717700 1 -- 172.21.15.53:0/2495179845 shutdown_connections 2023-11-06T01:26:28.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.364+0000 7fb52a717700 1 -- 172.21.15.53:0/2495179845 wait complete. 2023-11-06T01:26:28.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.365+0000 7fb52a717700 1 Processor -- start 2023-11-06T01:26:28.343 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.365+0000 7fb52a717700 1 -- start start 2023-11-06T01:26:28.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.365+0000 7fb52a717700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb52419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:28.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.365+0000 7fb52a717700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52419a6c0 con 0x7fb5241053f0 2023-11-06T01:26:28.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.366+0000 7fb523fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb52419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:28.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.366+0000 7fb523fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb52419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36926/0 (socket says 172.21.15.53:36926) 2023-11-06T01:26:28.344 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.366+0000 7fb523fff700 1 -- 172.21.15.53:0/2538639504 learned_addr learned my addr 172.21.15.53:0/2538639504 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:28.345 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.366+0000 7fb523fff700 1 -- 172.21.15.53:0/2538639504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb514009550 con 0x7fb5241053f0 2023-11-06T01:26:28.345 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.366+0000 7fb523fff700 1 --2- 172.21.15.53:0/2538639504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb52419a180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb514000c00 tx=0x7fb514011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:28.345 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.366+0000 7fb5217fa700 1 -- 172.21.15.53:0/2538639504 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5140039f0 con 0x7fb5241053f0 2023-11-06T01:26:28.345 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.366+0000 7fb52a717700 1 -- 172.21.15.53:0/2538639504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52419a8c0 con 0x7fb5241053f0 2023-11-06T01:26:28.345 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.366+0000 7fb5217fa700 1 -- 172.21.15.53:0/2538639504 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb514003b50 con 0x7fb5241053f0 2023-11-06T01:26:28.346 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.366+0000 7fb5217fa700 1 -- 172.21.15.53:0/2538639504 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb51401a780 con 0x7fb5241053f0 2023-11-06T01:26:28.346 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.366+0000 7fb52a717700 1 -- 172.21.15.53:0/2538639504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52419ad60 con 0x7fb5241053f0 2023-11-06T01:26:28.346 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.367+0000 7fb5217fa700 1 -- 172.21.15.53:0/2538639504 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 11) v1 ==== 45175+0+0 (secure 0 0 0) 0x7fb51401aa40 con 0x7fb5241053f0 2023-11-06T01:26:28.347 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.367+0000 7fb5217fa700 1 --2- 172.21.15.53:0/2538639504 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb50c038220 0x7fb50c03a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:28.347 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.368+0000 7fb5217fa700 1 -- 172.21.15.53:0/2538639504 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb51404c1a0 con 0x7fb5241053f0 2023-11-06T01:26:28.347 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.368+0000 7fb5237fe700 1 --2- 172.21.15.53:0/2538639504 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb50c038220 0x7fb50c03a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:28.348 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.368+0000 7fb52a717700 1 -- 172.21.15.53:0/2538639504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5240623c0 con 0x7fb5241053f0 2023-11-06T01:26:28.348 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.368+0000 7fb5237fe700 1 --2- 172.21.15.53:0/2538639504 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb50c038220 0x7fb50c03a6e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb518006fd0 tx=0x7fb518006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:28.348 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.373+0000 7fb5217fa700 1 -- 172.21.15.53:0/2538639504 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb51401ad40 con 0x7fb5241053f0 2023-11-06T01:26:28.349 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.765+0000 7fb52a717700 1 -- 172.21.15.53:0/2538639504 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb524102b80 con 0x7fb50c038220 2023-11-06T01:26:28.349 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.924+0000 7fb5217fa700 1 -- 172.21.15.53:0/2538639504 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb524102b80 con 0x7fb50c038220 2023-11-06T01:26:28.349 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.926+0000 7fb52a717700 1 -- 172.21.15.53:0/2538639504 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb50c038220 msgr2=0x7fb50c03a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:28.350 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.926+0000 7fb52a717700 1 --2- 172.21.15.53:0/2538639504 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb50c038220 0x7fb50c03a6e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb518006fd0 tx=0x7fb518006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:28.350 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.926+0000 7fb52a717700 1 -- 172.21.15.53:0/2538639504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 msgr2=0x7fb52419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:28.351 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.926+0000 7fb52a717700 1 --2- 172.21.15.53:0/2538639504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb52419a180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb514000c00 tx=0x7fb514011a30 comp rx=0 tx=0).stop 2023-11-06T01:26:28.351 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.926+0000 7fb52a717700 1 -- 172.21.15.53:0/2538639504 shutdown_connections 2023-11-06T01:26:28.351 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.926+0000 7fb52a717700 1 --2- 172.21.15.53:0/2538639504 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb50c038220 0x7fb50c03a6e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:28.352 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.926+0000 7fb52a717700 1 --2- 172.21.15.53:0/2538639504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5241053f0 0x7fb52419a180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:28.352 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.926+0000 7fb52a717700 1 -- 172.21.15.53:0/2538639504 >> 172.21.15.53:0/2538639504 conn(0x7fb5241009f0 msgr2=0x7fb524079840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:28.352 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.927+0000 7fb52a717700 1 -- 172.21.15.53:0/2538639504 shutdown_connections 2023-11-06T01:26:28.353 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:27.927+0000 7fb52a717700 1 -- 172.21.15.53:0/2538639504 wait complete. 2023-11-06T01:26:28.353 INFO:teuthology.orchestra.run.smithi053.stdout:Creating initial admin user... 2023-11-06T01:26:29.311 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:29 smithi053 ceph-mon[99472]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:29.311 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:29 smithi053 ceph-mon[99472]: mgrmap e12: x(active, since 2s) 2023-11-06T01:26:29.881 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1TmvDXi7tLZsrFYirKzmuewnFroGJlSVw565OjMNpytmEbXV/oBHe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1699233989, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-11-06T01:26:29.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.864+0000 7f492d2dd700 1 Processor -- start 2023-11-06T01:26:29.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.864+0000 7f492d2dd700 1 -- start start 2023-11-06T01:26:29.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.864+0000 7f492d2dd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f4928106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:29.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.864+0000 7f492d2dd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4928106ee0 con 0x7f49281064f0 2023-11-06T01:26:29.882 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.864+0000 7f4926ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f4928106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:29.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.864+0000 7f4926ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f4928106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36930/0 (socket says 172.21.15.53:36930) 2023-11-06T01:26:29.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.864+0000 7f4926ffd700 1 -- 172.21.15.53:0/2334485611 learned_addr learned my addr 172.21.15.53:0/2334485611 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:29.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.865+0000 7f4926ffd700 1 -- 172.21.15.53:0/2334485611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49281076f0 con 0x7f49281064f0 2023-11-06T01:26:29.883 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.865+0000 7f4926ffd700 1 --2- 172.21.15.53:0/2334485611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f4928106910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4918009e80 tx=0x7f491800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535646373ce00ea0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:29.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.865+0000 7f4925ffb700 1 -- 172.21.15.53:0/2334485611 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f491800e070 con 0x7f49281064f0 2023-11-06T01:26:29.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.865+0000 7f4925ffb700 1 -- 172.21.15.53:0/2334485611 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f491800b8c0 con 0x7f49281064f0 2023-11-06T01:26:29.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.865+0000 7f4925ffb700 1 -- 172.21.15.53:0/2334485611 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49180043e0 con 0x7f49281064f0 2023-11-06T01:26:29.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.866+0000 7f492d2dd700 1 -- 172.21.15.53:0/2334485611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 msgr2=0x7f4928106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:29.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.866+0000 7f492d2dd700 1 --2- 172.21.15.53:0/2334485611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f4928106910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4918009e80 tx=0x7f491800b330 comp rx=0 tx=0).stop 2023-11-06T01:26:29.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.866+0000 7f492d2dd700 1 -- 172.21.15.53:0/2334485611 shutdown_connections 2023-11-06T01:26:29.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.866+0000 7f492d2dd700 1 --2- 172.21.15.53:0/2334485611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f4928106910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:29.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.866+0000 7f492d2dd700 1 -- 172.21.15.53:0/2334485611 >> 172.21.15.53:0/2334485611 conn(0x7f4928101ad0 msgr2=0x7f4928103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:29.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.866+0000 7f492d2dd700 1 -- 172.21.15.53:0/2334485611 shutdown_connections 2023-11-06T01:26:29.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.866+0000 7f492d2dd700 1 -- 172.21.15.53:0/2334485611 wait complete. 2023-11-06T01:26:29.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.867+0000 7f492d2dd700 1 Processor -- start 2023-11-06T01:26:29.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.867+0000 7f492d2dd700 1 -- start start 2023-11-06T01:26:29.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.867+0000 7f492d2dd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f492819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:29.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.867+0000 7f492d2dd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492819a660 con 0x7f49281064f0 2023-11-06T01:26:29.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.867+0000 7f4926ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f492819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:29.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.867+0000 7f4926ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f492819a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36932/0 (socket says 172.21.15.53:36932) 2023-11-06T01:26:29.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.867+0000 7f4926ffd700 1 -- 172.21.15.53:0/1606561212 learned_addr learned my addr 172.21.15.53:0/1606561212 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:29.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.868+0000 7f4926ffd700 1 -- 172.21.15.53:0/1606561212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4918009b30 con 0x7f49281064f0 2023-11-06T01:26:29.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.868+0000 7f4926ffd700 1 --2- 172.21.15.53:0/1606561212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f492819a120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4918009530 tx=0x7f4918003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:29.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.868+0000 7f491ffff700 1 -- 172.21.15.53:0/1606561212 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f491800e040 con 0x7f49281064f0 2023-11-06T01:26:29.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.868+0000 7f492d2dd700 1 -- 172.21.15.53:0/1606561212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492819a860 con 0x7f49281064f0 2023-11-06T01:26:29.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.868+0000 7f491ffff700 1 -- 172.21.15.53:0/1606561212 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f491800bd70 con 0x7f49281064f0 2023-11-06T01:26:29.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.868+0000 7f491ffff700 1 -- 172.21.15.53:0/1606561212 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f491801c700 con 0x7f49281064f0 2023-11-06T01:26:29.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.868+0000 7f492d2dd700 1 -- 172.21.15.53:0/1606561212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492819ad60 con 0x7f49281064f0 2023-11-06T01:26:29.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.869+0000 7f491ffff700 1 -- 172.21.15.53:0/1606561212 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 12) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f491801c940 con 0x7f49281064f0 2023-11-06T01:26:29.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.869+0000 7f491ffff700 1 --2- 172.21.15.53:0/1606561212 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4914038350 0x7f491403a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:29.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.869+0000 7f491ffff700 1 -- 172.21.15.53:0/1606561212 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f491804c410 con 0x7f49281064f0 2023-11-06T01:26:29.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.869+0000 7f492d2dd700 1 -- 172.21.15.53:0/1606561212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492810a920 con 0x7f49281064f0 2023-11-06T01:26:29.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.869+0000 7f49267fc700 1 --2- 172.21.15.53:0/1606561212 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4914038350 0x7f491403a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:29.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.870+0000 7f49267fc700 1 --2- 172.21.15.53:0/1606561212 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4914038350 0x7f491403a810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4910006fd0 tx=0x7f4910006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:29.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:28.874+0000 7f491ffff700 1 -- 172.21.15.53:0/1606561212 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4918017080 con 0x7f49281064f0 2023-11-06T01:26:29.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.278+0000 7f492d2dd700 1 -- 172.21.15.53:0/1606561212 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f492810ab10 con 0x7f4914038350 2023-11-06T01:26:29.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.524+0000 7f491ffff700 1 -- 172.21.15.53:0/1606561212 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f492810ab10 con 0x7f4914038350 2023-11-06T01:26:29.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.526+0000 7f492d2dd700 1 -- 172.21.15.53:0/1606561212 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4914038350 msgr2=0x7f491403a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:29.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.526+0000 7f492d2dd700 1 --2- 172.21.15.53:0/1606561212 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4914038350 0x7f491403a810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4910006fd0 tx=0x7f4910006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:29.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.526+0000 7f492d2dd700 1 -- 172.21.15.53:0/1606561212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 msgr2=0x7f492819a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:29.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.526+0000 7f492d2dd700 1 --2- 172.21.15.53:0/1606561212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f492819a120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4918009530 tx=0x7f4918003d00 comp rx=0 tx=0).stop 2023-11-06T01:26:29.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.527+0000 7f492d2dd700 1 -- 172.21.15.53:0/1606561212 shutdown_connections 2023-11-06T01:26:29.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.527+0000 7f492d2dd700 1 --2- 172.21.15.53:0/1606561212 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4914038350 0x7f491403a810 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:29.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.527+0000 7f492d2dd700 1 --2- 172.21.15.53:0/1606561212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49281064f0 0x7f492819a120 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:29.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.527+0000 7f492d2dd700 1 -- 172.21.15.53:0/1606561212 >> 172.21.15.53:0/1606561212 conn(0x7f4928101ad0 msgr2=0x7f4928102840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:29.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.527+0000 7f492d2dd700 1 -- 172.21.15.53:0/1606561212 shutdown_connections 2023-11-06T01:26:29.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:29.527+0000 7f492d2dd700 1 -- 172.21.15.53:0/1606561212 wait complete. 2023-11-06T01:26:29.892 INFO:teuthology.orchestra.run.smithi053.stdout:Fetching dashboard port number... 2023-11-06T01:26:30.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:30 smithi053 ceph-mon[99472]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:30.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:31.026 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 8443 2023-11-06T01:26:31.026 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.332+0000 7f8b4e499700 1 Processor -- start 2023-11-06T01:26:31.026 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.332+0000 7f8b4e499700 1 -- start start 2023-11-06T01:26:31.027 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.332+0000 7f8b4e499700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b48104c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:31.027 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.332+0000 7f8b4e499700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481051e0 con 0x7f8b481047f0 2023-11-06T01:26:31.027 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.332+0000 7f8b47fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b48104c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:31.027 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.332+0000 7f8b47fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b48104c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36936/0 (socket says 172.21.15.53:36936) 2023-11-06T01:26:31.028 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.332+0000 7f8b47fff700 1 -- 172.21.15.53:0/3790758544 learned_addr learned my addr 172.21.15.53:0/3790758544 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:31.028 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.333+0000 7f8b47fff700 1 -- 172.21.15.53:0/3790758544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b48107780 con 0x7f8b481047f0 2023-11-06T01:26:31.028 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.333+0000 7f8b47fff700 1 --2- 172.21.15.53:0/3790758544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b48104c10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b380098a0 tx=0x7f8b38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a9c9c10f52bfbf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:31.028 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.333+0000 7f8b46ffd700 1 -- 172.21.15.53:0/3790758544 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b38004160 con 0x7f8b481047f0 2023-11-06T01:26:31.029 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.333+0000 7f8b46ffd700 1 -- 172.21.15.53:0/3790758544 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b380042c0 con 0x7f8b481047f0 2023-11-06T01:26:31.029 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.333+0000 7f8b46ffd700 1 -- 172.21.15.53:0/3790758544 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b380119b0 con 0x7f8b481047f0 2023-11-06T01:26:31.029 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.334+0000 7f8b4e499700 1 -- 172.21.15.53:0/3790758544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 msgr2=0x7f8b48104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:31.029 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.334+0000 7f8b4e499700 1 --2- 172.21.15.53:0/3790758544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b48104c10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b380098a0 tx=0x7f8b38009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:31.029 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.334+0000 7f8b4e499700 1 -- 172.21.15.53:0/3790758544 shutdown_connections 2023-11-06T01:26:31.030 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.334+0000 7f8b4e499700 1 --2- 172.21.15.53:0/3790758544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b48104c10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:31.030 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.334+0000 7f8b4e499700 1 -- 172.21.15.53:0/3790758544 >> 172.21.15.53:0/3790758544 conn(0x7f8b48100140 msgr2=0x7f8b48102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:31.030 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.334+0000 7f8b4e499700 1 -- 172.21.15.53:0/3790758544 shutdown_connections 2023-11-06T01:26:31.030 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.334+0000 7f8b4e499700 1 -- 172.21.15.53:0/3790758544 wait complete. 2023-11-06T01:26:31.030 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.334+0000 7f8b4e499700 1 Processor -- start 2023-11-06T01:26:31.031 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.334+0000 7f8b4e499700 1 -- start start 2023-11-06T01:26:31.031 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.335+0000 7f8b4e499700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b4819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:31.031 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.335+0000 7f8b4e499700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4819a6d0 con 0x7f8b481047f0 2023-11-06T01:26:31.031 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.335+0000 7f8b47fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b4819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:31.032 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.335+0000 7f8b47fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b4819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36938/0 (socket says 172.21.15.53:36938) 2023-11-06T01:26:31.032 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.335+0000 7f8b47fff700 1 -- 172.21.15.53:0/1483989577 learned_addr learned my addr 172.21.15.53:0/1483989577 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:31.032 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.335+0000 7f8b47fff700 1 -- 172.21.15.53:0/1483989577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38009550 con 0x7f8b481047f0 2023-11-06T01:26:31.032 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.335+0000 7f8b47fff700 1 --2- 172.21.15.53:0/1483989577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b4819a190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b38000c00 tx=0x7f8b38011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:31.033 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.336+0000 7f8b457fa700 1 -- 172.21.15.53:0/1483989577 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b380039f0 con 0x7f8b481047f0 2023-11-06T01:26:31.033 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.336+0000 7f8b4e499700 1 -- 172.21.15.53:0/1483989577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4819a8d0 con 0x7f8b481047f0 2023-11-06T01:26:31.033 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.336+0000 7f8b457fa700 1 -- 172.21.15.53:0/1483989577 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b38003b50 con 0x7f8b481047f0 2023-11-06T01:26:31.034 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.336+0000 7f8b457fa700 1 -- 172.21.15.53:0/1483989577 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b3801a780 con 0x7f8b481047f0 2023-11-06T01:26:31.034 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.336+0000 7f8b4e499700 1 -- 172.21.15.53:0/1483989577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4819ad70 con 0x7f8b481047f0 2023-11-06T01:26:31.034 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.337+0000 7f8b457fa700 1 -- 172.21.15.53:0/1483989577 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 12) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f8b3801a9f0 con 0x7f8b481047f0 2023-11-06T01:26:31.034 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.337+0000 7f8b457fa700 1 --2- 172.21.15.53:0/1483989577 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8b300383a0 0x7f8b3003a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:31.035 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.337+0000 7f8b457fa700 1 -- 172.21.15.53:0/1483989577 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8b3804c5a0 con 0x7f8b481047f0 2023-11-06T01:26:31.035 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.337+0000 7f8b477fe700 1 --2- 172.21.15.53:0/1483989577 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8b300383a0 0x7f8b3003a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:31.035 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.337+0000 7f8b4e499700 1 -- 172.21.15.53:0/1483989577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b480623c0 con 0x7f8b481047f0 2023-11-06T01:26:31.035 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.338+0000 7f8b477fe700 1 --2- 172.21.15.53:0/1483989577 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8b300383a0 0x7f8b3003a860 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c006fd0 tx=0x7f8b3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:31.036 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.342+0000 7f8b457fa700 1 -- 172.21.15.53:0/1483989577 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8b3801acf0 con 0x7f8b481047f0 2023-11-06T01:26:31.036 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.712+0000 7f8b4e499700 1 -- 172.21.15.53:0/1483989577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f8b4819d730 con 0x7f8b481047f0 2023-11-06T01:26:31.036 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.712+0000 7f8b457fa700 1 -- 172.21.15.53:0/1483989577 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8b3802c350 con 0x7f8b481047f0 2023-11-06T01:26:31.036 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.715+0000 7f8b4e499700 1 -- 172.21.15.53:0/1483989577 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8b300383a0 msgr2=0x7f8b3003a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:31.037 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.715+0000 7f8b4e499700 1 --2- 172.21.15.53:0/1483989577 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8b300383a0 0x7f8b3003a860 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c006fd0 tx=0x7f8b3c006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:31.037 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.715+0000 7f8b4e499700 1 -- 172.21.15.53:0/1483989577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 msgr2=0x7f8b4819a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:31.037 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.715+0000 7f8b4e499700 1 --2- 172.21.15.53:0/1483989577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b4819a190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b38000c00 tx=0x7f8b38011a30 comp rx=0 tx=0).stop 2023-11-06T01:26:31.038 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.715+0000 7f8b4e499700 1 -- 172.21.15.53:0/1483989577 shutdown_connections 2023-11-06T01:26:31.038 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.715+0000 7f8b4e499700 1 --2- 172.21.15.53:0/1483989577 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8b300383a0 0x7f8b3003a860 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:31.038 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.715+0000 7f8b4e499700 1 --2- 172.21.15.53:0/1483989577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481047f0 0x7f8b4819a190 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:31.038 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.715+0000 7f8b4e499700 1 -- 172.21.15.53:0/1483989577 >> 172.21.15.53:0/1483989577 conn(0x7f8b48100140 msgr2=0x7f8b48100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:31.039 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.715+0000 7f8b4e499700 1 -- 172.21.15.53:0/1483989577 shutdown_connections 2023-11-06T01:26:31.039 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:30.715+0000 7f8b4e499700 1 -- 172.21.15.53:0/1483989577 wait complete. 2023-11-06T01:26:31.040 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-06T01:26:31.040 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2023-11-06T01:26:31.041 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-06T01:26:31.041 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2023-11-06T01:26:31.041 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2023-11-06T01:26:31.041 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-11-06T01:26:31.042 INFO:teuthology.orchestra.run.smithi053.stdout:Ceph Dashboard is now available at: 2023-11-06T01:26:31.042 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:31.043 INFO:teuthology.orchestra.run.smithi053.stdout: URL: https://smithi053.front.sepia.ceph.com:8443/ 2023-11-06T01:26:31.043 INFO:teuthology.orchestra.run.smithi053.stdout: User: admin 2023-11-06T01:26:31.043 INFO:teuthology.orchestra.run.smithi053.stdout: Password: gznaeetc63 2023-11-06T01:26:31.043 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:31.043 INFO:teuthology.orchestra.run.smithi053.stdout:Saving cluster configuration to /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config directory 2023-11-06T01:26:31.044 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling autotune for osd_memory_target 2023-11-06T01:26:31.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:31 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1483989577' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-11-06T01:26:32.288 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.568+0000 7f4a356f0700 1 Processor -- start 2023-11-06T01:26:32.289 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.568+0000 7f4a356f0700 1 -- start start 2023-11-06T01:26:32.289 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.568+0000 7f4a356f0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a30104c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:32.289 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.568+0000 7f4a356f0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a301051e0 con 0x7f4a301047f0 2023-11-06T01:26:32.289 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.568+0000 7f4a2effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a30104c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:32.290 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.569+0000 7f4a2effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a30104c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36946/0 (socket says 172.21.15.53:36946) 2023-11-06T01:26:32.290 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.569+0000 7f4a2effd700 1 -- 172.21.15.53:0/1890161740 learned_addr learned my addr 172.21.15.53:0/1890161740 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:32.290 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.569+0000 7f4a2effd700 1 -- 172.21.15.53:0/1890161740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30107780 con 0x7f4a301047f0 2023-11-06T01:26:32.290 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.569+0000 7f4a2effd700 1 --2- 172.21.15.53:0/1890161740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a30104c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4a180098a0 tx=0x7f4a18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9696f31fc60e0d43 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:32.291 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.569+0000 7f4a2dffb700 1 -- 172.21.15.53:0/1890161740 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a18004160 con 0x7f4a301047f0 2023-11-06T01:26:32.291 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.569+0000 7f4a2dffb700 1 -- 172.21.15.53:0/1890161740 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a180042c0 con 0x7f4a301047f0 2023-11-06T01:26:32.291 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.569+0000 7f4a2dffb700 1 -- 172.21.15.53:0/1890161740 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a180119b0 con 0x7f4a301047f0 2023-11-06T01:26:32.291 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.570+0000 7f4a356f0700 1 -- 172.21.15.53:0/1890161740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 msgr2=0x7f4a30104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:32.291 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.570+0000 7f4a356f0700 1 --2- 172.21.15.53:0/1890161740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a30104c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4a180098a0 tx=0x7f4a18009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:32.292 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.570+0000 7f4a356f0700 1 -- 172.21.15.53:0/1890161740 shutdown_connections 2023-11-06T01:26:32.292 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.570+0000 7f4a356f0700 1 --2- 172.21.15.53:0/1890161740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a30104c10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:32.292 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.570+0000 7f4a356f0700 1 -- 172.21.15.53:0/1890161740 >> 172.21.15.53:0/1890161740 conn(0x7f4a30100140 msgr2=0x7f4a30102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:32.292 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.570+0000 7f4a356f0700 1 -- 172.21.15.53:0/1890161740 shutdown_connections 2023-11-06T01:26:32.293 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.570+0000 7f4a356f0700 1 -- 172.21.15.53:0/1890161740 wait complete. 2023-11-06T01:26:32.293 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.570+0000 7f4a356f0700 1 Processor -- start 2023-11-06T01:26:32.293 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.571+0000 7f4a356f0700 1 -- start start 2023-11-06T01:26:32.294 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.571+0000 7f4a356f0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a3019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:32.294 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.571+0000 7f4a356f0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3019a6d0 con 0x7f4a301047f0 2023-11-06T01:26:32.294 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.571+0000 7f4a2effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a3019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:32.294 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.571+0000 7f4a2effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a3019a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36948/0 (socket says 172.21.15.53:36948) 2023-11-06T01:26:32.294 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.571+0000 7f4a2effd700 1 -- 172.21.15.53:0/4289322439 learned_addr learned my addr 172.21.15.53:0/4289322439 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:32.295 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.571+0000 7f4a2effd700 1 -- 172.21.15.53:0/4289322439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a18009550 con 0x7f4a301047f0 2023-11-06T01:26:32.295 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.571+0000 7f4a2effd700 1 --2- 172.21.15.53:0/4289322439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a3019a190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a18003e60 tx=0x7f4a18011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:32.295 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.572+0000 7f4a27fff700 1 -- 172.21.15.53:0/4289322439 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a18003970 con 0x7f4a301047f0 2023-11-06T01:26:32.295 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.572+0000 7f4a27fff700 1 -- 172.21.15.53:0/4289322439 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a18003ad0 con 0x7f4a301047f0 2023-11-06T01:26:32.295 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.572+0000 7f4a356f0700 1 -- 172.21.15.53:0/4289322439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3019a8d0 con 0x7f4a301047f0 2023-11-06T01:26:32.296 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.572+0000 7f4a27fff700 1 -- 172.21.15.53:0/4289322439 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a180195e0 con 0x7f4a301047f0 2023-11-06T01:26:32.296 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.572+0000 7f4a356f0700 1 -- 172.21.15.53:0/4289322439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3019ad70 con 0x7f4a301047f0 2023-11-06T01:26:32.296 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.572+0000 7f4a27fff700 1 -- 172.21.15.53:0/4289322439 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 12) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f4a18019850 con 0x7f4a301047f0 2023-11-06T01:26:32.296 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.573+0000 7f4a27fff700 1 --2- 172.21.15.53:0/4289322439 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a1c038350 0x7f4a1c03a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:32.296 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.573+0000 7f4a27fff700 1 -- 172.21.15.53:0/4289322439 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4a1804c4a0 con 0x7f4a301047f0 2023-11-06T01:26:32.297 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.573+0000 7f4a2e7fc700 1 --2- 172.21.15.53:0/4289322439 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a1c038350 0x7f4a1c03a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:32.297 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.573+0000 7f4a356f0700 1 -- 172.21.15.53:0/4289322439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a300623c0 con 0x7f4a301047f0 2023-11-06T01:26:32.297 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.573+0000 7f4a2e7fc700 1 --2- 172.21.15.53:0/4289322439 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a1c038350 0x7f4a1c03a810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4a20006fd0 tx=0x7f4a20006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:32.297 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.578+0000 7f4a27fff700 1 -- 172.21.15.53:0/4289322439 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4a18045e60 con 0x7f4a301047f0 2023-11-06T01:26:32.297 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.587+0000 7f4a27fff700 1 -- 172.21.15.53:0/4289322439 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f4a1802d9b0 con 0x7f4a301047f0 2023-11-06T01:26:32.298 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.940+0000 7f4a356f0700 1 -- 172.21.15.53:0/4289322439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f4a3019d730 con 0x7f4a301047f0 2023-11-06T01:26:32.298 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.941+0000 7f4a27fff700 1 -- 172.21.15.53:0/4289322439 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4a18018d20 con 0x7f4a301047f0 2023-11-06T01:26:32.298 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.943+0000 7f4a356f0700 1 -- 172.21.15.53:0/4289322439 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a1c038350 msgr2=0x7f4a1c03a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:32.298 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.943+0000 7f4a356f0700 1 --2- 172.21.15.53:0/4289322439 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a1c038350 0x7f4a1c03a810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4a20006fd0 tx=0x7f4a20006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:32.298 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.943+0000 7f4a356f0700 1 -- 172.21.15.53:0/4289322439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 msgr2=0x7f4a3019a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:32.299 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.943+0000 7f4a356f0700 1 --2- 172.21.15.53:0/4289322439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a3019a190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a18003e60 tx=0x7f4a18011aa0 comp rx=0 tx=0).stop 2023-11-06T01:26:32.299 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.943+0000 7f4a356f0700 1 -- 172.21.15.53:0/4289322439 shutdown_connections 2023-11-06T01:26:32.299 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.943+0000 7f4a356f0700 1 --2- 172.21.15.53:0/4289322439 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a1c038350 0x7f4a1c03a810 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:32.299 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.943+0000 7f4a356f0700 1 --2- 172.21.15.53:0/4289322439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301047f0 0x7f4a3019a190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:32.299 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.943+0000 7f4a356f0700 1 -- 172.21.15.53:0/4289322439 >> 172.21.15.53:0/4289322439 conn(0x7f4a30100140 msgr2=0x7f4a30100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:32.300 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.943+0000 7f4a356f0700 1 -- 172.21.15.53:0/4289322439 shutdown_connections 2023-11-06T01:26:32.300 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:31.943+0000 7f4a356f0700 1 -- 172.21.15.53:0/4289322439 wait complete. 2023-11-06T01:26:32.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:32 smithi053 ceph-mon[99472]: mgrmap e13: x(active, since 6s) 2023-11-06T01:26:33.737 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.740+0000 7f2561f41700 1 Processor -- start 2023-11-06T01:26:33.737 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.740+0000 7f2561f41700 1 -- start start 2023-11-06T01:26:33.737 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.740+0000 7f2561f41700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c1093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:33.738 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.740+0000 7f2561f41700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c079e10 con 0x7f255c106fe0 2023-11-06T01:26:33.738 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.740+0000 7f255b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c1093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:33.738 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.740+0000 7f255b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c1093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36952/0 (socket says 172.21.15.53:36952) 2023-11-06T01:26:33.738 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.740+0000 7f255b7fe700 1 -- 172.21.15.53:0/1028125783 learned_addr learned my addr 172.21.15.53:0/1028125783 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:33.738 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.740+0000 7f255b7fe700 1 -- 172.21.15.53:0/1028125783 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c109910 con 0x7f255c106fe0 2023-11-06T01:26:33.739 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.741+0000 7f255b7fe700 1 --2- 172.21.15.53:0/1028125783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c1093d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f254c0098a0 tx=0x7f254c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76fc0aef5eaad0f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:33.739 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.741+0000 7f255a7fc700 1 -- 172.21.15.53:0/1028125783 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f254c004160 con 0x7f255c106fe0 2023-11-06T01:26:33.739 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.741+0000 7f255a7fc700 1 -- 172.21.15.53:0/1028125783 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254c0042c0 con 0x7f255c106fe0 2023-11-06T01:26:33.739 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.741+0000 7f255a7fc700 1 -- 172.21.15.53:0/1028125783 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f254c0119b0 con 0x7f255c106fe0 2023-11-06T01:26:33.740 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.741+0000 7f2561f41700 1 -- 172.21.15.53:0/1028125783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 msgr2=0x7f255c1093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:33.740 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.741+0000 7f2561f41700 1 --2- 172.21.15.53:0/1028125783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c1093d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f254c0098a0 tx=0x7f254c009bb0 comp rx=0 tx=0).stop 2023-11-06T01:26:33.740 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.741+0000 7f2561f41700 1 -- 172.21.15.53:0/1028125783 shutdown_connections 2023-11-06T01:26:33.740 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.741+0000 7f2561f41700 1 --2- 172.21.15.53:0/1028125783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c1093d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:33.740 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.741+0000 7f2561f41700 1 -- 172.21.15.53:0/1028125783 >> 172.21.15.53:0/1028125783 conn(0x7f255c1009b0 msgr2=0x7f255c102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:33.741 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.741+0000 7f2561f41700 1 -- 172.21.15.53:0/1028125783 shutdown_connections 2023-11-06T01:26:33.741 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.742+0000 7f2561f41700 1 -- 172.21.15.53:0/1028125783 wait complete. 2023-11-06T01:26:33.741 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.742+0000 7f2561f41700 1 Processor -- start 2023-11-06T01:26:33.741 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.742+0000 7f2561f41700 1 -- start start 2023-11-06T01:26:33.741 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f2561f41700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:33.742 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f2561f41700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c19a670 con 0x7f255c106fe0 2023-11-06T01:26:33.742 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f255b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:33.742 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f255b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36954/0 (socket says 172.21.15.53:36954) 2023-11-06T01:26:33.742 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f255b7fe700 1 -- 172.21.15.53:0/4170974394 learned_addr learned my addr 172.21.15.53:0/4170974394 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:33.742 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f255b7fe700 1 -- 172.21.15.53:0/4170974394 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c009550 con 0x7f255c106fe0 2023-11-06T01:26:33.743 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f255b7fe700 1 --2- 172.21.15.53:0/4170974394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c19a130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f254c011980 tx=0x7f254c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:33.743 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f2558ff9700 1 -- 172.21.15.53:0/4170974394 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f254c003a60 con 0x7f255c106fe0 2023-11-06T01:26:33.743 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f2558ff9700 1 -- 172.21.15.53:0/4170974394 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254c003bc0 con 0x7f255c106fe0 2023-11-06T01:26:33.743 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f2561f41700 1 -- 172.21.15.53:0/4170974394 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255c19a870 con 0x7f255c106fe0 2023-11-06T01:26:33.743 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f2558ff9700 1 -- 172.21.15.53:0/4170974394 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f254c01a670 con 0x7f255c106fe0 2023-11-06T01:26:33.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.743+0000 7f2561f41700 1 -- 172.21.15.53:0/4170974394 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255c19ad10 con 0x7f255c106fe0 2023-11-06T01:26:33.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.744+0000 7f2558ff9700 1 -- 172.21.15.53:0/4170974394 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f254c01a8e0 con 0x7f255c106fe0 2023-11-06T01:26:33.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.744+0000 7f2558ff9700 1 --2- 172.21.15.53:0/4170974394 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2544038440 0x7f254403a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:33.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.744+0000 7f2561f41700 1 -- 172.21.15.53:0/4170974394 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255c10a6a0 con 0x7f255c106fe0 2023-11-06T01:26:33.744 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.745+0000 7f2558ff9700 1 -- 172.21.15.53:0/4170974394 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f254c04c5b0 con 0x7f255c106fe0 2023-11-06T01:26:33.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.745+0000 7f255affd700 1 --2- 172.21.15.53:0/4170974394 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2544038440 0x7f254403a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:33.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.745+0000 7f255affd700 1 --2- 172.21.15.53:0/4170974394 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2544038440 0x7f254403a900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2550006fd0 tx=0x7f2550006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:33.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:32.749+0000 7f2558ff9700 1 -- 172.21.15.53:0/4170974394 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f254c018b40 con 0x7f255c106fe0 2023-11-06T01:26:33.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.327+0000 7f2561f41700 1 -- 172.21.15.53:0/4170974394 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f255c19daf0 con 0x7f255c106fe0 2023-11-06T01:26:33.745 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.330+0000 7f2558ff9700 1 -- 172.21.15.53:0/4170974394 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f254c018b40 con 0x7f255c106fe0 2023-11-06T01:26:33.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.332+0000 7f2561f41700 1 -- 172.21.15.53:0/4170974394 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2544038440 msgr2=0x7f254403a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:33.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.332+0000 7f2561f41700 1 --2- 172.21.15.53:0/4170974394 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2544038440 0x7f254403a900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2550006fd0 tx=0x7f2550006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:33.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.332+0000 7f2561f41700 1 -- 172.21.15.53:0/4170974394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 msgr2=0x7f255c19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:33.746 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.332+0000 7f2561f41700 1 --2- 172.21.15.53:0/4170974394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c19a130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f254c011980 tx=0x7f254c011a60 comp rx=0 tx=0).stop 2023-11-06T01:26:33.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.332+0000 7f2561f41700 1 -- 172.21.15.53:0/4170974394 shutdown_connections 2023-11-06T01:26:33.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.332+0000 7f2561f41700 1 --2- 172.21.15.53:0/4170974394 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2544038440 0x7f254403a900 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:33.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.332+0000 7f2561f41700 1 --2- 172.21.15.53:0/4170974394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c106fe0 0x7f255c19a130 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:33.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.332+0000 7f2561f41700 1 -- 172.21.15.53:0/4170974394 >> 172.21.15.53:0/4170974394 conn(0x7f255c1009b0 msgr2=0x7f255c101720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:33.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.332+0000 7f2561f41700 1 -- 172.21.15.53:0/4170974394 shutdown_connections 2023-11-06T01:26:33.747 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2023-11-06T01:26:33.332+0000 7f2561f41700 1 -- 172.21.15.53:0/4170974394 wait complete. 2023-11-06T01:26:33.748 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-11-06T01:26:33.748 INFO:teuthology.orchestra.run.smithi053.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-11-06T01:26:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-11-06T01:26:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:33.749 INFO:teuthology.orchestra.run.smithi053.stdout:Or, if you are only running a single cluster on this host: 2023-11-06T01:26:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-11-06T01:26:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:33.749 INFO:teuthology.orchestra.run.smithi053.stdout:Please consider enabling telemetry to help improve Ceph: 2023-11-06T01:26:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: ceph telemetry on 2023-11-06T01:26:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:33.750 INFO:teuthology.orchestra.run.smithi053.stdout:For more information see: 2023-11-06T01:26:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-11-06T01:26:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:33.751 INFO:teuthology.orchestra.run.smithi053.stdout:Bootstrap complete. 2023-11-06T01:26:33.770 INFO:tasks.cephadm:Fetching config... 2023-11-06T01:26:33.771 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:26:33.771 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-11-06T01:26:33.789 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-11-06T01:26:33.789 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:26:33.790 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-11-06T01:26:33.848 INFO:tasks.cephadm:Fetching mon keyring... 2023-11-06T01:26:33.849 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:26:33.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-11-06T01:26:33.918 INFO:tasks.cephadm:Fetching pub ssh key... 2023-11-06T01:26:33.919 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:26:33.919 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-11-06T01:26:33.976 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-11-06T01:26:33.977 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgjBbf6Kp8aICaklGQQMvixLLKgGoExmEJdJwu19q6vSKgZRfS2FRiGwTMeHaGZTFiw7QnLLRBGAm+wYyVTWbV/H1E6aZMhCvBxPXpz99LMvenUaW7hO+E4LO8ipR6BAN9VWuYFspN/42KyGxR7dldOpfLJ66hk/Tv8UJ15TN3YRhmalZXajGR2p1GWRvPLTztiINVyN/tgwQnqCCr0/P0b96SQPOgt/0QQ2qg+H2sZW1vS2AmnMETTRXg5Rm+zvO5Sx73duya5w9JX8LIZGJEgD5adEzjwSccK1TKRWeRVzVB8804G9e9P+U+IpSUt1uHd26gOkpXyDSHbMvVNIWxYRxQtu4LNjAbkuJfWXadUV7qtfYqd8cvzANLPrvG8lQdgaYNCmjpFbHQToVH9OcTzCKV5QmAxshfvAL0VRZYrfndSdh3gQ3LScJezXMfGBImpzaJtf5mCeiIcohyD5mLpTocyHWXDmagUX15qO+zPv/yFZtC8BzPNvtUQoTl/u0= ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-06T01:26:34.056 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgjBbf6Kp8aICaklGQQMvixLLKgGoExmEJdJwu19q6vSKgZRfS2FRiGwTMeHaGZTFiw7QnLLRBGAm+wYyVTWbV/H1E6aZMhCvBxPXpz99LMvenUaW7hO+E4LO8ipR6BAN9VWuYFspN/42KyGxR7dldOpfLJ66hk/Tv8UJ15TN3YRhmalZXajGR2p1GWRvPLTztiINVyN/tgwQnqCCr0/P0b96SQPOgt/0QQ2qg+H2sZW1vS2AmnMETTRXg5Rm+zvO5Sx73duya5w9JX8LIZGJEgD5adEzjwSccK1TKRWeRVzVB8804G9e9P+U+IpSUt1uHd26gOkpXyDSHbMvVNIWxYRxQtu4LNjAbkuJfWXadUV7qtfYqd8cvzANLPrvG8lQdgaYNCmjpFbHQToVH9OcTzCKV5QmAxshfvAL0VRZYrfndSdh3gQ3LScJezXMfGBImpzaJtf5mCeiIcohyD5mLpTocyHWXDmagUX15qO+zPv/yFZtC8BzPNvtUQoTl/u0= ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:26:34.068 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgjBbf6Kp8aICaklGQQMvixLLKgGoExmEJdJwu19q6vSKgZRfS2FRiGwTMeHaGZTFiw7QnLLRBGAm+wYyVTWbV/H1E6aZMhCvBxPXpz99LMvenUaW7hO+E4LO8ipR6BAN9VWuYFspN/42KyGxR7dldOpfLJ66hk/Tv8UJ15TN3YRhmalZXajGR2p1GWRvPLTztiINVyN/tgwQnqCCr0/P0b96SQPOgt/0QQ2qg+H2sZW1vS2AmnMETTRXg5Rm+zvO5Sx73duya5w9JX8LIZGJEgD5adEzjwSccK1TKRWeRVzVB8804G9e9P+U+IpSUt1uHd26gOkpXyDSHbMvVNIWxYRxQtu4LNjAbkuJfWXadUV7qtfYqd8cvzANLPrvG8lQdgaYNCmjpFbHQToVH9OcTzCKV5QmAxshfvAL0VRZYrfndSdh3gQ3LScJezXMfGBImpzaJtf5mCeiIcohyD5mLpTocyHWXDmagUX15qO+zPv/yFZtC8BzPNvtUQoTl/u0= ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-06T01:26:34.116 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgjBbf6Kp8aICaklGQQMvixLLKgGoExmEJdJwu19q6vSKgZRfS2FRiGwTMeHaGZTFiw7QnLLRBGAm+wYyVTWbV/H1E6aZMhCvBxPXpz99LMvenUaW7hO+E4LO8ipR6BAN9VWuYFspN/42KyGxR7dldOpfLJ66hk/Tv8UJ15TN3YRhmalZXajGR2p1GWRvPLTztiINVyN/tgwQnqCCr0/P0b96SQPOgt/0QQ2qg+H2sZW1vS2AmnMETTRXg5Rm+zvO5Sx73duya5w9JX8LIZGJEgD5adEzjwSccK1TKRWeRVzVB8804G9e9P+U+IpSUt1uHd26gOkpXyDSHbMvVNIWxYRxQtu4LNjAbkuJfWXadUV7qtfYqd8cvzANLPrvG8lQdgaYNCmjpFbHQToVH9OcTzCKV5QmAxshfvAL0VRZYrfndSdh3gQ3LScJezXMfGBImpzaJtf5mCeiIcohyD5mLpTocyHWXDmagUX15qO+zPv/yFZtC8BzPNvtUQoTl/u0= ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:26:34.130 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgjBbf6Kp8aICaklGQQMvixLLKgGoExmEJdJwu19q6vSKgZRfS2FRiGwTMeHaGZTFiw7QnLLRBGAm+wYyVTWbV/H1E6aZMhCvBxPXpz99LMvenUaW7hO+E4LO8ipR6BAN9VWuYFspN/42KyGxR7dldOpfLJ66hk/Tv8UJ15TN3YRhmalZXajGR2p1GWRvPLTztiINVyN/tgwQnqCCr0/P0b96SQPOgt/0QQ2qg+H2sZW1vS2AmnMETTRXg5Rm+zvO5Sx73duya5w9JX8LIZGJEgD5adEzjwSccK1TKRWeRVzVB8804G9e9P+U+IpSUt1uHd26gOkpXyDSHbMvVNIWxYRxQtu4LNjAbkuJfWXadUV7qtfYqd8cvzANLPrvG8lQdgaYNCmjpFbHQToVH9OcTzCKV5QmAxshfvAL0VRZYrfndSdh3gQ3LScJezXMfGBImpzaJtf5mCeiIcohyD5mLpTocyHWXDmagUX15qO+zPv/yFZtC8BzPNvtUQoTl/u0= ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-06T01:26:34.177 INFO:teuthology.orchestra.run.smithi163.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgjBbf6Kp8aICaklGQQMvixLLKgGoExmEJdJwu19q6vSKgZRfS2FRiGwTMeHaGZTFiw7QnLLRBGAm+wYyVTWbV/H1E6aZMhCvBxPXpz99LMvenUaW7hO+E4LO8ipR6BAN9VWuYFspN/42KyGxR7dldOpfLJ66hk/Tv8UJ15TN3YRhmalZXajGR2p1GWRvPLTztiINVyN/tgwQnqCCr0/P0b96SQPOgt/0QQ2qg+H2sZW1vS2AmnMETTRXg5Rm+zvO5Sx73duya5w9JX8LIZGJEgD5adEzjwSccK1TKRWeRVzVB8804G9e9P+U+IpSUt1uHd26gOkpXyDSHbMvVNIWxYRxQtu4LNjAbkuJfWXadUV7qtfYqd8cvzANLPrvG8lQdgaYNCmjpFbHQToVH9OcTzCKV5QmAxshfvAL0VRZYrfndSdh3gQ3LScJezXMfGBImpzaJtf5mCeiIcohyD5mLpTocyHWXDmagUX15qO+zPv/yFZtC8BzPNvtUQoTl/u0= ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:26:34.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-11-06T01:26:34.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:34 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/4170974394' entity='client.admin' 2023-11-06T01:26:34.755 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:26:36.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.119+0000 7f09f6ecd700 1 -- 172.21.15.53:0/646534810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f00ff790 msgr2=0x7f09f00ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:36.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.119+0000 7f09f6ecd700 1 --2- 172.21.15.53:0/646534810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f00ff790 0x7f09f00ffbb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009910 tx=0x7f09e4009c20 comp rx=0 tx=0).stop 2023-11-06T01:26:36.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.119+0000 7f09f6ecd700 1 -- 172.21.15.53:0/646534810 shutdown_connections 2023-11-06T01:26:36.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.119+0000 7f09f6ecd700 1 --2- 172.21.15.53:0/646534810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f00ff790 0x7f09f00ffbb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:36.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.119+0000 7f09f6ecd700 1 -- 172.21.15.53:0/646534810 >> 172.21.15.53:0/646534810 conn(0x7f09f00fad30 msgr2=0x7f09f00fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:36.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.119+0000 7f09f6ecd700 1 -- 172.21.15.53:0/646534810 shutdown_connections 2023-11-06T01:26:36.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.119+0000 7f09f6ecd700 1 -- 172.21.15.53:0/646534810 wait complete. 2023-11-06T01:26:36.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.119+0000 7f09f6ecd700 1 Processor -- start 2023-11-06T01:26:36.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.119+0000 7f09f6ecd700 1 -- start start 2023-11-06T01:26:36.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.120+0000 7f09f6ecd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f00ff790 0x7f09f0195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:36.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.120+0000 7f09f6ecd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f0195eb0 con 0x7f09f00ff790 2023-11-06T01:26:36.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.120+0000 7f09f4c69700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f00ff790 0x7f09f0195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:36.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.120+0000 7f09f4c69700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f00ff790 0x7f09f0195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36960/0 (socket says 172.21.15.53:36960) 2023-11-06T01:26:36.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.120+0000 7f09f4c69700 1 -- 172.21.15.53:0/3430573779 learned_addr learned my addr 172.21.15.53:0/3430573779 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:36.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.120+0000 7f09f4c69700 1 -- 172.21.15.53:0/3430573779 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e40095f0 con 0x7f09f00ff790 2023-11-06T01:26:36.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.120+0000 7f09f4c69700 1 --2- 172.21.15.53:0/3430573779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f00ff790 0x7f09f0195970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f09e4006010 tx=0x7f09e40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:36.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.120+0000 7f09edffb700 1 -- 172.21.15.53:0/3430573779 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09e401d070 con 0x7f09f00ff790 2023-11-06T01:26:36.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.120+0000 7f09f6ecd700 1 -- 172.21.15.53:0/3430573779 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f01960b0 con 0x7f09f00ff790 2023-11-06T01:26:36.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.120+0000 7f09edffb700 1 -- 172.21.15.53:0/3430573779 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09e40044e0 con 0x7f09f00ff790 2023-11-06T01:26:36.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.120+0000 7f09edffb700 1 -- 172.21.15.53:0/3430573779 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09e40039b0 con 0x7f09f00ff790 2023-11-06T01:26:36.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.121+0000 7f09f6ecd700 1 -- 172.21.15.53:0/3430573779 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f01964f0 con 0x7f09f00ff790 2023-11-06T01:26:36.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.122+0000 7f09edffb700 1 -- 172.21.15.53:0/3430573779 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f09e4003c20 con 0x7f09f00ff790 2023-11-06T01:26:36.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.122+0000 7f09f6ecd700 1 -- 172.21.15.53:0/3430573779 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f018f2b0 con 0x7f09f00ff790 2023-11-06T01:26:36.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.122+0000 7f09edffb700 1 --2- 172.21.15.53:0/3430573779 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f09e00383f0 0x7f09e003a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:36.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.122+0000 7f09edffb700 1 -- 172.21.15.53:0/3430573779 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f09e404c850 con 0x7f09f00ff790 2023-11-06T01:26:36.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.122+0000 7f09effff700 1 --2- 172.21.15.53:0/3430573779 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f09e00383f0 0x7f09e003a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:36.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.122+0000 7f09effff700 1 --2- 172.21.15.53:0/3430573779 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f09e00383f0 0x7f09e003a8b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f09dc006fd0 tx=0x7f09dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:36.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.127+0000 7f09edffb700 1 -- 172.21.15.53:0/3430573779 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f09e402a950 con 0x7f09f00ff790 2023-11-06T01:26:36.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.327+0000 7f09f6ecd700 1 -- 172.21.15.53:0/3430573779 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f09f002d0e0 con 0x7f09f00ff790 2023-11-06T01:26:36.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.331+0000 7f09edffb700 1 -- 172.21.15.53:0/3430573779 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f09e402a360 con 0x7f09f00ff790 2023-11-06T01:26:36.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.333+0000 7f09f6ecd700 1 -- 172.21.15.53:0/3430573779 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f09e00383f0 msgr2=0x7f09e003a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:36.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.333+0000 7f09f6ecd700 1 --2- 172.21.15.53:0/3430573779 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f09e00383f0 0x7f09e003a8b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f09dc006fd0 tx=0x7f09dc006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:36.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.333+0000 7f09f6ecd700 1 -- 172.21.15.53:0/3430573779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f00ff790 msgr2=0x7f09f0195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:36.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.333+0000 7f09f6ecd700 1 --2- 172.21.15.53:0/3430573779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f00ff790 0x7f09f0195970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f09e4006010 tx=0x7f09e40057c0 comp rx=0 tx=0).stop 2023-11-06T01:26:36.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.333+0000 7f09f6ecd700 1 -- 172.21.15.53:0/3430573779 shutdown_connections 2023-11-06T01:26:36.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.333+0000 7f09f6ecd700 1 --2- 172.21.15.53:0/3430573779 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f09e00383f0 0x7f09e003a8b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:36.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.333+0000 7f09f6ecd700 1 --2- 172.21.15.53:0/3430573779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f00ff790 0x7f09f0195970 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:36.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.333+0000 7f09f6ecd700 1 -- 172.21.15.53:0/3430573779 >> 172.21.15.53:0/3430573779 conn(0x7f09f00fad30 msgr2=0x7f09f0069110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:36.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.333+0000 7f09f6ecd700 1 -- 172.21.15.53:0/3430573779 shutdown_connections 2023-11-06T01:26:36.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:36.333+0000 7f09f6ecd700 1 -- 172.21.15.53:0/3430573779 wait complete. 2023-11-06T01:26:36.741 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-11-06T01:26:36.742 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-11-06T01:26:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:26:37.583 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:37 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3430573779' entity='client.admin' 2023-11-06T01:26:37.583 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:37.583 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:37.583 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:26:37.584 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:37.584 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:37.584 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:38.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.507+0000 7f47010f5700 1 -- 172.21.15.53:0/1568355833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46fc0ff780 msgr2=0x7f46fc0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:38.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.507+0000 7f47010f5700 1 --2- 172.21.15.53:0/1568355833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46fc0ff780 0x7f46fc0ffba0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009910 tx=0x7f46e4009c20 comp rx=0 tx=0).stop 2023-11-06T01:26:38.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.507+0000 7f47010f5700 1 -- 172.21.15.53:0/1568355833 shutdown_connections 2023-11-06T01:26:38.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.507+0000 7f47010f5700 1 --2- 172.21.15.53:0/1568355833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46fc0ff780 0x7f46fc0ffba0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:38.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.507+0000 7f47010f5700 1 -- 172.21.15.53:0/1568355833 >> 172.21.15.53:0/1568355833 conn(0x7f46fc0fad20 msgr2=0x7f46fc0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:38.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.507+0000 7f47010f5700 1 -- 172.21.15.53:0/1568355833 shutdown_connections 2023-11-06T01:26:38.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.507+0000 7f47010f5700 1 -- 172.21.15.53:0/1568355833 wait complete. 2023-11-06T01:26:38.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.508+0000 7f47010f5700 1 Processor -- start 2023-11-06T01:26:38.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.508+0000 7f47010f5700 1 -- start start 2023-11-06T01:26:38.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.508+0000 7f47010f5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46fc0ff780 0x7f46fc068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:38.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.508+0000 7f47010f5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc1986c0 con 0x7f46fc0ff780 2023-11-06T01:26:38.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.508+0000 7f46fad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46fc0ff780 0x7f46fc068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:38.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.508+0000 7f46fad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46fc0ff780 0x7f46fc068b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36966/0 (socket says 172.21.15.53:36966) 2023-11-06T01:26:38.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.508+0000 7f46fad9d700 1 -- 172.21.15.53:0/1620962332 learned_addr learned my addr 172.21.15.53:0/1620962332 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:38.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.508+0000 7f46fad9d700 1 -- 172.21.15.53:0/1620962332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e40095f0 con 0x7f46fc0ff780 2023-11-06T01:26:38.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.509+0000 7f46fad9d700 1 --2- 172.21.15.53:0/1620962332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46fc0ff780 0x7f46fc068b10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f46e4006010 tx=0x7f46e40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:38.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.509+0000 7f46f3fff700 1 -- 172.21.15.53:0/1620962332 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46e401d070 con 0x7f46fc0ff780 2023-11-06T01:26:38.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.509+0000 7f46f3fff700 1 -- 172.21.15.53:0/1620962332 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46e40044e0 con 0x7f46fc0ff780 2023-11-06T01:26:38.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.509+0000 7f47010f5700 1 -- 172.21.15.53:0/1620962332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc198800 con 0x7f46fc0ff780 2023-11-06T01:26:38.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.509+0000 7f46f3fff700 1 -- 172.21.15.53:0/1620962332 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46e40039b0 con 0x7f46fc0ff780 2023-11-06T01:26:38.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.509+0000 7f47010f5700 1 -- 172.21.15.53:0/1620962332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc198c40 con 0x7f46fc0ff780 2023-11-06T01:26:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.510+0000 7f47010f5700 1 -- 172.21.15.53:0/1620962332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc18e290 con 0x7f46fc0ff780 2023-11-06T01:26:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.510+0000 7f46f3fff700 1 -- 172.21.15.53:0/1620962332 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f46e400baf0 con 0x7f46fc0ff780 2023-11-06T01:26:38.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.510+0000 7f46f3fff700 1 --2- 172.21.15.53:0/1620962332 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f46e80383a0 0x7f46e803a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:38.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.510+0000 7f46f3fff700 1 -- 172.21.15.53:0/1620962332 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f46e404c890 con 0x7f46fc0ff780 2023-11-06T01:26:38.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.510+0000 7f46fa59c700 1 --2- 172.21.15.53:0/1620962332 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f46e80383a0 0x7f46e803a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:38.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.511+0000 7f46fa59c700 1 --2- 172.21.15.53:0/1620962332 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f46e80383a0 0x7f46e803a860 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f46ec006fd0 tx=0x7f46ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.515+0000 7f46f3fff700 1 -- 172.21.15.53:0/1620962332 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f46e400e9b0 con 0x7f46fc0ff780 2023-11-06T01:26:38.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.716+0000 7f47010f5700 1 -- 172.21.15.53:0/1620962332 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f46fc061260 con 0x7f46e80383a0 2023-11-06T01:26:38.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.720+0000 7f46f3fff700 1 -- 172.21.15.53:0/1620962332 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f46fc061260 con 0x7f46e80383a0 2023-11-06T01:26:38.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.722+0000 7f47010f5700 1 -- 172.21.15.53:0/1620962332 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f46e80383a0 msgr2=0x7f46e803a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:38.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.722+0000 7f47010f5700 1 --2- 172.21.15.53:0/1620962332 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f46e80383a0 0x7f46e803a860 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f46ec006fd0 tx=0x7f46ec006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:38.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.722+0000 7f47010f5700 1 -- 172.21.15.53:0/1620962332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46fc0ff780 msgr2=0x7f46fc068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:38.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.722+0000 7f47010f5700 1 --2- 172.21.15.53:0/1620962332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46fc0ff780 0x7f46fc068b10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f46e4006010 tx=0x7f46e40057c0 comp rx=0 tx=0).stop 2023-11-06T01:26:38.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.722+0000 7f47010f5700 1 -- 172.21.15.53:0/1620962332 shutdown_connections 2023-11-06T01:26:38.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.722+0000 7f47010f5700 1 --2- 172.21.15.53:0/1620962332 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f46e80383a0 0x7f46e803a860 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:38.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.722+0000 7f47010f5700 1 --2- 172.21.15.53:0/1620962332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46fc0ff780 0x7f46fc068b10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:38.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.722+0000 7f47010f5700 1 -- 172.21.15.53:0/1620962332 >> 172.21.15.53:0/1620962332 conn(0x7f46fc0fad20 msgr2=0x7f46fc0fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:38.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.722+0000 7f47010f5700 1 -- 172.21.15.53:0/1620962332 shutdown_connections 2023-11-06T01:26:38.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:38.722+0000 7f47010f5700 1 -- 172.21.15.53:0/1620962332 wait complete. 2023-11-06T01:26:39.176 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi062 2023-11-06T01:26:39.176 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-11-06T01:26:39.176 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/etc/ceph/ceph.conf 2023-11-06T01:26:39.198 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-11-06T01:26:39.198 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:26:39.258 INFO:tasks.cephadm:Adding host smithi062 to orchestrator... 2023-11-06T01:26:39.259 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch host add smithi062 2023-11-06T01:26:39.835 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:26:40.016 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:39 smithi053 ceph-mon[99472]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:40.016 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:40.016 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:40.017 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:26:40.017 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:26:40.017 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:39 smithi053 ceph-mon[99472]: Updating smithi053:/etc/ceph/ceph.conf 2023-11-06T01:26:40.017 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:39 smithi053 ceph-mon[99472]: Updating smithi053:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:26:40.017 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:39 smithi053 ceph-mon[99472]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:26:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.023+0000 7ffbb7d40700 1 -- 172.21.15.53:0/1834588922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0105780 msgr2=0x7ffbb0105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.023+0000 7ffbb7d40700 1 --2- 172.21.15.53:0/1834588922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0105780 0x7ffbb0105b60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009910 tx=0x7ffbac009c20 comp rx=0 tx=0).stop 2023-11-06T01:26:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.023+0000 7ffbb7d40700 1 -- 172.21.15.53:0/1834588922 shutdown_connections 2023-11-06T01:26:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.023+0000 7ffbb7d40700 1 --2- 172.21.15.53:0/1834588922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0105780 0x7ffbb0105b60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:41.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.023+0000 7ffbb7d40700 1 -- 172.21.15.53:0/1834588922 >> 172.21.15.53:0/1834588922 conn(0x7ffbb0075290 msgr2=0x7ffbb00756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:41.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.023+0000 7ffbb7d40700 1 -- 172.21.15.53:0/1834588922 shutdown_connections 2023-11-06T01:26:41.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.023+0000 7ffbb7d40700 1 -- 172.21.15.53:0/1834588922 wait complete. 2023-11-06T01:26:41.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.024+0000 7ffbb7d40700 1 Processor -- start 2023-11-06T01:26:41.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.024+0000 7ffbb7d40700 1 -- start start 2023-11-06T01:26:41.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.024+0000 7ffbb7d40700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0105780 0x7ffbb0196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:41.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.024+0000 7ffbb7d40700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb01970c0 con 0x7ffbb0105780 2023-11-06T01:26:41.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.024+0000 7ffbb5adc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0105780 0x7ffbb0196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:41.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.024+0000 7ffbb5adc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0105780 0x7ffbb0196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36972/0 (socket says 172.21.15.53:36972) 2023-11-06T01:26:41.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.024+0000 7ffbb5adc700 1 -- 172.21.15.53:0/3359321946 learned_addr learned my addr 172.21.15.53:0/3359321946 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:41.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.025+0000 7ffbb5adc700 1 -- 172.21.15.53:0/3359321946 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac0095f0 con 0x7ffbb0105780 2023-11-06T01:26:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.025+0000 7ffbb5adc700 1 --2- 172.21.15.53:0/3359321946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0105780 0x7ffbb0196b80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffbac006010 tx=0x7ffbac004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.025+0000 7ffba6ffd700 1 -- 172.21.15.53:0/3359321946 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac01d070 con 0x7ffbb0105780 2023-11-06T01:26:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.025+0000 7ffba6ffd700 1 -- 172.21.15.53:0/3359321946 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbac0044a0 con 0x7ffbb0105780 2023-11-06T01:26:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.025+0000 7ffbb7d40700 1 -- 172.21.15.53:0/3359321946 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb01972c0 con 0x7ffbb0105780 2023-11-06T01:26:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.025+0000 7ffba6ffd700 1 -- 172.21.15.53:0/3359321946 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac003a70 con 0x7ffbb0105780 2023-11-06T01:26:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.025+0000 7ffbb7d40700 1 -- 172.21.15.53:0/3359321946 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb01976e0 con 0x7ffbb0105780 2023-11-06T01:26:41.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.026+0000 7ffba6ffd700 1 -- 172.21.15.53:0/3359321946 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7ffbac003c60 con 0x7ffbb0105780 2023-11-06T01:26:41.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.026+0000 7ffba6ffd700 1 --2- 172.21.15.53:0/3359321946 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffb9c0383f0 0x7ffb9c03a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:41.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.026+0000 7ffbb7d40700 1 -- 172.21.15.53:0/3359321946 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb01039c0 con 0x7ffbb0105780 2023-11-06T01:26:41.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.026+0000 7ffba6ffd700 1 -- 172.21.15.53:0/3359321946 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffbac04c160 con 0x7ffbb0105780 2023-11-06T01:26:41.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.026+0000 7ffbb52db700 1 --2- 172.21.15.53:0/3359321946 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffb9c0383f0 0x7ffb9c03a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:41.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.027+0000 7ffbb52db700 1 --2- 172.21.15.53:0/3359321946 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffb9c0383f0 0x7ffb9c03a8b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffba0006fd0 tx=0x7ffba0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:41.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.031+0000 7ffba6ffd700 1 -- 172.21.15.53:0/3359321946 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffbac026030 con 0x7ffbb0105780 2023-11-06T01:26:41.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:40 smithi053 ceph-mon[99472]: Updating smithi053:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-06T01:26:41.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:41.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:41.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:41.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:41.235+0000 7ffbb7d40700 1 -- 172.21.15.53:0/3359321946 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi062", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbb00685a0 con 0x7ffb9c0383f0 2023-11-06T01:26:42.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:41 smithi053 ceph-mon[99472]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:43.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:42 smithi053 ceph-mon[99472]: Deploying cephadm binary to smithi062 2023-11-06T01:26:44.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:44.080+0000 7ffba6ffd700 1 -- 172.21.15.53:0/3359321946 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7ffbb00685a0 con 0x7ffb9c0383f0 2023-11-06T01:26:44.081 INFO:teuthology.orchestra.run.smithi053.stdout:Added host 'smithi062' with addr '172.21.15.62' 2023-11-06T01:26:44.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:44.082+0000 7ffbb7d40700 1 -- 172.21.15.53:0/3359321946 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffb9c0383f0 msgr2=0x7ffb9c03a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:44.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:44.082+0000 7ffbb7d40700 1 --2- 172.21.15.53:0/3359321946 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffb9c0383f0 0x7ffb9c03a8b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffba0006fd0 tx=0x7ffba0006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:44.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:44.082+0000 7ffbb7d40700 1 -- 172.21.15.53:0/3359321946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0105780 msgr2=0x7ffbb0196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:44.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:44.082+0000 7ffbb7d40700 1 --2- 172.21.15.53:0/3359321946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0105780 0x7ffbb0196b80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffbac006010 tx=0x7ffbac004ff0 comp rx=0 tx=0).stop 2023-11-06T01:26:44.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:44.082+0000 7ffbb7d40700 1 -- 172.21.15.53:0/3359321946 shutdown_connections 2023-11-06T01:26:44.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:44.082+0000 7ffbb7d40700 1 --2- 172.21.15.53:0/3359321946 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffb9c0383f0 0x7ffb9c03a8b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:44.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:44.082+0000 7ffbb7d40700 1 --2- 172.21.15.53:0/3359321946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0105780 0x7ffbb0196b80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:44.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:44.082+0000 7ffbb7d40700 1 -- 172.21.15.53:0/3359321946 >> 172.21.15.53:0/3359321946 conn(0x7ffbb0075290 msgr2=0x7ffbb00fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:44.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:44.082+0000 7ffbb7d40700 1 -- 172.21.15.53:0/3359321946 shutdown_connections 2023-11-06T01:26:44.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:44.083+0000 7ffbb7d40700 1 -- 172.21.15.53:0/3359321946 wait complete. 2023-11-06T01:26:44.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch host ls --format=json 2023-11-06T01:26:45.053 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:26:45.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:45.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:45 smithi053 ceph-mon[99472]: Added host smithi062 2023-11-06T01:26:45.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:45.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:46.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.105+0000 7f227d5f8700 1 -- 172.21.15.53:0/3006596293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2278105780 msgr2=0x7f2278105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:46.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.105+0000 7f227d5f8700 1 --2- 172.21.15.53:0/3006596293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2278105780 0x7f2278105b60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2268009910 tx=0x7f2268009c20 comp rx=0 tx=0).stop 2023-11-06T01:26:46.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.105+0000 7f227d5f8700 1 -- 172.21.15.53:0/3006596293 shutdown_connections 2023-11-06T01:26:46.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.105+0000 7f227d5f8700 1 --2- 172.21.15.53:0/3006596293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2278105780 0x7f2278105b60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:46.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.105+0000 7f227d5f8700 1 -- 172.21.15.53:0/3006596293 >> 172.21.15.53:0/3006596293 conn(0x7f2278075290 msgr2=0x7f22780756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:46.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.105+0000 7f227d5f8700 1 -- 172.21.15.53:0/3006596293 shutdown_connections 2023-11-06T01:26:46.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.105+0000 7f227d5f8700 1 -- 172.21.15.53:0/3006596293 wait complete. 2023-11-06T01:26:46.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.106+0000 7f227d5f8700 1 Processor -- start 2023-11-06T01:26:46.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.106+0000 7f227d5f8700 1 -- start start 2023-11-06T01:26:46.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.106+0000 7f227d5f8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2278105780 0x7f2278195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:46.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.106+0000 7f227d5f8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2278196330 con 0x7f2278105780 2023-11-06T01:26:46.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.106+0000 7f2276ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2278105780 0x7f2278195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:46.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.106+0000 7f2276ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2278105780 0x7f2278195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36998/0 (socket says 172.21.15.53:36998) 2023-11-06T01:26:46.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.106+0000 7f2276ffd700 1 -- 172.21.15.53:0/1660307649 learned_addr learned my addr 172.21.15.53:0/1660307649 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:46.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.106+0000 7f2276ffd700 1 -- 172.21.15.53:0/1660307649 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22680095f0 con 0x7f2278105780 2023-11-06T01:26:46.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.107+0000 7f2276ffd700 1 --2- 172.21.15.53:0/1660307649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2278105780 0x7f2278195df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2268005850 tx=0x7f2268004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:46.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.107+0000 7f226ffff700 1 -- 172.21.15.53:0/1660307649 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f226801d070 con 0x7f2278105780 2023-11-06T01:26:46.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.107+0000 7f226ffff700 1 -- 172.21.15.53:0/1660307649 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2268004370 con 0x7f2278105780 2023-11-06T01:26:46.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.107+0000 7f227d5f8700 1 -- 172.21.15.53:0/1660307649 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2278196530 con 0x7f2278105780 2023-11-06T01:26:46.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.107+0000 7f226ffff700 1 -- 172.21.15.53:0/1660307649 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2268003ab0 con 0x7f2278105780 2023-11-06T01:26:46.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.107+0000 7f227d5f8700 1 -- 172.21.15.53:0/1660307649 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2278196950 con 0x7f2278105780 2023-11-06T01:26:46.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.108+0000 7f226ffff700 1 -- 172.21.15.53:0/1660307649 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f2268003d20 con 0x7f2278105780 2023-11-06T01:26:46.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.108+0000 7f227d5f8700 1 -- 172.21.15.53:0/1660307649 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22780623c0 con 0x7f2278105780 2023-11-06T01:26:46.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.108+0000 7f226ffff700 1 --2- 172.21.15.53:0/1660307649 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f22640383f0 0x7f226403a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:46.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.108+0000 7f226ffff700 1 -- 172.21.15.53:0/1660307649 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f226804c160 con 0x7f2278105780 2023-11-06T01:26:46.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.108+0000 7f22767fc700 1 --2- 172.21.15.53:0/1660307649 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f22640383f0 0x7f226403a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:46.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.109+0000 7f22767fc700 1 --2- 172.21.15.53:0/1660307649 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f22640383f0 0x7f226403a8b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2260006fd0 tx=0x7f2260006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:46.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.113+0000 7f226ffff700 1 -- 172.21.15.53:0/1660307649 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2268029770 con 0x7f2278105780 2023-11-06T01:26:46.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.318+0000 7f227d5f8700 1 -- 172.21.15.53:0/1660307649 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22780686a0 con 0x7f22640383f0 2023-11-06T01:26:46.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.320+0000 7f226ffff700 1 -- 172.21.15.53:0/1660307649 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f22780686a0 con 0x7f22640383f0 2023-11-06T01:26:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:46.321 INFO:teuthology.orchestra.run.smithi053.stdout:[{"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}] 2023-11-06T01:26:46.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.322+0000 7f227d5f8700 1 -- 172.21.15.53:0/1660307649 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f22640383f0 msgr2=0x7f226403a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.322+0000 7f227d5f8700 1 --2- 172.21.15.53:0/1660307649 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f22640383f0 0x7f226403a8b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2260006fd0 tx=0x7f2260006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.322+0000 7f227d5f8700 1 -- 172.21.15.53:0/1660307649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2278105780 msgr2=0x7f2278195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.322+0000 7f227d5f8700 1 --2- 172.21.15.53:0/1660307649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2278105780 0x7f2278195df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2268005850 tx=0x7f2268004ff0 comp rx=0 tx=0).stop 2023-11-06T01:26:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.322+0000 7f227d5f8700 1 -- 172.21.15.53:0/1660307649 shutdown_connections 2023-11-06T01:26:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.322+0000 7f227d5f8700 1 --2- 172.21.15.53:0/1660307649 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f22640383f0 0x7f226403a8b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:46.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.322+0000 7f227d5f8700 1 --2- 172.21.15.53:0/1660307649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2278105780 0x7f2278195df0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:46.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.322+0000 7f227d5f8700 1 -- 172.21.15.53:0/1660307649 >> 172.21.15.53:0/1660307649 conn(0x7f2278075290 msgr2=0x7f2278067f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:46.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.322+0000 7f227d5f8700 1 -- 172.21.15.53:0/1660307649 shutdown_connections 2023-11-06T01:26:46.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:46.322+0000 7f227d5f8700 1 -- 172.21.15.53:0/1660307649 wait complete. 2023-11-06T01:26:46.467 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:46.698 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi163 2023-11-06T01:26:46.698 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-11-06T01:26:46.699 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.conf 2023-11-06T01:26:46.720 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-11-06T01:26:46.720 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:26:46.779 INFO:tasks.cephadm:Adding host smithi163 to orchestrator... 2023-11-06T01:26:46.780 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch host add smithi163 2023-11-06T01:26:47.338 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:26:47.523 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:47 smithi053 ceph-mon[99472]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:26:47.524 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:47 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.526+0000 7f115af0e700 1 -- 172.21.15.53:0/1697088797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1154105780 msgr2=0x7f1154105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.526+0000 7f115af0e700 1 --2- 172.21.15.53:0/1697088797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1154105780 0x7f1154105b60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1148009910 tx=0x7f1148009c20 comp rx=0 tx=0).stop 2023-11-06T01:26:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.526+0000 7f115af0e700 1 -- 172.21.15.53:0/1697088797 shutdown_connections 2023-11-06T01:26:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.526+0000 7f115af0e700 1 --2- 172.21.15.53:0/1697088797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1154105780 0x7f1154105b60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.526+0000 7f115af0e700 1 -- 172.21.15.53:0/1697088797 >> 172.21.15.53:0/1697088797 conn(0x7f1154075290 msgr2=0x7f11540756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:48.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.526+0000 7f115af0e700 1 -- 172.21.15.53:0/1697088797 shutdown_connections 2023-11-06T01:26:48.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.526+0000 7f115af0e700 1 -- 172.21.15.53:0/1697088797 wait complete. 2023-11-06T01:26:48.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.526+0000 7f115af0e700 1 Processor -- start 2023-11-06T01:26:48.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.526+0000 7f115af0e700 1 -- start start 2023-11-06T01:26:48.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.527+0000 7f115af0e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1154105780 0x7f1154195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:48.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.527+0000 7f115af0e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1154196330 con 0x7f1154105780 2023-11-06T01:26:48.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.527+0000 7f1158caa700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1154105780 0x7f1154195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:48.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.527+0000 7f1158caa700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1154105780 0x7f1154195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37004/0 (socket says 172.21.15.53:37004) 2023-11-06T01:26:48.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.527+0000 7f1158caa700 1 -- 172.21.15.53:0/1015200838 learned_addr learned my addr 172.21.15.53:0/1015200838 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:48.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.527+0000 7f1158caa700 1 -- 172.21.15.53:0/1015200838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11480095f0 con 0x7f1154105780 2023-11-06T01:26:48.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.527+0000 7f1158caa700 1 --2- 172.21.15.53:0/1015200838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1154105780 0x7f1154195df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1148005850 tx=0x7f1148004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:48.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.527+0000 7f1151ffb700 1 -- 172.21.15.53:0/1015200838 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f114801d070 con 0x7f1154105780 2023-11-06T01:26:48.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.527+0000 7f115af0e700 1 -- 172.21.15.53:0/1015200838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1154196530 con 0x7f1154105780 2023-11-06T01:26:48.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.527+0000 7f1151ffb700 1 -- 172.21.15.53:0/1015200838 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1148004370 con 0x7f1154105780 2023-11-06T01:26:48.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.527+0000 7f1151ffb700 1 -- 172.21.15.53:0/1015200838 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1148003ab0 con 0x7f1154105780 2023-11-06T01:26:48.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.528+0000 7f115af0e700 1 -- 172.21.15.53:0/1015200838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1154196950 con 0x7f1154105780 2023-11-06T01:26:48.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.528+0000 7f1151ffb700 1 -- 172.21.15.53:0/1015200838 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f114800f460 con 0x7f1154105780 2023-11-06T01:26:48.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.528+0000 7f115af0e700 1 -- 172.21.15.53:0/1015200838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11540623c0 con 0x7f1154105780 2023-11-06T01:26:48.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.528+0000 7f1151ffb700 1 --2- 172.21.15.53:0/1015200838 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f11440383a0 0x7f114403a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:48.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.529+0000 7f1151ffb700 1 -- 172.21.15.53:0/1015200838 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f114804c180 con 0x7f1154105780 2023-11-06T01:26:48.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.529+0000 7f1153fff700 1 --2- 172.21.15.53:0/1015200838 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f11440383a0 0x7f114403a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:48.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.529+0000 7f1153fff700 1 --2- 172.21.15.53:0/1015200838 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f11440383a0 0x7f114403a860 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1140006fd0 tx=0x7f1140006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:48.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.533+0000 7f1151ffb700 1 -- 172.21.15.53:0/1015200838 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1148029950 con 0x7f1154105780 2023-11-06T01:26:48.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:48 smithi053 ceph-mon[99472]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:26:48.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:48.737+0000 7f115af0e700 1 -- 172.21.15.53:0/1015200838 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}) v1 -- 0x7f11540686a0 con 0x7f11440383a0 2023-11-06T01:26:49.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:49 smithi053 ceph-mon[99472]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:26:50.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:50 smithi053 ceph-mon[99472]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:26:50.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:50 smithi053 ceph-mon[99472]: Deploying cephadm binary to smithi163 2023-11-06T01:26:50.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:50.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:50.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:50.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:50.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:26:50.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:26:50.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:26:51.577 INFO:teuthology.orchestra.run.smithi053.stdout:Added host 'smithi163' with addr '172.21.15.163' 2023-11-06T01:26:51.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:51.575+0000 7f1151ffb700 1 -- 172.21.15.53:0/1015200838 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f11540686a0 con 0x7f11440383a0 2023-11-06T01:26:51.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:51.578+0000 7f115af0e700 1 -- 172.21.15.53:0/1015200838 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f11440383a0 msgr2=0x7f114403a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:51.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:51.578+0000 7f115af0e700 1 --2- 172.21.15.53:0/1015200838 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f11440383a0 0x7f114403a860 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1140006fd0 tx=0x7f1140006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:51.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:51.578+0000 7f115af0e700 1 -- 172.21.15.53:0/1015200838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1154105780 msgr2=0x7f1154195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:51.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:51.578+0000 7f115af0e700 1 --2- 172.21.15.53:0/1015200838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1154105780 0x7f1154195df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1148005850 tx=0x7f1148004ff0 comp rx=0 tx=0).stop 2023-11-06T01:26:51.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:51.578+0000 7f115af0e700 1 -- 172.21.15.53:0/1015200838 shutdown_connections 2023-11-06T01:26:51.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:51.578+0000 7f115af0e700 1 --2- 172.21.15.53:0/1015200838 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f11440383a0 0x7f114403a860 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:51.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:51.578+0000 7f115af0e700 1 --2- 172.21.15.53:0/1015200838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1154105780 0x7f1154195df0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:51.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:51.578+0000 7f115af0e700 1 -- 172.21.15.53:0/1015200838 >> 172.21.15.53:0/1015200838 conn(0x7f1154075290 msgr2=0x7f1154067f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:51.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:51.578+0000 7f115af0e700 1 -- 172.21.15.53:0/1015200838 shutdown_connections 2023-11-06T01:26:51.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:51.578+0000 7f115af0e700 1 -- 172.21.15.53:0/1015200838 wait complete. 2023-11-06T01:26:51.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:51 smithi053 ceph-mon[99472]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:26:51.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:51 smithi053 ceph-mon[99472]: Updating smithi062:/etc/ceph/ceph.conf 2023-11-06T01:26:51.902 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch host ls --format=json 2023-11-06T01:26:52.461 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:26:52.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:52 smithi053 ceph-mon[99472]: Updating smithi062:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:26:52.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:52.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:52.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:52.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:52.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:26:53.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.554+0000 7fe29674d700 1 -- 172.21.15.53:0/415947987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe290105780 msgr2=0x7fe290105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:53.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.554+0000 7fe29674d700 1 --2- 172.21.15.53:0/415947987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe290105780 0x7fe290105b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe280009910 tx=0x7fe280009c20 comp rx=0 tx=0).stop 2023-11-06T01:26:53.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.555+0000 7fe29674d700 1 -- 172.21.15.53:0/415947987 shutdown_connections 2023-11-06T01:26:53.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.555+0000 7fe29674d700 1 --2- 172.21.15.53:0/415947987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe290105780 0x7fe290105b60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:53.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.555+0000 7fe29674d700 1 -- 172.21.15.53:0/415947987 >> 172.21.15.53:0/415947987 conn(0x7fe290075290 msgr2=0x7fe2900756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:53.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.555+0000 7fe29674d700 1 -- 172.21.15.53:0/415947987 shutdown_connections 2023-11-06T01:26:53.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.555+0000 7fe29674d700 1 -- 172.21.15.53:0/415947987 wait complete. 2023-11-06T01:26:53.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.555+0000 7fe29674d700 1 Processor -- start 2023-11-06T01:26:53.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.555+0000 7fe29674d700 1 -- start start 2023-11-06T01:26:53.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe29674d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe290105780 0x7fe290195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:53.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe29674d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901962a0 con 0x7fe290105780 2023-11-06T01:26:53.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe28ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe290105780 0x7fe290195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:53.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe28ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe290105780 0x7fe290195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37030/0 (socket says 172.21.15.53:37030) 2023-11-06T01:26:53.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe28ffff700 1 -- 172.21.15.53:0/3053403818 learned_addr learned my addr 172.21.15.53:0/3053403818 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe28ffff700 1 -- 172.21.15.53:0/3053403818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2800095f0 con 0x7fe290105780 2023-11-06T01:26:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe28ffff700 1 --2- 172.21.15.53:0/3053403818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe290105780 0x7fe290195d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe280005850 tx=0x7fe280004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe28dffb700 1 -- 172.21.15.53:0/3053403818 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe28001d070 con 0x7fe290105780 2023-11-06T01:26:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe28dffb700 1 -- 172.21.15.53:0/3053403818 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe280004370 con 0x7fe290105780 2023-11-06T01:26:53.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe29674d700 1 -- 172.21.15.53:0/3053403818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2901964a0 con 0x7fe290105780 2023-11-06T01:26:53.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe28dffb700 1 -- 172.21.15.53:0/3053403818 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe280003ab0 con 0x7fe290105780 2023-11-06T01:26:53.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.556+0000 7fe29674d700 1 -- 172.21.15.53:0/3053403818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2901968c0 con 0x7fe290105780 2023-11-06T01:26:53.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.557+0000 7fe28dffb700 1 -- 172.21.15.53:0/3053403818 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7fe280003d20 con 0x7fe290105780 2023-11-06T01:26:53.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.557+0000 7fe29674d700 1 -- 172.21.15.53:0/3053403818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2900623c0 con 0x7fe290105780 2023-11-06T01:26:53.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.557+0000 7fe28dffb700 1 --2- 172.21.15.53:0/3053403818 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe27c0383f0 0x7fe27c03a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:53.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.557+0000 7fe28dffb700 1 -- 172.21.15.53:0/3053403818 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe28004c120 con 0x7fe290105780 2023-11-06T01:26:53.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.558+0000 7fe287fff700 1 --2- 172.21.15.53:0/3053403818 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe27c0383f0 0x7fe27c03a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:53.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.558+0000 7fe287fff700 1 --2- 172.21.15.53:0/3053403818 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe27c0383f0 0x7fe27c03a8b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe278006fd0 tx=0x7fe278006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:53.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.562+0000 7fe28dffb700 1 -- 172.21.15.53:0/3053403818 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe280029950 con 0x7fe290105780 2023-11-06T01:26:53.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:53 smithi053 ceph-mon[99472]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:26:53.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:53 smithi053 ceph-mon[99472]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:26:53.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:53 smithi053 ceph-mon[99472]: Updating smithi062:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-06T01:26:53.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:53 smithi053 ceph-mon[99472]: Added host smithi163 2023-11-06T01:26:53.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:53.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:53.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.768+0000 7fe29674d700 1 -- 172.21.15.53:0/3053403818 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2900686a0 con 0x7fe27c0383f0 2023-11-06T01:26:53.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.770+0000 7fe28dffb700 1 -- 172.21.15.53:0/3053403818 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7fe2900686a0 con 0x7fe27c0383f0 2023-11-06T01:26:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:26:53.772 INFO:teuthology.orchestra.run.smithi053.stdout:[{"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.163", "hostname": "smithi163", "labels": [], "status": ""}] 2023-11-06T01:26:53.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.771+0000 7fe29674d700 1 -- 172.21.15.53:0/3053403818 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe27c0383f0 msgr2=0x7fe27c03a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:53.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.771+0000 7fe29674d700 1 --2- 172.21.15.53:0/3053403818 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe27c0383f0 0x7fe27c03a8b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe278006fd0 tx=0x7fe278006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:53.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.772+0000 7fe29674d700 1 -- 172.21.15.53:0/3053403818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe290105780 msgr2=0x7fe290195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:53.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.772+0000 7fe29674d700 1 --2- 172.21.15.53:0/3053403818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe290105780 0x7fe290195d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe280005850 tx=0x7fe280004ff0 comp rx=0 tx=0).stop 2023-11-06T01:26:53.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.772+0000 7fe29674d700 1 -- 172.21.15.53:0/3053403818 shutdown_connections 2023-11-06T01:26:53.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.772+0000 7fe29674d700 1 --2- 172.21.15.53:0/3053403818 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe27c0383f0 0x7fe27c03a8b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:53.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.772+0000 7fe29674d700 1 --2- 172.21.15.53:0/3053403818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe290105780 0x7fe290195d60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:53.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.772+0000 7fe29674d700 1 -- 172.21.15.53:0/3053403818 >> 172.21.15.53:0/3053403818 conn(0x7fe290075290 msgr2=0x7fe290067f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:53.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.772+0000 7fe29674d700 1 -- 172.21.15.53:0/3053403818 shutdown_connections 2023-11-06T01:26:53.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:53.772+0000 7fe29674d700 1 -- 172.21.15.53:0/3053403818 wait complete. 2023-11-06T01:26:54.124 INFO:tasks.cephadm:Setting crush tunables to default 2023-11-06T01:26:54.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd crush tunables default 2023-11-06T01:26:54.685 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:26:55.523 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:55 smithi053 ceph-mon[99472]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:26:55.523 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:55 smithi053 ceph-mon[99472]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:26:55.523 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:55.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.968+0000 7fa743194700 1 -- 172.21.15.53:0/3208690642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa73c105780 msgr2=0x7fa73c105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:55.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.968+0000 7fa743194700 1 --2- 172.21.15.53:0/3208690642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa73c105780 0x7fa73c105b60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa730009910 tx=0x7fa730009c20 comp rx=0 tx=0).stop 2023-11-06T01:26:55.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.968+0000 7fa743194700 1 -- 172.21.15.53:0/3208690642 shutdown_connections 2023-11-06T01:26:55.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.968+0000 7fa743194700 1 --2- 172.21.15.53:0/3208690642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa73c105780 0x7fa73c105b60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:55.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.968+0000 7fa743194700 1 -- 172.21.15.53:0/3208690642 >> 172.21.15.53:0/3208690642 conn(0x7fa73c075290 msgr2=0x7fa73c0756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:55.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.968+0000 7fa743194700 1 -- 172.21.15.53:0/3208690642 shutdown_connections 2023-11-06T01:26:55.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.968+0000 7fa743194700 1 -- 172.21.15.53:0/3208690642 wait complete. 2023-11-06T01:26:55.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.969+0000 7fa743194700 1 Processor -- start 2023-11-06T01:26:55.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.969+0000 7fa743194700 1 -- start start 2023-11-06T01:26:55.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.969+0000 7fa743194700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa73c105780 0x7fa73c196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:55.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.969+0000 7fa743194700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c197080 con 0x7fa73c105780 2023-11-06T01:26:55.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.969+0000 7fa740f30700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa73c105780 0x7fa73c196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:55.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.969+0000 7fa740f30700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa73c105780 0x7fa73c196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37036/0 (socket says 172.21.15.53:37036) 2023-11-06T01:26:55.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.969+0000 7fa740f30700 1 -- 172.21.15.53:0/130866531 learned_addr learned my addr 172.21.15.53:0/130866531 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:26:55.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.969+0000 7fa740f30700 1 -- 172.21.15.53:0/130866531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7300095f0 con 0x7fa73c105780 2023-11-06T01:26:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.969+0000 7fa740f30700 1 --2- 172.21.15.53:0/130866531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa73c105780 0x7fa73c196b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa730005850 tx=0x7fa730004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.969+0000 7fa739ffb700 1 -- 172.21.15.53:0/130866531 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa73001d070 con 0x7fa73c105780 2023-11-06T01:26:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.970+0000 7fa743194700 1 -- 172.21.15.53:0/130866531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c197280 con 0x7fa73c105780 2023-11-06T01:26:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.970+0000 7fa739ffb700 1 -- 172.21.15.53:0/130866531 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa730004370 con 0x7fa73c105780 2023-11-06T01:26:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.970+0000 7fa739ffb700 1 -- 172.21.15.53:0/130866531 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa730003ab0 con 0x7fa73c105780 2023-11-06T01:26:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.970+0000 7fa743194700 1 -- 172.21.15.53:0/130866531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c1976a0 con 0x7fa73c105780 2023-11-06T01:26:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.971+0000 7fa739ffb700 1 -- 172.21.15.53:0/130866531 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7fa73000f460 con 0x7fa73c105780 2023-11-06T01:26:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.971+0000 7fa739ffb700 1 --2- 172.21.15.53:0/130866531 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa72c0383a0 0x7fa72c03a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.971+0000 7fa739ffb700 1 -- 172.21.15.53:0/130866531 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa73004c040 con 0x7fa73c105780 2023-11-06T01:26:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.971+0000 7fa73bfff700 1 --2- 172.21.15.53:0/130866531 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa72c0383a0 0x7fa72c03a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.971+0000 7fa743194700 1 -- 172.21.15.53:0/130866531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c1039c0 con 0x7fa73c105780 2023-11-06T01:26:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.971+0000 7fa73bfff700 1 --2- 172.21.15.53:0/130866531 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa72c0383a0 0x7fa72c03a860 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa728006fd0 tx=0x7fa728006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:55.976+0000 7fa739ffb700 1 -- 172.21.15.53:0/130866531 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa730026070 con 0x7fa73c105780 2023-11-06T01:26:56.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:56.188+0000 7fa743194700 1 -- 172.21.15.53:0/130866531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa73c1979b0 con 0x7fa73c105780 2023-11-06T01:26:57.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:57.180+0000 7fa739ffb700 1 -- 172.21.15.53:0/130866531 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa73c1979b0 con 0x7fa73c105780 2023-11-06T01:26:57.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:57.183+0000 7fa743194700 1 -- 172.21.15.53:0/130866531 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa72c0383a0 msgr2=0x7fa72c03a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:57.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:57.183+0000 7fa743194700 1 --2- 172.21.15.53:0/130866531 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa72c0383a0 0x7fa72c03a860 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa728006fd0 tx=0x7fa728006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:57.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:57.183+0000 7fa743194700 1 -- 172.21.15.53:0/130866531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa73c105780 msgr2=0x7fa73c196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:57.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:57.183+0000 7fa743194700 1 --2- 172.21.15.53:0/130866531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa73c105780 0x7fa73c196b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa730005850 tx=0x7fa730004ff0 comp rx=0 tx=0).stop 2023-11-06T01:26:57.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:57.184+0000 7fa743194700 1 -- 172.21.15.53:0/130866531 shutdown_connections 2023-11-06T01:26:57.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:57.184+0000 7fa743194700 1 --2- 172.21.15.53:0/130866531 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa72c0383a0 0x7fa72c03a860 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:57.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:57.184+0000 7fa743194700 1 --2- 172.21.15.53:0/130866531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa73c105780 0x7fa73c196b40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:57.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:57.184+0000 7fa743194700 1 -- 172.21.15.53:0/130866531 >> 172.21.15.53:0/130866531 conn(0x7fa73c075290 msgr2=0x7fa73c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:57.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:57.184+0000 7fa743194700 1 -- 172.21.15.53:0/130866531 shutdown_connections 2023-11-06T01:26:57.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:26:57.184+0000 7fa743194700 1 -- 172.21.15.53:0/130866531 wait complete. 2023-11-06T01:26:57.188 INFO:teuthology.orchestra.run.smithi053.stderr:adjusted tunables profile to default 2023-11-06T01:26:57.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:57 smithi053 ceph-mon[99472]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:26:57.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:57 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/130866531' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-06T01:26:57.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:57.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:57.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:57.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:57.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:26:57.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:26:57.537 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:26:57.538 INFO:tasks.cephadm:Adding mon.a on smithi053 2023-11-06T01:26:57.538 INFO:tasks.cephadm:Adding mon.b on smithi062 2023-11-06T01:26:57.539 INFO:tasks.cephadm:Adding mon.c on smithi163 2023-11-06T01:26:57.539 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch apply mon '3;smithi053:172.21.15.53=a;smithi062:172.21.15.62=b;smithi163:172.21.15.163=c' 2023-11-06T01:26:58.143 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:26:58.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:58 smithi053 ceph-mon[99472]: Updating smithi163:/etc/ceph/ceph.conf 2023-11-06T01:26:58.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:58 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/130866531' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-06T01:26:58.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:58 smithi053 ceph-mon[99472]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:26:58.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:58.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:58.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:26:59.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.541+0000 7f798b4f2700 1 -- 172.21.15.163:0/2391805708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7984105700 msgr2=0x7f7984105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:59.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.541+0000 7f798b4f2700 1 --2- 172.21.15.163:0/2391805708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7984105700 0x7f7984105ae0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7974009910 tx=0x7f7974009c20 comp rx=0 tx=0).stop 2023-11-06T01:26:59.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.541+0000 7f798b4f2700 1 -- 172.21.15.163:0/2391805708 shutdown_connections 2023-11-06T01:26:59.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.541+0000 7f798b4f2700 1 --2- 172.21.15.163:0/2391805708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7984105700 0x7f7984105ae0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:59.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.541+0000 7f798b4f2700 1 -- 172.21.15.163:0/2391805708 >> 172.21.15.163:0/2391805708 conn(0x7f79840fb210 msgr2=0x7f79840fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:59.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.541+0000 7f798b4f2700 1 -- 172.21.15.163:0/2391805708 shutdown_connections 2023-11-06T01:26:59.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.541+0000 7f798b4f2700 1 -- 172.21.15.163:0/2391805708 wait complete. 2023-11-06T01:26:59.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.541+0000 7f798b4f2700 1 Processor -- start 2023-11-06T01:26:59.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.542+0000 7f798b4f2700 1 -- start start 2023-11-06T01:26:59.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.542+0000 7f798b4f2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7984105700 0x7f798419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:59.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.542+0000 7f798b4f2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798419aea0 con 0x7f7984105700 2023-11-06T01:26:59.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.542+0000 7f798928e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7984105700 0x7f798419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:59.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.542+0000 7f798928e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7984105700 0x7f798419a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.163:42558/0 (socket says 172.21.15.163:42558) 2023-11-06T01:26:59.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.542+0000 7f798928e700 1 -- 172.21.15.163:0/2764197422 learned_addr learned my addr 172.21.15.163:0/2764197422 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-11-06T01:26:59.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.542+0000 7f798928e700 1 -- 172.21.15.163:0/2764197422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79740095f0 con 0x7f7984105700 2023-11-06T01:26:59.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.543+0000 7f798928e700 1 --2- 172.21.15.163:0/2764197422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7984105700 0x7f798419a960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7974006010 tx=0x7f7974004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:59.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.543+0000 7f797a7fc700 1 -- 172.21.15.163:0/2764197422 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f797401f070 con 0x7f7984105700 2023-11-06T01:26:59.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.543+0000 7f798b4f2700 1 -- 172.21.15.163:0/2764197422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798419b0a0 con 0x7f7984105700 2023-11-06T01:26:59.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.543+0000 7f797a7fc700 1 -- 172.21.15.163:0/2764197422 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f797400bbb0 con 0x7f7984105700 2023-11-06T01:26:59.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.543+0000 7f797a7fc700 1 -- 172.21.15.163:0/2764197422 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7974003930 con 0x7f7984105700 2023-11-06T01:26:59.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.543+0000 7f798b4f2700 1 -- 172.21.15.163:0/2764197422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79841949e0 con 0x7f7984105700 2023-11-06T01:26:59.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.544+0000 7f797a7fc700 1 -- 172.21.15.163:0/2764197422 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f7974003b20 con 0x7f7984105700 2023-11-06T01:26:59.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.544+0000 7f797a7fc700 1 --2- 172.21.15.163:0/2764197422 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f79700383f0 0x7f797003a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:26:59.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.544+0000 7f798b4f2700 1 -- 172.21.15.163:0/2764197422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79840623c0 con 0x7f7984105700 2023-11-06T01:26:59.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.544+0000 7f797a7fc700 1 -- 172.21.15.163:0/2764197422 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f797404e2b0 con 0x7f7984105700 2023-11-06T01:26:59.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.544+0000 7f7988a8d700 1 --2- 172.21.15.163:0/2764197422 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f79700383f0 0x7f797003a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:26:59.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.545+0000 7f7988a8d700 1 --2- 172.21.15.163:0/2764197422 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f79700383f0 0x7f797003a8b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7980006fd0 tx=0x7f7980006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:26:59.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.549+0000 7f797a7fc700 1 -- 172.21.15.163:0/2764197422 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f797402b950 con 0x7f7984105700 2023-11-06T01:26:59.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:59 smithi053 ceph-mon[99472]: Updating smithi163:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:26:59.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:59 smithi053 ceph-mon[99472]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:26:59.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:59 smithi053 ceph-mon[99472]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:26:59.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:26:59 smithi053 ceph-mon[99472]: Updating smithi163:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-06T01:26:59.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.747+0000 7f798b4f2700 1 -- 172.21.15.163:0/2764197422 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi053:172.21.15.53=a;smithi062:172.21.15.62=b;smithi163:172.21.15.163=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f7984195410 con 0x7f79700383f0 2023-11-06T01:26:59.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.754+0000 7f797a7fc700 1 -- 172.21.15.163:0/2764197422 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7984195410 con 0x7f79700383f0 2023-11-06T01:26:59.757 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled mon update... 2023-11-06T01:26:59.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.756+0000 7f798b4f2700 1 -- 172.21.15.163:0/2764197422 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f79700383f0 msgr2=0x7f797003a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:59.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.756+0000 7f798b4f2700 1 --2- 172.21.15.163:0/2764197422 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f79700383f0 0x7f797003a8b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7980006fd0 tx=0x7f7980006e40 comp rx=0 tx=0).stop 2023-11-06T01:26:59.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.756+0000 7f798b4f2700 1 -- 172.21.15.163:0/2764197422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7984105700 msgr2=0x7f798419a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:26:59.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.756+0000 7f798b4f2700 1 --2- 172.21.15.163:0/2764197422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7984105700 0x7f798419a960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7974006010 tx=0x7f7974004ff0 comp rx=0 tx=0).stop 2023-11-06T01:26:59.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.756+0000 7f798b4f2700 1 -- 172.21.15.163:0/2764197422 shutdown_connections 2023-11-06T01:26:59.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.756+0000 7f798b4f2700 1 --2- 172.21.15.163:0/2764197422 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f79700383f0 0x7f797003a8b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:59.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.756+0000 7f798b4f2700 1 --2- 172.21.15.163:0/2764197422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7984105700 0x7f798419a960 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:26:59.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.756+0000 7f798b4f2700 1 -- 172.21.15.163:0/2764197422 >> 172.21.15.163:0/2764197422 conn(0x7f79840fb210 msgr2=0x7f7984107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:26:59.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.756+0000 7f798b4f2700 1 -- 172.21.15.163:0/2764197422 shutdown_connections 2023-11-06T01:26:59.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:26:59.756+0000 7f798b4f2700 1 -- 172.21.15.163:0/2764197422 wait complete. 2023-11-06T01:27:00.117 DEBUG:teuthology.orchestra.run.smithi062:mon.b> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.b.service 2023-11-06T01:27:00.120 DEBUG:teuthology.orchestra.run.smithi163:mon.c> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.c.service 2023-11-06T01:27:00.123 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-06T01:27:00.123 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph mon dump -f json 2023-11-06T01:27:00.151 INFO:journalctl@ceph.mon.c.smithi163.stdout:-- Logs begin at Mon 2023-11-06 01:10:53 UTC. -- 2023-11-06T01:27:00.157 INFO:journalctl@ceph.mon.b.smithi062.stdout:-- Logs begin at Mon 2023-11-06 01:10:07 UTC. -- 2023-11-06T01:27:01.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:00 smithi053 ceph-mon[99472]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:01.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:00 smithi053 ceph-mon[99472]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi053:172.21.15.53=a;smithi062:172.21.15.62=b;smithi163:172.21.15.163=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:01.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:00 smithi053 ceph-mon[99472]: Saving service mon spec with placement smithi053:172.21.15.53=a;smithi062:172.21.15.62=b;smithi163:172.21.15.163=c;count:3 2023-11-06T01:27:01.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:01.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:01.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:01.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:01.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:01.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:01.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:01.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:00 smithi053 ceph-mon[99472]: Deploying daemon mon.c on smithi163 2023-11-06T01:27:01.141 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:02.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.710+0000 7f91ff016700 1 -- 172.21.15.163:0/1896645763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f8105700 msgr2=0x7f91f8105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:02.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.710+0000 7f91ff016700 1 --2- 172.21.15.163:0/1896645763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f8105700 0x7f91f8105ae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009910 tx=0x7f91ec009c20 comp rx=0 tx=0).stop 2023-11-06T01:27:02.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.710+0000 7f91ff016700 1 -- 172.21.15.163:0/1896645763 shutdown_connections 2023-11-06T01:27:02.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.710+0000 7f91ff016700 1 --2- 172.21.15.163:0/1896645763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f8105700 0x7f91f8105ae0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:02.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.710+0000 7f91ff016700 1 -- 172.21.15.163:0/1896645763 >> 172.21.15.163:0/1896645763 conn(0x7f91f80fb210 msgr2=0x7f91f80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:02.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.710+0000 7f91ff016700 1 -- 172.21.15.163:0/1896645763 shutdown_connections 2023-11-06T01:27:02.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.710+0000 7f91ff016700 1 -- 172.21.15.163:0/1896645763 wait complete. 2023-11-06T01:27:02.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.711+0000 7f91ff016700 1 Processor -- start 2023-11-06T01:27:02.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.711+0000 7f91ff016700 1 -- start start 2023-11-06T01:27:02.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.711+0000 7f91ff016700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f8105700 0x7f91f81012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:02.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.711+0000 7f91ff016700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f8103250 con 0x7f91f8105700 2023-11-06T01:27:02.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.711+0000 7f91fcdb2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f8105700 0x7f91f81012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:02.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.711+0000 7f91fcdb2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f8105700 0x7f91f81012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.163:42564/0 (socket says 172.21.15.163:42564) 2023-11-06T01:27:02.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.711+0000 7f91fcdb2700 1 -- 172.21.15.163:0/1185888522 learned_addr learned my addr 172.21.15.163:0/1185888522 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-11-06T01:27:02.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.712+0000 7f91fcdb2700 1 -- 172.21.15.163:0/1185888522 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec0095f0 con 0x7f91f8105700 2023-11-06T01:27:02.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.712+0000 7f91fcdb2700 1 --2- 172.21.15.163:0/1185888522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f8105700 0x7f91f81012d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91ec006010 tx=0x7f91ec004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:02.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.712+0000 7f91f5ffb700 1 -- 172.21.15.163:0/1185888522 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91ec01d070 con 0x7f91f8105700 2023-11-06T01:27:02.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.712+0000 7f91ff016700 1 -- 172.21.15.163:0/1185888522 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f8101810 con 0x7f91f8105700 2023-11-06T01:27:02.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.712+0000 7f91f5ffb700 1 -- 172.21.15.163:0/1185888522 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91ec00bc10 con 0x7f91f8105700 2023-11-06T01:27:02.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.712+0000 7f91f5ffb700 1 -- 172.21.15.163:0/1185888522 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91ec003910 con 0x7f91f8105700 2023-11-06T01:27:02.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.712+0000 7f91ff016700 1 -- 172.21.15.163:0/1185888522 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f8101c90 con 0x7f91f8105700 2023-11-06T01:27:02.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.713+0000 7f91f5ffb700 1 -- 172.21.15.163:0/1185888522 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f91ec003bd0 con 0x7f91f8105700 2023-11-06T01:27:02.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.713+0000 7f91ff016700 1 -- 172.21.15.163:0/1185888522 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f80623c0 con 0x7f91f8105700 2023-11-06T01:27:02.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.713+0000 7f91f5ffb700 1 --2- 172.21.15.163:0/1185888522 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f91e80383a0 0x7f91e803a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:02.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.713+0000 7f91f5ffb700 1 -- 172.21.15.163:0/1185888522 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f91ec04c030 con 0x7f91f8105700 2023-11-06T01:27:02.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.714+0000 7f91f7fff700 1 --2- 172.21.15.163:0/1185888522 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f91e80383a0 0x7f91e803a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:02.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.714+0000 7f91f7fff700 1 --2- 172.21.15.163:0/1185888522 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f91e80383a0 0x7f91e803a860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f91e4006fd0 tx=0x7f91e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:02.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:02.719+0000 7f91f5ffb700 1 -- 172.21.15.163:0/1185888522 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f91ec00ba70 con 0x7f91f8105700 2023-11-06T01:27:03.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.036+0000 7f91ff016700 1 -- 172.21.15.163:0/1185888522 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f91f8102030 con 0x7f91f8105700 2023-11-06T01:27:03.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.037+0000 7f91f5ffb700 1 -- 172.21.15.163:0/1185888522 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f91ec026070 con 0x7f91f8105700 2023-11-06T01:27:03.040 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:27:03.041 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","modified":"2023-11-06T01:25:23.107939Z","created":"2023-11-06T01:25:23.107939Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-11-06T01:27:03.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.039+0000 7f91ff016700 1 -- 172.21.15.163:0/1185888522 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f91e80383a0 msgr2=0x7f91e803a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:03.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.039+0000 7f91ff016700 1 --2- 172.21.15.163:0/1185888522 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f91e80383a0 0x7f91e803a860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f91e4006fd0 tx=0x7f91e4006e40 comp rx=0 tx=0).stop 2023-11-06T01:27:03.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.039+0000 7f91ff016700 1 -- 172.21.15.163:0/1185888522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f8105700 msgr2=0x7f91f81012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:03.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.039+0000 7f91ff016700 1 --2- 172.21.15.163:0/1185888522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f8105700 0x7f91f81012d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91ec006010 tx=0x7f91ec004ff0 comp rx=0 tx=0).stop 2023-11-06T01:27:03.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.039+0000 7f91ff016700 1 -- 172.21.15.163:0/1185888522 shutdown_connections 2023-11-06T01:27:03.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.039+0000 7f91ff016700 1 --2- 172.21.15.163:0/1185888522 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f91e80383a0 0x7f91e803a860 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:03.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.039+0000 7f91ff016700 1 --2- 172.21.15.163:0/1185888522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f8105700 0x7f91f81012d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.039+0000 7f91ff016700 1 -- 172.21.15.163:0/1185888522 >> 172.21.15.163:0/1185888522 conn(0x7f91f80fb210 msgr2=0x7f91f8107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.039+0000 7f91ff016700 1 -- 172.21.15.163:0/1185888522 shutdown_connections 2023-11-06T01:27:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:03.039+0000 7f91ff016700 1 -- 172.21.15.163:0/1185888522 wait complete. 2023-11-06T01:27:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2023-11-06T01:27:03.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:02 smithi053 ceph-mon[99472]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:03.694 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 podman[104374]: 2023-11-06T01:27:04.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:03 smithi053 ceph-mon[99472]: from='client.? 172.21.15.163:0/1185888522' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-06T01:27:04.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: set uid:gid to 167:167 (ceph:ceph) 2023-11-06T01:27:04.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable), process ceph-mon, pid 7 2023-11-06T01:27:04.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 bash[104374]: 6690a5c213684d69fbc2d54895fec51cd1fa9ceec0c49fd03c45fd7c2b61add1 2023-11-06T01:27:04.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: pidfile_write: ignore empty --pid-file 2023-11-06T01:27:04.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: load: jerasure load: lrc 2023-11-06T01:27:04.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: RocksDB version: 7.9.2 2023-11-06T01:27:04.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Git sha 0 2023-11-06T01:27:04.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Compile date 2023-11-04 15:03:31 2023-11-06T01:27:04.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: DB SUMMARY 2023-11-06T01:27:04.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: DB Session ID: ZQFODL6LQQ36Z5O41UMA 2023-11-06T01:27:04.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: CURRENT file: CURRENT 2023-11-06T01:27:04.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: IDENTITY file: IDENTITY 2023-11-06T01:27:04.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-06T01:27:04.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-11-06T01:27:04.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-11-06T01:27:04.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.error_if_exists: 0 2023-11-06T01:27:04.245 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.create_if_missing: 0 2023-11-06T01:27:04.245 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.paranoid_checks: 1 2023-11-06T01:27:04.246 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-06T01:27:04.246 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-06T01:27:04.246 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-06T01:27:04.247 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.env: 0x56391de5b720 2023-11-06T01:27:04.247 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.fs: PosixFileSystem 2023-11-06T01:27:04.247 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.info_log: 0x56391f607440 2023-11-06T01:27:04.247 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_file_opening_threads: 16 2023-11-06T01:27:04.248 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.statistics: (nil) 2023-11-06T01:27:04.248 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.use_fsync: 0 2023-11-06T01:27:04.248 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_log_file_size: 0 2023-11-06T01:27:04.248 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-06T01:27:04.249 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-06T01:27:04.249 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.keep_log_file_num: 1000 2023-11-06T01:27:04.249 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.recycle_log_file_num: 0 2023-11-06T01:27:04.249 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.allow_fallocate: 1 2023-11-06T01:27:04.250 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.allow_mmap_reads: 0 2023-11-06T01:27:04.250 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.allow_mmap_writes: 0 2023-11-06T01:27:04.250 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.use_direct_reads: 0 2023-11-06T01:27:04.251 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-06T01:27:04.251 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.create_missing_column_families: 0 2023-11-06T01:27:04.251 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.db_log_dir: 2023-11-06T01:27:04.251 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.wal_dir: 2023-11-06T01:27:04.252 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-06T01:27:04.252 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-06T01:27:04.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-06T01:27:04.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-06T01:27:04.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-06T01:27:04.254 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-06T01:27:04.254 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.advise_random_on_open: 1 2023-11-06T01:27:04.255 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.db_write_buffer_size: 0 2023-11-06T01:27:04.255 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.write_buffer_manager: 0x56391e898320 2023-11-06T01:27:04.255 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-06T01:27:04.255 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-06T01:27:04.256 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-06T01:27:04.256 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.rate_limiter: (nil) 2023-11-06T01:27:04.256 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-06T01:27:04.256 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.wal_recovery_mode: 2 2023-11-06T01:27:04.257 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.enable_thread_tracking: 0 2023-11-06T01:27:04.257 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.enable_pipelined_write: 0 2023-11-06T01:27:04.257 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.unordered_write: 0 2023-11-06T01:27:04.258 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-06T01:27:04.258 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-06T01:27:04.258 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-06T01:27:04.258 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-06T01:27:04.259 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.row_cache: None 2023-11-06T01:27:04.259 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.wal_filter: None 2023-11-06T01:27:04.259 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-06T01:27:04.259 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.allow_ingest_behind: 0 2023-11-06T01:27:04.259 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.two_write_queues: 0 2023-11-06T01:27:04.260 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.manual_wal_flush: 0 2023-11-06T01:27:04.260 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.wal_compression: 0 2023-11-06T01:27:04.260 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.atomic_flush: 0 2023-11-06T01:27:04.260 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-06T01:27:04.260 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-06T01:27:04.261 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-06T01:27:04.261 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.log_readahead_size: 0 2023-11-06T01:27:04.261 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-06T01:27:04.261 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.best_efforts_recovery: 0 2023-11-06T01:27:04.262 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-06T01:27:04.262 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-06T01:27:04.262 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.allow_data_in_errors: 0 2023-11-06T01:27:04.262 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.db_host_id: __hostname__ 2023-11-06T01:27:04.263 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-06T01:27:04.263 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_background_jobs: 2 2023-11-06T01:27:04.263 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_background_compactions: -1 2023-11-06T01:27:04.263 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_subcompactions: 1 2023-11-06T01:27:04.263 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-06T01:27:04.264 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-06T01:27:04.264 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-06T01:27:04.264 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_total_wal_size: 0 2023-11-06T01:27:04.264 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-06T01:27:04.264 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-06T01:27:04.265 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-06T01:27:04.265 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-06T01:27:04.265 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_open_files: -1 2023-11-06T01:27:04.265 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bytes_per_sync: 0 2023-11-06T01:27:04.265 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-06T01:27:04.266 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-06T01:27:04.266 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_readahead_size: 0 2023-11-06T01:27:04.266 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_background_flushes: -1 2023-11-06T01:27:04.266 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Compression algorithms supported: 2023-11-06T01:27:04.266 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-06T01:27:04.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: kZSTD supported: 0 2023-11-06T01:27:04.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: kXpressCompression supported: 0 2023-11-06T01:27:04.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: kLZ4HCCompression supported: 1 2023-11-06T01:27:04.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: kZlibCompression supported: 1 2023-11-06T01:27:04.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: kSnappyCompression supported: 1 2023-11-06T01:27:04.268 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: kLZ4Compression supported: 1 2023-11-06T01:27:04.268 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: kBZip2Compression supported: 0 2023-11-06T01:27:04.268 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-06T01:27:04.268 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-06T01:27:04.269 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-11-06T01:27:04.269 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-06T01:27:04.269 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-06T01:27:04.270 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.merge_operator: 2023-11-06T01:27:04.270 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_filter: None 2023-11-06T01:27:04.270 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_filter_factory: None 2023-11-06T01:27:04.270 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.sst_partitioner_factory: None 2023-11-06T01:27:04.271 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-06T01:27:04.271 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-06T01:27:04.271 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56391f607540) 2023-11-06T01:27:04.271 INFO:journalctl@ceph.mon.c.smithi163.stdout: cache_index_and_filter_blocks: 1 2023-11-06T01:27:04.271 INFO:journalctl@ceph.mon.c.smithi163.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-06T01:27:04.272 INFO:journalctl@ceph.mon.c.smithi163.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-06T01:27:04.272 INFO:journalctl@ceph.mon.c.smithi163.stdout: pin_top_level_index_and_filter: 1 2023-11-06T01:27:04.272 INFO:journalctl@ceph.mon.c.smithi163.stdout: index_type: 0 2023-11-06T01:27:04.272 INFO:journalctl@ceph.mon.c.smithi163.stdout: data_block_index_type: 0 2023-11-06T01:27:04.273 INFO:journalctl@ceph.mon.c.smithi163.stdout: index_shortening: 1 2023-11-06T01:27:04.273 INFO:journalctl@ceph.mon.c.smithi163.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-06T01:27:04.273 INFO:journalctl@ceph.mon.c.smithi163.stdout: checksum: 4 2023-11-06T01:27:04.273 INFO:journalctl@ceph.mon.c.smithi163.stdout: no_block_cache: 0 2023-11-06T01:27:04.274 INFO:journalctl@ceph.mon.c.smithi163.stdout: block_cache: 0x56391e9191f0 2023-11-06T01:27:04.274 INFO:journalctl@ceph.mon.c.smithi163.stdout: block_cache_name: BinnedLRUCache 2023-11-06T01:27:04.274 INFO:journalctl@ceph.mon.c.smithi163.stdout: block_cache_options: 2023-11-06T01:27:04.274 INFO:journalctl@ceph.mon.c.smithi163.stdout: capacity : 536870912 2023-11-06T01:27:04.274 INFO:journalctl@ceph.mon.c.smithi163.stdout: num_shard_bits : 4 2023-11-06T01:27:04.275 INFO:journalctl@ceph.mon.c.smithi163.stdout: strict_capacity_limit : 0 2023-11-06T01:27:04.275 INFO:journalctl@ceph.mon.c.smithi163.stdout: high_pri_pool_ratio: 0.000 2023-11-06T01:27:04.276 INFO:journalctl@ceph.mon.c.smithi163.stdout: block_cache_compressed: (nil) 2023-11-06T01:27:04.276 INFO:journalctl@ceph.mon.c.smithi163.stdout: persistent_cache: (nil) 2023-11-06T01:27:04.276 INFO:journalctl@ceph.mon.c.smithi163.stdout: block_size: 4096 2023-11-06T01:27:04.277 INFO:journalctl@ceph.mon.c.smithi163.stdout: block_size_deviation: 10 2023-11-06T01:27:04.277 INFO:journalctl@ceph.mon.c.smithi163.stdout: block_restart_interval: 16 2023-11-06T01:27:04.277 INFO:journalctl@ceph.mon.c.smithi163.stdout: index_block_restart_interval: 1 2023-11-06T01:27:04.278 INFO:journalctl@ceph.mon.c.smithi163.stdout: metadata_block_size: 4096 2023-11-06T01:27:04.279 INFO:journalctl@ceph.mon.c.smithi163.stdout: partition_filters: 0 2023-11-06T01:27:04.279 INFO:journalctl@ceph.mon.c.smithi163.stdout: use_delta_encoding: 1 2023-11-06T01:27:04.280 INFO:journalctl@ceph.mon.c.smithi163.stdout: filter_policy: bloomfilter 2023-11-06T01:27:04.280 INFO:journalctl@ceph.mon.c.smithi163.stdout: whole_key_filtering: 1 2023-11-06T01:27:04.281 INFO:journalctl@ceph.mon.c.smithi163.stdout: verify_compression: 0 2023-11-06T01:27:04.281 INFO:journalctl@ceph.mon.c.smithi163.stdout: read_amp_bytes_per_bit: 0 2023-11-06T01:27:04.282 INFO:journalctl@ceph.mon.c.smithi163.stdout: format_version: 5 2023-11-06T01:27:04.282 INFO:journalctl@ceph.mon.c.smithi163.stdout: enable_index_compression: 1 2023-11-06T01:27:04.283 INFO:journalctl@ceph.mon.c.smithi163.stdout: block_align: 0 2023-11-06T01:27:04.285 INFO:journalctl@ceph.mon.c.smithi163.stdout: max_auto_readahead_size: 262144 2023-11-06T01:27:04.286 INFO:journalctl@ceph.mon.c.smithi163.stdout: prepopulate_block_cache: 0 2023-11-06T01:27:04.286 INFO:journalctl@ceph.mon.c.smithi163.stdout: initial_auto_readahead_size: 8192 2023-11-06T01:27:04.287 INFO:journalctl@ceph.mon.c.smithi163.stdout: num_file_reads_for_auto_readahead: 2 2023-11-06T01:27:04.287 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.write_buffer_size: 33554432 2023-11-06T01:27:04.287 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_write_buffer_number: 2 2023-11-06T01:27:04.288 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compression: NoCompression 2023-11-06T01:27:04.288 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bottommost_compression: Disabled 2023-11-06T01:27:04.289 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.prefix_extractor: nullptr 2023-11-06T01:27:04.289 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-06T01:27:04.290 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.num_levels: 7 2023-11-06T01:27:04.291 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-06T01:27:04.291 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-06T01:27:04.291 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-06T01:27:04.292 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-06T01:27:04.292 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-06T01:27:04.292 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-06T01:27:04.292 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-06T01:27:04.293 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-06T01:27:04.293 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-06T01:27:04.293 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-06T01:27:04.293 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-06T01:27:04.294 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-06T01:27:04.294 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-06T01:27:04.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compression_opts.level: 32767 2023-11-06T01:27:04.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compression_opts.strategy: 0 2023-11-06T01:27:04.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-06T01:27:04.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-06T01:27:04.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-06T01:27:04.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-06T01:27:04.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compression_opts.enabled: false 2023-11-06T01:27:04.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-06T01:27:04.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-06T01:27:04.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-06T01:27:04.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-06T01:27:04.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.target_file_size_base: 67108864 2023-11-06T01:27:04.301 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-06T01:27:04.301 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-06T01:27:04.302 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-06T01:27:04.302 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-06T01:27:04.302 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-06T01:27:04.303 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-06T01:27:04.303 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-06T01:27:04.303 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-06T01:27:04.303 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-06T01:27:04.303 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-06T01:27:04.304 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-06T01:27:04.304 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-06T01:27:04.304 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-06T01:27:04.304 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-06T01:27:04.304 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.arena_block_size: 1048576 2023-11-06T01:27:04.305 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-06T01:27:04.305 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-06T01:27:04.305 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.disable_auto_compactions: 0 2023-11-06T01:27:04.308 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-06T01:27:04.308 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-06T01:27:04.308 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-06T01:27:04.309 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-06T01:27:04.309 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-06T01:27:04.309 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-06T01:27:04.310 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-06T01:27:04.310 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-06T01:27:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-06T01:27:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-06T01:27:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.table_properties_collectors: 2023-11-06T01:27:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.inplace_update_support: 0 2023-11-06T01:27:04.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-06T01:27:04.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-06T01:27:04.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-06T01:27:04.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-06T01:27:04.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.bloom_locality: 0 2023-11-06T01:27:04.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.max_successive_merges: 0 2023-11-06T01:27:04.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-06T01:27:04.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.paranoid_file_checks: 0 2023-11-06T01:27:04.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.force_consistency_checks: 1 2023-11-06T01:27:04.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.report_bg_io_stats: 0 2023-11-06T01:27:04.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.ttl: 2592000 2023-11-06T01:27:04.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-06T01:27:04.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-06T01:27:04.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-06T01:27:04.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.enable_blob_files: false 2023-11-06T01:27:04.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.min_blob_size: 0 2023-11-06T01:27:04.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.blob_file_size: 268435456 2023-11-06T01:27:04.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-06T01:27:04.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-06T01:27:04.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-06T01:27:04.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-06T01:27:04.317 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-06T01:27:04.317 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.blob_file_starting_level: 0 2023-11-06T01:27:04.317 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-06T01:27:04.317 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-06T01:27:04.318 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-06T01:27:04.318 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 86370aae-930d-433f-aad3-208477569495 2023-11-06T01:27:04.318 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: EVENT_LOG_v1 {"time_micros": 1699234023781242, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-06T01:27:04.318 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-06T01:27:04.318 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: EVENT_LOG_v1 {"time_micros": 1699234023781661, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1699234023, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "86370aae-930d-433f-aad3-208477569495", "db_session_id": "ZQFODL6LQQ36Z5O41UMA", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-06T01:27:04.319 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: EVENT_LOG_v1 {"time_micros": 1699234023781727, "job": 1, "event": "recovery_finished"} 2023-11-06T01:27:04.319 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-06T01:27:04.319 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-06T01:27:04.319 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56391e9b6000 2023-11-06T01:27:04.319 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: DB pointer 0x56391e9a2000 2023-11-06T01:27:04.320 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-06T01:27:04.320 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-06T01:27:04.320 INFO:journalctl@ceph.mon.c.smithi163.stdout: ** DB Stats ** 2023-11-06T01:27:04.320 INFO:journalctl@ceph.mon.c.smithi163.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-06T01:27:04.321 INFO:journalctl@ceph.mon.c.smithi163.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-06T01:27:04.321 INFO:journalctl@ceph.mon.c.smithi163.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-06T01:27:04.321 INFO:journalctl@ceph.mon.c.smithi163.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-06T01:27:04.321 INFO:journalctl@ceph.mon.c.smithi163.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-06T01:27:04.321 INFO:journalctl@ceph.mon.c.smithi163.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-06T01:27:04.322 INFO:journalctl@ceph.mon.c.smithi163.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-06T01:27:04.323 INFO:journalctl@ceph.mon.c.smithi163.stdout: 2023-11-06T01:27:04.323 INFO:journalctl@ceph.mon.c.smithi163.stdout: ** Compaction Stats [default] ** 2023-11-06T01:27:04.323 INFO:journalctl@ceph.mon.c.smithi163.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-06T01:27:04.324 INFO:journalctl@ceph.mon.c.smithi163.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-06T01:27:04.324 INFO:journalctl@ceph.mon.c.smithi163.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-06T01:27:04.325 INFO:journalctl@ceph.mon.c.smithi163.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-06T01:27:04.325 INFO:journalctl@ceph.mon.c.smithi163.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-06T01:27:04.325 INFO:journalctl@ceph.mon.c.smithi163.stdout: 2023-11-06T01:27:04.325 INFO:journalctl@ceph.mon.c.smithi163.stdout: ** Compaction Stats [default] ** 2023-11-06T01:27:04.325 INFO:journalctl@ceph.mon.c.smithi163.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-06T01:27:04.326 INFO:journalctl@ceph.mon.c.smithi163.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-06T01:27:04.326 INFO:journalctl@ceph.mon.c.smithi163.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-06T01:27:04.326 INFO:journalctl@ceph.mon.c.smithi163.stdout: 2023-11-06T01:27:04.326 INFO:journalctl@ceph.mon.c.smithi163.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-06T01:27:04.326 INFO:journalctl@ceph.mon.c.smithi163.stdout: 2023-11-06T01:27:04.327 INFO:journalctl@ceph.mon.c.smithi163.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-06T01:27:04.327 INFO:journalctl@ceph.mon.c.smithi163.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-06T01:27:04.327 INFO:journalctl@ceph.mon.c.smithi163.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-06T01:27:04.327 INFO:journalctl@ceph.mon.c.smithi163.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-06T01:27:04.327 INFO:journalctl@ceph.mon.c.smithi163.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-06T01:27:04.328 INFO:journalctl@ceph.mon.c.smithi163.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-06T01:27:04.328 INFO:journalctl@ceph.mon.c.smithi163.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-06T01:27:04.328 INFO:journalctl@ceph.mon.c.smithi163.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-06T01:27:04.328 INFO:journalctl@ceph.mon.c.smithi163.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-06T01:27:04.328 INFO:journalctl@ceph.mon.c.smithi163.stdout: Block cache BinnedLRUCache@0x56391e9191f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-11-06T01:27:04.329 INFO:journalctl@ceph.mon.c.smithi163.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-06T01:27:04.329 INFO:journalctl@ceph.mon.c.smithi163.stdout: 2023-11-06T01:27:04.329 INFO:journalctl@ceph.mon.c.smithi163.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-06T01:27:04.329 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-11-06T01:27:04.329 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: using public_addr v2:172.21.15.163:0/0 -> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] 2023-11-06T01:27:04.330 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: starting mon.c rank -1 at public addrs [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] at bind addrs [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:27:04.330 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(???) e0 preinit fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:27:04.330 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).mds e1 new map 2023-11-06T01:27:04.330 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).mds e1 print_map 2023-11-06T01:27:04.330 INFO:journalctl@ceph.mon.c.smithi163.stdout: e1 2023-11-06T01:27:04.331 INFO:journalctl@ceph.mon.c.smithi163.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-06T01:27:04.331 INFO:journalctl@ceph.mon.c.smithi163.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-06T01:27:04.331 INFO:journalctl@ceph.mon.c.smithi163.stdout: legacy client fscid: -1 2023-11-06T01:27:04.331 INFO:journalctl@ceph.mon.c.smithi163.stdout: 2023-11-06T01:27:04.331 INFO:journalctl@ceph.mon.c.smithi163.stdout: No filesystems configured 2023-11-06T01:27:04.332 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-06T01:27:04.332 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-06T01:27:04.332 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-06T01:27:04.332 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-06T01:27:04.333 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-06T01:27:04.333 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-06T01:27:04.333 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-06T01:27:04.333 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-06T01:27:04.333 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-06T01:27:04.333 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-06T01:27:04.334 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:04.334 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Deploying cephadm binary to smithi163 2023-11-06T01:27:04.334 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.334 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.334 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.335 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.335 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:27:04.335 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:04.335 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:04.335 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:04.336 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Updating smithi062:/etc/ceph/ceph.conf 2023-11-06T01:27:04.336 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Updating smithi062:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:04.336 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.336 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.336 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.337 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.337 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:04.337 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:27:04.337 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:04.337 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Updating smithi062:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-06T01:27:04.338 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Added host smithi163 2023-11-06T01:27:04.338 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.338 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.338 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:04.338 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:27:04.339 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.339 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:04.339 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/130866531' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-06T01:27:04.339 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.339 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.339 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.340 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.340 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:27:04.340 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:04.340 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:04.340 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Updating smithi163:/etc/ceph/ceph.conf 2023-11-06T01:27:04.341 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/130866531' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-06T01:27:04.341 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:27:04.341 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.341 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.341 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.342 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Updating smithi163:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:04.342 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:04.342 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:27:04.342 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Updating smithi163:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-06T01:27:04.342 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:04.343 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi053:172.21.15.53=a;smithi062:172.21.15.62=b;smithi163:172.21.15.163=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:04.343 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Saving service mon spec with placement smithi053:172.21.15.53=a;smithi062:172.21.15.62=b;smithi163:172.21.15.163=c;count:3 2023-11-06T01:27:04.343 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.343 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:04.343 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:04.344 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:04.344 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:04.344 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:04.344 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:04.344 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: Deploying daemon mon.c on smithi163 2023-11-06T01:27:04.345 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:04.345 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: from='client.? 172.21.15.163:0/1185888522' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-06T01:27:04.345 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-06T01:27:04.345 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-11-06T01:27:04.345 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta expand map: {default=false} 2023-11-06T01:27:04.346 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta from 'false' to 'false' 2023-11-06T01:27:04.346 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta expanded map: {default=false} 2023-11-06T01:27:04.346 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta expand map: {default=info} 2023-11-06T01:27:04.346 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta from 'info' to 'info' 2023-11-06T01:27:04.346 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta expanded map: {default=info} 2023-11-06T01:27:04.346 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta expand map: {default=daemon} 2023-11-06T01:27:04.347 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-06T01:27:04.347 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta expanded map: {default=daemon} 2023-11-06T01:27:04.347 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta expand map: {default=debug} 2023-11-06T01:27:04.347 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta from 'debug' to 'debug' 2023-11-06T01:27:04.347 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 ceph-mon[104416]: expand_channel_meta expanded map: {default=debug} 2023-11-06T01:27:04.348 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:03 smithi163 systemd[1]: Started Ceph mon.c for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:27:04.458 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-06T01:27:04.459 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph mon dump -f json 2023-11-06T01:27:05.013 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:27:09.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.169+0000 7facc81e2700 1 -- 172.21.15.163:0/3644560081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0057d0 msgr2=0x7facac007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:09.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.169+0000 7facc81e2700 1 --2- 172.21.15.163:0/3644560081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0057d0 0x7facac007c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7facc00ff790 tx=0x7facbc00feb0 comp rx=0 tx=0).stop 2023-11-06T01:27:09.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.169+0000 7facc81e2700 1 -- 172.21.15.163:0/3644560081 shutdown_connections 2023-11-06T01:27:09.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.169+0000 7facc81e2700 1 --2- 172.21.15.163:0/3644560081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0057d0 0x7facac007c60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:09.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.169+0000 7facc81e2700 1 -- 172.21.15.163:0/3644560081 >> 172.21.15.163:0/3644560081 conn(0x7facc00fb390 msgr2=0x7facc00fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.169+0000 7facc81e2700 1 -- 172.21.15.163:0/3644560081 shutdown_connections 2023-11-06T01:27:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.169+0000 7facc81e2700 1 -- 172.21.15.163:0/3644560081 wait complete. 2023-11-06T01:27:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.170+0000 7facc81e2700 1 Processor -- start 2023-11-06T01:27:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.170+0000 7facc81e2700 1 -- start start 2023-11-06T01:27:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc81e2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0057d0 0x7facc0193ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc81e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facc01941e0 0x7facc01975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc81e2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc0194770 con 0x7facac0057d0 2023-11-06T01:27:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc81e2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01948b0 con 0x7facc01941e0 2023-11-06T01:27:09.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc577d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facc01941e0 0x7facc01975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:09.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc577d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facc01941e0 0x7facc01975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46614/0 (socket says 172.21.15.163:46614) 2023-11-06T01:27:09.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc577d700 1 -- 172.21.15.163:0/2139635520 learned_addr learned my addr 172.21.15.163:0/2139635520 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-11-06T01:27:09.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc5f7e700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0057d0 0x7facc0193ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:09.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc577d700 1 -- 172.21.15.163:0/2139635520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facc01941e0 msgr2=0x7facc01975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-11-06T01:27:09.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc577d700 1 -- 172.21.15.163:0/2139635520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facc01941e0 msgr2=0x7facc01975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-06T01:27:09.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc577d700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facc01941e0 0x7facc01975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T01:27:09.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.171+0000 7facc577d700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facc01941e0 0x7facc01975a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T01:27:09.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.172+0000 7facc5f7e700 1 -- 172.21.15.163:0/2139635520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facc01941e0 msgr2=0x7facc01975a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:27:09.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.172+0000 7facc5f7e700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facc01941e0 0x7facc01975a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:09.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.172+0000 7facc5f7e700 1 -- 172.21.15.163:0/2139635520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc00f900 con 0x7facac0057d0 2023-11-06T01:27:09.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.172+0000 7facc5f7e700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0057d0 0x7facc0193ca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7facbc010010 tx=0x7facbc0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:09.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.172+0000 7facb6ffd700 1 -- 172.21.15.163:0/2139635520 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7facbc013960 con 0x7facac0057d0 2023-11-06T01:27:09.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.172+0000 7facb6ffd700 1 -- 172.21.15.163:0/2139635520 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facbc004030 con 0x7facac0057d0 2023-11-06T01:27:09.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.172+0000 7facc81e2700 1 -- 172.21.15.163:0/2139635520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc0197ae0 con 0x7facac0057d0 2023-11-06T01:27:09.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.172+0000 7facb6ffd700 1 -- 172.21.15.163:0/2139635520 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7facbc003af0 con 0x7facac0057d0 2023-11-06T01:27:09.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.172+0000 7facc81e2700 1 -- 172.21.15.163:0/2139635520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc0198030 con 0x7facac0057d0 2023-11-06T01:27:09.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.173+0000 7facb6ffd700 1 -- 172.21.15.163:0/2139635520 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7facbc011540 con 0x7facac0057d0 2023-11-06T01:27:09.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.173+0000 7facb6ffd700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faca4038220 0x7faca403a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:09.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.173+0000 7facc81e2700 1 -- 172.21.15.163:0/2139635520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc00623c0 con 0x7facac0057d0 2023-11-06T01:27:09.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.173+0000 7facb6ffd700 1 -- 172.21.15.163:0/2139635520 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7facbc000e40 con 0x7facac0057d0 2023-11-06T01:27:09.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.173+0000 7facc577d700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faca4038220 0x7faca403a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:09.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.174+0000 7facc577d700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faca4038220 0x7faca403a6e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7facb0009180 tx=0x7facb0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:09.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.179+0000 7facb6ffd700 1 -- 172.21.15.163:0/2139635520 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7facbc01a980 con 0x7facac0057d0 2023-11-06T01:27:09.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:09.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:27:09.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: mon.a calling monitor election 2023-11-06T01:27:09.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:09.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: mon.c calling monitor election 2023-11-06T01:27:09.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:09.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-06T01:27:09.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: monmap e2: 2 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-11-06T01:27:09.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: fsmap 2023-11-06T01:27:09.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:27:09.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: mgrmap e13: x(active, since 43s) 2023-11-06T01:27:09.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: overall HEALTH_OK 2023-11-06T01:27:09.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:09.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:09.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:09.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:09.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:09.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:09.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:27:09.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: mon.a calling monitor election 2023-11-06T01:27:09.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:09.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: mon.c calling monitor election 2023-11-06T01:27:09.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:09.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:09.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-06T01:27:09.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: monmap e2: 2 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-11-06T01:27:09.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: fsmap 2023-11-06T01:27:09.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:27:09.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: mgrmap e13: x(active, since 43s) 2023-11-06T01:27:09.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: overall HEALTH_OK 2023-11-06T01:27:09.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:09.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:09.349 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:09.349 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:09.349 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:09.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.446+0000 7facc81e2700 1 -- 172.21.15.163:0/2139635520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7facc00008d0 con 0x7facac0057d0 2023-11-06T01:27:09.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.446+0000 7facb6ffd700 1 -- 172.21.15.163:0/2139635520 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7facbc035b90 con 0x7facac0057d0 2023-11-06T01:27:09.449 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:27:09.449 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":2,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","modified":"2023-11-06T01:27:03.810306Z","created":"2023-11-06T01:25:23.107939Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:3300","nonce":0},{"type":"v1","addr":"172.21.15.163:6789","nonce":0}]},"addr":"172.21.15.163:6789/0","public_addr":"172.21.15.163:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-11-06T01:27:09.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.449+0000 7facc81e2700 1 -- 172.21.15.163:0/2139635520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faca4038220 msgr2=0x7faca403a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:09.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.449+0000 7facc81e2700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faca4038220 0x7faca403a6e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7facb0009180 tx=0x7facb0008040 comp rx=0 tx=0).stop 2023-11-06T01:27:09.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.449+0000 7facc81e2700 1 -- 172.21.15.163:0/2139635520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0057d0 msgr2=0x7facc0193ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:09.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.449+0000 7facc81e2700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0057d0 0x7facc0193ca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7facbc010010 tx=0x7facbc0047a0 comp rx=0 tx=0).stop 2023-11-06T01:27:09.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.449+0000 7facc81e2700 1 -- 172.21.15.163:0/2139635520 shutdown_connections 2023-11-06T01:27:09.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.449+0000 7facc81e2700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faca4038220 0x7faca403a6e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:09.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.449+0000 7facc81e2700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facac0057d0 0x7facc0193ca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:09.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.449+0000 7facc81e2700 1 --2- 172.21.15.163:0/2139635520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facc01941e0 0x7facc01975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:09.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.449+0000 7facc81e2700 1 -- 172.21.15.163:0/2139635520 >> 172.21.15.163:0/2139635520 conn(0x7facc00fb390 msgr2=0x7facc018cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:09.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.449+0000 7facc81e2700 1 -- 172.21.15.163:0/2139635520 shutdown_connections 2023-11-06T01:27:09.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:09.449+0000 7facc81e2700 1 -- 172.21.15.163:0/2139635520 wait complete. 2023-11-06T01:27:09.453 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 2 2023-11-06T01:27:10.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:09 smithi163 ceph-mon[104416]: Deploying daemon mon.b on smithi062 2023-11-06T01:27:10.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:09 smithi163 ceph-mon[104416]: from='client.? 172.21.15.163:0/2139635520' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-06T01:27:10.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:09 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:10.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:09 smithi053 ceph-mon[99472]: Deploying daemon mon.b on smithi062 2023-11-06T01:27:10.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:09 smithi053 ceph-mon[99472]: from='client.? 172.21.15.163:0/2139635520' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-06T01:27:10.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:09 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:10.775 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-06T01:27:10.776 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph mon dump -f json 2023-11-06T01:27:11.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:10 smithi053 ceph-mon[99472]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:11.095 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:27:10 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:27:10.809+0000 7fb073a2f700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-06T01:27:11.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:10 smithi163 ceph-mon[104416]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:11.341 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:27:12.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:11 smithi062 systemd[1]: Starting Ceph mon.b for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:27:12.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 podman[103884]: 2023-11-06T01:27:12.367 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: set uid:gid to 167:167 (ceph:ceph) 2023-11-06T01:27:12.367 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable), process ceph-mon, pid 7 2023-11-06T01:27:12.367 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pidfile_write: ignore empty --pid-file 2023-11-06T01:27:12.367 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: load: jerasure load: lrc 2023-11-06T01:27:12.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: RocksDB version: 7.9.2 2023-11-06T01:27:12.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Git sha 0 2023-11-06T01:27:12.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Compile date 2023-11-04 15:03:31 2023-11-06T01:27:12.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: DB SUMMARY 2023-11-06T01:27:12.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: DB Session ID: 73D55AE7OWYLOWX9BDOA 2023-11-06T01:27:12.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: CURRENT file: CURRENT 2023-11-06T01:27:12.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: IDENTITY file: IDENTITY 2023-11-06T01:27:12.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-06T01:27:12.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-11-06T01:27:12.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-11-06T01:27:12.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.error_if_exists: 0 2023-11-06T01:27:12.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.create_if_missing: 0 2023-11-06T01:27:12.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.paranoid_checks: 1 2023-11-06T01:27:12.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-06T01:27:12.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-06T01:27:12.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-06T01:27:12.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.env: 0x55e59c45e720 2023-11-06T01:27:12.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.fs: PosixFileSystem 2023-11-06T01:27:12.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.info_log: 0x55e59eafd440 2023-11-06T01:27:12.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_file_opening_threads: 16 2023-11-06T01:27:12.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.statistics: (nil) 2023-11-06T01:27:12.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.use_fsync: 0 2023-11-06T01:27:12.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_log_file_size: 0 2023-11-06T01:27:12.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-06T01:27:12.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-06T01:27:12.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.keep_log_file_num: 1000 2023-11-06T01:27:12.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.recycle_log_file_num: 0 2023-11-06T01:27:12.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.allow_fallocate: 1 2023-11-06T01:27:12.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.allow_mmap_reads: 0 2023-11-06T01:27:12.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.allow_mmap_writes: 0 2023-11-06T01:27:12.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.use_direct_reads: 0 2023-11-06T01:27:12.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-06T01:27:12.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.create_missing_column_families: 0 2023-11-06T01:27:12.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.db_log_dir: 2023-11-06T01:27:12.653 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.wal_dir: 2023-11-06T01:27:12.653 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-06T01:27:12.653 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-06T01:27:12.653 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-06T01:27:12.654 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-06T01:27:12.656 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-06T01:27:12.657 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-06T01:27:12.657 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.advise_random_on_open: 1 2023-11-06T01:27:12.657 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.db_write_buffer_size: 0 2023-11-06T01:27:12.657 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.write_buffer_manager: 0x55e59dd8e320 2023-11-06T01:27:12.657 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-06T01:27:12.658 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-06T01:27:12.658 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-06T01:27:12.658 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.rate_limiter: (nil) 2023-11-06T01:27:12.658 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-06T01:27:12.659 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.wal_recovery_mode: 2 2023-11-06T01:27:12.659 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.enable_thread_tracking: 0 2023-11-06T01:27:12.659 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.enable_pipelined_write: 0 2023-11-06T01:27:12.659 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.unordered_write: 0 2023-11-06T01:27:12.659 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-06T01:27:12.660 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-06T01:27:12.660 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-06T01:27:12.660 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-06T01:27:12.660 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.row_cache: None 2023-11-06T01:27:12.660 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.wal_filter: None 2023-11-06T01:27:12.661 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-06T01:27:12.661 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.allow_ingest_behind: 0 2023-11-06T01:27:12.661 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.two_write_queues: 0 2023-11-06T01:27:12.661 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.manual_wal_flush: 0 2023-11-06T01:27:12.661 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.wal_compression: 0 2023-11-06T01:27:12.662 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.atomic_flush: 0 2023-11-06T01:27:12.662 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-06T01:27:12.662 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-06T01:27:12.662 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-06T01:27:12.663 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.log_readahead_size: 0 2023-11-06T01:27:12.663 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-06T01:27:12.663 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.best_efforts_recovery: 0 2023-11-06T01:27:12.663 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-06T01:27:12.664 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-06T01:27:12.664 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.allow_data_in_errors: 0 2023-11-06T01:27:12.665 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.db_host_id: __hostname__ 2023-11-06T01:27:12.665 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-06T01:27:12.665 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_background_jobs: 2 2023-11-06T01:27:12.665 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_background_compactions: -1 2023-11-06T01:27:12.665 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_subcompactions: 1 2023-11-06T01:27:12.666 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-06T01:27:12.666 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-06T01:27:12.666 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-06T01:27:12.666 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_total_wal_size: 0 2023-11-06T01:27:12.667 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-06T01:27:12.667 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-06T01:27:12.668 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-06T01:27:12.668 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-06T01:27:12.668 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_open_files: -1 2023-11-06T01:27:12.668 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bytes_per_sync: 0 2023-11-06T01:27:12.668 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-06T01:27:12.669 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-06T01:27:12.669 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_readahead_size: 0 2023-11-06T01:27:12.669 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_background_flushes: -1 2023-11-06T01:27:12.669 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Compression algorithms supported: 2023-11-06T01:27:12.669 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-06T01:27:12.669 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: kZSTD supported: 0 2023-11-06T01:27:12.670 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: kXpressCompression supported: 0 2023-11-06T01:27:12.670 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: kLZ4HCCompression supported: 1 2023-11-06T01:27:12.670 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: kZlibCompression supported: 1 2023-11-06T01:27:12.670 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: kSnappyCompression supported: 1 2023-11-06T01:27:12.671 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: kLZ4Compression supported: 1 2023-11-06T01:27:12.671 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: kBZip2Compression supported: 0 2023-11-06T01:27:12.671 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-06T01:27:12.671 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-06T01:27:12.671 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-11-06T01:27:12.672 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-06T01:27:12.672 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-06T01:27:12.672 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.merge_operator: 2023-11-06T01:27:12.672 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_filter: None 2023-11-06T01:27:12.672 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_filter_factory: None 2023-11-06T01:27:12.673 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.sst_partitioner_factory: None 2023-11-06T01:27:12.673 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-06T01:27:12.673 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-06T01:27:12.673 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e59eafd540) 2023-11-06T01:27:12.673 INFO:journalctl@ceph.mon.b.smithi062.stdout: cache_index_and_filter_blocks: 1 2023-11-06T01:27:12.674 INFO:journalctl@ceph.mon.b.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-06T01:27:12.674 INFO:journalctl@ceph.mon.b.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-06T01:27:12.674 INFO:journalctl@ceph.mon.b.smithi062.stdout: pin_top_level_index_and_filter: 1 2023-11-06T01:27:12.674 INFO:journalctl@ceph.mon.b.smithi062.stdout: index_type: 0 2023-11-06T01:27:12.674 INFO:journalctl@ceph.mon.b.smithi062.stdout: data_block_index_type: 0 2023-11-06T01:27:12.675 INFO:journalctl@ceph.mon.b.smithi062.stdout: index_shortening: 1 2023-11-06T01:27:12.675 INFO:journalctl@ceph.mon.b.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-06T01:27:12.675 INFO:journalctl@ceph.mon.b.smithi062.stdout: checksum: 4 2023-11-06T01:27:12.675 INFO:journalctl@ceph.mon.b.smithi062.stdout: no_block_cache: 0 2023-11-06T01:27:12.675 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache: 0x55e59de0f1f0 2023-11-06T01:27:12.676 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache_name: BinnedLRUCache 2023-11-06T01:27:12.676 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache_options: 2023-11-06T01:27:12.676 INFO:journalctl@ceph.mon.b.smithi062.stdout: capacity : 536870912 2023-11-06T01:27:12.676 INFO:journalctl@ceph.mon.b.smithi062.stdout: num_shard_bits : 4 2023-11-06T01:27:12.676 INFO:journalctl@ceph.mon.b.smithi062.stdout: strict_capacity_limit : 0 2023-11-06T01:27:12.677 INFO:journalctl@ceph.mon.b.smithi062.stdout: high_pri_pool_ratio: 0.000 2023-11-06T01:27:12.677 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache_compressed: (nil) 2023-11-06T01:27:12.677 INFO:journalctl@ceph.mon.b.smithi062.stdout: persistent_cache: (nil) 2023-11-06T01:27:12.677 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_size: 4096 2023-11-06T01:27:12.677 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_size_deviation: 10 2023-11-06T01:27:12.678 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_restart_interval: 16 2023-11-06T01:27:12.678 INFO:journalctl@ceph.mon.b.smithi062.stdout: index_block_restart_interval: 1 2023-11-06T01:27:12.678 INFO:journalctl@ceph.mon.b.smithi062.stdout: metadata_block_size: 4096 2023-11-06T01:27:12.678 INFO:journalctl@ceph.mon.b.smithi062.stdout: partition_filters: 0 2023-11-06T01:27:12.678 INFO:journalctl@ceph.mon.b.smithi062.stdout: use_delta_encoding: 1 2023-11-06T01:27:12.679 INFO:journalctl@ceph.mon.b.smithi062.stdout: filter_policy: bloomfilter 2023-11-06T01:27:12.679 INFO:journalctl@ceph.mon.b.smithi062.stdout: whole_key_filtering: 1 2023-11-06T01:27:12.679 INFO:journalctl@ceph.mon.b.smithi062.stdout: verify_compression: 0 2023-11-06T01:27:12.679 INFO:journalctl@ceph.mon.b.smithi062.stdout: read_amp_bytes_per_bit: 0 2023-11-06T01:27:12.679 INFO:journalctl@ceph.mon.b.smithi062.stdout: format_version: 5 2023-11-06T01:27:12.680 INFO:journalctl@ceph.mon.b.smithi062.stdout: enable_index_compression: 1 2023-11-06T01:27:12.680 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_align: 0 2023-11-06T01:27:12.680 INFO:journalctl@ceph.mon.b.smithi062.stdout: max_auto_readahead_size: 262144 2023-11-06T01:27:12.680 INFO:journalctl@ceph.mon.b.smithi062.stdout: prepopulate_block_cache: 0 2023-11-06T01:27:12.680 INFO:journalctl@ceph.mon.b.smithi062.stdout: initial_auto_readahead_size: 8192 2023-11-06T01:27:12.681 INFO:journalctl@ceph.mon.b.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2023-11-06T01:27:12.681 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.write_buffer_size: 33554432 2023-11-06T01:27:12.681 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_write_buffer_number: 2 2023-11-06T01:27:12.681 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compression: NoCompression 2023-11-06T01:27:12.681 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bottommost_compression: Disabled 2023-11-06T01:27:12.682 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.prefix_extractor: nullptr 2023-11-06T01:27:12.682 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-06T01:27:12.682 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.num_levels: 7 2023-11-06T01:27:12.682 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-06T01:27:12.682 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-06T01:27:12.683 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-06T01:27:12.683 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-06T01:27:12.683 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-06T01:27:12.683 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-06T01:27:12.683 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-06T01:27:12.684 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-06T01:27:12.684 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-06T01:27:12.684 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-06T01:27:12.684 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-06T01:27:12.684 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-06T01:27:12.685 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-06T01:27:12.685 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compression_opts.level: 32767 2023-11-06T01:27:12.685 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compression_opts.strategy: 0 2023-11-06T01:27:12.685 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-06T01:27:12.685 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-06T01:27:12.686 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-06T01:27:12.686 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-06T01:27:12.686 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compression_opts.enabled: false 2023-11-06T01:27:12.686 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-06T01:27:12.686 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-06T01:27:12.687 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-06T01:27:12.687 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-06T01:27:12.687 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.target_file_size_base: 67108864 2023-11-06T01:27:12.687 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-06T01:27:12.687 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-06T01:27:12.688 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-06T01:27:12.688 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-06T01:27:12.688 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-06T01:27:12.688 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-06T01:27:12.688 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-06T01:27:12.689 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-06T01:27:12.689 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-06T01:27:12.689 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-06T01:27:12.689 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-06T01:27:12.689 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-06T01:27:12.690 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-06T01:27:12.690 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-06T01:27:12.690 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.arena_block_size: 1048576 2023-11-06T01:27:12.690 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-06T01:27:12.690 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-06T01:27:12.691 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.disable_auto_compactions: 0 2023-11-06T01:27:12.691 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-06T01:27:12.691 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-06T01:27:12.691 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-06T01:27:12.691 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-06T01:27:12.692 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-06T01:27:12.692 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-06T01:27:12.692 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-06T01:27:12.692 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-06T01:27:12.692 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-06T01:27:12.693 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-06T01:27:12.693 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.table_properties_collectors: 2023-11-06T01:27:12.693 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.inplace_update_support: 0 2023-11-06T01:27:12.693 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-06T01:27:12.694 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-06T01:27:12.694 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-06T01:27:12.694 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-06T01:27:12.694 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.bloom_locality: 0 2023-11-06T01:27:12.694 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.max_successive_merges: 0 2023-11-06T01:27:12.695 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-06T01:27:12.695 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.paranoid_file_checks: 0 2023-11-06T01:27:12.695 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.force_consistency_checks: 1 2023-11-06T01:27:12.695 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.report_bg_io_stats: 0 2023-11-06T01:27:12.695 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.ttl: 2592000 2023-11-06T01:27:12.696 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-06T01:27:12.696 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-06T01:27:12.696 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-06T01:27:12.696 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.enable_blob_files: false 2023-11-06T01:27:12.696 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.min_blob_size: 0 2023-11-06T01:27:12.697 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.blob_file_size: 268435456 2023-11-06T01:27:12.697 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-06T01:27:12.697 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-06T01:27:12.697 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-06T01:27:12.697 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-06T01:27:12.698 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-06T01:27:12.698 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.blob_file_starting_level: 0 2023-11-06T01:27:12.698 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-06T01:27:12.698 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-06T01:27:12.698 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-06T01:27:12.699 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bbad7347-5d20-4328-9361-67349e503a73 2023-11-06T01:27:12.699 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: EVENT_LOG_v1 {"time_micros": 1699234032367903, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-06T01:27:12.699 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-06T01:27:12.699 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: EVENT_LOG_v1 {"time_micros": 1699234032368317, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1699234032, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bbad7347-5d20-4328-9361-67349e503a73", "db_session_id": "73D55AE7OWYLOWX9BDOA", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-06T01:27:12.699 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: EVENT_LOG_v1 {"time_micros": 1699234032368386, "job": 1, "event": "recovery_finished"} 2023-11-06T01:27:12.700 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-06T01:27:12.700 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-06T01:27:12.700 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e59deac000 2023-11-06T01:27:12.700 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: DB pointer 0x55e59de98000 2023-11-06T01:27:12.700 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-06T01:27:12.701 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-06T01:27:12.701 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** DB Stats ** 2023-11-06T01:27:12.701 INFO:journalctl@ceph.mon.b.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-06T01:27:12.701 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-06T01:27:12.701 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-06T01:27:12.702 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-06T01:27:12.702 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-06T01:27:12.702 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-06T01:27:12.702 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-06T01:27:12.702 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2023-11-06T01:27:12.703 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** Compaction Stats [default] ** 2023-11-06T01:27:12.703 INFO:journalctl@ceph.mon.b.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-06T01:27:12.703 INFO:journalctl@ceph.mon.b.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-06T01:27:12.703 INFO:journalctl@ceph.mon.b.smithi062.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-06T01:27:12.703 INFO:journalctl@ceph.mon.b.smithi062.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-06T01:27:12.704 INFO:journalctl@ceph.mon.b.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-06T01:27:12.704 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2023-11-06T01:27:12.704 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** Compaction Stats [default] ** 2023-11-06T01:27:12.704 INFO:journalctl@ceph.mon.b.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-06T01:27:12.704 INFO:journalctl@ceph.mon.b.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-06T01:27:12.705 INFO:journalctl@ceph.mon.b.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-06T01:27:12.705 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2023-11-06T01:27:12.705 INFO:journalctl@ceph.mon.b.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-06T01:27:12.705 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2023-11-06T01:27:12.705 INFO:journalctl@ceph.mon.b.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-06T01:27:12.706 INFO:journalctl@ceph.mon.b.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-06T01:27:12.706 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-06T01:27:12.706 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-06T01:27:12.706 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-06T01:27:12.706 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-06T01:27:12.707 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-06T01:27:12.707 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-06T01:27:12.707 INFO:journalctl@ceph.mon.b.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-06T01:27:12.707 INFO:journalctl@ceph.mon.b.smithi062.stdout: Block cache BinnedLRUCache@0x55e59de0f1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-11-06T01:27:12.707 INFO:journalctl@ceph.mon.b.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-06T01:27:12.708 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2023-11-06T01:27:12.708 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-06T01:27:12.708 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-11-06T01:27:12.708 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: using public_addr v2:172.21.15.62:0/0 -> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] 2023-11-06T01:27:12.708 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: starting mon.b rank -1 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:27:12.709 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(???) e0 preinit fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:27:12.709 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).mds e1 new map 2023-11-06T01:27:12.709 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).mds e1 print_map 2023-11-06T01:27:12.709 INFO:journalctl@ceph.mon.b.smithi062.stdout: e1 2023-11-06T01:27:12.709 INFO:journalctl@ceph.mon.b.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-06T01:27:12.710 INFO:journalctl@ceph.mon.b.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-06T01:27:12.710 INFO:journalctl@ceph.mon.b.smithi062.stdout: legacy client fscid: -1 2023-11-06T01:27:12.710 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2023-11-06T01:27:12.710 INFO:journalctl@ceph.mon.b.smithi062.stdout: No filesystems configured 2023-11-06T01:27:12.711 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-06T01:27:12.711 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-06T01:27:12.711 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-06T01:27:12.711 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-06T01:27:12.711 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-06T01:27:12.712 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-06T01:27:12.713 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-06T01:27:12.713 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-06T01:27:12.713 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-06T01:27:12.714 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-06T01:27:12.714 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:12.714 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Deploying cephadm binary to smithi163 2023-11-06T01:27:12.714 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.714 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.715 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.715 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.715 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:27:12.715 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:12.715 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:12.716 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:12.716 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Updating smithi062:/etc/ceph/ceph.conf 2023-11-06T01:27:12.716 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Updating smithi062:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:12.716 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.716 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.717 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.717 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.717 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:12.717 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:27:12.717 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:12.718 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Updating smithi062:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-06T01:27:12.718 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Added host smithi163 2023-11-06T01:27:12.718 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.718 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.718 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:12.719 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:27:12.719 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.719 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:12.719 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/130866531' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-06T01:27:12.719 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.720 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.720 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.720 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.720 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:27:12.720 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:12.721 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:12.721 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Updating smithi163:/etc/ceph/ceph.conf 2023-11-06T01:27:12.721 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/130866531' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-06T01:27:12.721 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:27:12.721 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.722 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.722 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.722 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Updating smithi163:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:12.722 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:12.722 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:27:12.723 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Updating smithi163:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-06T01:27:12.723 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:12.723 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi053:172.21.15.53=a;smithi062:172.21.15.62=b;smithi163:172.21.15.163=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:12.723 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Saving service mon spec with placement smithi053:172.21.15.53=a;smithi062:172.21.15.62=b;smithi163:172.21.15.163=c;count:3 2023-11-06T01:27:12.723 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.724 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:12.724 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:12.724 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:12.724 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.725 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:12.725 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:12.725 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Deploying daemon mon.c on smithi163 2023-11-06T01:27:12.725 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:12.725 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='client.? 172.21.15.163:0/1185888522' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-06T01:27:12.726 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:12.726 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:27:12.726 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:12.726 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.a calling monitor election 2023-11-06T01:27:12.727 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:12.727 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:12.727 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:12.727 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.c calling monitor election 2023-11-06T01:27:12.727 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:12.728 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:12.728 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:12.728 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:12.728 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-06T01:27:12.728 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: monmap e2: 2 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-11-06T01:27:12.728 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: fsmap 2023-11-06T01:27:12.729 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:27:12.729 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mgrmap e13: x(active, since 43s) 2023-11-06T01:27:12.729 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: overall HEALTH_OK 2023-11-06T01:27:12.729 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 bash[103884]: 28bde106559b6a801cba3c4ea642c9def107033422c8b59512ed93fdbb6edcfd 2023-11-06T01:27:12.730 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.730 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.730 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:12.730 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:12.730 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:12.731 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: Deploying daemon mon.b on smithi062 2023-11-06T01:27:12.731 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='client.? 172.21.15.163:0/2139635520' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-06T01:27:12.731 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:12.731 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:12.731 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-06T01:27:12.732 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-11-06T01:27:12.732 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta expand map: {default=false} 2023-11-06T01:27:12.732 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta from 'false' to 'false' 2023-11-06T01:27:12.732 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta expanded map: {default=false} 2023-11-06T01:27:12.732 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta expand map: {default=info} 2023-11-06T01:27:12.733 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta from 'info' to 'info' 2023-11-06T01:27:12.733 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta expanded map: {default=info} 2023-11-06T01:27:12.733 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta expand map: {default=daemon} 2023-11-06T01:27:12.733 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-06T01:27:12.733 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta expanded map: {default=daemon} 2023-11-06T01:27:12.733 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta expand map: {default=debug} 2023-11-06T01:27:12.734 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta from 'debug' to 'debug' 2023-11-06T01:27:12.734 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 ceph-mon[103924]: expand_channel_meta expanded map: {default=debug} 2023-11-06T01:27:12.734 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:12 smithi062 systemd[1]: Started Ceph mon.b for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:27:17.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: mon.c calling monitor election 2023-11-06T01:27:17.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:27:17.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: mon.a calling monitor election 2023-11-06T01:27:17.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:17.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:17.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:17.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-06T01:27:17.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: monmap e3: 3 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-11-06T01:27:17.743 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: fsmap 2023-11-06T01:27:17.743 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:27:17.743 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: mgrmap e13: x(active, since 52s) 2023-11-06T01:27:17.743 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-06T01:27:17.743 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-06T01:27:17.744 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-06T01:27:17.744 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: mon.b (rank 2) addr [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] is down (out of quorum) 2023-11-06T01:27:17.744 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:17.744 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:17.744 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:17.745 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:17.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: mon.c calling monitor election 2023-11-06T01:27:17.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:27:17.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: mon.a calling monitor election 2023-11-06T01:27:17.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:17.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:17.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:17.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:17.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-06T01:27:17.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: monmap e3: 3 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-11-06T01:27:17.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: fsmap 2023-11-06T01:27:17.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:27:17.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: mgrmap e13: x(active, since 52s) 2023-11-06T01:27:17.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-06T01:27:17.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-06T01:27:17.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-06T01:27:17.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: mon.b (rank 2) addr [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] is down (out of quorum) 2023-11-06T01:27:17.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:17.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:17.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:17.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:18.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.530+0000 7eff8b5e3700 1 -- 172.21.15.163:0/667981341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70007ed0 msgr2=0x7eff700082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:18.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.530+0000 7eff8b5e3700 1 --2- 172.21.15.163:0/667981341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70007ed0 0x7eff700082f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff84100980 tx=0x7eff7400ff20 comp rx=0 tx=0).stop 2023-11-06T01:27:18.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.530+0000 7eff8b5e3700 1 -- 172.21.15.163:0/667981341 shutdown_connections 2023-11-06T01:27:18.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.530+0000 7eff8b5e3700 1 --2- 172.21.15.163:0/667981341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70007ed0 0x7eff700082f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:18.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.530+0000 7eff8b5e3700 1 -- 172.21.15.163:0/667981341 >> 172.21.15.163:0/667981341 conn(0x7eff84075340 msgr2=0x7eff84075750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:18.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.530+0000 7eff8b5e3700 1 -- 172.21.15.163:0/667981341 shutdown_connections 2023-11-06T01:27:18.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.531+0000 7eff8b5e3700 1 -- 172.21.15.163:0/667981341 wait complete. 2023-11-06T01:27:18.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.532+0000 7eff8b5e3700 1 Processor -- start 2023-11-06T01:27:18.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.532+0000 7eff8b5e3700 1 -- start start 2023-11-06T01:27:18.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.532+0000 7eff8b5e3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70007ed0 0x7eff84193e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:18.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.532+0000 7eff8b5e3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff84194340 0x7eff841987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:18.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.532+0000 7eff8b5e3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff84198cf0 0x7eff841991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:18.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.532+0000 7eff8b5e3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff84194a30 con 0x7eff70007ed0 2023-11-06T01:27:18.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.532+0000 7eff8b5e3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff841a0a60 con 0x7eff84194340 2023-11-06T01:27:18.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.532+0000 7eff8b5e3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff841a0ba0 con 0x7eff84198cf0 2023-11-06T01:27:18.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.532+0000 7eff89b80700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff84198cf0 0x7eff841991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:18.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.532+0000 7eff89b80700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff84198cf0 0x7eff841991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46646/0 (socket says 172.21.15.163:46646) 2023-11-06T01:27:18.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.532+0000 7eff89b80700 1 -- 172.21.15.163:0/1831682073 learned_addr learned my addr 172.21.15.163:0/1831682073 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-11-06T01:27:18.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff8937f700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70007ed0 0x7eff84193e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:18.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff89b80700 1 -- 172.21.15.163:0/1831682073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff84198cf0 msgr2=0x7eff841991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-11-06T01:27:18.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff89b80700 1 -- 172.21.15.163:0/1831682073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff84198cf0 msgr2=0x7eff841991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-06T01:27:18.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff89b80700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff84198cf0 0x7eff841991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T01:27:18.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff89b80700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff84198cf0 0x7eff841991a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T01:27:18.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff88b7e700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff84194340 0x7eff841987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:18.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff88b7e700 1 -- 172.21.15.163:0/1831682073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff84194340 msgr2=0x7eff841987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-11-06T01:27:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff88b7e700 1 -- 172.21.15.163:0/1831682073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff84194340 msgr2=0x7eff841987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-06T01:27:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff88b7e700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff84194340 0x7eff841987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T01:27:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff88b7e700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff84194340 0x7eff841987b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T01:27:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff8937f700 1 -- 172.21.15.163:0/1831682073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff84198cf0 msgr2=0x7eff841991a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:27:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff8937f700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff84198cf0 0x7eff841991a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff8937f700 1 -- 172.21.15.163:0/1831682073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff84194340 msgr2=0x7eff841987b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:27:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff8937f700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff84194340 0x7eff841987b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff8937f700 1 -- 172.21.15.163:0/1831682073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7400f970 con 0x7eff70007ed0 2023-11-06T01:27:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff8937f700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70007ed0 0x7eff84193e00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7eff74000c00 tx=0x7eff74012c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff7a7fc700 1 -- 172.21.15.163:0/1831682073 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff74021040 con 0x7eff70007ed0 2023-11-06T01:27:18.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff7a7fc700 1 -- 172.21.15.163:0/1831682073 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff74004ae0 con 0x7eff70007ed0 2023-11-06T01:27:18.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff8b5e3700 1 -- 172.21.15.163:0/1831682073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff841a0e20 con 0x7eff70007ed0 2023-11-06T01:27:18.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff7a7fc700 1 -- 172.21.15.163:0/1831682073 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7401d650 con 0x7eff70007ed0 2023-11-06T01:27:18.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.533+0000 7eff8b5e3700 1 -- 172.21.15.163:0/1831682073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff841a1240 con 0x7eff70007ed0 2023-11-06T01:27:18.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.534+0000 7eff7a7fc700 1 -- 172.21.15.163:0/1831682073 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7eff7401d7b0 con 0x7eff70007ed0 2023-11-06T01:27:18.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.534+0000 7eff8b5e3700 1 -- 172.21.15.163:0/1831682073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff840623c0 con 0x7eff70007ed0 2023-11-06T01:27:18.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.534+0000 7eff7a7fc700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff68038870 0x7eff6803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:18.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.535+0000 7eff7a7fc700 1 -- 172.21.15.163:0/1831682073 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7eff7404dd00 con 0x7eff70007ed0 2023-11-06T01:27:18.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.535+0000 7eff88b7e700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff68038870 0x7eff6803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:18.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.535+0000 7eff88b7e700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff68038870 0x7eff6803ad30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7eff80009850 tx=0x7eff80008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:18.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.540+0000 7eff7a7fc700 1 -- 172.21.15.163:0/1831682073 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7eff74037350 con 0x7eff70007ed0 2023-11-06T01:27:18.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:18 smithi163 ceph-mon[104416]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:18.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:18.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:18.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.838+0000 7eff8b5e3700 1 -- 172.21.15.163:0/1831682073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7eff84106720 con 0x7eff70007ed0 2023-11-06T01:27:18.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.838+0000 7eff7a7fc700 1 -- 172.21.15.163:0/1831682073 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7eff7402a030 con 0x7eff70007ed0 2023-11-06T01:27:18.840 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-11-06T01:27:18.840 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":3,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","modified":"2023-11-06T01:27:12.395713Z","created":"2023-11-06T01:25:23.107939Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:3300","nonce":0},{"type":"v1","addr":"172.21.15.163:6789","nonce":0}]},"addr":"172.21.15.163:6789/0","public_addr":"172.21.15.163:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-11-06T01:27:18.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 -- 172.21.15.163:0/1831682073 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff68038870 msgr2=0x7eff6803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:18.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff68038870 0x7eff6803ad30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7eff80009850 tx=0x7eff80008040 comp rx=0 tx=0).stop 2023-11-06T01:27:18.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 -- 172.21.15.163:0/1831682073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70007ed0 msgr2=0x7eff84193e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:18.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70007ed0 0x7eff84193e00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7eff74000c00 tx=0x7eff74012c60 comp rx=0 tx=0).stop 2023-11-06T01:27:18.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 -- 172.21.15.163:0/1831682073 shutdown_connections 2023-11-06T01:27:18.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff68038870 0x7eff6803ad30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:18.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff84198cf0 0x7eff841991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:18.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70007ed0 0x7eff84193e00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:18.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 --2- 172.21.15.163:0/1831682073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff84194340 0x7eff841987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:18.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 -- 172.21.15.163:0/1831682073 >> 172.21.15.163:0/1831682073 conn(0x7eff84075340 msgr2=0x7eff84101440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:18.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 -- 172.21.15.163:0/1831682073 shutdown_connections 2023-11-06T01:27:18.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:18.841+0000 7eff8b5e3700 1 -- 172.21.15.163:0/1831682073 wait complete. 2023-11-06T01:27:18.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:18 smithi053 ceph-mon[99472]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:18.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:18.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:18.846 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 3 2023-11-06T01:27:19.213 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-11-06T01:27:19.213 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph config generate-minimal-conf 2023-11-06T01:27:19.801 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:27:19.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: mon.c calling monitor election 2023-11-06T01:27:19.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:27:19.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: mon.a calling monitor election 2023-11-06T01:27:19.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:19.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:27:19.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:19.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:19.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:19.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:19.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:19.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:19.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:19.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-06T01:27:19.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: monmap e3: 3 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-11-06T01:27:19.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: fsmap 2023-11-06T01:27:19.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:27:19.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: mgrmap e13: x(active, since 52s) 2023-11-06T01:27:19.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-06T01:27:19.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-06T01:27:19.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-06T01:27:19.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: mon.b (rank 2) addr [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] is down (out of quorum) 2023-11-06T01:27:19.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:19.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:19.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:19.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:19.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:19.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:19.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:20.229 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: mon.b calling monitor election 2023-11-06T01:27:20.230 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: mon.b calling monitor election 2023-11-06T01:27:20.230 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: mon.c calling monitor election 2023-11-06T01:27:20.230 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: mon.a calling monitor election 2023-11-06T01:27:20.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-06T01:27:20.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: monmap e3: 3 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-11-06T01:27:20.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: fsmap 2023-11-06T01:27:20.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:27:20.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: mgrmap e13: x(active, since 54s) 2023-11-06T01:27:20.231 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-06T01:27:20.232 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: Cluster is now healthy 2023-11-06T01:27:20.232 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: overall HEALTH_OK 2023-11-06T01:27:20.232 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.232 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.232 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.233 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.233 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.233 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.233 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.233 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.234 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.234 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.234 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.234 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:20.234 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-06T01:27:20.235 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:20.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: mon.b calling monitor election 2023-11-06T01:27:20.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: mon.b calling monitor election 2023-11-06T01:27:20.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: mon.c calling monitor election 2023-11-06T01:27:20.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: mon.a calling monitor election 2023-11-06T01:27:20.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-06T01:27:20.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: monmap e3: 3 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-11-06T01:27:20.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: fsmap 2023-11-06T01:27:20.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:27:20.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: mgrmap e13: x(active, since 54s) 2023-11-06T01:27:20.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-06T01:27:20.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: Cluster is now healthy 2023-11-06T01:27:20.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: overall HEALTH_OK 2023-11-06T01:27:20.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:20.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-06T01:27:20.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:20.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: mon.b calling monitor election 2023-11-06T01:27:20.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: mon.b calling monitor election 2023-11-06T01:27:20.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: mon.c calling monitor election 2023-11-06T01:27:20.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: mon.a calling monitor election 2023-11-06T01:27:20.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-06T01:27:20.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: monmap e3: 3 mons at {a=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-11-06T01:27:20.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: fsmap 2023-11-06T01:27:20.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: osdmap e4: 0 total, 0 up, 0 in 2023-11-06T01:27:20.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: mgrmap e13: x(active, since 54s) 2023-11-06T01:27:20.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-06T01:27:20.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: Cluster is now healthy 2023-11-06T01:27:20.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: overall HEALTH_OK 2023-11-06T01:27:20.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:20.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:20.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-06T01:27:20.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:21.245 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:21 smithi053 ceph-mon[99472]: Updating smithi062:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:21.246 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:21 smithi053 ceph-mon[99472]: Updating smithi163:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:21.246 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:21 smithi053 ceph-mon[99472]: Updating smithi053:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:21.246 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:21 smithi053 ceph-mon[99472]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:21.246 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:21 smithi053 ceph-mon[99472]: Reconfiguring mon.a (unknown last config time)... 2023-11-06T01:27:21.247 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:21 smithi053 ceph-mon[99472]: Reconfiguring daemon mon.a on smithi053 2023-11-06T01:27:21.247 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:21.247 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:21.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.289+0000 7f7517c00700 1 -- 172.21.15.53:0/1305427448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7510106820 msgr2=0x7f7510106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:21.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.289+0000 7f7517c00700 1 --2- 172.21.15.53:0/1305427448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7510106820 0x7f7510106c00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f750c009910 tx=0x7f750c009c20 comp rx=0 tx=0).stop 2023-11-06T01:27:21.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.289+0000 7f7517c00700 1 -- 172.21.15.53:0/1305427448 shutdown_connections 2023-11-06T01:27:21.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.289+0000 7f7517c00700 1 --2- 172.21.15.53:0/1305427448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7510106820 0x7f7510106c00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:21.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.289+0000 7f7517c00700 1 -- 172.21.15.53:0/1305427448 >> 172.21.15.53:0/1305427448 conn(0x7f75100fc350 msgr2=0x7f75100fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:21.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.289+0000 7f7517c00700 1 -- 172.21.15.53:0/1305427448 shutdown_connections 2023-11-06T01:27:21.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.289+0000 7f7517c00700 1 -- 172.21.15.53:0/1305427448 wait complete. 2023-11-06T01:27:21.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.290+0000 7f7517c00700 1 Processor -- start 2023-11-06T01:27:21.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.290+0000 7f7517c00700 1 -- start start 2023-11-06T01:27:21.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.290+0000 7f7517c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7510106820 0x7f7510195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:21.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.290+0000 7f7517c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7510196370 0x7f751019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:21.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.290+0000 7f7517c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f751019ad20 0x7f751019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:21.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.290+0000 7f7517c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7510196a60 con 0x7f7510196370 2023-11-06T01:27:21.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.290+0000 7f7517c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75101a2a90 con 0x7f7510106820 2023-11-06T01:27:21.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.290+0000 7f7517c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75101a2bd0 con 0x7f751019ad20 2023-11-06T01:27:21.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.290+0000 7f751519b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7510196370 0x7f751019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:21.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.290+0000 7f751519b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7510196370 0x7f751019a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37044/0 (socket says 172.21.15.53:37044) 2023-11-06T01:27:21.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.290+0000 7f751519b700 1 -- 172.21.15.53:0/4159824734 learned_addr learned my addr 172.21.15.53:0/4159824734 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:27:21.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f751599c700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7510106820 0x7f7510195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:21.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f751619d700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f751019ad20 0x7f751019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:21.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f751519b700 1 -- 172.21.15.53:0/4159824734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f751019ad20 msgr2=0x7f751019b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:21.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f751519b700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f751019ad20 0x7f751019b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:21.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f751519b700 1 -- 172.21.15.53:0/4159824734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7510106820 msgr2=0x7f7510195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:21.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f751519b700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7510106820 0x7f7510195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:21.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f751519b700 1 -- 172.21.15.53:0/4159824734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c0095f0 con 0x7f7510196370 2023-11-06T01:27:21.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f751599c700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7510106820 0x7f7510195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:27:21.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f751519b700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7510196370 0x7f751019a7e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f750000b780 tx=0x7f750000bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:21.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f7506ffd700 1 -- 172.21.15.53:0/4159824734 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500004230 con 0x7f7510196370 2023-11-06T01:27:21.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f7517c00700 1 -- 172.21.15.53:0/4159824734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75101a2eb0 con 0x7f7510196370 2023-11-06T01:27:21.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f7506ffd700 1 -- 172.21.15.53:0/4159824734 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7500004390 con 0x7f7510196370 2023-11-06T01:27:21.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f7506ffd700 1 -- 172.21.15.53:0/4159824734 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500011b30 con 0x7f7510196370 2023-11-06T01:27:21.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.291+0000 7f7517c00700 1 -- 172.21.15.53:0/4159824734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75101a3370 con 0x7f7510196370 2023-11-06T01:27:21.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.292+0000 7f7506ffd700 1 -- 172.21.15.53:0/4159824734 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f7500011cd0 con 0x7f7510196370 2023-11-06T01:27:21.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.292+0000 7f7506ffd700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f74fc038870 0x7f74fc03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:21.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.292+0000 7f7506ffd700 1 -- 172.21.15.53:0/4159824734 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f750004a850 con 0x7f7510196370 2023-11-06T01:27:21.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.292+0000 7f751599c700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f74fc038870 0x7f74fc03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:21.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.293+0000 7f751599c700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f74fc038870 0x7f74fc03ad30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f750c006010 tx=0x7f750c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:21.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.293+0000 7f7517c00700 1 -- 172.21.15.53:0/4159824734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75100623c0 con 0x7f7510196370 2023-11-06T01:27:21.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.298+0000 7f7506ffd700 1 -- 172.21.15.53:0/4159824734 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f75000193a0 con 0x7f7510196370 2023-11-06T01:27:21.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:21 smithi062 ceph-mon[103924]: Updating smithi062:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:21.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:21 smithi062 ceph-mon[103924]: Updating smithi163:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:21.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:21 smithi062 ceph-mon[103924]: Updating smithi053:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:21.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:21 smithi062 ceph-mon[103924]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:21.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:21 smithi062 ceph-mon[103924]: Reconfiguring mon.a (unknown last config time)... 2023-11-06T01:27:21.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:21 smithi062 ceph-mon[103924]: Reconfiguring daemon mon.a on smithi053 2023-11-06T01:27:21.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:21.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:21.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:21 smithi163 ceph-mon[104416]: Updating smithi062:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:21.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:21 smithi163 ceph-mon[104416]: Updating smithi163:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:21.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:21 smithi163 ceph-mon[104416]: Updating smithi053:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-11-06T01:27:21.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:21 smithi163 ceph-mon[104416]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:21.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:21 smithi163 ceph-mon[104416]: Reconfiguring mon.a (unknown last config time)... 2023-11-06T01:27:21.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:21 smithi163 ceph-mon[104416]: Reconfiguring daemon mon.a on smithi053 2023-11-06T01:27:21.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:27:21.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:21.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.501+0000 7f7517c00700 1 -- 172.21.15.53:0/4159824734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f75101a3690 con 0x7f7510196370 2023-11-06T01:27:21.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.501+0000 7f7506ffd700 1 -- 172.21.15.53:0/4159824734 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f7500037030 con 0x7f7510196370 2023-11-06T01:27:21.504 INFO:teuthology.orchestra.run.smithi053.stdout:# minimal ceph.conf for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:27:21.504 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2023-11-06T01:27:21.504 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:27:21.505 INFO:teuthology.orchestra.run.smithi053.stdout: mon_host = [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] 2023-11-06T01:27:21.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.503+0000 7f7517c00700 1 -- 172.21.15.53:0/4159824734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f74fc038870 msgr2=0x7f74fc03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:21.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.503+0000 7f7517c00700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f74fc038870 0x7f74fc03ad30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f750c006010 tx=0x7f750c0058e0 comp rx=0 tx=0).stop 2023-11-06T01:27:21.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.503+0000 7f7517c00700 1 -- 172.21.15.53:0/4159824734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7510196370 msgr2=0x7f751019a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:21.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.503+0000 7f7517c00700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7510196370 0x7f751019a7e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f750000b780 tx=0x7f750000bb40 comp rx=0 tx=0).stop 2023-11-06T01:27:21.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.504+0000 7f7517c00700 1 -- 172.21.15.53:0/4159824734 shutdown_connections 2023-11-06T01:27:21.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.504+0000 7f7517c00700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f74fc038870 0x7f74fc03ad30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:21.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.504+0000 7f7517c00700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7510106820 0x7f7510195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:21.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.504+0000 7f7517c00700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f751019ad20 0x7f751019b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:21.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.504+0000 7f7517c00700 1 --2- 172.21.15.53:0/4159824734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7510196370 0x7f751019a7e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:21.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.504+0000 7f7517c00700 1 -- 172.21.15.53:0/4159824734 >> 172.21.15.53:0/4159824734 conn(0x7f75100fc350 msgr2=0x7f7510102320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:21.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.504+0000 7f7517c00700 1 -- 172.21.15.53:0/4159824734 shutdown_connections 2023-11-06T01:27:21.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:21.504+0000 7f7517c00700 1 -- 172.21.15.53:0/4159824734 wait complete. 2023-11-06T01:27:21.863 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-11-06T01:27:21.863 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:27:21.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2023-11-06T01:27:21.894 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:27:21.895 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:27:21.962 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-11-06T01:27:21.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2023-11-06T01:27:21.993 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-11-06T01:27:21.993 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:27:22.061 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-11-06T01:27:22.061 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.conf 2023-11-06T01:27:22.096 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-11-06T01:27:22.097 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-06T01:27:22.167 INFO:tasks.cephadm:Adding mgr.x on smithi053 2023-11-06T01:27:22.167 INFO:tasks.cephadm:Adding mgr.y on smithi062 2023-11-06T01:27:22.168 INFO:tasks.cephadm:Adding mgr.z on smithi163 2023-11-06T01:27:22.168 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch apply mgr '3;smithi053=x;smithi062=y;smithi163=z' 2023-11-06T01:27:22.467 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:22.467 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:22.467 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:22 smithi062 ceph-mon[103924]: Reconfiguring mon.b (monmap changed)... 2023-11-06T01:27:22.468 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:22.468 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-06T01:27:22.469 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:22.469 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:22 smithi062 ceph-mon[103924]: Reconfiguring daemon mon.b on smithi062 2023-11-06T01:27:22.469 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:22 smithi062 ceph-mon[103924]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:22.470 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:22 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/4159824734' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:22.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:22.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:22.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:22 smithi163 ceph-mon[104416]: Reconfiguring mon.b (monmap changed)... 2023-11-06T01:27:22.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:22.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-06T01:27:22.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:22.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:22 smithi163 ceph-mon[104416]: Reconfiguring daemon mon.b on smithi062 2023-11-06T01:27:22.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:22 smithi163 ceph-mon[104416]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:22.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:22 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/4159824734' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:22.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:22.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:22.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:22 smithi053 ceph-mon[99472]: Reconfiguring mon.b (monmap changed)... 2023-11-06T01:27:22.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:22.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-06T01:27:22.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:22.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:22 smithi053 ceph-mon[99472]: Reconfiguring daemon mon.b on smithi062 2023-11-06T01:27:22.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:22 smithi053 ceph-mon[99472]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:22.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:22 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/4159824734' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:22.762 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:27:23.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:23 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:24.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:23 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:24.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:23 smithi053 ceph-mon[99472]: Reconfiguring mon.c (monmap changed)... 2023-11-06T01:27:24.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:23 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:24.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:23 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-06T01:27:24.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:23 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:24.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:23 smithi053 ceph-mon[99472]: Reconfiguring daemon mon.c on smithi163 2023-11-06T01:27:24.096 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:23 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:24.097 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:23 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:24.097 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:23 smithi163 ceph-mon[104416]: Reconfiguring mon.c (monmap changed)... 2023-11-06T01:27:24.097 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:23 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:24.098 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:23 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-06T01:27:24.098 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:23 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:24.098 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:23 smithi163 ceph-mon[104416]: Reconfiguring daemon mon.c on smithi163 2023-11-06T01:27:24.099 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:23 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:24.100 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:23 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:24.100 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:23 smithi062 ceph-mon[103924]: Reconfiguring mon.c (monmap changed)... 2023-11-06T01:27:24.100 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:23 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-06T01:27:24.101 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:23 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-06T01:27:24.101 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:23 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:24.101 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:23 smithi062 ceph-mon[103924]: Reconfiguring daemon mon.c on smithi163 2023-11-06T01:27:24.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.315+0000 7f4442bb4700 1 -- 172.21.15.163:0/601389371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f443c0ffd20 msgr2=0x7f443c100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:24.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.315+0000 7f4442bb4700 1 --2- 172.21.15.163:0/601389371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f443c0ffd20 0x7f443c100140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4430009990 tx=0x7f4430009ca0 comp rx=0 tx=0).stop 2023-11-06T01:27:24.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.315+0000 7f4442bb4700 1 -- 172.21.15.163:0/601389371 shutdown_connections 2023-11-06T01:27:24.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.315+0000 7f4442bb4700 1 --2- 172.21.15.163:0/601389371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c1018c0 0x7f443c108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:24.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.315+0000 7f4442bb4700 1 --2- 172.21.15.163:0/601389371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f443c0ffd20 0x7f443c100140 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:24.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.315+0000 7f4442bb4700 1 --2- 172.21.15.163:0/601389371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f443c100f20 0x7f443c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:24.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.315+0000 7f4442bb4700 1 -- 172.21.15.163:0/601389371 >> 172.21.15.163:0/601389371 conn(0x7f443c0fb2a0 msgr2=0x7f443c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:24.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.315+0000 7f4442bb4700 1 -- 172.21.15.163:0/601389371 shutdown_connections 2023-11-06T01:27:24.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.315+0000 7f4442bb4700 1 -- 172.21.15.163:0/601389371 wait complete. 2023-11-06T01:27:24.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f4442bb4700 1 Processor -- start 2023-11-06T01:27:24.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f4442bb4700 1 -- start start 2023-11-06T01:27:24.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f4442bb4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c0ffd20 0x7f443c196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:24.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f4442bb4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f443c100f20 0x7f443c1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:24.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f4442bb4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f443c1018c0 0x7f443c19e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:24.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f4442bb4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c19e730 con 0x7f443c1018c0 2023-11-06T01:27:24.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f4442bb4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c19e870 con 0x7f443c0ffd20 2023-11-06T01:27:24.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f4442bb4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c19ea00 con 0x7f443c100f20 2023-11-06T01:27:24.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f443bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f443c100f20 0x7f443c1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:24.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f443bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f443c100f20 0x7f443c1970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46658/0 (socket says 172.21.15.163:46658) 2023-11-06T01:27:24.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f443bfff700 1 -- 172.21.15.163:0/1896378716 learned_addr learned my addr 172.21.15.163:0/1896378716 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-11-06T01:27:24.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f4440950700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c0ffd20 0x7f443c196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:24.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f4441151700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f443c1018c0 0x7f443c19e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:24.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f443bfff700 1 -- 172.21.15.163:0/1896378716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c0ffd20 msgr2=0x7f443c196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:24.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f443bfff700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c0ffd20 0x7f443c196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:24.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f443bfff700 1 -- 172.21.15.163:0/1896378716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f443c1018c0 msgr2=0x7f443c19e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:24.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f443bfff700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f443c1018c0 0x7f443c19e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:24.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f443bfff700 1 -- 172.21.15.163:0/1896378716 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44300095f0 con 0x7f443c100f20 2023-11-06T01:27:24.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.316+0000 7f4441151700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f443c1018c0 0x7f443c19e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:27:24.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.317+0000 7f443bfff700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f443c100f20 0x7f443c1970e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f442800c950 tx=0x7f442800cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:24.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.317+0000 7f4439ffb700 1 -- 172.21.15.163:0/1896378716 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428007ac0 con 0x7f443c100f20 2023-11-06T01:27:24.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.317+0000 7f4442bb4700 1 -- 172.21.15.163:0/1896378716 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c19ece0 con 0x7f443c100f20 2023-11-06T01:27:24.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.317+0000 7f4439ffb700 1 -- 172.21.15.163:0/1896378716 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4428004cf0 con 0x7f443c100f20 2023-11-06T01:27:24.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.317+0000 7f4439ffb700 1 -- 172.21.15.163:0/1896378716 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428005580 con 0x7f443c100f20 2023-11-06T01:27:24.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.317+0000 7f4442bb4700 1 -- 172.21.15.163:0/1896378716 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443c19f210 con 0x7f443c100f20 2023-11-06T01:27:24.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.318+0000 7f4439ffb700 1 -- 172.21.15.163:0/1896378716 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f442800f450 con 0x7f443c100f20 2023-11-06T01:27:24.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.318+0000 7f4439ffb700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f442c0385d0 0x7f442c03aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:24.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.318+0000 7f4439ffb700 1 -- 172.21.15.163:0/1896378716 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f442804aaa0 con 0x7f443c100f20 2023-11-06T01:27:24.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.318+0000 7f4442bb4700 1 -- 172.21.15.163:0/1896378716 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443c10b5f0 con 0x7f443c100f20 2023-11-06T01:27:24.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.318+0000 7f4440950700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f442c0385d0 0x7f442c03aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:24.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.318+0000 7f4440950700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f442c0385d0 0x7f442c03aa90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f443000b5c0 tx=0x7f4430005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:24.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.323+0000 7f4439ffb700 1 -- 172.21.15.163:0/1896378716 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f44280173b0 con 0x7f443c100f20 2023-11-06T01:27:24.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.522+0000 7f4442bb4700 1 -- 172.21.15.163:0/1896378716 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi053=x;smithi062=y;smithi163=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f443c068fa0 con 0x7f442c0385d0 2023-11-06T01:27:24.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.529+0000 7f4439ffb700 1 -- 172.21.15.163:0/1896378716 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f443c068fa0 con 0x7f442c0385d0 2023-11-06T01:27:24.531 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled mgr update... 2023-11-06T01:27:24.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 -- 172.21.15.163:0/1896378716 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f442c0385d0 msgr2=0x7f442c03aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:24.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f442c0385d0 0x7f442c03aa90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f443000b5c0 tx=0x7f4430005fb0 comp rx=0 tx=0).stop 2023-11-06T01:27:24.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 -- 172.21.15.163:0/1896378716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f443c100f20 msgr2=0x7f443c1970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:24.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f443c100f20 0x7f443c1970e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f442800c950 tx=0x7f442800cd10 comp rx=0 tx=0).stop 2023-11-06T01:27:24.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 -- 172.21.15.163:0/1896378716 shutdown_connections 2023-11-06T01:27:24.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f442c0385d0 0x7f442c03aa90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:24.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c0ffd20 0x7f443c196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:24.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f443c1018c0 0x7f443c19e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:24.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 --2- 172.21.15.163:0/1896378716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f443c100f20 0x7f443c1970e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:24.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 -- 172.21.15.163:0/1896378716 >> 172.21.15.163:0/1896378716 conn(0x7f443c0fb2a0 msgr2=0x7f443c068890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:24.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 -- 172.21.15.163:0/1896378716 shutdown_connections 2023-11-06T01:27:24.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:27:24.532+0000 7f4442bb4700 1 -- 172.21.15.163:0/1896378716 wait complete. 2023-11-06T01:27:24.917 DEBUG:teuthology.orchestra.run.smithi062:mgr.y> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.y.service 2023-11-06T01:27:24.919 DEBUG:teuthology.orchestra.run.smithi163:mgr.z> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.z.service 2023-11-06T01:27:24.922 INFO:tasks.cephadm:Deploying OSDs... 2023-11-06T01:27:24.922 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:27:24.923 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2023-11-06T01:27:24.944 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-06T01:27:24.944 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2023-11-06T01:27:24.952 INFO:journalctl@ceph.mgr.z.smithi163.stdout:-- Logs begin at Mon 2023-11-06 01:10:53 UTC. -- 2023-11-06T01:27:24.954 INFO:journalctl@ceph.mgr.y.smithi062.stdout:-- Logs begin at Mon 2023-11-06 01:10:07 UTC. -- 2023-11-06T01:27:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-06T01:27:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:25.006 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 389842 Links: 1 2023-11-06T01:27:25.006 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:25.007 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:25.007 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-06 01:26:36.243226003 +0000 2023-11-06T01:27:25.007 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-06 01:20:47.017552346 +0000 2023-11-06T01:27:25.007 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-06 01:20:47.017552346 +0000 2023-11-06T01:27:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2023-11-06T01:27:25.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-06T01:27:25.077 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-06T01:27:25.077 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-06T01:27:25.078 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000127049 s, 4.0 MB/s 2023-11-06T01:27:25.079 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-06T01:27:25.140 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2023-11-06T01:27:25.198 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-06T01:27:25.199 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:25.199 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 380906 Links: 1 2023-11-06T01:27:25.199 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:25.199 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:25.200 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-06 01:26:36.301224850 +0000 2023-11-06T01:27:25.200 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-06 01:20:47.547541800 +0000 2023-11-06T01:27:25.200 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-06 01:20:47.547541800 +0000 2023-11-06T01:27:25.200 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2023-11-06T01:27:25.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-06T01:27:25.221 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:25.221 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.221 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.221 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:25.222 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:25.222 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:25.222 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.222 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.222 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:25.223 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:25.223 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:25.223 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.223 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-06T01:27:25.223 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-06T01:27:25.223 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-06T01:27:25.224 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:25.267 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-06T01:27:25.267 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-06T01:27:25.267 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000126972 s, 4.0 MB/s 2023-11-06T01:27:25.268 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-06T01:27:25.328 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2023-11-06T01:27:25.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:25.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:25.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:25.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:25.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:25.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:25.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:25.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-06T01:27:25.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-06T01:27:25.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-06T01:27:25.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-06T01:27:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:25.366 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 394288 Links: 1 2023-11-06T01:27:25.366 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:25.367 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:25.367 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-06 01:26:36.362223637 +0000 2023-11-06T01:27:25.367 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-06 01:20:48.080531194 +0000 2023-11-06T01:27:25.367 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-06 01:20:48.080531194 +0000 2023-11-06T01:27:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2023-11-06T01:27:25.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-06T01:27:25.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:25.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:25.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:25.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:25.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:25.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:25.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:25.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:25.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-06T01:27:25.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-06T01:27:25.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-06T01:27:25.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:25.434 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-06T01:27:25.435 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-06T01:27:25.435 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.00010053 s, 5.1 MB/s 2023-11-06T01:27:25.436 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-06T01:27:25.495 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2023-11-06T01:27:25.553 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-06T01:27:25.554 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:25.554 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 398346 Links: 1 2023-11-06T01:27:25.554 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:25.555 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:25.555 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-06 01:26:36.430222285 +0000 2023-11-06T01:27:25.555 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-06 01:20:48.591521026 +0000 2023-11-06T01:27:25.555 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-06 01:20:48.591521026 +0000 2023-11-06T01:27:25.555 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2023-11-06T01:27:25.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-06T01:27:25.620 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-06T01:27:25.621 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-06T01:27:25.621 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000111726 s, 4.6 MB/s 2023-11-06T01:27:25.622 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-06T01:27:25.681 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-11-06T01:27:25.681 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2023-11-06T01:27:25.703 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-06T01:27:25.703 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2023-11-06T01:27:25.764 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-06T01:27:25.764 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:25.764 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 389605 Links: 1 2023-11-06T01:27:25.765 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:25.765 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:25.765 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-11-06 01:27:11.766570348 +0000 2023-11-06T01:27:25.765 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-11-06 01:20:50.141778981 +0000 2023-11-06T01:27:25.765 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-11-06 01:20:50.141778981 +0000 2023-11-06T01:27:25.766 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-11-06T01:27:25.766 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-06T01:27:25.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-11-06T01:27:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-11-06T01:27:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000257684 s, 2.0 MB/s 2023-11-06T01:27:25.836 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-06T01:27:25.896 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2023-11-06T01:27:25.956 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-06T01:27:25.957 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:25.957 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 395297 Links: 1 2023-11-06T01:27:25.957 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:25.958 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:25.958 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-11-06 01:27:11.766570348 +0000 2023-11-06T01:27:25.958 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-11-06 01:20:50.679770399 +0000 2023-11-06T01:27:25.958 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-11-06 01:20:50.679770399 +0000 2023-11-06T01:27:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-11-06T01:27:25.959 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-06T01:27:26.025 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-11-06T01:27:26.026 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-11-06T01:27:26.026 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000222681 s, 2.3 MB/s 2023-11-06T01:27:26.027 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-06T01:27:26.086 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2023-11-06T01:27:26.146 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-06T01:27:26.146 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:26.146 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 395432 Links: 1 2023-11-06T01:27:26.147 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:26.147 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:26.147 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-11-06 01:27:11.766570348 +0000 2023-11-06T01:27:26.147 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-11-06 01:20:51.200762087 +0000 2023-11-06T01:27:26.148 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-11-06 01:20:51.200762087 +0000 2023-11-06T01:27:26.148 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-11-06T01:27:26.148 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-06T01:27:26.214 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-11-06T01:27:26.214 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-11-06T01:27:26.214 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000314576 s, 1.6 MB/s 2023-11-06T01:27:26.216 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-06T01:27:26.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:26 smithi163 ceph-mon[104416]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi053=x;smithi062=y;smithi163=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:26.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:26 smithi163 ceph-mon[104416]: Saving service mgr spec with placement smithi053=x;smithi062=y;smithi163=z;count:3 2023-11-06T01:27:26.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:26 smithi163 ceph-mon[104416]: Deploying daemon mgr.z on smithi163 2023-11-06T01:27:26.241 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:26 smithi163 systemd[1]: Starting Ceph mgr.z for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:27:26.275 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2023-11-06T01:27:26.334 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-06T01:27:26.334 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:26.335 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 392488 Links: 1 2023-11-06T01:27:26.335 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:26.335 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:26.335 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-11-06 01:27:11.766570348 +0000 2023-11-06T01:27:26.335 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-11-06 01:20:51.739753488 +0000 2023-11-06T01:27:26.336 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-11-06 01:20:51.739753488 +0000 2023-11-06T01:27:26.336 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-11-06T01:27:26.336 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-06T01:27:26.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:26 smithi053 ceph-mon[99472]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi053=x;smithi062=y;smithi163=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:26.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:26 smithi053 ceph-mon[99472]: Saving service mgr spec with placement smithi053=x;smithi062=y;smithi163=z;count:3 2023-11-06T01:27:26.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:26 smithi053 ceph-mon[99472]: Deploying daemon mgr.z on smithi163 2023-11-06T01:27:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:26 smithi062 ceph-mon[103924]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi053=x;smithi062=y;smithi163=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:26 smithi062 ceph-mon[103924]: Saving service mgr spec with placement smithi053=x;smithi062=y;smithi163=z;count:3 2023-11-06T01:27:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:26 smithi062 ceph-mon[103924]: Deploying daemon mgr.z on smithi163 2023-11-06T01:27:26.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-11-06T01:27:26.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-11-06T01:27:26.401 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000191417 s, 2.7 MB/s 2023-11-06T01:27:26.402 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-06T01:27:26.460 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-11-06T01:27:26.461 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2023-11-06T01:27:26.480 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-06T01:27:26.480 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2023-11-06T01:27:26.537 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-06T01:27:26.538 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:26.538 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 379460 Links: 1 2023-11-06T01:27:26.538 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:26.538 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:26.538 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-11-06 01:27:26.020261061 +0000 2023-11-06T01:27:26.539 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-11-06 01:20:39.949967378 +0000 2023-11-06T01:27:26.539 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-11-06 01:20:39.949967378 +0000 2023-11-06T01:27:26.539 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-11-06T01:27:26.540 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-06T01:27:26.607 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-11-06T01:27:26.608 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-11-06T01:27:26.608 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000262046 s, 2.0 MB/s 2023-11-06T01:27:26.609 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-06T01:27:26.654 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:26 smithi163 podman[106280]: 2023-11-06T01:27:26.654 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:26 smithi163 bash[106280]: 4fb48f8bee1a245c41643b8816b154deca02c83b84518216953552b85a6da079 2023-11-06T01:27:26.654 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:26 smithi163 ceph-mgr[106379]: -- 172.21.15.163:0/3625647211 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556030fef1e0 con 0x556030fc4c00 2023-11-06T01:27:26.667 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2023-11-06T01:27:26.726 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-06T01:27:26.726 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:26.727 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 388671 Links: 1 2023-11-06T01:27:26.727 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:26.727 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:26.727 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-11-06 01:27:26.020261061 +0000 2023-11-06T01:27:26.727 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-11-06 01:20:40.450955768 +0000 2023-11-06T01:27:26.728 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-11-06 01:20:40.450955768 +0000 2023-11-06T01:27:26.728 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-11-06T01:27:26.728 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-06T01:27:26.794 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-11-06T01:27:26.795 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-11-06T01:27:26.795 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000248115 s, 2.1 MB/s 2023-11-06T01:27:26.796 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-06T01:27:26.853 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2023-11-06T01:27:26.909 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-06T01:27:26.910 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:26.910 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 390649 Links: 1 2023-11-06T01:27:26.910 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:26.910 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:26.911 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-11-06 01:27:26.020261061 +0000 2023-11-06T01:27:26.911 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-11-06 01:20:41.010942790 +0000 2023-11-06T01:27:26.911 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-11-06 01:20:41.010942790 +0000 2023-11-06T01:27:26.911 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-11-06T01:27:26.911 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-06T01:27:26.941 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:26 smithi163 systemd[1]: Started Ceph mgr.z for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:27:26.941 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:26 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:26.764+0000 7f42a0b66200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-06T01:27:26.941 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:26 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:26.848+0000 7f42a0b66200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T01:27:26.941 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:26 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:26.939+0000 7f42a0b66200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T01:27:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-11-06T01:27:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-11-06T01:27:26.967 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000243768 s, 2.1 MB/s 2023-11-06T01:27:26.968 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-06T01:27:27.025 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2023-11-06T01:27:27.082 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-06T01:27:27.083 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T01:27:27.083 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 390779 Links: 1 2023-11-06T01:27:27.083 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T01:27:27.084 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T01:27:27.084 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-11-06 01:27:26.020261061 +0000 2023-11-06T01:27:27.084 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-11-06 01:20:41.533930670 +0000 2023-11-06T01:27:27.084 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-11-06 01:20:41.533930670 +0000 2023-11-06T01:27:27.084 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-11-06T01:27:27.085 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-06T01:27:27.148 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-11-06T01:27:27.148 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-11-06T01:27:27.148 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000204652 s, 2.5 MB/s 2023-11-06T01:27:27.149 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-06T01:27:27.208 INFO:tasks.cephadm:Deploying osd.0 on smithi053 with /dev/vg_nvme/lv_4... 2023-11-06T01:27:27.209 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-06T01:27:27.218 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:27.039+0000 7f42a0b66200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T01:27:27.219 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-mon[104416]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:27.219 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:27.219 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:27.219 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:27.220 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-06T01:27:27.220 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-06T01:27:27.220 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-06T01:27:27.220 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:27.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:27 smithi053 ceph-mon[99472]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:27.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:27.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:27.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:27.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-06T01:27:27.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-06T01:27:27.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-06T01:27:27.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:27.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:27 smithi062 ceph-mon[103924]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:27.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:27.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:27.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:27.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-06T01:27:27.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-06T01:27:27.367 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-06T01:27:27.367 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:27.490 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:27.217+0000 7f42a0b66200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T01:27:27.778 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:27:27.897 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:27.580+0000 7f42a0b66200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T01:27:27.897 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:27.714+0000 7f42a0b66200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T01:27:27.897 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:27.802+0000 7f42a0b66200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T01:27:27.898 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:27.896+0000 7f42a0b66200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T01:27:28.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:28 smithi163 ceph-mon[104416]: Deploying daemon mgr.y on smithi062 2023-11-06T01:27:28.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:28 smithi053 ceph-mon[99472]: Deploying daemon mgr.y on smithi062 2023-11-06T01:27:28.359 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:28 smithi062 ceph-mon[103924]: Deploying daemon mgr.y on smithi062 2023-11-06T01:27:28.645 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:28 smithi062 systemd[1]: Starting Ceph mgr.y for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:27:28.739 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:28 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:28.314+0000 7f42a0b66200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T01:27:28.953 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:28 smithi062 podman[105396]: 2023-11-06T01:27:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-06T01:27:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-06T01:27:29.147 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2023-11-06T01:27:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2023-11-06T01:27:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00802832 s, 1.3 GB/s 2023-11-06T01:27:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2023-11-06T01:27:29.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:29 smithi163 ceph-mon[104416]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:29.240 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:28 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:28.978+0000 7f42a0b66200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T01:27:29.240 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:29 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:29.059+0000 7f42a0b66200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T01:27:29.299 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:29 smithi062 ceph-mon[103924]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:29.299 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:28 smithi062 ceph-mgr[105436]: -- 172.21.15.62:0/3501399127 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563928dcb1e0 con 0x563928da0800 2023-11-06T01:27:29.300 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:29 smithi062 bash[105396]: 3ace7424ff4fa49af6d75e01905b3d4c69a2dcf395830d8ddf8b08e18797be0c 2023-11-06T01:27:29.300 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:29 smithi062 systemd[1]: Started Ceph mgr.y for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:27:29.300 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:29 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:29.091+0000 7fe8b84d0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-06T01:27:29.300 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:29 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:29.188+0000 7fe8b84d0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T01:27:29.300 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:29 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:29.297+0000 7fe8b84d0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T01:27:29.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:29 smithi053 ceph-mon[99472]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:29.535 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2023-11-06T01:27:29.625 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:29 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:29.411+0000 7fe8b84d0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T01:27:29.895 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:29 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:29.623+0000 7fe8b84d0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T01:27:30.095 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:27:30.147 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:29 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:29.862+0000 7f42a0b66200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T01:27:30.147 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:29 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:29.955+0000 7f42a0b66200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-06T01:27:30.147 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.148 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.148 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.148 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.148 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:30.368 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.369 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.369 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.369 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.369 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:30.394 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:30 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:30.114+0000 7fe8b84d0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T01:27:30.394 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:30 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:30.282+0000 7fe8b84d0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T01:27:30.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:30.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:30.444 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:30.145+0000 7f42a0b66200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T01:27:30.645 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:30 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:30.392+0000 7fe8b84d0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T01:27:30.645 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:30 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:30.513+0000 7fe8b84d0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T01:27:30.739 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:30.442+0000 7f42a0b66200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-06T01:27:30.740 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:30.679+0000 7f42a0b66200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T01:27:31.112 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:30.843+0000 7f42a0b66200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T01:27:31.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.358+0000 7f302fb8e700 1 -- 172.21.15.53:0/3089858871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30281018c0 msgr2=0x7f3028108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:31.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.358+0000 7f302fb8e700 1 --2- 172.21.15.53:0/3089858871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30281018c0 0x7f3028108170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f302000b0d0 tx=0x7f302000b3e0 comp rx=0 tx=0).stop 2023-11-06T01:27:31.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.358+0000 7f302fb8e700 1 -- 172.21.15.53:0/3089858871 shutdown_connections 2023-11-06T01:27:31.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.358+0000 7f302fb8e700 1 --2- 172.21.15.53:0/3089858871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30281018c0 0x7f3028108170 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:31.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.358+0000 7f302fb8e700 1 --2- 172.21.15.53:0/3089858871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30280ffd20 0x7f3028100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:31.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.358+0000 7f302fb8e700 1 --2- 172.21.15.53:0/3089858871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3028100f20 0x7f3028101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:31.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.358+0000 7f302fb8e700 1 -- 172.21.15.53:0/3089858871 >> 172.21.15.53:0/3089858871 conn(0x7f30280fb2a0 msgr2=0x7f30280fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:31.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.358+0000 7f302fb8e700 1 -- 172.21.15.53:0/3089858871 shutdown_connections 2023-11-06T01:27:31.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.358+0000 7f302fb8e700 1 -- 172.21.15.53:0/3089858871 wait complete. 2023-11-06T01:27:31.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302fb8e700 1 Processor -- start 2023-11-06T01:27:31.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302fb8e700 1 -- start start 2023-11-06T01:27:31.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302fb8e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30280ffd20 0x7f3028069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:31.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302fb8e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3028100f20 0x7f302819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:31.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302fb8e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30281018c0 0x7f302819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:31.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302fb8e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302819e6f0 con 0x7f3028100f20 2023-11-06T01:27:31.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302fb8e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302819e860 con 0x7f30281018c0 2023-11-06T01:27:31.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302fb8e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302819ea20 con 0x7f30280ffd20 2023-11-06T01:27:31.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302d129700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3028100f20 0x7f302819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:31.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302d92a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30280ffd20 0x7f3028069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:31.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302d129700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3028100f20 0x7f302819b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37058/0 (socket says 172.21.15.53:37058) 2023-11-06T01:27:31.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302e12b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30281018c0 0x7f302819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:31.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.359+0000 7f302d129700 1 -- 172.21.15.53:0/912802489 learned_addr learned my addr 172.21.15.53:0/912802489 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:27:31.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f302d129700 1 -- 172.21.15.53:0/912802489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30280ffd20 msgr2=0x7f3028069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:31.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f302d129700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30280ffd20 0x7f3028069410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:31.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f302d129700 1 -- 172.21.15.53:0/912802489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30281018c0 msgr2=0x7f302819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:31.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f302d129700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30281018c0 0x7f302819df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:31.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f302d129700 1 -- 172.21.15.53:0/912802489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3020009d00 con 0x7f3028100f20 2023-11-06T01:27:31.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f302e12b700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30281018c0 0x7f302819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:27:31.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f302d129700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3028100f20 0x7f302819b8c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f301800e9a0 tx=0x7f301800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:31.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f301effd700 1 -- 172.21.15.53:0/912802489 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018009790 con 0x7f3028100f20 2023-11-06T01:27:31.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f302fb8e700 1 -- 172.21.15.53:0/912802489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302819ed00 con 0x7f3028100f20 2023-11-06T01:27:31.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f301effd700 1 -- 172.21.15.53:0/912802489 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3018010430 con 0x7f3028100f20 2023-11-06T01:27:31.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f301effd700 1 -- 172.21.15.53:0/912802489 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018018680 con 0x7f3028100f20 2023-11-06T01:27:31.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.360+0000 7f302fb8e700 1 -- 172.21.15.53:0/912802489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302819f1a0 con 0x7f3028100f20 2023-11-06T01:27:31.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.361+0000 7f301effd700 1 -- 172.21.15.53:0/912802489 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f301800ca10 con 0x7f3028100f20 2023-11-06T01:27:31.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.361+0000 7f302fb8e700 1 -- 172.21.15.53:0/912802489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3028107280 con 0x7f3028100f20 2023-11-06T01:27:31.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.361+0000 7f301effd700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30140385d0 0x7f301403aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:31.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.361+0000 7f301effd700 1 -- 172.21.15.53:0/912802489 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3018014070 con 0x7f3028100f20 2023-11-06T01:27:31.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.361+0000 7f302d92a700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30140385d0 0x7f301403aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:31.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.362+0000 7f302d92a700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30140385d0 0x7f301403aa90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3028100d80 tx=0x7f3024007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:31.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.366+0000 7f301effd700 1 -- 172.21.15.53:0/912802489 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f301801f780 con 0x7f3028100f20 2023-11-06T01:27:31.395 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:31.035+0000 7fe8b84d0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T01:27:31.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:31.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:31.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:31.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-06T01:27:31.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-06T01:27:31.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:31.489 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:31.111+0000 7f42a0b66200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T01:27:31.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:31.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:31.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:31.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-06T01:27:31.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-06T01:27:31.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:31.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:31.578+0000 7f302fb8e700 1 -- 172.21.15.53:0/912802489 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f30280611d0 con 0x7f30140385d0 2023-11-06T01:27:31.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:31.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:31.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:31.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:31.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-06T01:27:31.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-06T01:27:31.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:31 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:32.134 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:31.811+0000 7f42a0b66200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T01:27:32.134 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:31 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:31.940+0000 7f42a0b66200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T01:27:32.134 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:32 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:32.133+0000 7f42a0b66200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T01:27:32.145 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:31.831+0000 7fe8b84d0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T01:27:32.145 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:31 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:31.929+0000 7fe8b84d0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T01:27:32.468 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:32 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:32.248+0000 7f42a0b66200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T01:27:32.468 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:32 smithi163 ceph-mon[104416]: Reconfiguring mgr.x (unknown last config time)... 2023-11-06T01:27:32.469 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:32 smithi163 ceph-mon[104416]: Reconfiguring daemon mgr.x on smithi053 2023-11-06T01:27:32.469 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:27:32.469 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:27:32.469 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:32.470 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:32.508 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:32 smithi053 ceph-mon[99472]: Reconfiguring mgr.x (unknown last config time)... 2023-11-06T01:27:32.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:32 smithi053 ceph-mon[99472]: Reconfiguring daemon mgr.x on smithi053 2023-11-06T01:27:32.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:27:32.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:27:32.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:32.509 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:32.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:32 smithi062 ceph-mon[103924]: Reconfiguring mgr.x (unknown last config time)... 2023-11-06T01:27:32.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:32 smithi062 ceph-mon[103924]: Reconfiguring daemon mgr.x on smithi053 2023-11-06T01:27:32.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:27:32.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:27:32.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:32.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:32.739 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:32 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:32.467+0000 7f42a0b66200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T01:27:33.176 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:32 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:32.902+0000 7fe8b84d0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T01:27:33.176 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:33 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:33.014+0000 7fe8b84d0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-06T01:27:33.208 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:33 smithi053 ceph-mon[99472]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:33.208 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:33 smithi053 ceph-mon[99472]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:33.209 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:33.209 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:33.209 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:33.209 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:33.209 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:33.489 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:27:33 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z[106296]: 2023-11-06T01:27:33.320+0000 7f42a0b66200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:33 smithi163 ceph-mon[104416]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:33 smithi163 ceph-mon[104416]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:33.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:33.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:33.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:33.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:33.588 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:33 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:33.235+0000 7fe8b84d0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T01:27:33.589 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:33 smithi062 ceph-mon[103924]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:33.589 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:33 smithi062 ceph-mon[103924]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:33.589 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:33.589 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:33.589 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:33.590 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:33.590 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:33.841 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:33 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:33.587+0000 7fe8b84d0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-06T01:27:34.145 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:33 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:33.840+0000 7fe8b84d0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T01:27:34.145 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:34.022+0000 7fe8b84d0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T01:27:34.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:34.183+0000 7f301effd700 1 -- 172.21.15.53:0/912802489 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 14) v1 ==== 90169+0+0 (secure 0 0 0) 0x7f301800fd00 con 0x7f3028100f20 2023-11-06T01:27:34.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:34 smithi163 ceph-mon[104416]: Standby manager daemon z started 2023-11-06T01:27:34.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:34 smithi163 ceph-mon[104416]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-06T01:27:34.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:34 smithi163 ceph-mon[104416]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-06T01:27:34.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:34 smithi163 ceph-mon[104416]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-06T01:27:34.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:34 smithi163 ceph-mon[104416]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-06T01:27:34.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:34 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3661283875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b30174e-feb1-42fd-8b43-0ffd686e01ee"}]: dispatch 2023-11-06T01:27:34.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:34 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3661283875' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b30174e-feb1-42fd-8b43-0ffd686e01ee"}]': finished 2023-11-06T01:27:34.495 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:34 smithi163 ceph-mon[104416]: osdmap e5: 1 total, 0 up, 1 in 2023-11-06T01:27:34.496 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:34 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:34.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:34 smithi053 ceph-mon[99472]: Standby manager daemon z started 2023-11-06T01:27:34.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:34 smithi053 ceph-mon[99472]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-06T01:27:34.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:34 smithi053 ceph-mon[99472]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-06T01:27:34.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:34 smithi053 ceph-mon[99472]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-06T01:27:34.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:34 smithi053 ceph-mon[99472]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-06T01:27:34.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:34 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3661283875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b30174e-feb1-42fd-8b43-0ffd686e01ee"}]: dispatch 2023-11-06T01:27:34.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:34 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3661283875' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b30174e-feb1-42fd-8b43-0ffd686e01ee"}]': finished 2023-11-06T01:27:34.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:34 smithi053 ceph-mon[99472]: osdmap e5: 1 total, 0 up, 1 in 2023-11-06T01:27:34.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:34 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:34.645 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:34.352+0000 7fe8b84d0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T01:27:34.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:34 smithi062 ceph-mon[103924]: Standby manager daemon z started 2023-11-06T01:27:34.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:34 smithi062 ceph-mon[103924]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-06T01:27:34.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:34 smithi062 ceph-mon[103924]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-06T01:27:34.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:34 smithi062 ceph-mon[103924]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-06T01:27:34.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:34 smithi062 ceph-mon[103924]: from='mgr.? 172.21.15.163:0/3696936213' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-06T01:27:34.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:34 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3661283875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b30174e-feb1-42fd-8b43-0ffd686e01ee"}]: dispatch 2023-11-06T01:27:34.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:34 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3661283875' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b30174e-feb1-42fd-8b43-0ffd686e01ee"}]': finished 2023-11-06T01:27:34.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:34 smithi062 ceph-mon[103924]: osdmap e5: 1 total, 0 up, 1 in 2023-11-06T01:27:34.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:34 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:35.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:35 smithi163 ceph-mon[104416]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:35.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:35 smithi163 ceph-mon[104416]: mgrmap e14: x(active, since 68s), standbys: z 2023-11-06T01:27:35.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-06T01:27:35.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:35 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/784582744' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:27:35.589 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:35 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:35.181+0000 7fe8b84d0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T01:27:35.590 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:35 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:35.341+0000 7fe8b84d0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T01:27:35.590 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:35 smithi062 ceph-mon[103924]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:35.590 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:35 smithi062 ceph-mon[103924]: mgrmap e14: x(active, since 68s), standbys: z 2023-11-06T01:27:35.590 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-06T01:27:35.591 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:35 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/784582744' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:27:35.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:35 smithi053 ceph-mon[99472]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:35.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:35 smithi053 ceph-mon[99472]: mgrmap e14: x(active, since 68s), standbys: z 2023-11-06T01:27:35.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-06T01:27:35.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:35 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/784582744' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:27:35.894 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:35 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:35.588+0000 7fe8b84d0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T01:27:35.895 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:35 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:35.725+0000 7fe8b84d0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T01:27:36.395 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:36 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:35.999+0000 7fe8b84d0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T01:27:37.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:37.192+0000 7f301effd700 1 -- 172.21.15.53:0/912802489 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 15) v1 ==== 135014+0+0 (secure 0 0 0) 0x7f30180187e0 con 0x7f3028100f20 2023-11-06T01:27:37.395 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:27:37 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y[105412]: 2023-11-06T01:27:37.041+0000 7fe8b84d0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T01:27:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:37 smithi062 ceph-mon[103924]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:37 smithi062 ceph-mon[103924]: Standby manager daemon y started 2023-11-06T01:27:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:37 smithi062 ceph-mon[103924]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-06T01:27:37.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:37 smithi062 ceph-mon[103924]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-06T01:27:37.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:37 smithi062 ceph-mon[103924]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-06T01:27:37.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:37 smithi062 ceph-mon[103924]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-06T01:27:37.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:37 smithi163 ceph-mon[104416]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:37.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:37 smithi163 ceph-mon[104416]: Standby manager daemon y started 2023-11-06T01:27:37.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:37 smithi163 ceph-mon[104416]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-06T01:27:37.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:37 smithi163 ceph-mon[104416]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-06T01:27:37.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:37 smithi163 ceph-mon[104416]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-06T01:27:37.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:37 smithi163 ceph-mon[104416]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-06T01:27:37.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:37 smithi053 ceph-mon[99472]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:37.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:37 smithi053 ceph-mon[99472]: Standby manager daemon y started 2023-11-06T01:27:37.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:37 smithi053 ceph-mon[99472]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-06T01:27:37.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:37 smithi053 ceph-mon[99472]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-06T01:27:37.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:37 smithi053 ceph-mon[99472]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-06T01:27:37.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:37 smithi053 ceph-mon[99472]: from='mgr.? 172.21.15.62:0/1377607913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-06T01:27:38.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:38 smithi053 ceph-mon[99472]: mgrmap e15: x(active, since 71s), standbys: y, z 2023-11-06T01:27:38.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-06T01:27:38.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:38 smithi163 ceph-mon[104416]: mgrmap e15: x(active, since 71s), standbys: y, z 2023-11-06T01:27:38.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-06T01:27:38.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:38 smithi062 ceph-mon[103924]: mgrmap e15: x(active, since 71s), standbys: y, z 2023-11-06T01:27:38.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-06T01:27:39.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:39 smithi053 ceph-mon[99472]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:39.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:39 smithi163 ceph-mon[104416]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:39.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:39 smithi062 ceph-mon[103924]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:41.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:41 smithi163 ceph-mon[104416]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-06T01:27:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:41 smithi053 ceph-mon[99472]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-06T01:27:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:41.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:41 smithi062 ceph-mon[103924]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:41.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-06T01:27:41.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:42.309 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:42 smithi053 ceph-mon[99472]: Deploying daemon osd.0 on smithi053 2023-11-06T01:27:42.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:42 smithi163 ceph-mon[104416]: Deploying daemon osd.0 on smithi053 2023-11-06T01:27:42.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:42 smithi062 ceph-mon[103924]: Deploying daemon osd.0 on smithi053 2023-11-06T01:27:43.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:43 smithi053 ceph-mon[99472]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:43.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:43 smithi163 ceph-mon[104416]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:43.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:43 smithi062 ceph-mon[103924]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:45.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:45 smithi163 ceph-mon[104416]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:45.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:45 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:45.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:45 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:45.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:45 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:45.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:45 smithi053 ceph-mon[99472]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:45.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:45.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:45.510 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:45 smithi062 ceph-mon[103924]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:45 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:45 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:45 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:46.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.878+0000 7f301effd700 1 -- 172.21.15.53:0/912802489 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f30280611d0 con 0x7f30140385d0 2023-11-06T01:27:46.880 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 0 on host 'smithi053' 2023-11-06T01:27:46.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.880+0000 7f302fb8e700 1 -- 172.21.15.53:0/912802489 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30140385d0 msgr2=0x7f301403aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:46.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.880+0000 7f302fb8e700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30140385d0 0x7f301403aa90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3028100d80 tx=0x7f3024007400 comp rx=0 tx=0).stop 2023-11-06T01:27:46.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.880+0000 7f302fb8e700 1 -- 172.21.15.53:0/912802489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3028100f20 msgr2=0x7f302819b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:46.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.880+0000 7f302fb8e700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3028100f20 0x7f302819b8c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f301800e9a0 tx=0x7f301800ecb0 comp rx=0 tx=0).stop 2023-11-06T01:27:46.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.880+0000 7f302fb8e700 1 -- 172.21.15.53:0/912802489 shutdown_connections 2023-11-06T01:27:46.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.880+0000 7f302fb8e700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30140385d0 0x7f301403aa90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:46.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.880+0000 7f302fb8e700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30281018c0 0x7f302819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:46.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.880+0000 7f302fb8e700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30280ffd20 0x7f3028069410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:46.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.880+0000 7f302fb8e700 1 --2- 172.21.15.53:0/912802489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3028100f20 0x7f302819b8c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:46.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.880+0000 7f302fb8e700 1 -- 172.21.15.53:0/912802489 >> 172.21.15.53:0/912802489 conn(0x7f30280fb2a0 msgr2=0x7f30281043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:46.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.881+0000 7f302fb8e700 1 -- 172.21.15.53:0/912802489 shutdown_connections 2023-11-06T01:27:46.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:27:46.881+0000 7f302fb8e700 1 -- 172.21.15.53:0/912802489 wait complete. 2023-11-06T01:27:47.302 DEBUG:teuthology.orchestra.run.smithi053:osd.0> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.0.service 2023-11-06T01:27:47.305 INFO:tasks.cephadm:Deploying osd.1 on smithi062 with /dev/vg_nvme/lv_4... 2023-11-06T01:27:47.305 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-06T01:27:47.342 INFO:journalctl@ceph.osd.0.smithi053.stdout:-- Logs begin at Mon 2023-11-06 01:09:30 UTC. -- 2023-11-06T01:27:47.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:47 smithi163 ceph-mon[104416]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:47.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:47 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:47 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:47 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:47.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:47 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:47.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:47 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:47 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:47 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:47 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:47 smithi053 ceph-mon[99472]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:47 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:47 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:47 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:47.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:47 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:47.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:47 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:47 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:47 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:47 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:47 smithi062 ceph-mon[103924]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:47 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:47 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:47 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:47.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:47 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:47.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:47 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:47 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:47 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:47.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:47 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:27:47.864 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:27:48.603 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:27:48 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-0[118800]: 2023-11-06T01:27:48.343+0000 7fb4d8990640 -1 osd.0 0 log_to_monitors true 2023-11-06T01:27:49.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:49 smithi062 ceph-mon[103924]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:49.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:49 smithi062 ceph-mon[103924]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-06T01:27:49.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:49 smithi163 ceph-mon[104416]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:49.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:49 smithi163 ceph-mon[104416]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-06T01:27:49.497 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-06T01:27:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-06T01:27:49.498 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2023-11-06T01:27:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2023-11-06T01:27:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110392 s, 950 MB/s 2023-11-06T01:27:49.499 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2023-11-06T01:27:49.531 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:49 smithi053 ceph-mon[99472]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:49.531 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:49 smithi053 ceph-mon[99472]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-06T01:27:49.913 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2023-11-06T01:27:50.475 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:27:50.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:50 smithi163 ceph-mon[104416]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-06T01:27:50.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:50 smithi163 ceph-mon[104416]: osdmap e6: 1 total, 0 up, 1 in 2023-11-06T01:27:50.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:50 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:50.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:50 smithi163 ceph-mon[104416]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:27:50.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:50 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:50.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:50 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:50.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:50 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:27:50.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:50 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:50.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:50 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:50.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:50 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:50.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:50 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:50.594 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-0[118800]: 2023-11-06T01:27:50.207+0000 7fb4ce007700 -1 osd.0 0 waiting for initial osdmap 2023-11-06T01:27:50.594 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-0[118800]: 2023-11-06T01:27:50.211+0000 7fb4c85f9700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:27:50.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-mon[99472]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-06T01:27:50.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-mon[99472]: osdmap e6: 1 total, 0 up, 1 in 2023-11-06T01:27:50.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:50.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-mon[99472]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:27:50.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:50.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:50.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:27:50.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:50.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:50.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:50.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:50.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:50 smithi062 ceph-mon[103924]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-06T01:27:50.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:50 smithi062 ceph-mon[103924]: osdmap e6: 1 total, 0 up, 1 in 2023-11-06T01:27:50.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:50 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:50.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:50 smithi062 ceph-mon[103924]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:27:50.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:50 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:50.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:50 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:50.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:50 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:27:50.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:50 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:50.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:50 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:50.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:50 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:27:50.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:50 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:27:51.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:51 smithi062 ceph-mon[103924]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:51.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:51 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi053 2023-11-06T01:27:51.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:51 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi053 to 17173M 2023-11-06T01:27:51.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:51 smithi062 ceph-mon[103924]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:27:51.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:51 smithi062 ceph-mon[103924]: osdmap e7: 1 total, 0 up, 1 in 2023-11-06T01:27:51.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:51 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:51.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:51 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:51.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:51 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:51.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:51 smithi163 ceph-mon[104416]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:51.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:51 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi053 2023-11-06T01:27:51.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:51 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi053 to 17173M 2023-11-06T01:27:51.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:51 smithi163 ceph-mon[104416]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:27:51.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:51 smithi163 ceph-mon[104416]: osdmap e7: 1 total, 0 up, 1 in 2023-11-06T01:27:51.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:51 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:51.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:51 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:51.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:51 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:51.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:51 smithi053 ceph-mon[99472]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:51.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:51 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi053 2023-11-06T01:27:51.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:51 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi053 to 17173M 2023-11-06T01:27:51.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:51 smithi053 ceph-mon[99472]: from='osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:27:51.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:51 smithi053 ceph-mon[99472]: osdmap e7: 1 total, 0 up, 1 in 2023-11-06T01:27:51.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:51 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:51.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:51 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:51.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:51 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:51.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.721+0000 7f9056949700 1 -- 172.21.15.62:0/1631115616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90500ffd20 msgr2=0x7f9050100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:51.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.721+0000 7f9056949700 1 --2- 172.21.15.62:0/1631115616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90500ffd20 0x7f9050100140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f90400099c0 tx=0x7f9040009cd0 comp rx=0 tx=0).stop 2023-11-06T01:27:51.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.721+0000 7f9056949700 1 -- 172.21.15.62:0/1631115616 shutdown_connections 2023-11-06T01:27:51.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.721+0000 7f9056949700 1 --2- 172.21.15.62:0/1631115616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90500ffd20 0x7f9050100140 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:51.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.721+0000 7f9056949700 1 --2- 172.21.15.62:0/1631115616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90501018c0 0x7f9050108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:51.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.721+0000 7f9056949700 1 --2- 172.21.15.62:0/1631115616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9050100f20 0x7f9050101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:51.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.721+0000 7f9056949700 1 -- 172.21.15.62:0/1631115616 >> 172.21.15.62:0/1631115616 conn(0x7f90500fb2a0 msgr2=0x7f90500fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:27:51.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.721+0000 7f9056949700 1 -- 172.21.15.62:0/1631115616 shutdown_connections 2023-11-06T01:27:51.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.721+0000 7f9056949700 1 -- 172.21.15.62:0/1631115616 wait complete. 2023-11-06T01:27:51.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.722+0000 7f9056949700 1 Processor -- start 2023-11-06T01:27:51.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.722+0000 7f9056949700 1 -- start start 2023-11-06T01:27:51.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f9056949700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90500ffd20 0x7f9050069c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:51.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f9056949700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9050100f20 0x7f905019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:51.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f9056949700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90501018c0 0x7f905019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:51.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f9056949700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905019e520 con 0x7f9050100f20 2023-11-06T01:27:51.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f9056949700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905019e690 con 0x7f90500ffd20 2023-11-06T01:27:51.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f9056949700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905019e850 con 0x7f90501018c0 2023-11-06T01:27:51.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90500ffd20 0x7f9050069c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:51.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90500ffd20 0x7f9050069c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42354/0 (socket says 172.21.15.62:42354) 2023-11-06T01:27:51.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904ffff700 1 -- 172.21.15.62:0/2118611998 learned_addr learned my addr 172.21.15.62:0/2118611998 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-11-06T01:27:51.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f9054ee6700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90501018c0 0x7f905019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:51.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904f7fe700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9050100f20 0x7f905019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:51.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904ffff700 1 -- 172.21.15.62:0/2118611998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90501018c0 msgr2=0x7f905019dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:51.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904ffff700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90501018c0 0x7f905019dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:51.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904ffff700 1 -- 172.21.15.62:0/2118611998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9050100f20 msgr2=0x7f905019b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:27:51.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904ffff700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9050100f20 0x7f905019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:27:51.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904ffff700 1 -- 172.21.15.62:0/2118611998 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90400095f0 con 0x7f90500ffd20 2023-11-06T01:27:51.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904ffff700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90500ffd20 0x7f9050069c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9040009990 tx=0x7f9040003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:51.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904d7fa700 1 -- 172.21.15.62:0/2118611998 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904001d070 con 0x7f90500ffd20 2023-11-06T01:27:51.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904d7fa700 1 -- 172.21.15.62:0/2118611998 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9040004640 con 0x7f90500ffd20 2023-11-06T01:27:51.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f9056949700 1 -- 172.21.15.62:0/2118611998 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905019ead0 con 0x7f90500ffd20 2023-11-06T01:27:51.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f904d7fa700 1 -- 172.21.15.62:0/2118611998 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904000f460 con 0x7f90500ffd20 2023-11-06T01:27:51.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.723+0000 7f9056949700 1 -- 172.21.15.62:0/2118611998 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905019ef70 con 0x7f90500ffd20 2023-11-06T01:27:51.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.724+0000 7f9056949700 1 -- 172.21.15.62:0/2118611998 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905010b520 con 0x7f90500ffd20 2023-11-06T01:27:51.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.726+0000 7f904d7fa700 1 -- 172.21.15.62:0/2118611998 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135014+0+0 (secure 0 0 0) 0x7f9040004050 con 0x7f90500ffd20 2023-11-06T01:27:51.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.726+0000 7f904d7fa700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90380a0490 0x7f90380a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:27:51.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.726+0000 7f904d7fa700 1 -- 172.21.15.62:0/2118611998 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f90400cba00 con 0x7f90500ffd20 2023-11-06T01:27:51.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.727+0000 7f904f7fe700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90380a0490 0x7f90380a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:27:51.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.727+0000 7f904f7fe700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90380a0490 0x7f90380a2950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f905006a1c0 tx=0x7f9044005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:27:51.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.730+0000 7f904d7fa700 1 -- 172.21.15.62:0/2118611998 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f90400970a0 con 0x7f90500ffd20 2023-11-06T01:27:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:27:51.936+0000 7f9056949700 1 -- 172.21.15.62:0/2118611998 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f90500611d0 con 0x7f90380a0490 2023-11-06T01:27:52.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:52 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:27:52.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:52 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:27:52.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:52 smithi163 ceph-mon[104416]: osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073] boot 2023-11-06T01:27:52.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:52 smithi163 ceph-mon[104416]: osdmap e8: 1 total, 1 up, 1 in 2023-11-06T01:27:52.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:52 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:52.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:52 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:27:52.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:52 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:27:52.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:52 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:52.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:52 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:27:52.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:52 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:27:52.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:52 smithi053 ceph-mon[99472]: osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073] boot 2023-11-06T01:27:52.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:52 smithi053 ceph-mon[99472]: osdmap e8: 1 total, 1 up, 1 in 2023-11-06T01:27:52.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:52.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:27:52.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:27:52.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:52.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:52 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:27:52.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:52 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:27:52.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:52 smithi062 ceph-mon[103924]: osd.0 [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073] boot 2023-11-06T01:27:52.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:52 smithi062 ceph-mon[103924]: osdmap e8: 1 total, 1 up, 1 in 2023-11-06T01:27:52.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:52 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-06T01:27:52.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:52 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:27:52.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:52 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:27:52.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:52 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:27:53.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:53 smithi163 ceph-mon[104416]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:53.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:53 smithi163 ceph-mon[104416]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:53.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:53 smithi163 ceph-mon[104416]: osdmap e9: 1 total, 1 up, 1 in 2023-11-06T01:27:53.549 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:53 smithi062 ceph-mon[103924]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:53.549 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:53 smithi062 ceph-mon[103924]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:53.550 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:53 smithi062 ceph-mon[103924]: osdmap e9: 1 total, 1 up, 1 in 2023-11-06T01:27:53.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:53 smithi053 ceph-mon[99472]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-06T01:27:53.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:53 smithi053 ceph-mon[99472]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:27:53.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:53 smithi053 ceph-mon[99472]: osdmap e9: 1 total, 1 up, 1 in 2023-11-06T01:27:55.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:55 smithi163 ceph-mon[104416]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:27:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:55 smithi163 ceph-mon[104416]: from='client.? 172.21.15.62:0/1123338031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a75b02-34df-4def-b5f6-e29f1190c1e8"}]: dispatch 2023-11-06T01:27:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:55 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a75b02-34df-4def-b5f6-e29f1190c1e8"}]: dispatch 2023-11-06T01:27:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:55 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73a75b02-34df-4def-b5f6-e29f1190c1e8"}]': finished 2023-11-06T01:27:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:55 smithi163 ceph-mon[104416]: osdmap e10: 2 total, 1 up, 2 in 2023-11-06T01:27:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:27:55.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:55 smithi053 ceph-mon[99472]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:27:55.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:55 smithi053 ceph-mon[99472]: from='client.? 172.21.15.62:0/1123338031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a75b02-34df-4def-b5f6-e29f1190c1e8"}]: dispatch 2023-11-06T01:27:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:55 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a75b02-34df-4def-b5f6-e29f1190c1e8"}]: dispatch 2023-11-06T01:27:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:55 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73a75b02-34df-4def-b5f6-e29f1190c1e8"}]': finished 2023-11-06T01:27:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:55 smithi053 ceph-mon[99472]: osdmap e10: 2 total, 1 up, 2 in 2023-11-06T01:27:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:27:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:55 smithi062 ceph-mon[103924]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:27:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:55 smithi062 ceph-mon[103924]: from='client.? 172.21.15.62:0/1123338031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a75b02-34df-4def-b5f6-e29f1190c1e8"}]: dispatch 2023-11-06T01:27:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:55 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a75b02-34df-4def-b5f6-e29f1190c1e8"}]: dispatch 2023-11-06T01:27:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:55 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73a75b02-34df-4def-b5f6-e29f1190c1e8"}]': finished 2023-11-06T01:27:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:55 smithi062 ceph-mon[103924]: osdmap e10: 2 total, 1 up, 2 in 2023-11-06T01:27:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:27:56.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:56 smithi163 ceph-mon[104416]: from='client.? 172.21.15.62:0/3206219072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:27:56.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:56 smithi053 ceph-mon[99472]: from='client.? 172.21.15.62:0/3206219072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:27:56.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:56 smithi062 ceph-mon[103924]: from='client.? 172.21.15.62:0/3206219072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:27:57.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:57 smithi053 ceph-mon[99472]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:27:57.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:57 smithi062 ceph-mon[103924]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:27:57.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:57 smithi163 ceph-mon[104416]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:27:59.495 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:27:59 smithi062 ceph-mon[103924]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:27:59.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:27:59 smithi053 ceph-mon[99472]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:27:59.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:27:59 smithi163 ceph-mon[104416]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:01.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:01 smithi053 ceph-mon[99472]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:01.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:01 smithi062 ceph-mon[103924]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:01.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:01 smithi163 ceph-mon[104416]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:02.566 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:02 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-06T01:28:02.567 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:02 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:02.567 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:02 smithi062 ceph-mon[103924]: Deploying daemon osd.1 on smithi062 2023-11-06T01:28:02.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:02 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-06T01:28:02.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:02 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:02.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:02 smithi053 ceph-mon[99472]: Deploying daemon osd.1 on smithi062 2023-11-06T01:28:02.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:02 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-06T01:28:02.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:02 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:02.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:02 smithi163 ceph-mon[104416]: Deploying daemon osd.1 on smithi062 2023-11-06T01:28:03.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:03 smithi053 ceph-mon[99472]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:03.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:03 smithi062 ceph-mon[103924]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:03.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:03 smithi163 ceph-mon[104416]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:05.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:05 smithi062 ceph-mon[103924]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:05.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:05 smithi053 ceph-mon[99472]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:05.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:05 smithi163 ceph-mon[104416]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:07.560 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:07 smithi062 ceph-mon[103924]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:07.560 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:07 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:07.561 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:07 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:07.561 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:07 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:07.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:07 smithi053 ceph-mon[99472]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:07.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:07 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:07.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:07 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:07.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:07 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:07.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:07 smithi163 ceph-mon[104416]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:07.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:07 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:07.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:07 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:07.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:07 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:09.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:09 smithi053 ceph-mon[99472]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:09.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:09 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:09.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:09 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:09.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:09 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:09.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:09 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:09.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:09 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:09.638 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:09 smithi062 ceph-mon[103924]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:09.639 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:09 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:09.639 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:09 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:09.639 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:09 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:09.639 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:09 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:09.640 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:09 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:09.712 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 1 on host 'smithi062' 2023-11-06T01:28:09.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.710+0000 7f904d7fa700 1 -- 172.21.15.62:0/2118611998 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f90500611d0 con 0x7f90380a0490 2023-11-06T01:28:09.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 -- 172.21.15.62:0/2118611998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90380a0490 msgr2=0x7f90380a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:09.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90380a0490 0x7f90380a2950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f905006a1c0 tx=0x7f9044005d50 comp rx=0 tx=0).stop 2023-11-06T01:28:09.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 -- 172.21.15.62:0/2118611998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90500ffd20 msgr2=0x7f9050069c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:09.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90500ffd20 0x7f9050069c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9040009990 tx=0x7f9040003ba0 comp rx=0 tx=0).stop 2023-11-06T01:28:09.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 -- 172.21.15.62:0/2118611998 shutdown_connections 2023-11-06T01:28:09.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90380a0490 0x7f90380a2950 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:09.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90500ffd20 0x7f9050069c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:09.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90501018c0 0x7f905019dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:09.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 --2- 172.21.15.62:0/2118611998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9050100f20 0x7f905019b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:09.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 -- 172.21.15.62:0/2118611998 >> 172.21.15.62:0/2118611998 conn(0x7f90500fb2a0 msgr2=0x7f90500fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:28:09.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 -- 172.21.15.62:0/2118611998 shutdown_connections 2023-11-06T01:28:09.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:09.713+0000 7f9056949700 1 -- 172.21.15.62:0/2118611998 wait complete. 2023-11-06T01:28:09.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:09 smithi163 ceph-mon[104416]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:09.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:09 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:09.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:09 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:09.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:09 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:09.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:09 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:09.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:09 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:10.177 DEBUG:teuthology.orchestra.run.smithi062:osd.1> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.1.service 2023-11-06T01:28:10.179 INFO:tasks.cephadm:Deploying osd.2 on smithi163 with /dev/vg_nvme/lv_4... 2023-11-06T01:28:10.180 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-06T01:28:10.211 INFO:journalctl@ceph.osd.1.smithi062.stdout:-- Logs begin at Mon 2023-11-06 01:10:07 UTC. -- 2023-11-06T01:28:10.710 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:28:10 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-1[109002]: 2023-11-06T01:28:10.426+0000 7f0aa751f640 -1 osd.1 0 log_to_monitors true 2023-11-06T01:28:10.710 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:10 smithi062 ceph-mon[103924]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:10.711 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:10 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:10.711 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:10 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:10.711 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:10 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:10.712 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:10 smithi062 ceph-mon[103924]: from='osd.1 [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-06T01:28:10.712 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:10 smithi062 ceph-mon[103924]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-06T01:28:10.743 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:28:10.938 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:10 smithi163 ceph-mon[104416]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:10.939 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:10 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:10.939 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:10 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:10.939 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:10 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:10.939 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:10 smithi163 ceph-mon[104416]: from='osd.1 [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-06T01:28:10.940 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:10 smithi163 ceph-mon[104416]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-06T01:28:11.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:10 smithi053 ceph-mon[99472]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:11.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:10 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:11.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:10 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:11.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:10 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:11.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:10 smithi053 ceph-mon[99472]: from='osd.1 [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-06T01:28:11.099 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:10 smithi053 ceph-mon[99472]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-06T01:28:12.203 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-06T01:28:12.203 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-06T01:28:12.203 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2023-11-06T01:28:12.203 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2023-11-06T01:28:12.204 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110906 s, 945 MB/s 2023-11-06T01:28:12.204 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2023-11-06T01:28:12.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:12 smithi163 ceph-mon[104416]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-06T01:28:12.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:12 smithi163 ceph-mon[104416]: osdmap e11: 2 total, 1 up, 2 in 2023-11-06T01:28:12.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:12 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:12.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:12 smithi163 ceph-mon[104416]: from='osd.1 [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:28:12.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:12 smithi163 ceph-mon[104416]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:28:12.543 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:28:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-1[109002]: 2023-11-06T01:28:12.269+0000 7f0a9cb96700 -1 osd.1 0 waiting for initial osdmap 2023-11-06T01:28:12.543 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:28:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-1[109002]: 2023-11-06T01:28:12.274+0000 7f0a96186700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:28:12.543 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:12 smithi062 ceph-mon[103924]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-06T01:28:12.544 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:12 smithi062 ceph-mon[103924]: osdmap e11: 2 total, 1 up, 2 in 2023-11-06T01:28:12.544 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:12.544 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:12 smithi062 ceph-mon[103924]: from='osd.1 [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:28:12.544 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:12 smithi062 ceph-mon[103924]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:28:12.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:12 smithi053 ceph-mon[99472]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-06T01:28:12.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:12 smithi053 ceph-mon[99472]: osdmap e11: 2 total, 1 up, 2 in 2023-11-06T01:28:12.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:12 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:12.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:12 smithi053 ceph-mon[99472]: from='osd.1 [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:28:12.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:12 smithi053 ceph-mon[99472]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:28:12.596 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi163:vg_nvme/lv_4 2023-11-06T01:28:13.144 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:28:13.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:13.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:28:13.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: osdmap e12: 2 total, 1 up, 2 in 2023-11-06T01:28:13.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:13.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:13.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:13.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:13.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:13.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:13.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:13.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:13.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:13.555 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:13.555 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:28:13.555 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: osdmap e12: 2 total, 1 up, 2 in 2023-11-06T01:28:13.556 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:13.556 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:13.556 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:13.556 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:13.557 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:13.557 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:13.557 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:13.557 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:13.557 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:13.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:13.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:28:13.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: osdmap e12: 2 total, 1 up, 2 in 2023-11-06T01:28:13.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:13.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:13.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:13.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:13.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:13.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:13.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:13.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:13.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:14.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.311+0000 7f650f4d2700 1 -- 172.21.15.163:0/3383838017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6508101940 msgr2=0x7f6508103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:14.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.311+0000 7f650f4d2700 1 --2- 172.21.15.163:0/3383838017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6508101940 0x7f6508103d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009990 tx=0x7f64f8009ca0 comp rx=0 tx=0).stop 2023-11-06T01:28:14.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.311+0000 7f650f4d2700 1 -- 172.21.15.163:0/3383838017 shutdown_connections 2023-11-06T01:28:14.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.311+0000 7f650f4d2700 1 --2- 172.21.15.163:0/3383838017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508106ba0 0x7f6508108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:14.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.311+0000 7f650f4d2700 1 --2- 172.21.15.163:0/3383838017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6508101940 0x7f6508103d30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:14.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.311+0000 7f650f4d2700 1 --2- 172.21.15.163:0/3383838017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6508104270 0x7f6508106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:14.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.311+0000 7f650f4d2700 1 -- 172.21.15.163:0/3383838017 >> 172.21.15.163:0/3383838017 conn(0x7f65080fb2d0 msgr2=0x7f65080fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:28:14.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.311+0000 7f650f4d2700 1 -- 172.21.15.163:0/3383838017 shutdown_connections 2023-11-06T01:28:14.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.311+0000 7f650f4d2700 1 -- 172.21.15.163:0/3383838017 wait complete. 2023-11-06T01:28:14.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650f4d2700 1 Processor -- start 2023-11-06T01:28:14.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650f4d2700 1 -- start start 2023-11-06T01:28:14.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650f4d2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508101940 0x7f6508196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:14.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650f4d2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6508104270 0x7f65081970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:14.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650f4d2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6508106ba0 0x7f650819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:14.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650f4d2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819e710 con 0x7f6508104270 2023-11-06T01:28:14.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650f4d2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819e850 con 0x7f6508101940 2023-11-06T01:28:14.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650f4d2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819e9e0 con 0x7f6508106ba0 2023-11-06T01:28:14.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650da6f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6508106ba0 0x7f650819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:14.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650da6f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6508106ba0 0x7f650819e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46690/0 (socket says 172.21.15.163:46690) 2023-11-06T01:28:14.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650da6f700 1 -- 172.21.15.163:0/1222049617 learned_addr learned my addr 172.21.15.163:0/1222049617 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-11-06T01:28:14.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.312+0000 7f650d26e700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508101940 0x7f6508196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:14.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f650ca6d700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6508104270 0x7f65081970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:14.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f650da6f700 1 -- 172.21.15.163:0/1222049617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508101940 msgr2=0x7f6508196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:14.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f650da6f700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508101940 0x7f6508196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:14.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f650da6f700 1 -- 172.21.15.163:0/1222049617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6508104270 msgr2=0x7f65081970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:14.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f650da6f700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6508104270 0x7f65081970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:14.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f650da6f700 1 -- 172.21.15.163:0/1222049617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f80095f0 con 0x7f6508106ba0 2023-11-06T01:28:14.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f650ca6d700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6508104270 0x7f65081970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:28:14.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f650da6f700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6508106ba0 0x7f650819e140 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6500007ee0 tx=0x7f650000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:28:14.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f64fe7fc700 1 -- 172.21.15.163:0/1222049617 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500004db0 con 0x7f6508106ba0 2023-11-06T01:28:14.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f64fe7fc700 1 -- 172.21.15.163:0/1222049617 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f650000f040 con 0x7f6508106ba0 2023-11-06T01:28:14.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f650f4d2700 1 -- 172.21.15.163:0/1222049617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650819ecc0 con 0x7f6508106ba0 2023-11-06T01:28:14.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f64fe7fc700 1 -- 172.21.15.163:0/1222049617 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500005910 con 0x7f6508106ba0 2023-11-06T01:28:14.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.313+0000 7f650f4d2700 1 -- 172.21.15.163:0/1222049617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650819f1c0 con 0x7f6508106ba0 2023-11-06T01:28:14.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.314+0000 7f650f4d2700 1 -- 172.21.15.163:0/1222049617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65080fce50 con 0x7f6508106ba0 2023-11-06T01:28:14.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.315+0000 7f64fe7fc700 1 -- 172.21.15.163:0/1222049617 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 15) v1 ==== 135014+0+0 (secure 0 0 0) 0x7f6500005020 con 0x7f6508106ba0 2023-11-06T01:28:14.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.315+0000 7f64fe7fc700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f64f40a0510 0x7f64f40a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:14.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.316+0000 7f64fe7fc700 1 -- 172.21.15.163:0/1222049617 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f65000c9b10 con 0x7f6508106ba0 2023-11-06T01:28:14.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.316+0000 7f650d26e700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f64f40a0510 0x7f64f40a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:14.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.316+0000 7f650d26e700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f64f40a0510 0x7f64f40a29d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64f800b5c0 tx=0x7f64f8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:28:14.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.320+0000 7f64fe7fc700 1 -- 172.21.15.163:0/1222049617 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f650004f0e0 con 0x7f6508106ba0 2023-11-06T01:28:14.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:14.515+0000 7f650f4d2700 1 -- 172.21.15.163:0/1222049617 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f65080611d0 con 0x7f64f40a0510 2023-11-06T01:28:14.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:14 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:28:14.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:14 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:28:14.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:14 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi062 2023-11-06T01:28:14.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:14 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi062 to 17168M 2023-11-06T01:28:14.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:14 smithi053 ceph-mon[99472]: osd.1 [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393] boot 2023-11-06T01:28:14.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:14 smithi053 ceph-mon[99472]: osdmap e13: 2 total, 2 up, 2 in 2023-11-06T01:28:14.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:14.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:14 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:28:14.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:14 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:28:14.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:14 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi062 2023-11-06T01:28:14.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:14 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi062 to 17168M 2023-11-06T01:28:14.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:14 smithi062 ceph-mon[103924]: osd.1 [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393] boot 2023-11-06T01:28:14.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:14 smithi062 ceph-mon[103924]: osdmap e13: 2 total, 2 up, 2 in 2023-11-06T01:28:14.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:14.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:14 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:28:14.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:14 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:28:14.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:14 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi062 2023-11-06T01:28:14.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:14 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi062 to 17168M 2023-11-06T01:28:14.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:14 smithi163 ceph-mon[104416]: osd.1 [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393] boot 2023-11-06T01:28:14.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:14 smithi163 ceph-mon[104416]: osdmap e13: 2 total, 2 up, 2 in 2023-11-06T01:28:14.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-06T01:28:15.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:15 smithi053 ceph-mon[99472]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:15.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:15 smithi053 ceph-mon[99472]: osdmap e14: 2 total, 2 up, 2 in 2023-11-06T01:28:15.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:15 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:28:15.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:15 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:28:15.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:15 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:15.640 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:15 smithi163 ceph-mon[104416]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:15.641 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:15 smithi163 ceph-mon[104416]: osdmap e14: 2 total, 2 up, 2 in 2023-11-06T01:28:15.641 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:15 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:28:15.641 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:15 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:28:15.641 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:15 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:15.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:15 smithi062 ceph-mon[103924]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-11-06T01:28:15.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:15 smithi062 ceph-mon[103924]: osdmap e14: 2 total, 2 up, 2 in 2023-11-06T01:28:15.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:15 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:28:15.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:15 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:28:15.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:15 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:16.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:16 smithi163 ceph-mon[104416]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:28:16.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:16 smithi053 ceph-mon[99472]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:28:16.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:16 smithi062 ceph-mon[103924]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:28:17.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:17 smithi053 ceph-mon[99472]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:17.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:17 smithi053 ceph-mon[99472]: from='client.? 172.21.15.163:0/3512936765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6650ac65-e6f8-4a01-b2bd-60b0c7995997"}]: dispatch 2023-11-06T01:28:17.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:17 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6650ac65-e6f8-4a01-b2bd-60b0c7995997"}]: dispatch 2023-11-06T01:28:17.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:17 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6650ac65-e6f8-4a01-b2bd-60b0c7995997"}]': finished 2023-11-06T01:28:17.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:17 smithi053 ceph-mon[99472]: osdmap e15: 3 total, 2 up, 3 in 2023-11-06T01:28:17.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:17.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:17 smithi062 ceph-mon[103924]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:17.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:17 smithi062 ceph-mon[103924]: from='client.? 172.21.15.163:0/3512936765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6650ac65-e6f8-4a01-b2bd-60b0c7995997"}]: dispatch 2023-11-06T01:28:17.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:17 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6650ac65-e6f8-4a01-b2bd-60b0c7995997"}]: dispatch 2023-11-06T01:28:17.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:17 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6650ac65-e6f8-4a01-b2bd-60b0c7995997"}]': finished 2023-11-06T01:28:17.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:17 smithi062 ceph-mon[103924]: osdmap e15: 3 total, 2 up, 3 in 2023-11-06T01:28:17.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:17 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:17.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:17 smithi163 ceph-mon[104416]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:17.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:17 smithi163 ceph-mon[104416]: from='client.? 172.21.15.163:0/3512936765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6650ac65-e6f8-4a01-b2bd-60b0c7995997"}]: dispatch 2023-11-06T01:28:17.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:17 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6650ac65-e6f8-4a01-b2bd-60b0c7995997"}]: dispatch 2023-11-06T01:28:17.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:17 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6650ac65-e6f8-4a01-b2bd-60b0c7995997"}]': finished 2023-11-06T01:28:17.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:17 smithi163 ceph-mon[104416]: osdmap e15: 3 total, 2 up, 3 in 2023-11-06T01:28:17.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:18.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:18 smithi053 ceph-mon[99472]: from='client.? 172.21.15.163:0/1199088307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:28:18.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:18 smithi062 ceph-mon[103924]: from='client.? 172.21.15.163:0/1199088307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:28:18.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:18 smithi163 ceph-mon[104416]: from='client.? 172.21.15.163:0/1199088307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:28:19.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:19 smithi053 ceph-mon[99472]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:19.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:19 smithi062 ceph-mon[103924]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:19.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:19 smithi163 ceph-mon[104416]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:21 smithi053 ceph-mon[99472]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:21.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:21 smithi062 ceph-mon[103924]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:21.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:21 smithi163 ceph-mon[104416]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:23.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:23 smithi053 ceph-mon[99472]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:23.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:23 smithi062 ceph-mon[103924]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:23.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:23 smithi163 ceph-mon[104416]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:24.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-06T01:28:24.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:24.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-06T01:28:24.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:24.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-06T01:28:24.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:25.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:25 smithi163 ceph-mon[104416]: Deploying daemon osd.2 on smithi163 2023-11-06T01:28:25.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:25 smithi163 ceph-mon[104416]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:25.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:25 smithi053 ceph-mon[99472]: Deploying daemon osd.2 on smithi163 2023-11-06T01:28:25.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:25 smithi053 ceph-mon[99472]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:25.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:25 smithi062 ceph-mon[103924]: Deploying daemon osd.2 on smithi163 2023-11-06T01:28:25.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:25 smithi062 ceph-mon[103924]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:26.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:26 smithi062 ceph-mon[103924]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:26.897 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:26 smithi163 ceph-mon[104416]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:27.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:26 smithi053 ceph-mon[99472]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:28.916 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:28 smithi163 ceph-mon[104416]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:28.916 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:28.916 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:28.917 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:29.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:28 smithi053 ceph-mon[99472]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:29.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:29.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:29.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:29.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:28 smithi062 ceph-mon[103924]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:29.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:29.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:29.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:30.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.476+0000 7f64fe7fc700 1 -- 172.21.15.163:0/1222049617 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f65080611d0 con 0x7f64f40a0510 2023-11-06T01:28:30.480 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 2 on host 'smithi163' 2023-11-06T01:28:30.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 -- 172.21.15.163:0/1222049617 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f64f40a0510 msgr2=0x7f64f40a29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:30.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f64f40a0510 0x7f64f40a29d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64f800b5c0 tx=0x7f64f8005fd0 comp rx=0 tx=0).stop 2023-11-06T01:28:30.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 -- 172.21.15.163:0/1222049617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6508106ba0 msgr2=0x7f650819e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:30.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6508106ba0 0x7f650819e140 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6500007ee0 tx=0x7f650000d3b0 comp rx=0 tx=0).stop 2023-11-06T01:28:30.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 -- 172.21.15.163:0/1222049617 shutdown_connections 2023-11-06T01:28:30.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f64f40a0510 0x7f64f40a29d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:30.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508101940 0x7f6508196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:30.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6508106ba0 0x7f650819e140 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:30.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 --2- 172.21.15.163:0/1222049617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6508104270 0x7f65081970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:30.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 -- 172.21.15.163:0/1222049617 >> 172.21.15.163:0/1222049617 conn(0x7f65080fb2d0 msgr2=0x7f65081025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:28:30.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 -- 172.21.15.163:0/1222049617 shutdown_connections 2023-11-06T01:28:30.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:28:30.479+0000 7f650f4d2700 1 -- 172.21.15.163:0/1222049617 wait complete. 2023-11-06T01:28:30.932 DEBUG:teuthology.orchestra.run.smithi163:osd.2> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.2.service 2023-11-06T01:28:30.935 INFO:tasks.cephadm:Deploying osd.3 on smithi053 with /dev/vg_nvme/lv_3... 2023-11-06T01:28:30.936 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-06T01:28:30.969 INFO:journalctl@ceph.osd.2.smithi163.stdout:-- Logs begin at Mon 2023-11-06 01:10:53 UTC. -- 2023-11-06T01:28:30.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:30 smithi163 ceph-mon[104416]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:30.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:30.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:30.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:30.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:30.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:30.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:30.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:30.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:31.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:30 smithi053 ceph-mon[99472]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:31.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:31.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:31.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:31.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:31.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:31.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:31.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:31.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:31.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:30 smithi062 ceph-mon[103924]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:31.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:31.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:31.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:31.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:31.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:31.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:31.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:31.147 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:31.495 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:28:31.953 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:31 smithi053 ceph-mon[99472]: from='osd.2 [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-06T01:28:31.954 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:31 smithi053 ceph-mon[99472]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-06T01:28:31.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:31 smithi163 ceph-mon[104416]: from='osd.2 [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-06T01:28:31.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:31 smithi163 ceph-mon[104416]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-06T01:28:31.990 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:28:31 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-2[110234]: 2023-11-06T01:28:31.704+0000 7f3606352640 -1 osd.2 0 log_to_monitors true 2023-11-06T01:28:32.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:31 smithi062 ceph-mon[103924]: from='osd.2 [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-06T01:28:32.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:31 smithi062 ceph-mon[103924]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-06T01:28:32.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:32 smithi053 ceph-mon[99472]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:32.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:32 smithi053 ceph-mon[99472]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-06T01:28:32.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:32 smithi053 ceph-mon[99472]: osdmap e16: 3 total, 2 up, 3 in 2023-11-06T01:28:32.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:32.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:32 smithi053 ceph-mon[99472]: from='osd.2 [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:28:32.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:32 smithi053 ceph-mon[99472]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:28:33.037 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:32 smithi163 ceph-mon[104416]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:33.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:32 smithi163 ceph-mon[104416]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-06T01:28:33.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:32 smithi163 ceph-mon[104416]: osdmap e16: 3 total, 2 up, 3 in 2023-11-06T01:28:33.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:33.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:32 smithi163 ceph-mon[104416]: from='osd.2 [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:28:33.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:32 smithi163 ceph-mon[104416]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:28:33.039 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:28:32 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-2[110234]: 2023-11-06T01:28:32.747+0000 7f35fb9c9700 -1 osd.2 0 waiting for initial osdmap 2023-11-06T01:28:33.039 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:28:32 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-2[110234]: 2023-11-06T01:28:32.752+0000 7f35f5fbb700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:28:33.047 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-06T01:28:33.047 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-06T01:28:33.047 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2023-11-06T01:28:33.047 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2023-11-06T01:28:33.048 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00814481 s, 1.3 GB/s 2023-11-06T01:28:33.048 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2023-11-06T01:28:33.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:32 smithi062 ceph-mon[103924]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:33.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:32 smithi062 ceph-mon[103924]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-06T01:28:33.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:32 smithi062 ceph-mon[103924]: osdmap e16: 3 total, 2 up, 3 in 2023-11-06T01:28:33.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:33.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:32 smithi062 ceph-mon[103924]: from='osd.2 [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:28:33.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:32 smithi062 ceph-mon[103924]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:28:33.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2023-11-06T01:28:33.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:28:33.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: osdmap e17: 3 total, 2 up, 3 in 2023-11-06T01:28:33.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:33.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:33.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi163 2023-11-06T01:28:33.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:33.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:33.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:33.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi163 to 17168M 2023-11-06T01:28:33.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:33.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:33.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:33.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:34.113 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:28:34.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:28:34.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: osdmap e17: 3 total, 2 up, 3 in 2023-11-06T01:28:34.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:34.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:34.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi163 2023-11-06T01:28:34.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:34.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:34.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:34.147 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi163 to 17168M 2023-11-06T01:28:34.147 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:34.147 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:34.147 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:34.147 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:34.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:28:34.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: osdmap e17: 3 total, 2 up, 3 in 2023-11-06T01:28:34.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:34.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:34.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi163 2023-11-06T01:28:34.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:34.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:34.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:34.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi163 to 17168M 2023-11-06T01:28:34.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:34.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:34.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:34.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:34.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:34 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:28:34.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:34 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:28:34.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:34 smithi053 ceph-mon[99472]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:34.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:34 smithi053 ceph-mon[99472]: osd.2 [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016] boot 2023-11-06T01:28:34.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:34 smithi053 ceph-mon[99472]: osdmap e18: 3 total, 3 up, 3 in 2023-11-06T01:28:34.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:34 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:35.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:34 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:28:35.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:34 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:28:35.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:34 smithi062 ceph-mon[103924]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:35.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:34 smithi062 ceph-mon[103924]: osd.2 [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016] boot 2023-11-06T01:28:35.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:34 smithi062 ceph-mon[103924]: osdmap e18: 3 total, 3 up, 3 in 2023-11-06T01:28:35.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:34 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:35.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:34 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:28:35.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:34 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:28:35.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:34 smithi163 ceph-mon[104416]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-11-06T01:28:35.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:34 smithi163 ceph-mon[104416]: osd.2 [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016] boot 2023-11-06T01:28:35.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:34 smithi163 ceph-mon[104416]: osdmap e18: 3 total, 3 up, 3 in 2023-11-06T01:28:35.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:34 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-06T01:28:35.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.404+0000 7f7e0af54700 1 -- 172.21.15.53:0/1758833569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04100000 msgr2=0x7f7e04100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:35.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.404+0000 7f7e0af54700 1 --2- 172.21.15.53:0/1758833569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04100000 0x7f7e04100480 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009990 tx=0x7f7df0009ca0 comp rx=0 tx=0).stop 2023-11-06T01:28:35.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.404+0000 7f7e0af54700 1 -- 172.21.15.53:0/1758833569 shutdown_connections 2023-11-06T01:28:35.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.404+0000 7f7e0af54700 1 --2- 172.21.15.53:0/1758833569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e040ff6a0 0x7f7e040ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:35.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.404+0000 7f7e0af54700 1 --2- 172.21.15.53:0/1758833569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e041009c0 0x7f7e04100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:35.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.404+0000 7f7e0af54700 1 --2- 172.21.15.53:0/1758833569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04100000 0x7f7e04100480 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:35.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.404+0000 7f7e0af54700 1 -- 172.21.15.53:0/1758833569 >> 172.21.15.53:0/1758833569 conn(0x7f7e040fb2a0 msgr2=0x7f7e040fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:28:35.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.404+0000 7f7e0af54700 1 -- 172.21.15.53:0/1758833569 shutdown_connections 2023-11-06T01:28:35.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.404+0000 7f7e0af54700 1 -- 172.21.15.53:0/1758833569 wait complete. 2023-11-06T01:28:35.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e0af54700 1 Processor -- start 2023-11-06T01:28:35.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e0af54700 1 -- start start 2023-11-06T01:28:35.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e0af54700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e040ff6a0 0x7f7e04198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:35.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e0af54700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04100000 0x7f7e04199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:35.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e0af54700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e041009c0 0x7f7e041a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:35.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e0af54700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e041a0840 con 0x7f7e04100000 2023-11-06T01:28:35.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e0af54700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e041a0980 con 0x7f7e040ff6a0 2023-11-06T01:28:35.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e0af54700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e041a0b40 con 0x7f7e041009c0 2023-11-06T01:28:35.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e03fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04100000 0x7f7e04199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:35.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e03fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04100000 0x7f7e04199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37132/0 (socket says 172.21.15.53:37132) 2023-11-06T01:28:35.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e03fff700 1 -- 172.21.15.53:0/553203338 learned_addr learned my addr 172.21.15.53:0/553203338 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:28:35.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e094f1700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e041009c0 0x7f7e041a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:35.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e08cf0700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e040ff6a0 0x7f7e04198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:35.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e03fff700 1 -- 172.21.15.53:0/553203338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e041009c0 msgr2=0x7f7e041a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:35.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e03fff700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e041009c0 0x7f7e041a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:35.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e03fff700 1 -- 172.21.15.53:0/553203338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e040ff6a0 msgr2=0x7f7e04198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:35.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e03fff700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e040ff6a0 0x7f7e04198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:35.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e03fff700 1 -- 172.21.15.53:0/553203338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df00095f0 con 0x7f7e04100000 2023-11-06T01:28:35.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e08cf0700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e040ff6a0 0x7f7e04198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:28:35.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.405+0000 7f7e03fff700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04100000 0x7f7e04199230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7df0006010 tx=0x7f7df000f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:28:35.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.406+0000 7f7e01ffb700 1 -- 172.21.15.53:0/553203338 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df001d070 con 0x7f7e04100000 2023-11-06T01:28:35.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.406+0000 7f7e01ffb700 1 -- 172.21.15.53:0/553203338 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7df000fca0 con 0x7f7e04100000 2023-11-06T01:28:35.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.406+0000 7f7e01ffb700 1 -- 172.21.15.53:0/553203338 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df00176e0 con 0x7f7e04100000 2023-11-06T01:28:35.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.406+0000 7f7e0af54700 1 -- 172.21.15.53:0/553203338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e041a0dc0 con 0x7f7e04100000 2023-11-06T01:28:35.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.406+0000 7f7e0af54700 1 -- 172.21.15.53:0/553203338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e041a1180 con 0x7f7e04100000 2023-11-06T01:28:35.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.407+0000 7f7e0af54700 1 -- 172.21.15.53:0/553203338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0410a480 con 0x7f7e04100000 2023-11-06T01:28:35.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.408+0000 7f7e01ffb700 1 -- 172.21.15.53:0/553203338 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) v1 ==== 135014+0+0 (secure 0 0 0) 0x7f7df0017840 con 0x7f7e04100000 2023-11-06T01:28:35.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.408+0000 7f7e01ffb700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7df40a0590 0x7f7df40a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:35.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.408+0000 7f7e08cf0700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7df40a0590 0x7f7df40a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:35.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.409+0000 7f7e01ffb700 1 -- 172.21.15.53:0/553203338 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f7df00cb610 con 0x7f7e04100000 2023-11-06T01:28:35.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.409+0000 7f7e08cf0700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7df40a0590 0x7f7df40a2a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7e0419a1d0 tx=0x7f7df8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:28:35.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.413+0000 7f7e01ffb700 1 -- 172.21.15.53:0/553203338 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7df00967d0 con 0x7f7e04100000 2023-11-06T01:28:35.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:35.625+0000 7f7e0af54700 1 -- 172.21.15.53:0/553203338 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e040611d0 con 0x7f7df40a0590 2023-11-06T01:28:36.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:36 smithi053 ceph-mon[99472]: osdmap e19: 3 total, 3 up, 3 in 2023-11-06T01:28:36.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-06T01:28:36.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:28:36.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:28:36.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:36.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:36 smithi062 ceph-mon[103924]: osdmap e19: 3 total, 3 up, 3 in 2023-11-06T01:28:36.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-06T01:28:36.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:28:36.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:28:36.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:36.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:36 smithi163 ceph-mon[104416]: osdmap e19: 3 total, 3 up, 3 in 2023-11-06T01:28:36.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-06T01:28:36.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:28:36.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:28:36.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:37.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:37 smithi053 ceph-mon[99472]: pgmap v74: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:37.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:37 smithi053 ceph-mon[99472]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:28:37.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-06T01:28:37.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:37 smithi053 ceph-mon[99472]: osdmap e20: 3 total, 3 up, 3 in 2023-11-06T01:28:37.536 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-06T01:28:37.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:37 smithi062 ceph-mon[103924]: pgmap v74: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:37.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:37 smithi062 ceph-mon[103924]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:28:37.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:37 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-06T01:28:37.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:37 smithi062 ceph-mon[103924]: osdmap e20: 3 total, 3 up, 3 in 2023-11-06T01:28:37.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:37 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-06T01:28:37.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:37 smithi163 ceph-mon[104416]: pgmap v74: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:37.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:37 smithi163 ceph-mon[104416]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:28:37.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:37 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-06T01:28:37.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:37 smithi163 ceph-mon[104416]: osdmap e20: 3 total, 3 up, 3 in 2023-11-06T01:28:37.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:37 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-06T01:28:38.094 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:28:37 smithi053 sudo[125197]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-06T01:28:38.095 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:28:37 smithi053 sudo[125197]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:38.095 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:28:37 smithi053 sudo[125197]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:38.095 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:28:37 smithi053 sudo[125197]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:38.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:38.275+0000 7f7e01ffb700 1 -- 172.21.15.53:0/553203338 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f7df0096ba0 con 0x7f7e04100000 2023-11-06T01:28:38.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-06T01:28:38.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:38 smithi053 ceph-mon[99472]: osdmap e21: 3 total, 3 up, 3 in 2023-11-06T01:28:38.595 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:28:38 smithi053 sudo[125240]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-06T01:28:38.595 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:28:38 smithi053 sudo[125240]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:38.595 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:28:38 smithi053 sudo[125240]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:38.595 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:28:38 smithi053 sudo[125240]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:38.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-06T01:28:38.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:38 smithi062 ceph-mon[103924]: osdmap e21: 3 total, 3 up, 3 in 2023-11-06T01:28:38.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-06T01:28:38.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:38 smithi163 ceph-mon[104416]: osdmap e21: 3 total, 3 up, 3 in 2023-11-06T01:28:39.144 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:28:38 smithi062 sudo[113715]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-06T01:28:39.145 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:28:38 smithi062 sudo[113715]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:39.145 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:28:38 smithi062 sudo[113715]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:39.145 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:28:38 smithi062 sudo[113715]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:39.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:39 smithi053 ceph-mon[99472]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:39.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:39 smithi053 ceph-mon[99472]: osdmap e22: 3 total, 3 up, 3 in 2023-11-06T01:28:39.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:39 smithi053 ceph-mon[99472]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-11-06T01:28:39.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:39.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:39 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2733877423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0faf95d-f034-418c-8604-278ed2d8a98c"}]: dispatch 2023-11-06T01:28:39.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:39 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2733877423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0faf95d-f034-418c-8604-278ed2d8a98c"}]': finished 2023-11-06T01:28:39.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:39 smithi053 ceph-mon[99472]: osdmap e23: 4 total, 3 up, 4 in 2023-11-06T01:28:39.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:39.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:39 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/735841899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:28:39.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:39.645 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:28:39 smithi062 sudo[113718]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-06T01:28:39.645 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:28:39 smithi062 sudo[113718]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:39.645 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:28:39 smithi062 sudo[113718]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:39.646 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:28:39 smithi062 sudo[113718]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:39.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:39 smithi062 ceph-mon[103924]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:39.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:39 smithi062 ceph-mon[103924]: osdmap e22: 3 total, 3 up, 3 in 2023-11-06T01:28:39.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:39 smithi062 ceph-mon[103924]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-11-06T01:28:39.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:39.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:39 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2733877423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0faf95d-f034-418c-8604-278ed2d8a98c"}]: dispatch 2023-11-06T01:28:39.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:39 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2733877423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0faf95d-f034-418c-8604-278ed2d8a98c"}]': finished 2023-11-06T01:28:39.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:39 smithi062 ceph-mon[103924]: osdmap e23: 4 total, 3 up, 4 in 2023-11-06T01:28:39.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:39.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:39 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/735841899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:28:39.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:39.702 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:28:39 smithi163 sudo[115032]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-06T01:28:39.703 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:28:39 smithi163 sudo[115032]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:39.703 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:28:39 smithi163 sudo[115032]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:39.703 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:28:39 smithi163 sudo[115032]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:39.704 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:39 smithi163 ceph-mon[104416]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:39.704 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:39 smithi163 ceph-mon[104416]: osdmap e22: 3 total, 3 up, 3 in 2023-11-06T01:28:39.704 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:39 smithi163 ceph-mon[104416]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-11-06T01:28:39.704 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:39.704 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:39 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2733877423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0faf95d-f034-418c-8604-278ed2d8a98c"}]: dispatch 2023-11-06T01:28:39.705 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:39 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2733877423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0faf95d-f034-418c-8604-278ed2d8a98c"}]': finished 2023-11-06T01:28:39.705 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:39 smithi163 ceph-mon[104416]: osdmap e23: 4 total, 3 up, 4 in 2023-11-06T01:28:39.705 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:39.705 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:39 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/735841899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:28:39.705 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:39.989 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:28:39 smithi163 sudo[115035]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-06T01:28:39.990 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:28:39 smithi163 sudo[115035]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:39.990 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:28:39 smithi163 sudo[115035]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:39.990 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:28:39 smithi163 sudo[115035]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:40.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:40 smithi053 sudo[125700]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-06T01:28:40.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:40 smithi053 sudo[125700]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:40.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:40 smithi053 sudo[125700]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:40.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:40 smithi053 sudo[125700]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:41.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:40 smithi053 sudo[126072]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-06T01:28:41.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:40 smithi053 sudo[126072]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:41.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:40 smithi053 sudo[126072]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:41.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:40 smithi053 sudo[126072]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:41.124 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:40 smithi163 sudo[115038]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-06T01:28:41.124 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:40 smithi163 sudo[115038]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:41.124 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:40 smithi163 sudo[115038]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:41.124 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:40 smithi163 sudo[115038]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 ceph-mon[103924]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 ceph-mon[103924]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-06T01:28:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 ceph-mon[103924]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-06T01:28:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:28:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:28:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:28:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:28:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:28:41.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:28:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 sudo[115041]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-06T01:28:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 sudo[115041]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 sudo[115041]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 sudo[115041]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:41.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 ceph-mon[104416]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:41.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:41.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 ceph-mon[104416]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-06T01:28:41.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 ceph-mon[104416]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-06T01:28:41.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:28:41.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:28:41.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:28:41.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:28:41.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:28:41.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:28:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:41 smithi053 ceph-mon[99472]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:41 smithi053 ceph-mon[99472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-06T01:28:41.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:41 smithi053 ceph-mon[99472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-06T01:28:41.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:28:41.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:28:41.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:28:41.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:28:41.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:28:41.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:28:41.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 sudo[113721]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-06T01:28:41.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 sudo[113721]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:41.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 sudo[113721]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:41.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:41 smithi062 sudo[113721]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:42.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:42 smithi053 ceph-mon[99472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-06T01:28:42.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:42 smithi053 ceph-mon[99472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-06T01:28:42.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:42 smithi053 ceph-mon[99472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-06T01:28:42.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:42 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:28:42.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:42 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:28:42.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:42 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:28:42.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:42 smithi053 ceph-mon[99472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-06T01:28:42.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:42 smithi062 sudo[113724]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-06T01:28:42.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:42 smithi062 sudo[113724]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-06T01:28:42.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:42 smithi062 sudo[113724]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-06T01:28:42.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:42 smithi062 sudo[113724]: pam_unix(sudo:session): session closed for user root 2023-11-06T01:28:42.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:42 smithi062 ceph-mon[103924]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-06T01:28:42.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:42 smithi062 ceph-mon[103924]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-06T01:28:42.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:42 smithi062 ceph-mon[103924]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-06T01:28:42.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:42 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:28:42.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:42 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:28:42.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:42 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:28:42.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:42 smithi062 ceph-mon[103924]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-06T01:28:42.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:42 smithi163 ceph-mon[104416]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-06T01:28:42.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:42 smithi163 ceph-mon[104416]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-06T01:28:42.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:42 smithi163 ceph-mon[104416]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-06T01:28:42.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:42 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-06T01:28:42.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:42 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-06T01:28:42.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:42 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-06T01:28:42.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:42 smithi163 ceph-mon[104416]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-06T01:28:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:43 smithi062 ceph-mon[103924]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:43.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:43 smithi163 ceph-mon[104416]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:43.501 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:43 smithi053 ceph-mon[99472]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:45.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:45 smithi062 ceph-mon[103924]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:45.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:45 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-06T01:28:45.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:45 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:45.405 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:45 smithi053 ceph-mon[99472]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:45.406 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-06T01:28:45.406 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:45.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:45 smithi163 ceph-mon[104416]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-11-06T01:28:45.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:45 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-06T01:28:45.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:45 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:46.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:46 smithi062 ceph-mon[103924]: Deploying daemon osd.3 on smithi053 2023-11-06T01:28:46.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:46 smithi163 ceph-mon[104416]: Deploying daemon osd.3 on smithi053 2023-11-06T01:28:46.562 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:46 smithi053 ceph-mon[99472]: Deploying daemon osd.3 on smithi053 2023-11-06T01:28:47.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:47 smithi163 ceph-mon[104416]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-11-06T01:28:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:47 smithi053 ceph-mon[99472]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-11-06T01:28:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:47 smithi062 ceph-mon[103924]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-11-06T01:28:49.450 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:49 smithi053 ceph-mon[99472]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-11-06T01:28:49.451 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:49 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:49.451 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:49 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:49.451 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:49 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:49.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:49 smithi163 ceph-mon[104416]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-11-06T01:28:49.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:49 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:49.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:49 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:49.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:49 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:49.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:49 smithi062 ceph-mon[103924]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-11-06T01:28:49.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:49 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:49.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:49 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:49.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:49 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:51.420 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:51 smithi053 ceph-mon[99472]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-11-06T01:28:51.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:51 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:51.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:51 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:51.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:51 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:51.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:51 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:51.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:51 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:51.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:51 smithi163 ceph-mon[104416]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-11-06T01:28:51.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:51 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:51.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:51 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:51.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:51 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:51.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:51 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:51.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:51 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:51.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.523+0000 7f7e01ffb700 1 -- 172.21.15.53:0/553203338 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7e040611d0 con 0x7f7df40a0590 2023-11-06T01:28:51.526 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 3 on host 'smithi053' 2023-11-06T01:28:51.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.526+0000 7f7e0af54700 1 -- 172.21.15.53:0/553203338 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7df40a0590 msgr2=0x7f7df40a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:51.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.526+0000 7f7e0af54700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7df40a0590 0x7f7df40a2a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7e0419a1d0 tx=0x7f7df8008040 comp rx=0 tx=0).stop 2023-11-06T01:28:51.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.526+0000 7f7e0af54700 1 -- 172.21.15.53:0/553203338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04100000 msgr2=0x7f7e04199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:51.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.526+0000 7f7e0af54700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04100000 0x7f7e04199230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7df0006010 tx=0x7f7df000f7b0 comp rx=0 tx=0).stop 2023-11-06T01:28:51.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.526+0000 7f7e0af54700 1 -- 172.21.15.53:0/553203338 shutdown_connections 2023-11-06T01:28:51.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.526+0000 7f7e0af54700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7df40a0590 0x7f7df40a2a50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:51.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.526+0000 7f7e0af54700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e040ff6a0 0x7f7e04198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:51.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.526+0000 7f7e0af54700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e041009c0 0x7f7e041a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:51.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.526+0000 7f7e0af54700 1 --2- 172.21.15.53:0/553203338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04100000 0x7f7e04199230 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:51.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.526+0000 7f7e0af54700 1 -- 172.21.15.53:0/553203338 >> 172.21.15.53:0/553203338 conn(0x7f7e040fb2a0 msgr2=0x7f7e040fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:28:51.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.527+0000 7f7e0af54700 1 -- 172.21.15.53:0/553203338 shutdown_connections 2023-11-06T01:28:51.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:28:51.527+0000 7f7e0af54700 1 -- 172.21.15.53:0/553203338 wait complete. 2023-11-06T01:28:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:51 smithi062 ceph-mon[103924]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-11-06T01:28:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:51 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:51 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:51 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:51.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:51 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:51.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:51 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:51.903 DEBUG:teuthology.orchestra.run.smithi053:osd.3> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.3.service 2023-11-06T01:28:51.906 INFO:tasks.cephadm:Deploying osd.4 on smithi062 with /dev/vg_nvme/lv_3... 2023-11-06T01:28:51.906 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-06T01:28:51.940 INFO:journalctl@ceph.osd.3.smithi053.stdout:-- Logs begin at Mon 2023-11-06 01:09:30 UTC. -- 2023-11-06T01:28:52.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:52 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:52.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:52 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:52.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:52 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:52.481 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:28:52.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:52 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:52.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:52 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:52.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:52 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:52.514 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:28:52.514 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:52.515 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:52 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:53.090 INFO:journalctl@ceph.osd.3.smithi053.stdout:Nov 06 01:28:52 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-3[127479]: 2023-11-06T01:28:52.970+0000 7fee9a094640 -1 osd.3 0 log_to_monitors true 2023-11-06T01:28:53.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:53 smithi053 ceph-mon[99472]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-11-06T01:28:53.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:53 smithi053 ceph-mon[99472]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-06T01:28:53.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:53 smithi062 ceph-mon[103924]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-11-06T01:28:53.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:53 smithi062 ceph-mon[103924]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-06T01:28:53.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:53 smithi163 ceph-mon[104416]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-11-06T01:28:53.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:53 smithi163 ceph-mon[104416]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-06T01:28:54.176 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-06T01:28:54.176 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-06T01:28:54.176 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2023-11-06T01:28:54.177 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2023-11-06T01:28:54.177 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0146314 s, 717 MB/s 2023-11-06T01:28:54.177 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2023-11-06T01:28:54.344 INFO:journalctl@ceph.osd.3.smithi053.stdout:Nov 06 01:28:54 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-3[127479]: 2023-11-06T01:28:54.167+0000 7fee8f70b700 -1 osd.3 0 waiting for initial osdmap 2023-11-06T01:28:54.344 INFO:journalctl@ceph.osd.3.smithi053.stdout:Nov 06 01:28:54 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-3[127479]: 2023-11-06T01:28:54.174+0000 7fee88cfb700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:28:54.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:54 smithi053 ceph-mon[99472]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-06T01:28:54.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:54 smithi053 ceph-mon[99472]: osdmap e24: 4 total, 3 up, 4 in 2023-11-06T01:28:54.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:54 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:54.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:54 smithi053 ceph-mon[99472]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:28:54.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:54 smithi163 ceph-mon[104416]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-06T01:28:54.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:54 smithi163 ceph-mon[104416]: osdmap e24: 4 total, 3 up, 4 in 2023-11-06T01:28:54.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:54 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:54.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:54 smithi163 ceph-mon[104416]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:28:54.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:54 smithi062 ceph-mon[103924]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-06T01:28:54.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:54 smithi062 ceph-mon[103924]: osdmap e24: 4 total, 3 up, 4 in 2023-11-06T01:28:54.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:54 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:54.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:54 smithi062 ceph-mon[103924]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:28:54.630 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2023-11-06T01:28:55.189 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:28:55.438 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-11-06T01:28:55.439 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:28:55.439 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: osdmap e25: 4 total, 3 up, 4 in 2023-11-06T01:28:55.439 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:55.439 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:55.440 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.440 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.440 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:55.440 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:55.440 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.441 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:55.441 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:55.441 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.441 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345] boot 2023-11-06T01:28:55.442 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: osdmap e26: 4 total, 4 up, 4 in 2023-11-06T01:28:55.442 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-11-06T01:28:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:28:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: osdmap e25: 4 total, 3 up, 4 in 2023-11-06T01:28:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:55.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:55.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:55.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345] boot 2023-11-06T01:28:55.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: osdmap e26: 4 total, 4 up, 4 in 2023-11-06T01:28:55.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:55.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-11-06T01:28:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:28:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: osdmap e25: 4 total, 3 up, 4 in 2023-11-06T01:28:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:55.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:28:55.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:55.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:28:55.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:28:55.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: osd.3 [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345] boot 2023-11-06T01:28:55.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: osdmap e26: 4 total, 4 up, 4 in 2023-11-06T01:28:55.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-06T01:28:56.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:56 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:28:56.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:56 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:28:56.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:56 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi053 2023-11-06T01:28:56.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:56 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi053 to 8586M 2023-11-06T01:28:56.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:56 smithi163 ceph-mon[104416]: osdmap e27: 4 total, 4 up, 4 in 2023-11-06T01:28:56.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.547+0000 7f940cac5700 1 -- 172.21.15.62:0/2628030346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9408100f20 msgr2=0x7f9408101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:56.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.547+0000 7f940cac5700 1 --2- 172.21.15.62:0/2628030346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9408100f20 0x7f9408101380 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f93f00099c0 tx=0x7f93f0009cd0 comp rx=0 tx=0).stop 2023-11-06T01:28:56.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.547+0000 7f940cac5700 1 -- 172.21.15.62:0/2628030346 shutdown_connections 2023-11-06T01:28:56.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.547+0000 7f940cac5700 1 --2- 172.21.15.62:0/2628030346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94081018c0 0x7f9408108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:56.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.547+0000 7f940cac5700 1 --2- 172.21.15.62:0/2628030346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94080ffd20 0x7f9408100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:56.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.547+0000 7f940cac5700 1 --2- 172.21.15.62:0/2628030346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9408100f20 0x7f9408101380 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:56.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.547+0000 7f940cac5700 1 -- 172.21.15.62:0/2628030346 >> 172.21.15.62:0/2628030346 conn(0x7f94080fb2a0 msgr2=0x7f94080fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:28:56.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.547+0000 7f940cac5700 1 -- 172.21.15.62:0/2628030346 shutdown_connections 2023-11-06T01:28:56.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.547+0000 7f940cac5700 1 -- 172.21.15.62:0/2628030346 wait complete. 2023-11-06T01:28:56.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.548+0000 7f940cac5700 1 Processor -- start 2023-11-06T01:28:56.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.548+0000 7f940cac5700 1 -- start start 2023-11-06T01:28:56.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.548+0000 7f940cac5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94080ffd20 0x7f9408103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:56.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.548+0000 7f940cac5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9408100f20 0x7f9408102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:56.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.548+0000 7f940cac5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94081018c0 0x7f9408102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:56.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.548+0000 7f940cac5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9408102fc0 con 0x7f94080ffd20 2023-11-06T01:28:56.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.548+0000 7f940cac5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9408103130 con 0x7f94081018c0 2023-11-06T01:28:56.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.548+0000 7f940cac5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94081a0bc0 con 0x7f9408100f20 2023-11-06T01:28:56.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.548+0000 7f9406d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94081018c0 0x7f9408102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:56.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.548+0000 7f9406d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94081018c0 0x7f9408102840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42438/0 (socket says 172.21.15.62:42438) 2023-11-06T01:28:56.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.548+0000 7f9406d9d700 1 -- 172.21.15.62:0/3756335758 learned_addr learned my addr 172.21.15.62:0/3756335758 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-11-06T01:28:56.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f9405d9b700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9408100f20 0x7f9408102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:56.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f940659c700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94080ffd20 0x7f9408103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:56.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f9406d9d700 1 -- 172.21.15.62:0/3756335758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9408100f20 msgr2=0x7f9408102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:56.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f9406d9d700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9408100f20 0x7f9408102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:56.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f9406d9d700 1 -- 172.21.15.62:0/3756335758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94080ffd20 msgr2=0x7f9408103cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:28:56.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f9406d9d700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94080ffd20 0x7f9408103cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:28:56.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f9406d9d700 1 -- 172.21.15.62:0/3756335758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f00095f0 con 0x7f94081018c0 2023-11-06T01:28:56.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f9405d9b700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9408100f20 0x7f9408102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:28:56.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f940659c700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94080ffd20 0x7f9408103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:28:56.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f9406d9d700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94081018c0 0x7f9408102840 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9400007ee0 tx=0x7f940000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:28:56.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f93ff7fe700 1 -- 172.21.15.62:0/3756335758 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400017070 con 0x7f94081018c0 2023-11-06T01:28:56.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f940cac5700 1 -- 172.21.15.62:0/3756335758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94081a0ea0 con 0x7f94081018c0 2023-11-06T01:28:56.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f93ff7fe700 1 -- 172.21.15.62:0/3756335758 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f940000f040 con 0x7f94081018c0 2023-11-06T01:28:56.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f93ff7fe700 1 -- 172.21.15.62:0/3756335758 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400005740 con 0x7f94081018c0 2023-11-06T01:28:56.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.549+0000 7f940cac5700 1 -- 172.21.15.62:0/3756335758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94081a1310 con 0x7f94081018c0 2023-11-06T01:28:56.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.550+0000 7f940cac5700 1 -- 172.21.15.62:0/3756335758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940810b5c0 con 0x7f94081018c0 2023-11-06T01:28:56.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.551+0000 7f93ff7fe700 1 -- 172.21.15.62:0/3756335758 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f94000058a0 con 0x7f94081018c0 2023-11-06T01:28:56.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.552+0000 7f93ff7fe700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93f40a0520 0x7f93f40a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:28:56.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.552+0000 7f93ff7fe700 1 -- 172.21.15.62:0/3756335758 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f94000ca010 con 0x7f94081018c0 2023-11-06T01:28:56.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.552+0000 7f940659c700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93f40a0520 0x7f93f40a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:28:56.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.553+0000 7f940659c700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93f40a0520 0x7f93f40a29e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9408100d80 tx=0x7f93f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:28:56.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.557+0000 7f93ff7fe700 1 -- 172.21.15.62:0/3756335758 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9400094db0 con 0x7f94081018c0 2023-11-06T01:28:56.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:56 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:28:56.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:56 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:28:56.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:56 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi053 2023-11-06T01:28:56.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:56 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi053 to 8586M 2023-11-06T01:28:56.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:56 smithi053 ceph-mon[99472]: osdmap e27: 4 total, 4 up, 4 in 2023-11-06T01:28:56.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:56 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:28:56.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:56 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:28:56.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:56 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi053 2023-11-06T01:28:56.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:56 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi053 to 8586M 2023-11-06T01:28:56.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:56 smithi062 ceph-mon[103924]: osdmap e27: 4 total, 4 up, 4 in 2023-11-06T01:28:56.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:28:56.764+0000 7f940cac5700 1 -- 172.21.15.62:0/3756335758 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9408068fa0 con 0x7f93f40a0520 2023-11-06T01:28:57.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:57 smithi163 ceph-mon[104416]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:28:57.775 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:28:57.775 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:28:57.776 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:57.776 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:57 smithi163 ceph-mon[104416]: osdmap e28: 4 total, 4 up, 4 in 2023-11-06T01:28:57.777 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:57 smithi062 ceph-mon[103924]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:28:57.777 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:28:57.777 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:28:57.777 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:57.777 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:57 smithi062 ceph-mon[103924]: osdmap e28: 4 total, 4 up, 4 in 2023-11-06T01:28:57.778 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:57 smithi053 ceph-mon[99472]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:28:57.778 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:28:57.778 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:28:57.778 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:28:57.779 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:57 smithi053 ceph-mon[99472]: osdmap e28: 4 total, 4 up, 4 in 2023-11-06T01:28:58.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:58 smithi163 ceph-mon[104416]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:28:58.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:58 smithi163 ceph-mon[104416]: osdmap e29: 4 total, 4 up, 4 in 2023-11-06T01:28:58.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:58 smithi053 ceph-mon[99472]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:28:58.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:58 smithi053 ceph-mon[99472]: osdmap e29: 4 total, 4 up, 4 in 2023-11-06T01:28:58.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:58 smithi062 ceph-mon[103924]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:28:58.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:58 smithi062 ceph-mon[103924]: osdmap e29: 4 total, 4 up, 4 in 2023-11-06T01:28:59.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:59 smithi163 ceph-mon[104416]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:28:59.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:28:59 smithi163 ceph-mon[104416]: osdmap e30: 4 total, 4 up, 4 in 2023-11-06T01:28:59.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:59 smithi053 ceph-mon[99472]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:28:59.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:28:59 smithi053 ceph-mon[99472]: osdmap e30: 4 total, 4 up, 4 in 2023-11-06T01:28:59.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:59 smithi062 ceph-mon[103924]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:28:59.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:28:59 smithi062 ceph-mon[103924]: osdmap e30: 4 total, 4 up, 4 in 2023-11-06T01:29:00.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:00 smithi163 ceph-mon[104416]: from='client.? 172.21.15.62:0/3803164737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8542e9c-72e2-4551-b1f7-204be77127f0"}]: dispatch 2023-11-06T01:29:00.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:00 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8542e9c-72e2-4551-b1f7-204be77127f0"}]: dispatch 2023-11-06T01:29:00.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:00 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8542e9c-72e2-4551-b1f7-204be77127f0"}]': finished 2023-11-06T01:29:00.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:00 smithi163 ceph-mon[104416]: osdmap e31: 5 total, 4 up, 5 in 2023-11-06T01:29:00.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:00 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:00.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:00 smithi053 ceph-mon[99472]: from='client.? 172.21.15.62:0/3803164737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8542e9c-72e2-4551-b1f7-204be77127f0"}]: dispatch 2023-11-06T01:29:00.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:00 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8542e9c-72e2-4551-b1f7-204be77127f0"}]: dispatch 2023-11-06T01:29:00.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:00 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8542e9c-72e2-4551-b1f7-204be77127f0"}]': finished 2023-11-06T01:29:00.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:00 smithi053 ceph-mon[99472]: osdmap e31: 5 total, 4 up, 5 in 2023-11-06T01:29:00.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:00.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:00 smithi062 ceph-mon[103924]: from='client.? 172.21.15.62:0/3803164737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8542e9c-72e2-4551-b1f7-204be77127f0"}]: dispatch 2023-11-06T01:29:00.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:00 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8542e9c-72e2-4551-b1f7-204be77127f0"}]: dispatch 2023-11-06T01:29:00.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:00 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8542e9c-72e2-4551-b1f7-204be77127f0"}]': finished 2023-11-06T01:29:00.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:00 smithi062 ceph-mon[103924]: osdmap e31: 5 total, 4 up, 5 in 2023-11-06T01:29:00.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:00 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:01.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:01 smithi163 ceph-mon[104416]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-06T01:29:01.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:01 smithi163 ceph-mon[104416]: from='client.? 172.21.15.62:0/1513314807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:29:01.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:01 smithi053 ceph-mon[99472]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-06T01:29:01.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:01 smithi053 ceph-mon[99472]: from='client.? 172.21.15.62:0/1513314807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:29:01.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:01 smithi062 ceph-mon[103924]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-06T01:29:01.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:01 smithi062 ceph-mon[103924]: from='client.? 172.21.15.62:0/1513314807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:29:03.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:03 smithi163 ceph-mon[104416]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2023-11-06T01:29:03.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:03 smithi053 ceph-mon[99472]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2023-11-06T01:29:03.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:03 smithi062 ceph-mon[103924]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2023-11-06T01:29:05.478 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:05 smithi062 ceph-mon[103924]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-06T01:29:05.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:05 smithi163 ceph-mon[104416]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-06T01:29:05.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:05 smithi053 ceph-mon[99472]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-06T01:29:07.330 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:07 smithi062 ceph-mon[103924]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-11-06T01:29:07.330 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:07 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-06T01:29:07.331 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:07 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:07.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:07 smithi163 ceph-mon[104416]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-11-06T01:29:07.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:07 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-06T01:29:07.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:07 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:07.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:07 smithi053 ceph-mon[99472]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-11-06T01:29:07.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:07 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-06T01:29:07.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:07 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:08.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:08 smithi163 ceph-mon[104416]: Deploying daemon osd.4 on smithi062 2023-11-06T01:29:08.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:08 smithi053 ceph-mon[99472]: Deploying daemon osd.4 on smithi062 2023-11-06T01:29:08.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:08 smithi062 ceph-mon[103924]: Deploying daemon osd.4 on smithi062 2023-11-06T01:29:09.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:09 smithi062 ceph-mon[103924]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-11-06T01:29:09.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:09 smithi163 ceph-mon[104416]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-11-06T01:29:09.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:09 smithi053 ceph-mon[99472]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-11-06T01:29:11.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:11 smithi062 ceph-mon[103924]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:11.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:11.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:11.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:11 smithi163 ceph-mon[104416]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:11.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:11 smithi053 ceph-mon[99472]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:11.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:11.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:11.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:13.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:13 smithi163 ceph-mon[104416]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:13.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:13 smithi053 ceph-mon[99472]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:13.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:13 smithi062 ceph-mon[103924]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:14.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.128+0000 7f93ff7fe700 1 -- 172.21.15.62:0/3756335758 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9408068fa0 con 0x7f93f40a0520 2023-11-06T01:29:14.131 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 4 on host 'smithi062' 2023-11-06T01:29:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 -- 172.21.15.62:0/3756335758 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93f40a0520 msgr2=0x7f93f40a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93f40a0520 0x7f93f40a29e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9408100d80 tx=0x7f93f8008040 comp rx=0 tx=0).stop 2023-11-06T01:29:14.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 -- 172.21.15.62:0/3756335758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94081018c0 msgr2=0x7f9408102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:14.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94081018c0 0x7f9408102840 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9400007ee0 tx=0x7f940000d3b0 comp rx=0 tx=0).stop 2023-11-06T01:29:14.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 -- 172.21.15.62:0/3756335758 shutdown_connections 2023-11-06T01:29:14.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93f40a0520 0x7f93f40a29e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:14.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94081018c0 0x7f9408102840 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94080ffd20 0x7f9408103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 --2- 172.21.15.62:0/3756335758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9408100f20 0x7f9408102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 -- 172.21.15.62:0/3756335758 >> 172.21.15.62:0/3756335758 conn(0x7f94080fb2a0 msgr2=0x7f9408068890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:29:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 -- 172.21.15.62:0/3756335758 shutdown_connections 2023-11-06T01:29:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:14.131+0000 7f940cac5700 1 -- 172.21.15.62:0/3756335758 wait complete. 2023-11-06T01:29:14.583 DEBUG:teuthology.orchestra.run.smithi062:osd.4> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.4.service 2023-11-06T01:29:14.585 INFO:tasks.cephadm:Deploying osd.5 on smithi163 with /dev/vg_nvme/lv_3... 2023-11-06T01:29:14.586 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-06T01:29:14.615 INFO:journalctl@ceph.osd.4.smithi062.stdout:-- Logs begin at Mon 2023-11-06 01:10:07 UTC. -- 2023-11-06T01:29:14.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:14.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:14.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:14 smithi163 ceph-mon[104416]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:14.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:14.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:14.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:14.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:14 smithi053 ceph-mon[99472]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:14.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:14.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:14.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:14.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:14 smithi062 ceph-mon[103924]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:14.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:14.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:14.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:15.146 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:29:15.342 INFO:journalctl@ceph.osd.4.smithi062.stdout:Nov 06 01:29:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-4[116880]: 2023-11-06T01:29:14.988+0000 7f530688b640 -1 osd.4 0 log_to_monitors true 2023-11-06T01:29:15.638 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:15 smithi163 ceph-mon[104416]: from='osd.4 [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-06T01:29:15.638 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:15 smithi163 ceph-mon[104416]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-06T01:29:15.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:15 smithi062 ceph-mon[103924]: from='osd.4 [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-06T01:29:15.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:15 smithi062 ceph-mon[103924]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-06T01:29:15.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:15 smithi053 ceph-mon[99472]: from='osd.4 [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-06T01:29:15.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:15 smithi053 ceph-mon[99472]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-06T01:29:16.633 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-06T01:29:16.634 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-06T01:29:16.634 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2023-11-06T01:29:16.634 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2023-11-06T01:29:16.635 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0108877 s, 963 MB/s 2023-11-06T01:29:16.635 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2023-11-06T01:29:16.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:16 smithi163 ceph-mon[104416]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-06T01:29:16.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:16 smithi163 ceph-mon[104416]: osdmap e32: 5 total, 4 up, 5 in 2023-11-06T01:29:16.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:16 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:16.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:16 smithi163 ceph-mon[104416]: from='osd.4 [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:29:16.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:16 smithi163 ceph-mon[104416]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:29:16.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:16 smithi163 ceph-mon[104416]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:16.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:16 smithi053 ceph-mon[99472]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-06T01:29:16.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:16 smithi053 ceph-mon[99472]: osdmap e32: 5 total, 4 up, 5 in 2023-11-06T01:29:16.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:16 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:16.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:16 smithi053 ceph-mon[99472]: from='osd.4 [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:29:16.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:16 smithi053 ceph-mon[99472]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:29:16.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:16 smithi053 ceph-mon[99472]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:16.895 INFO:journalctl@ceph.osd.4.smithi062.stdout:Nov 06 01:29:16 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-4[116880]: 2023-11-06T01:29:16.426+0000 7f52fc703700 -1 osd.4 0 waiting for initial osdmap 2023-11-06T01:29:16.895 INFO:journalctl@ceph.osd.4.smithi062.stdout:Nov 06 01:29:16 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-4[116880]: 2023-11-06T01:29:16.433+0000 7f52f5cf3700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:29:16.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:16 smithi062 ceph-mon[103924]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-06T01:29:16.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:16 smithi062 ceph-mon[103924]: osdmap e32: 5 total, 4 up, 5 in 2023-11-06T01:29:16.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:16 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:16.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:16 smithi062 ceph-mon[103924]: from='osd.4 [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:29:16.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:16 smithi062 ceph-mon[103924]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:29:16.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:16 smithi062 ceph-mon[103924]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:17.088 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi163:vg_nvme/lv_3 2023-11-06T01:29:17.637 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:29:17.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:17 smithi062 ceph-mon[103924]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:29:17.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:17 smithi062 ceph-mon[103924]: osdmap e33: 5 total, 4 up, 5 in 2023-11-06T01:29:17.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:17 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:17.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:17 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:17.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:17 smithi163 ceph-mon[104416]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:29:17.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:17 smithi163 ceph-mon[104416]: osdmap e33: 5 total, 4 up, 5 in 2023-11-06T01:29:17.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:17.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:17.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:17 smithi053 ceph-mon[99472]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:29:17.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:17 smithi053 ceph-mon[99472]: osdmap e33: 5 total, 4 up, 5 in 2023-11-06T01:29:17.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:17.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:18.627 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:29:18.628 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:29:18.628 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: osd.4 [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718] boot 2023-11-06T01:29:18.628 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: osdmap e34: 5 total, 5 up, 5 in 2023-11-06T01:29:18.628 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:18.629 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:18.629 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi062 2023-11-06T01:29:18.629 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:18.629 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:18.629 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:18.629 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:18.630 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi062 to 8584M 2023-11-06T01:29:18.630 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:18.630 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:18.630 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:18.630 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.815+0000 7fa6f8810700 1 -- 172.21.15.163:0/1841589336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f40ffd30 msgr2=0x7fa6f4100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.815+0000 7fa6f8810700 1 --2- 172.21.15.163:0/1841589336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f40ffd30 0x7fa6f4100150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009990 tx=0x7fa6e8009ca0 comp rx=0 tx=0).stop 2023-11-06T01:29:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.815+0000 7fa6f8810700 1 -- 172.21.15.163:0/1841589336 shutdown_connections 2023-11-06T01:29:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.815+0000 7fa6f8810700 1 --2- 172.21.15.163:0/1841589336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018d0 0x7fa6f4108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.815+0000 7fa6f8810700 1 --2- 172.21.15.163:0/1841589336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f40ffd30 0x7fa6f4100150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.815+0000 7fa6f8810700 1 --2- 172.21.15.163:0/1841589336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f4100f30 0x7fa6f4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:18.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.815+0000 7fa6f8810700 1 -- 172.21.15.163:0/1841589336 >> 172.21.15.163:0/1841589336 conn(0x7fa6f40fb2d0 msgr2=0x7fa6f40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:29:18.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.815+0000 7fa6f8810700 1 -- 172.21.15.163:0/1841589336 shutdown_connections 2023-11-06T01:29:18.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.815+0000 7fa6f8810700 1 -- 172.21.15.163:0/1841589336 wait complete. 2023-11-06T01:29:18.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f8810700 1 Processor -- start 2023-11-06T01:29:18.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f8810700 1 -- start start 2023-11-06T01:29:18.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f8810700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f40ffd30 0x7fa6f4198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:18.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f8810700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f4100f30 0x7fa6f4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:18.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f8810700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018d0 0x7fa6f41a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:18.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f8810700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f41a0840 con 0x7fa6f4100f30 2023-11-06T01:29:18.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f8810700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f41a0980 con 0x7fa6f41018d0 2023-11-06T01:29:18.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f8810700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f41a0b40 con 0x7fa6f40ffd30 2023-11-06T01:29:18.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f40ffd30 0x7fa6f4198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:18.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f40ffd30 0x7fa6f4198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46770/0 (socket says 172.21.15.163:46770) 2023-11-06T01:29:18.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f259c700 1 -- 172.21.15.163:0/2563077443 learned_addr learned my addr 172.21.15.163:0/2563077443 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-11-06T01:29:18.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f1d9b700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f4100f30 0x7fa6f4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:18.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f2d9d700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018d0 0x7fa6f41a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:18.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f259c700 1 -- 172.21.15.163:0/2563077443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018d0 msgr2=0x7fa6f41a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:18.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f259c700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018d0 0x7fa6f41a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:18.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f259c700 1 -- 172.21.15.163:0/2563077443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f4100f30 msgr2=0x7fa6f4199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:18.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f259c700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f4100f30 0x7fa6f4199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:18.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f259c700 1 -- 172.21.15.163:0/2563077443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e80095f0 con 0x7fa6f40ffd30 2023-11-06T01:29:18.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f1d9b700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f4100f30 0x7fa6f4199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:29:18.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f2d9d700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018d0 0x7fa6f41a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:29:18.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.816+0000 7fa6f259c700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f40ffd30 0x7fa6f4198cf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800b680 tx=0x7fa6e8005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:29:18.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.817+0000 7fa6e37fe700 1 -- 172.21.15.163:0/2563077443 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e801d070 con 0x7fa6f40ffd30 2023-11-06T01:29:18.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.817+0000 7fa6e37fe700 1 -- 172.21.15.163:0/2563077443 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6e8004ca0 con 0x7fa6f40ffd30 2023-11-06T01:29:18.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.817+0000 7fa6f8810700 1 -- 172.21.15.163:0/2563077443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f41a0dc0 con 0x7fa6f40ffd30 2023-11-06T01:29:18.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.817+0000 7fa6e37fe700 1 -- 172.21.15.163:0/2563077443 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80055f0 con 0x7fa6f40ffd30 2023-11-06T01:29:18.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.817+0000 7fa6f8810700 1 -- 172.21.15.163:0/2563077443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f41a1180 con 0x7fa6f40ffd30 2023-11-06T01:29:18.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.818+0000 7fa6f8810700 1 -- 172.21.15.163:0/2563077443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f410b5a0 con 0x7fa6f40ffd30 2023-11-06T01:29:18.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.819+0000 7fa6e37fe700 1 -- 172.21.15.163:0/2563077443 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa6e800fc70 con 0x7fa6f40ffd30 2023-11-06T01:29:18.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.819+0000 7fa6e37fe700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6dc0a05a0 0x7fa6dc0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:18.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.819+0000 7fa6e37fe700 1 -- 172.21.15.163:0/2563077443 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fa6e80cbce0 con 0x7fa6f40ffd30 2023-11-06T01:29:18.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.819+0000 7fa6f1d9b700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6dc0a05a0 0x7fa6dc0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:18.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.820+0000 7fa6f1d9b700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6dc0a05a0 0x7fa6dc0a2a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009cb0 tx=0x7fa6e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:29:18.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:18.824+0000 7fa6e37fe700 1 -- 172.21.15.163:0/2563077443 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa6e805b0e0 con 0x7fa6f40ffd30 2023-11-06T01:29:18.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:29:18.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:29:18.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: osd.4 [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718] boot 2023-11-06T01:29:18.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: osdmap e34: 5 total, 5 up, 5 in 2023-11-06T01:29:18.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:18.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:18.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi062 2023-11-06T01:29:18.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:18.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:18.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:18.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:18.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi062 to 8584M 2023-11-06T01:29:18.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:18.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:18.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:18.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:18.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:29:18.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:29:18.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: osd.4 [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718] boot 2023-11-06T01:29:18.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: osdmap e34: 5 total, 5 up, 5 in 2023-11-06T01:29:18.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-06T01:29:18.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-11-06T01:29:18.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi062 2023-11-06T01:29:18.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:18.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:18.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:18.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:18.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi062 to 8584M 2023-11-06T01:29:18.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:18.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:18.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:18.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:19.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:19.020+0000 7fa6f8810700 1 -- 172.21.15.163:0/2563077443 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6f40698c0 con 0x7fa6dc0a05a0 2023-11-06T01:29:19.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:19 smithi163 ceph-mon[104416]: osdmap e35: 5 total, 5 up, 5 in 2023-11-06T01:29:19.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:19 smithi163 ceph-mon[104416]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:29:19.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:19 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:29:19.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:19 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:29:19.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:19 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:19.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:19 smithi053 ceph-mon[99472]: osdmap e35: 5 total, 5 up, 5 in 2023-11-06T01:29:19.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:19 smithi053 ceph-mon[99472]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:29:19.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:19 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:29:19.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:19 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:29:19.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:19 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:19.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:19 smithi062 ceph-mon[103924]: osdmap e35: 5 total, 5 up, 5 in 2023-11-06T01:29:19.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:19 smithi062 ceph-mon[103924]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:29:19.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:29:19.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:29:19.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:20.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:20 smithi163 ceph-mon[104416]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:20.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:20 smithi053 ceph-mon[99472]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:20.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:20 smithi062 ceph-mon[103924]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:22.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:21 smithi163 ceph-mon[104416]: from='client.? 172.21.15.163:0/3232263399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4adb65ab-f410-41a2-b4dc-7e12bcaba8ba"}]: dispatch 2023-11-06T01:29:22.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:21 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4adb65ab-f410-41a2-b4dc-7e12bcaba8ba"}]: dispatch 2023-11-06T01:29:22.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:21 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4adb65ab-f410-41a2-b4dc-7e12bcaba8ba"}]': finished 2023-11-06T01:29:22.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:21 smithi163 ceph-mon[104416]: osdmap e36: 6 total, 5 up, 6 in 2023-11-06T01:29:22.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:22.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:21 smithi053 ceph-mon[99472]: from='client.? 172.21.15.163:0/3232263399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4adb65ab-f410-41a2-b4dc-7e12bcaba8ba"}]: dispatch 2023-11-06T01:29:22.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:21 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4adb65ab-f410-41a2-b4dc-7e12bcaba8ba"}]: dispatch 2023-11-06T01:29:22.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:21 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4adb65ab-f410-41a2-b4dc-7e12bcaba8ba"}]': finished 2023-11-06T01:29:22.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:21 smithi053 ceph-mon[99472]: osdmap e36: 6 total, 5 up, 6 in 2023-11-06T01:29:22.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:22.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:21 smithi062 ceph-mon[103924]: from='client.? 172.21.15.163:0/3232263399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4adb65ab-f410-41a2-b4dc-7e12bcaba8ba"}]: dispatch 2023-11-06T01:29:22.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:21 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4adb65ab-f410-41a2-b4dc-7e12bcaba8ba"}]: dispatch 2023-11-06T01:29:22.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:21 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4adb65ab-f410-41a2-b4dc-7e12bcaba8ba"}]': finished 2023-11-06T01:29:22.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:21 smithi062 ceph-mon[103924]: osdmap e36: 6 total, 5 up, 6 in 2023-11-06T01:29:22.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:23.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:22 smithi163 ceph-mon[104416]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:23.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:22 smithi163 ceph-mon[104416]: from='client.? 172.21.15.163:0/1149214123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:29:23.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:22 smithi053 ceph-mon[99472]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:23.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:22 smithi053 ceph-mon[99472]: from='client.? 172.21.15.163:0/1149214123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:29:23.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:22 smithi062 ceph-mon[103924]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:23.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:22 smithi062 ceph-mon[103924]: from='client.? 172.21.15.163:0/1149214123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:29:25.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:24 smithi163 ceph-mon[104416]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:25.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:24 smithi053 ceph-mon[99472]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:25.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:24 smithi062 ceph-mon[103924]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:27.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:26 smithi163 ceph-mon[104416]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:27.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:26 smithi053 ceph-mon[99472]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:27.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:26 smithi062 ceph-mon[103924]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:28.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-06T01:29:28.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:28.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-06T01:29:28.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:28.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-06T01:29:28.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:29.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:28 smithi163 ceph-mon[104416]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:29.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:28 smithi163 ceph-mon[104416]: Deploying daemon osd.5 on smithi163 2023-11-06T01:29:29.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:28 smithi053 ceph-mon[99472]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:29.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:28 smithi053 ceph-mon[99472]: Deploying daemon osd.5 on smithi163 2023-11-06T01:29:29.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:28 smithi062 ceph-mon[103924]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:29.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:28 smithi062 ceph-mon[103924]: Deploying daemon osd.5 on smithi163 2023-11-06T01:29:31.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:30 smithi163 ceph-mon[104416]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:31.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:30 smithi053 ceph-mon[99472]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:31.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:30 smithi062 ceph-mon[103924]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:32.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:32 smithi163 ceph-mon[104416]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:32.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:32.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:32.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:33.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:32 smithi062 ceph-mon[103924]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:33.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:33.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:33.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:33.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:32 smithi053 ceph-mon[99472]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:33.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:33.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:33.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:34.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.387+0000 7fa6e37fe700 1 -- 172.21.15.163:0/2563077443 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa6f40698c0 con 0x7fa6dc0a05a0 2023-11-06T01:29:34.389 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 5 on host 'smithi163' 2023-11-06T01:29:34.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.389+0000 7fa6f8810700 1 -- 172.21.15.163:0/2563077443 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6dc0a05a0 msgr2=0x7fa6dc0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:34.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.389+0000 7fa6f8810700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6dc0a05a0 0x7fa6dc0a2a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009cb0 tx=0x7fa6e4009260 comp rx=0 tx=0).stop 2023-11-06T01:29:34.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.389+0000 7fa6f8810700 1 -- 172.21.15.163:0/2563077443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f40ffd30 msgr2=0x7fa6f4198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:34.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.389+0000 7fa6f8810700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f40ffd30 0x7fa6f4198cf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800b680 tx=0x7fa6e8005ef0 comp rx=0 tx=0).stop 2023-11-06T01:29:34.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.389+0000 7fa6f8810700 1 -- 172.21.15.163:0/2563077443 shutdown_connections 2023-11-06T01:29:34.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.389+0000 7fa6f8810700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6dc0a05a0 0x7fa6dc0a2a60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:34.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.389+0000 7fa6f8810700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018d0 0x7fa6f41a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:34.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.389+0000 7fa6f8810700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f40ffd30 0x7fa6f4198cf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:34.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.389+0000 7fa6f8810700 1 --2- 172.21.15.163:0/2563077443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f4100f30 0x7fa6f4199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:34.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.389+0000 7fa6f8810700 1 -- 172.21.15.163:0/2563077443 >> 172.21.15.163:0/2563077443 conn(0x7fa6f40fb2d0 msgr2=0x7fa6f40691b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:29:34.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.390+0000 7fa6f8810700 1 -- 172.21.15.163:0/2563077443 shutdown_connections 2023-11-06T01:29:34.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:29:34.390+0000 7fa6f8810700 1 -- 172.21.15.163:0/2563077443 wait complete. 2023-11-06T01:29:34.825 DEBUG:teuthology.orchestra.run.smithi163:osd.5> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.5.service 2023-11-06T01:29:34.828 INFO:tasks.cephadm:Deploying osd.6 on smithi053 with /dev/vg_nvme/lv_2... 2023-11-06T01:29:34.828 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-06T01:29:34.860 INFO:journalctl@ceph.osd.5.smithi163.stdout:-- Logs begin at Mon 2023-11-06 01:10:53 UTC. -- 2023-11-06T01:29:35.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:35 smithi053 ceph-mon[99472]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:35.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:35.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:35.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:35.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:35 smithi163 ceph-mon[104416]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:35.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:35.362 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:35.362 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.362 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.362 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.362 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:35.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:35 smithi062 ceph-mon[103924]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:35.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:35.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:35.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:35.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:35.402 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:29:36.050 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:29:35 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:29:35.890+0000 7f512d43d640 -1 osd.5 0 log_to_monitors true 2023-11-06T01:29:36.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:36 smithi053 ceph-mon[99472]: from='osd.5 [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-06T01:29:36.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:36 smithi053 ceph-mon[99472]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-06T01:29:36.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:36 smithi062 ceph-mon[103924]: from='osd.5 [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-06T01:29:36.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:36 smithi062 ceph-mon[103924]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-06T01:29:36.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:36 smithi163 ceph-mon[104416]: from='osd.5 [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-06T01:29:36.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:36 smithi163 ceph-mon[104416]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-06T01:29:36.900 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-06T01:29:36.901 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-06T01:29:36.901 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2023-11-06T01:29:36.901 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2023-11-06T01:29:36.901 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00796611 s, 1.3 GB/s 2023-11-06T01:29:36.902 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 2023-11-06T01:29:36.902 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2023-11-06T01:29:37.149 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:29:37 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:29:37.115+0000 7f5122ab4700 -1 osd.5 0 waiting for initial osdmap 2023-11-06T01:29:37.150 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:29:37 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:29:37.122+0000 7f511c8a5700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:29:37.150 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:37 smithi163 ceph-mon[104416]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:37.151 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:37 smithi163 ceph-mon[104416]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-06T01:29:37.151 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:37 smithi163 ceph-mon[104416]: osdmap e37: 6 total, 5 up, 6 in 2023-11-06T01:29:37.151 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:37 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:37.151 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:37 smithi163 ceph-mon[104416]: from='osd.5 [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:29:37.151 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:37 smithi163 ceph-mon[104416]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:29:37.263 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:37 smithi053 ceph-mon[99472]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:37.263 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:37 smithi053 ceph-mon[99472]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-06T01:29:37.263 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:37 smithi053 ceph-mon[99472]: osdmap e37: 6 total, 5 up, 6 in 2023-11-06T01:29:37.263 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:37.263 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:37 smithi053 ceph-mon[99472]: from='osd.5 [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:29:37.264 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:37 smithi053 ceph-mon[99472]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:29:37.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2023-11-06T01:29:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:37 smithi062 ceph-mon[103924]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:37 smithi062 ceph-mon[103924]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-06T01:29:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:37 smithi062 ceph-mon[103924]: osdmap e37: 6 total, 5 up, 6 in 2023-11-06T01:29:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:37 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:37.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:37 smithi062 ceph-mon[103924]: from='osd.5 [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:29:37.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:37 smithi062 ceph-mon[103924]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:29:37.839 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:29:38.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:29:38.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: osdmap e38: 6 total, 5 up, 6 in 2023-11-06T01:29:38.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:38.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:38.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi163 2023-11-06T01:29:38.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:38.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:38.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:38.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:38.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi163 to 8584M 2023-11-06T01:29:38.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:38.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:38.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:38.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:38.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:29:38.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: osdmap e38: 6 total, 5 up, 6 in 2023-11-06T01:29:38.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:38.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:38.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi163 2023-11-06T01:29:38.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:38.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:38.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:38.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:38.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi163 to 8584M 2023-11-06T01:29:38.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:38.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:38.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:38.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:38.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:29:38.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: osdmap e38: 6 total, 5 up, 6 in 2023-11-06T01:29:38.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:38.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:38.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi163 2023-11-06T01:29:38.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:38.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:38.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:38.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:38.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi163 to 8584M 2023-11-06T01:29:38.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:38.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:38.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:38.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:39.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.068+0000 7f2384cb2700 1 -- 172.21.15.53:0/3812563394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2380100e20 msgr2=0x7f2380101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:39.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.068+0000 7f2384cb2700 1 --2- 172.21.15.53:0/3812563394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2380100e20 0x7f2380101240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2370009990 tx=0x7f2370009ca0 comp rx=0 tx=0).stop 2023-11-06T01:29:39.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.068+0000 7f2384cb2700 1 -- 172.21.15.53:0/3812563394 shutdown_connections 2023-11-06T01:29:39.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.068+0000 7f2384cb2700 1 --2- 172.21.15.53:0/3812563394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2380102940 0x7f2380109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:39.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.068+0000 7f2384cb2700 1 --2- 172.21.15.53:0/3812563394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2380100e20 0x7f2380101240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:39.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.068+0000 7f2384cb2700 1 --2- 172.21.15.53:0/3812563394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380101f80 0x7f2380102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:39.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.068+0000 7f2384cb2700 1 -- 172.21.15.53:0/3812563394 >> 172.21.15.53:0/3812563394 conn(0x7f23800fc380 msgr2=0x7f23800fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:29:39.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.068+0000 7f2384cb2700 1 -- 172.21.15.53:0/3812563394 shutdown_connections 2023-11-06T01:29:39.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.068+0000 7f2384cb2700 1 -- 172.21.15.53:0/3812563394 wait complete. 2023-11-06T01:29:39.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.069+0000 7f2384cb2700 1 Processor -- start 2023-11-06T01:29:39.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.069+0000 7f2384cb2700 1 -- start start 2023-11-06T01:29:39.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.069+0000 7f2384cb2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380100e20 0x7f2380074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:39.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.069+0000 7f2384cb2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2380101f80 0x7f2380073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:39.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.069+0000 7f2384cb2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2380102940 0x7f23800736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:39.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.069+0000 7f2384cb2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2380073e40 con 0x7f2380102940 2023-11-06T01:29:39.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.069+0000 7f2384cb2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2380073fb0 con 0x7f2380101f80 2023-11-06T01:29:39.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.069+0000 7f2384cb2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23801a0c40 con 0x7f2380100e20 2023-11-06T01:29:39.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.069+0000 7f237ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2380102940 0x7f23800736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:39.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.069+0000 7f237ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2380102940 0x7f23800736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37260/0 (socket says 172.21.15.53:37260) 2023-11-06T01:29:39.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f237ed9d700 1 -- 172.21.15.53:0/956797265 learned_addr learned my addr 172.21.15.53:0/956797265 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:29:39.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f237e59c700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380100e20 0x7f2380074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:39.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f237dd9b700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2380101f80 0x7f2380073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:39.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f237ed9d700 1 -- 172.21.15.53:0/956797265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380100e20 msgr2=0x7f2380074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:39.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f237ed9d700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380100e20 0x7f2380074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:39.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f237ed9d700 1 -- 172.21.15.53:0/956797265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2380101f80 msgr2=0x7f2380073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:39.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f237ed9d700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2380101f80 0x7f2380073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:39.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f237ed9d700 1 -- 172.21.15.53:0/956797265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23700095f0 con 0x7f2380102940 2023-11-06T01:29:39.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f237e59c700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380100e20 0x7f2380074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:29:39.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f237dd9b700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2380101f80 0x7f2380073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:29:39.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f237ed9d700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2380102940 0x7f23800736c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f237800ada0 tx=0x7f237800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:29:39.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f23777fe700 1 -- 172.21.15.53:0/956797265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237800f070 con 0x7f2380102940 2023-11-06T01:29:39.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f23777fe700 1 -- 172.21.15.53:0/956797265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2378007b30 con 0x7f2380102940 2023-11-06T01:29:39.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f2384cb2700 1 -- 172.21.15.53:0/956797265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23801a0f20 con 0x7f2380102940 2023-11-06T01:29:39.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f23777fe700 1 -- 172.21.15.53:0/956797265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378005440 con 0x7f2380102940 2023-11-06T01:29:39.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.070+0000 7f2384cb2700 1 -- 172.21.15.53:0/956797265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23801a1340 con 0x7f2380102940 2023-11-06T01:29:39.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.071+0000 7f2384cb2700 1 -- 172.21.15.53:0/956797265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23801084b0 con 0x7f2380102940 2023-11-06T01:29:39.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.072+0000 7f23777fe700 1 -- 172.21.15.53:0/956797265 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f2378004ad0 con 0x7f2380102940 2023-11-06T01:29:39.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.073+0000 7f23777fe700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f236c0a0520 0x7f236c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:39.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.073+0000 7f237e59c700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f236c0a0520 0x7f236c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:39.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.073+0000 7f23777fe700 1 -- 172.21.15.53:0/956797265 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4566+0+0 (secure 0 0 0) 0x7f23780ca6a0 con 0x7f2380102940 2023-11-06T01:29:39.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.073+0000 7f237e59c700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f236c0a0520 0x7f236c0a29e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2370005f50 tx=0x7f2370005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:29:39.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.077+0000 7f23777fe700 1 -- 172.21.15.53:0/956797265 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2378095170 con 0x7f2380102940 2023-11-06T01:29:39.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:39.285+0000 7f2384cb2700 1 -- 172.21.15.53:0/956797265 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f23800611d0 con 0x7f236c0a0520 2023-11-06T01:29:39.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:39 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:29:39.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:39 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:29:39.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:39 smithi053 ceph-mon[99472]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:39.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:39.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:39 smithi053 ceph-mon[99472]: osd.5 [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439] boot 2023-11-06T01:29:39.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:39 smithi053 ceph-mon[99472]: osdmap e39: 6 total, 6 up, 6 in 2023-11-06T01:29:39.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:39 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:29:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:39 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:29:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:39 smithi062 ceph-mon[103924]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:39.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:39.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:39 smithi062 ceph-mon[103924]: osd.5 [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439] boot 2023-11-06T01:29:39.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:39 smithi062 ceph-mon[103924]: osdmap e39: 6 total, 6 up, 6 in 2023-11-06T01:29:39.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:39.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:39 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:29:39.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:39 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:29:39.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:39 smithi163 ceph-mon[104416]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-11-06T01:29:39.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:39.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:39 smithi163 ceph-mon[104416]: osd.5 [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439] boot 2023-11-06T01:29:39.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:39 smithi163 ceph-mon[104416]: osdmap e39: 6 total, 6 up, 6 in 2023-11-06T01:29:39.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-06T01:29:40.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:40 smithi053 ceph-mon[99472]: osdmap e40: 6 total, 6 up, 6 in 2023-11-06T01:29:40.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:40 smithi053 ceph-mon[99472]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:29:40.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:29:40.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:29:40.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:40.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:40 smithi062 ceph-mon[103924]: osdmap e40: 6 total, 6 up, 6 in 2023-11-06T01:29:40.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:40 smithi062 ceph-mon[103924]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:29:40.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:40 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:29:40.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:40 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:29:40.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:40 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:40.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:40 smithi163 ceph-mon[104416]: osdmap e40: 6 total, 6 up, 6 in 2023-11-06T01:29:40.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:40 smithi163 ceph-mon[104416]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:29:40.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:40 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:29:40.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:40 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:29:40.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:40 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:41 smithi062 ceph-mon[103924]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:41 smithi062 ceph-mon[103924]: osdmap e41: 6 total, 6 up, 6 in 2023-11-06T01:29:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:41 smithi062 ceph-mon[103924]: osdmap e42: 6 total, 6 up, 6 in 2023-11-06T01:29:41.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:41 smithi163 ceph-mon[104416]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:41 smithi163 ceph-mon[104416]: osdmap e41: 6 total, 6 up, 6 in 2023-11-06T01:29:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:41 smithi163 ceph-mon[104416]: osdmap e42: 6 total, 6 up, 6 in 2023-11-06T01:29:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:41 smithi053 ceph-mon[99472]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:41 smithi053 ceph-mon[99472]: osdmap e41: 6 total, 6 up, 6 in 2023-11-06T01:29:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:41 smithi053 ceph-mon[99472]: osdmap e42: 6 total, 6 up, 6 in 2023-11-06T01:29:42.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:42 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1168680278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b8502f9-4651-43b4-b212-d37785c4655a"}]: dispatch 2023-11-06T01:29:42.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:42 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1168680278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b8502f9-4651-43b4-b212-d37785c4655a"}]': finished 2023-11-06T01:29:42.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:42 smithi062 ceph-mon[103924]: osdmap e43: 7 total, 6 up, 7 in 2023-11-06T01:29:42.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:42 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:42.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:42 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1168680278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b8502f9-4651-43b4-b212-d37785c4655a"}]: dispatch 2023-11-06T01:29:42.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:42 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1168680278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b8502f9-4651-43b4-b212-d37785c4655a"}]': finished 2023-11-06T01:29:42.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:42 smithi163 ceph-mon[104416]: osdmap e43: 7 total, 6 up, 7 in 2023-11-06T01:29:42.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:42 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:42.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:42 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1168680278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b8502f9-4651-43b4-b212-d37785c4655a"}]: dispatch 2023-11-06T01:29:42.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:42 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1168680278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b8502f9-4651-43b4-b212-d37785c4655a"}]': finished 2023-11-06T01:29:42.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:42 smithi053 ceph-mon[99472]: osdmap e43: 7 total, 6 up, 7 in 2023-11-06T01:29:42.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:42 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:43 smithi062 ceph-mon[103924]: pgmap v130: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:43 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/690804631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:29:43.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:43 smithi163 ceph-mon[104416]: pgmap v130: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:43.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:43 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/690804631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:29:43.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:43 smithi053 ceph-mon[99472]: pgmap v130: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:43.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:43 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/690804631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:29:45.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:45 smithi062 ceph-mon[103924]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:45.418 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:45 smithi053 ceph-mon[99472]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:45.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:45 smithi163 ceph-mon[104416]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:47.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:47 smithi053 ceph-mon[99472]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-06T01:29:47.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:47 smithi062 ceph-mon[103924]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-06T01:29:47.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:47 smithi163 ceph-mon[104416]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-06T01:29:48.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:48 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-06T01:29:48.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:48 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:48.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:48 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-06T01:29:48.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:48 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:48.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:48 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-06T01:29:48.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:48 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:49.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:49 smithi053 ceph-mon[99472]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-11-06T01:29:49.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:49 smithi053 ceph-mon[99472]: Deploying daemon osd.6 on smithi053 2023-11-06T01:29:49.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:49 smithi163 ceph-mon[104416]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-11-06T01:29:49.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:49 smithi163 ceph-mon[104416]: Deploying daemon osd.6 on smithi053 2023-11-06T01:29:49.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:49 smithi062 ceph-mon[103924]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-11-06T01:29:49.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:49 smithi062 ceph-mon[103924]: Deploying daemon osd.6 on smithi053 2023-11-06T01:29:51.415 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:51 smithi053 ceph-mon[99472]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-11-06T01:29:51.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:51 smithi163 ceph-mon[104416]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-11-06T01:29:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:51 smithi062 ceph-mon[103924]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-11-06T01:29:53.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:53 smithi053 ceph-mon[99472]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-06T01:29:53.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:53.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:53.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:53.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:53 smithi163 ceph-mon[104416]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-06T01:29:53.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:53 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:53.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:53 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:53.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:53 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:53.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:53 smithi062 ceph-mon[103924]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-06T01:29:53.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:53 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:53.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:53 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:53.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:53 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:54.941 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 6 on host 'smithi053' 2023-11-06T01:29:54.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.939+0000 7f23777fe700 1 -- 172.21.15.53:0/956797265 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f23800611d0 con 0x7f236c0a0520 2023-11-06T01:29:54.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.941+0000 7f2384cb2700 1 -- 172.21.15.53:0/956797265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f236c0a0520 msgr2=0x7f236c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:54.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.941+0000 7f2384cb2700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f236c0a0520 0x7f236c0a29e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2370005f50 tx=0x7f2370005e80 comp rx=0 tx=0).stop 2023-11-06T01:29:54.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.942+0000 7f2384cb2700 1 -- 172.21.15.53:0/956797265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2380102940 msgr2=0x7f23800736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:54.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.942+0000 7f2384cb2700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2380102940 0x7f23800736c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f237800ada0 tx=0x7f237800c330 comp rx=0 tx=0).stop 2023-11-06T01:29:54.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.942+0000 7f2384cb2700 1 -- 172.21.15.53:0/956797265 shutdown_connections 2023-11-06T01:29:54.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.942+0000 7f2384cb2700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f236c0a0520 0x7f236c0a29e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:54.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.942+0000 7f2384cb2700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2380102940 0x7f23800736c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:54.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.942+0000 7f2384cb2700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380100e20 0x7f2380074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:54.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.942+0000 7f2384cb2700 1 --2- 172.21.15.53:0/956797265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2380101f80 0x7f2380073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:54.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.942+0000 7f2384cb2700 1 -- 172.21.15.53:0/956797265 >> 172.21.15.53:0/956797265 conn(0x7f23800fc380 msgr2=0x7f23801055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:29:54.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.942+0000 7f2384cb2700 1 -- 172.21.15.53:0/956797265 shutdown_connections 2023-11-06T01:29:54.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:29:54.942+0000 7f2384cb2700 1 -- 172.21.15.53:0/956797265 wait complete. 2023-11-06T01:29:55.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:55 smithi053 ceph-mon[99472]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-06T01:29:55.325 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.325 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.326 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:55.326 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:55.327 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.327 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.327 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.328 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:55.330 DEBUG:teuthology.orchestra.run.smithi053:osd.6> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.6.service 2023-11-06T01:29:55.334 INFO:tasks.cephadm:Deploying osd.7 on smithi062 with /dev/vg_nvme/lv_2... 2023-11-06T01:29:55.334 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-06T01:29:55.369 INFO:journalctl@ceph.osd.6.smithi053.stdout:-- Logs begin at Mon 2023-11-06 01:09:30 UTC. -- 2023-11-06T01:29:55.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:55 smithi163 ceph-mon[104416]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-06T01:29:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:55 smithi062 ceph-mon[103924]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-06T01:29:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:29:55.919 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:29:56.582 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:29:56 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:29:56.166+0000 7f257d93b640 -1 osd.6 0 log_to_monitors true 2023-11-06T01:29:57.063 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:56 smithi062 ceph-mon[103924]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-06T01:29:57.064 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:56 smithi062 ceph-mon[103924]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-06T01:29:57.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:56 smithi163 ceph-mon[104416]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-06T01:29:57.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:56 smithi163 ceph-mon[104416]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-06T01:29:57.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:56 smithi053 ceph-mon[99472]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-06T01:29:57.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:56 smithi053 ceph-mon[99472]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-06T01:29:57.567 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-06T01:29:57.567 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-06T01:29:57.567 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2023-11-06T01:29:57.568 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2023-11-06T01:29:57.568 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110084 s, 953 MB/s 2023-11-06T01:29:57.568 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2023-11-06T01:29:58.079 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2023-11-06T01:29:58.094 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:29:57.965+0000 7f2572fb2700 -1 osd.6 0 waiting for initial osdmap 2023-11-06T01:29:58.094 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:29:57.975+0000 7f256e5a6700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:29:58.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-06T01:29:58.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: osdmap e44: 7 total, 6 up, 7 in 2023-11-06T01:29:58.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:58.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:29:58.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.099 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:58.099 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:58.099 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:58.099 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.099 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:58.100 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:58.100 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-06T01:29:58.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: osdmap e44: 7 total, 6 up, 7 in 2023-11-06T01:29:58.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:58.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:29:58.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:58.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:58.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:58.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:58.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:58.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-06T01:29:58.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: osdmap e44: 7 total, 6 up, 7 in 2023-11-06T01:29:58.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:58.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:29:58.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:58.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:58.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:29:58.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:29:58.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:29:58.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:29:58.638 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:29:59.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:58 smithi062 ceph-mon[103924]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:59.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:58 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi053 2023-11-06T01:29:59.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:58 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi053 to 5724M 2023-11-06T01:29:59.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:58 smithi062 ceph-mon[103924]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:29:59.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:58 smithi062 ceph-mon[103924]: osdmap e45: 7 total, 6 up, 7 in 2023-11-06T01:29:59.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:58 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:59.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:58 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:59.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:58 smithi163 ceph-mon[104416]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:59.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:58 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi053 2023-11-06T01:29:59.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:58 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi053 to 5724M 2023-11-06T01:29:59.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:58 smithi163 ceph-mon[104416]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:29:59.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:58 smithi163 ceph-mon[104416]: osdmap e45: 7 total, 6 up, 7 in 2023-11-06T01:29:59.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:58 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:59.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:58 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:59.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:58 smithi053 ceph-mon[99472]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-11-06T01:29:59.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:58 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi053 2023-11-06T01:29:59.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:58 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi053 to 5724M 2023-11-06T01:29:59.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:58 smithi053 ceph-mon[99472]: from='osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:29:59.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:58 smithi053 ceph-mon[99472]: osdmap e45: 7 total, 6 up, 7 in 2023-11-06T01:29:59.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:59.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:29:59.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.900+0000 7fb9812f6700 1 -- 172.21.15.62:0/1747183426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb97c102940 msgr2=0x7fb97c109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:59.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.900+0000 7fb9812f6700 1 --2- 172.21.15.62:0/1747183426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb97c102940 0x7fb97c109260 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb97400b0a0 tx=0x7fb97400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:29:59.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.900+0000 7fb9812f6700 1 -- 172.21.15.62:0/1747183426 shutdown_connections 2023-11-06T01:29:59.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.900+0000 7fb9812f6700 1 --2- 172.21.15.62:0/1747183426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb97c102940 0x7fb97c109260 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:59.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.900+0000 7fb9812f6700 1 --2- 172.21.15.62:0/1747183426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c100e20 0x7fb97c101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:59.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.900+0000 7fb9812f6700 1 --2- 172.21.15.62:0/1747183426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c101f80 0x7fb97c102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:59.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.900+0000 7fb9812f6700 1 -- 172.21.15.62:0/1747183426 >> 172.21.15.62:0/1747183426 conn(0x7fb97c0fc380 msgr2=0x7fb97c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:29:59.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.900+0000 7fb9812f6700 1 -- 172.21.15.62:0/1747183426 shutdown_connections 2023-11-06T01:29:59.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.900+0000 7fb9812f6700 1 -- 172.21.15.62:0/1747183426 wait complete. 2023-11-06T01:29:59.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.900+0000 7fb9812f6700 1 Processor -- start 2023-11-06T01:29:59.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb9812f6700 1 -- start start 2023-11-06T01:29:59.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb9812f6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb97c100e20 0x7fb97c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:59.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb9812f6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c101f80 0x7fb97c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:59.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb9812f6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c102940 0x7fb97c0736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:59.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb9812f6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c073e40 con 0x7fb97c101f80 2023-11-06T01:29:59.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb9812f6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c073fb0 con 0x7fb97c100e20 2023-11-06T01:29:59.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb9812f6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1a0c10 con 0x7fb97c102940 2023-11-06T01:29:59.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb97c100e20 0x7fb97c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:59.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb97c100e20 0x7fb97c074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42548/0 (socket says 172.21.15.62:42548) 2023-11-06T01:29:59.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97affd700 1 -- 172.21.15.62:0/579253714 learned_addr learned my addr 172.21.15.62:0/579253714 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-11-06T01:29:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97a7fc700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c101f80 0x7fb97c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97affd700 1 -- 172.21.15.62:0/579253714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c102940 msgr2=0x7fb97c0736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97b7fe700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c102940 0x7fb97c0736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97affd700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c102940 0x7fb97c0736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97affd700 1 -- 172.21.15.62:0/579253714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c101f80 msgr2=0x7fb97c073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:29:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97affd700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c101f80 0x7fb97c073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:29:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97affd700 1 -- 172.21.15.62:0/579253714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb974009d00 con 0x7fb97c100e20 2023-11-06T01:29:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97b7fe700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c102940 0x7fb97c0736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:29:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.901+0000 7fb97affd700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb97c100e20 0x7fb97c074b30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00d950 tx=0x7fb96c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:29:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.902+0000 7fb963fff700 1 -- 172.21.15.62:0/579253714 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c009940 con 0x7fb97c100e20 2023-11-06T01:29:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.902+0000 7fb9812f6700 1 -- 172.21.15.62:0/579253714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c1a0ef0 con 0x7fb97c100e20 2023-11-06T01:29:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.902+0000 7fb963fff700 1 -- 172.21.15.62:0/579253714 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb96c010460 con 0x7fb97c100e20 2023-11-06T01:29:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.902+0000 7fb963fff700 1 -- 172.21.15.62:0/579253714 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00f620 con 0x7fb97c100e20 2023-11-06T01:29:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.902+0000 7fb9812f6700 1 -- 172.21.15.62:0/579253714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c1a1310 con 0x7fb97c100e20 2023-11-06T01:29:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.903+0000 7fb9812f6700 1 -- 172.21.15.62:0/579253714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c10c630 con 0x7fb97c100e20 2023-11-06T01:29:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.904+0000 7fb963fff700 1 -- 172.21.15.62:0/579253714 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fb96c0108d0 con 0x7fb97c100e20 2023-11-06T01:29:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.904+0000 7fb963fff700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9640a0520 0x7fb9640a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:29:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.904+0000 7fb963fff700 1 -- 172.21.15.62:0/579253714 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fb96c0cae60 con 0x7fb97c100e20 2023-11-06T01:29:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.904+0000 7fb97a7fc700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9640a0520 0x7fb9640a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:29:59.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.905+0000 7fb97a7fc700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9640a0520 0x7fb9640a29e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb970007be0 tx=0x7fb9700073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:29:59.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:29:59.908+0000 7fb963fff700 1 -- 172.21.15.62:0/579253714 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb96c095710 con 0x7fb97c100e20 2023-11-06T01:30:00.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:00.117+0000 7fb9812f6700 1 -- 172.21.15.62:0/579253714 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb97c0611d0 con 0x7fb9640a0520 2023-11-06T01:30:00.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:59 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:30:00.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:59 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:30:00.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:59 smithi163 ceph-mon[104416]: osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819] boot 2023-11-06T01:30:00.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:59 smithi163 ceph-mon[104416]: osdmap e46: 7 total, 7 up, 7 in 2023-11-06T01:30:00.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:29:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:30:00.327 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:59 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:30:00.327 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:59 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:30:00.328 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:59 smithi062 ceph-mon[103924]: osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819] boot 2023-11-06T01:30:00.328 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:59 smithi062 ceph-mon[103924]: osdmap e46: 7 total, 7 up, 7 in 2023-11-06T01:30:00.328 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:29:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:30:00.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:59 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:30:00.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:59 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:30:00.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:59 smithi053 ceph-mon[99472]: osd.6 [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819] boot 2023-11-06T01:30:00.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:59 smithi053 ceph-mon[99472]: osdmap e46: 7 total, 7 up, 7 in 2023-11-06T01:30:00.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:29:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-06T01:30:01.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:00 smithi163 ceph-mon[104416]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:01.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:00 smithi163 ceph-mon[104416]: osdmap e47: 7 total, 7 up, 7 in 2023-11-06T01:30:01.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:00 smithi163 ceph-mon[104416]: overall HEALTH_OK 2023-11-06T01:30:01.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:00 smithi163 ceph-mon[104416]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:30:01.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:00 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:30:01.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:00 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:30:01.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:00 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:01.264 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:00 smithi062 ceph-mon[103924]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:01.265 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:00 smithi062 ceph-mon[103924]: osdmap e47: 7 total, 7 up, 7 in 2023-11-06T01:30:01.265 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:00 smithi062 ceph-mon[103924]: overall HEALTH_OK 2023-11-06T01:30:01.265 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:00 smithi062 ceph-mon[103924]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:30:01.265 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:00 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:30:01.265 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:00 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:30:01.266 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:00 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:01.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:00 smithi053 ceph-mon[99472]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:01.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:00 smithi053 ceph-mon[99472]: osdmap e47: 7 total, 7 up, 7 in 2023-11-06T01:30:01.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:00 smithi053 ceph-mon[99472]: overall HEALTH_OK 2023-11-06T01:30:01.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:00 smithi053 ceph-mon[99472]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:30:01.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:30:01.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:30:01.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:03.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:02 smithi163 ceph-mon[104416]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:03.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:02 smithi163 ceph-mon[104416]: from='client.? 172.21.15.62:0/3850179412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6ac514c-4385-49cc-b408-f24f49a200ee"}]: dispatch 2023-11-06T01:30:03.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:02 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6ac514c-4385-49cc-b408-f24f49a200ee"}]: dispatch 2023-11-06T01:30:03.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:02 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6ac514c-4385-49cc-b408-f24f49a200ee"}]': finished 2023-11-06T01:30:03.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:02 smithi163 ceph-mon[104416]: osdmap e48: 8 total, 7 up, 8 in 2023-11-06T01:30:03.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:02 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:03.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:02 smithi053 ceph-mon[99472]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:03.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:02 smithi053 ceph-mon[99472]: from='client.? 172.21.15.62:0/3850179412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6ac514c-4385-49cc-b408-f24f49a200ee"}]: dispatch 2023-11-06T01:30:03.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:02 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6ac514c-4385-49cc-b408-f24f49a200ee"}]: dispatch 2023-11-06T01:30:03.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:02 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6ac514c-4385-49cc-b408-f24f49a200ee"}]': finished 2023-11-06T01:30:03.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:02 smithi053 ceph-mon[99472]: osdmap e48: 8 total, 7 up, 8 in 2023-11-06T01:30:03.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:02 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:03.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:02 smithi062 ceph-mon[103924]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:03.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:02 smithi062 ceph-mon[103924]: from='client.? 172.21.15.62:0/3850179412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6ac514c-4385-49cc-b408-f24f49a200ee"}]: dispatch 2023-11-06T01:30:03.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:02 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6ac514c-4385-49cc-b408-f24f49a200ee"}]: dispatch 2023-11-06T01:30:03.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:02 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6ac514c-4385-49cc-b408-f24f49a200ee"}]': finished 2023-11-06T01:30:03.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:02 smithi062 ceph-mon[103924]: osdmap e48: 8 total, 7 up, 8 in 2023-11-06T01:30:03.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:02 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:04.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:03 smithi163 ceph-mon[104416]: from='client.? 172.21.15.62:0/4129667319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:30:04.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:03 smithi053 ceph-mon[99472]: from='client.? 172.21.15.62:0/4129667319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:30:04.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:03 smithi062 ceph-mon[103924]: from='client.? 172.21.15.62:0/4129667319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:30:05.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:04 smithi163 ceph-mon[104416]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:05.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:04 smithi053 ceph-mon[99472]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:05.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:04 smithi062 ceph-mon[103924]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:07.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:06 smithi062 ceph-mon[103924]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:07.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:06 smithi163 ceph-mon[104416]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:07.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:06 smithi053 ceph-mon[99472]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:09.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:08 smithi062 ceph-mon[103924]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:09.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:08 smithi163 ceph-mon[104416]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:09.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:08 smithi053 ceph-mon[99472]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:10.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:09 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-06T01:30:10.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:09 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:10.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:09 smithi062 ceph-mon[103924]: Deploying daemon osd.7 on smithi062 2023-11-06T01:30:10.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:09 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-06T01:30:10.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:09 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:10.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:09 smithi053 ceph-mon[99472]: Deploying daemon osd.7 on smithi062 2023-11-06T01:30:10.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:09 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-06T01:30:10.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:09 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:10.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:09 smithi163 ceph-mon[104416]: Deploying daemon osd.7 on smithi062 2023-11-06T01:30:11.280 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:10 smithi062 ceph-mon[103924]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:11.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:10 smithi053 ceph-mon[99472]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:10 smithi163 ceph-mon[104416]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:13.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:12 smithi062 ceph-mon[103924]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:13.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:12 smithi053 ceph-mon[99472]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:13.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:12 smithi163 ceph-mon[104416]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:15.029 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:14 smithi062 ceph-mon[103924]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:15.030 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:15.030 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:15.030 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:15.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:14 smithi053 ceph-mon[99472]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:15.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:15.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:15.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:15.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:14 smithi163 ceph-mon[104416]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:15.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:15.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:15.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:16.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.734+0000 7fb963fff700 1 -- 172.21.15.62:0/579253714 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb97c0611d0 con 0x7fb9640a0520 2023-11-06T01:30:16.736 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 7 on host 'smithi062' 2023-11-06T01:30:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 -- 172.21.15.62:0/579253714 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9640a0520 msgr2=0x7fb9640a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9640a0520 0x7fb9640a29e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb970007be0 tx=0x7fb9700073d0 comp rx=0 tx=0).stop 2023-11-06T01:30:16.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 -- 172.21.15.62:0/579253714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb97c100e20 msgr2=0x7fb97c074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:16.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb97c100e20 0x7fb97c074b30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00d950 tx=0x7fb96c00dc60 comp rx=0 tx=0).stop 2023-11-06T01:30:16.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 -- 172.21.15.62:0/579253714 shutdown_connections 2023-11-06T01:30:16.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9640a0520 0x7fb9640a29e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:16.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb97c100e20 0x7fb97c074b30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:16.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c102940 0x7fb97c0736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:16.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 --2- 172.21.15.62:0/579253714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c101f80 0x7fb97c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:16.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 -- 172.21.15.62:0/579253714 >> 172.21.15.62:0/579253714 conn(0x7fb97c0fc380 msgr2=0x7fb97c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:30:16.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 -- 172.21.15.62:0/579253714 shutdown_connections 2023-11-06T01:30:16.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:30:16.736+0000 7fb9812f6700 1 -- 172.21.15.62:0/579253714 wait complete. 2023-11-06T01:30:16.955 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:16 smithi062 ceph-mon[103924]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:16.955 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:16 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:16.956 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:16 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:16.956 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:16 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:16.956 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:16 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:17.233 DEBUG:teuthology.orchestra.run.smithi062:osd.7> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.7.service 2023-11-06T01:30:17.235 INFO:tasks.cephadm:Deploying osd.8 on smithi163 with /dev/vg_nvme/lv_2... 2023-11-06T01:30:17.236 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-06T01:30:17.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:16 smithi163 ceph-mon[104416]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:17.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:16 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:17.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:16 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:17.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:16 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:17.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:16 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:17.269 INFO:journalctl@ceph.osd.7.smithi062.stdout:-- Logs begin at Mon 2023-11-06 01:10:07 UTC. -- 2023-11-06T01:30:17.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:16 smithi053 ceph-mon[99472]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:17.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:16 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:17.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:16 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:17.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:16 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:17.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:16 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:17.812 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:30:17.988 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:17 smithi062 ceph-mon[103924]: from='osd.7 [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-06T01:30:17.988 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:17 smithi062 ceph-mon[103924]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-06T01:30:17.989 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:30:17 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:30:17.728+0000 7f0627000640 -1 osd.7 0 log_to_monitors true 2023-11-06T01:30:18.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:17 smithi163 ceph-mon[104416]: from='osd.7 [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-06T01:30:18.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:17 smithi163 ceph-mon[104416]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-06T01:30:18.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:17 smithi053 ceph-mon[99472]: from='osd.7 [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-06T01:30:18.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:17 smithi053 ceph-mon[99472]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-06T01:30:18.983 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:30:18 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:30:18.971+0000 7f061c677700 -1 osd.7 0 waiting for initial osdmap 2023-11-06T01:30:18.984 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:18 smithi062 ceph-mon[103924]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:18.984 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:18 smithi062 ceph-mon[103924]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-06T01:30:18.984 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:18 smithi062 ceph-mon[103924]: osdmap e49: 8 total, 7 up, 8 in 2023-11-06T01:30:18.984 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:18.984 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:18 smithi062 ceph-mon[103924]: from='osd.7 [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:30:18.985 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:18 smithi062 ceph-mon[103924]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:30:19.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:18 smithi163 ceph-mon[104416]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:19.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:18 smithi163 ceph-mon[104416]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-06T01:30:19.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:18 smithi163 ceph-mon[104416]: osdmap e49: 8 total, 7 up, 8 in 2023-11-06T01:30:19.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:19.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:18 smithi163 ceph-mon[104416]: from='osd.7 [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:30:19.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:18 smithi163 ceph-mon[104416]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:30:19.282 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-06T01:30:19.283 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-06T01:30:19.283 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2023-11-06T01:30:19.283 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2023-11-06T01:30:19.283 INFO:teuthology.orchestra.run.smithi163.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0169368 s, 619 MB/s 2023-11-06T01:30:19.284 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2023-11-06T01:30:19.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:18 smithi053 ceph-mon[99472]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:19.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:18 smithi053 ceph-mon[99472]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-06T01:30:19.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:18 smithi053 ceph-mon[99472]: osdmap e49: 8 total, 7 up, 8 in 2023-11-06T01:30:19.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:19.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:18 smithi053 ceph-mon[99472]: from='osd.7 [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:30:19.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:18 smithi053 ceph-mon[99472]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:30:19.658 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi163:vg_nvme/lv_2 2023-11-06T01:30:19.973 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:30:18 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:30:18.981+0000 7f0615c67700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:30:20.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:19 smithi163 ceph-mon[104416]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:30:20.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:19 smithi163 ceph-mon[104416]: osdmap e50: 8 total, 7 up, 8 in 2023-11-06T01:30:20.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:19 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:20.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:19 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:20.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:19 smithi163 ceph-mon[104416]: osd.7 [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587] boot 2023-11-06T01:30:20.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:19 smithi163 ceph-mon[104416]: osdmap e51: 8 total, 8 up, 8 in 2023-11-06T01:30:20.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:19 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:20.253 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:30:20.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:19 smithi053 ceph-mon[99472]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:30:20.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:19 smithi053 ceph-mon[99472]: osdmap e50: 8 total, 7 up, 8 in 2023-11-06T01:30:20.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:19 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:20.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:19 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:20.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:19 smithi053 ceph-mon[99472]: osd.7 [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587] boot 2023-11-06T01:30:20.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:19 smithi053 ceph-mon[99472]: osdmap e51: 8 total, 8 up, 8 in 2023-11-06T01:30:20.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:19 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:20.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:19 smithi062 ceph-mon[103924]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:30:20.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:19 smithi062 ceph-mon[103924]: osdmap e50: 8 total, 7 up, 8 in 2023-11-06T01:30:20.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:20.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:20.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:19 smithi062 ceph-mon[103924]: osd.7 [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587] boot 2023-11-06T01:30:20.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:19 smithi062 ceph-mon[103924]: osdmap e51: 8 total, 8 up, 8 in 2023-11-06T01:30:20.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:19 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-06T01:30:21.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:30:21.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:30:21.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: pgmap v157: 1 pgs: 1 unknown; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:21.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi062 2023-11-06T01:30:21.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:21.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:21.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:21.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi062 to 5722M 2023-11-06T01:30:21.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:21.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:21.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:21.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:21.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:21.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:21 smithi163 ceph-mon[104416]: osdmap e52: 8 total, 8 up, 8 in 2023-11-06T01:30:21.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.306+0000 7f0813523700 1 -- 172.21.15.163:0/4031943055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080c100f20 msgr2=0x7f080c101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:21.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.306+0000 7f0813523700 1 --2- 172.21.15.163:0/4031943055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080c100f20 0x7f080c101380 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0808009990 tx=0x7f0808009ca0 comp rx=0 tx=0).stop 2023-11-06T01:30:21.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.306+0000 7f0813523700 1 -- 172.21.15.163:0/4031943055 shutdown_connections 2023-11-06T01:30:21.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.306+0000 7f0813523700 1 --2- 172.21.15.163:0/4031943055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f080c1018c0 0x7f080c108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:21.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.306+0000 7f0813523700 1 --2- 172.21.15.163:0/4031943055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c0ffd20 0x7f080c100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:21.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.306+0000 7f0813523700 1 --2- 172.21.15.163:0/4031943055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080c100f20 0x7f080c101380 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:21.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.306+0000 7f0813523700 1 -- 172.21.15.163:0/4031943055 >> 172.21.15.163:0/4031943055 conn(0x7f080c0fb2a0 msgr2=0x7f080c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:30:21.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.306+0000 7f0813523700 1 -- 172.21.15.163:0/4031943055 shutdown_connections 2023-11-06T01:30:21.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.306+0000 7f0813523700 1 -- 172.21.15.163:0/4031943055 wait complete. 2023-11-06T01:30:21.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f0813523700 1 Processor -- start 2023-11-06T01:30:21.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f0813523700 1 -- start start 2023-11-06T01:30:21.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f0813523700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080c0ffd20 0x7f080c196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:30:21.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f0813523700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c100f20 0x7f080c197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:30:21.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f0813523700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f080c1018c0 0x7f080c19e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:30:21.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f0813523700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c19e650 con 0x7f080c100f20 2023-11-06T01:30:21.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f0813523700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c19e790 con 0x7f080c1018c0 2023-11-06T01:30:21.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f0813523700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c19e920 con 0x7f080c0ffd20 2023-11-06T01:30:21.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f08112bf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080c0ffd20 0x7f080c196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:30:21.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f08112bf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080c0ffd20 0x7f080c196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46884/0 (socket says 172.21.15.163:46884) 2023-11-06T01:30:21.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f08112bf700 1 -- 172.21.15.163:0/185029778 learned_addr learned my addr 172.21.15.163:0/185029778 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-11-06T01:30:21.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f0810abe700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c100f20 0x7f080c197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:30:21.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f0811ac0700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f080c1018c0 0x7f080c19e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:30:21.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f08112bf700 1 -- 172.21.15.163:0/185029778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f080c1018c0 msgr2=0x7f080c19e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:21.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f08112bf700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f080c1018c0 0x7f080c19e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:21.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f08112bf700 1 -- 172.21.15.163:0/185029778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c100f20 msgr2=0x7f080c197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:21.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f08112bf700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c100f20 0x7f080c197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:21.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f08112bf700 1 -- 172.21.15.163:0/185029778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08080095f0 con 0x7f080c0ffd20 2023-11-06T01:30:21.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.307+0000 7f08112bf700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080c0ffd20 0x7f080c196ac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00d9b0 tx=0x7f07fc00dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:30:21.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.308+0000 7f08027fc700 1 -- 172.21.15.163:0/185029778 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00be80 con 0x7f080c0ffd20 2023-11-06T01:30:21.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.308+0000 7f0813523700 1 -- 172.21.15.163:0/185029778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080c19ec00 con 0x7f080c0ffd20 2023-11-06T01:30:21.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.308+0000 7f08027fc700 1 -- 172.21.15.163:0/185029778 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07fc0044e0 con 0x7f080c0ffd20 2023-11-06T01:30:21.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.308+0000 7f08027fc700 1 -- 172.21.15.163:0/185029778 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc003b40 con 0x7f080c0ffd20 2023-11-06T01:30:21.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.308+0000 7f0813523700 1 -- 172.21.15.163:0/185029778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080c19f100 con 0x7f080c0ffd20 2023-11-06T01:30:21.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.308+0000 7f0813523700 1 -- 172.21.15.163:0/185029778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080c107e60 con 0x7f080c0ffd20 2023-11-06T01:30:21.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.310+0000 7f08027fc700 1 -- 172.21.15.163:0/185029778 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f07fc009940 con 0x7f080c0ffd20 2023-11-06T01:30:21.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.310+0000 7f08027fc700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f07f80a0520 0x7f07f80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:30:21.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.310+0000 7f08027fc700 1 -- 172.21.15.163:0/185029778 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f07fc0caf70 con 0x7f080c0ffd20 2023-11-06T01:30:21.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.311+0000 7f0810abe700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f07f80a0520 0x7f07f80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:30:21.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.311+0000 7f0810abe700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f07f80a0520 0x7f07f80a29e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0808006010 tx=0x7f080800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:30:21.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.314+0000 7f08027fc700 1 -- 172.21.15.163:0/185029778 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f07fc095650 con 0x7f080c0ffd20 2023-11-06T01:30:21.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:30:21.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:30:21.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: pgmap v157: 1 pgs: 1 unknown; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:21.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi062 2023-11-06T01:30:21.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:21.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:21.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:21.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi062 to 5722M 2023-11-06T01:30:21.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:21.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:21.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:21.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:21.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:21.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:21 smithi062 ceph-mon[103924]: osdmap e52: 8 total, 8 up, 8 in 2023-11-06T01:30:21.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:21.512+0000 7f0813523700 1 -- 172.21.15.163:0/185029778 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f080c0611d0 con 0x7f07f80a0520 2023-11-06T01:30:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:30:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:30:21.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: pgmap v157: 1 pgs: 1 unknown; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-11-06T01:30:21.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi062 2023-11-06T01:30:21.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:21.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:21.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:21.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi062 to 5722M 2023-11-06T01:30:21.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:21.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:21.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:21.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:21.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:21.598 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:21.598 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:21 smithi053 ceph-mon[99472]: osdmap e52: 8 total, 8 up, 8 in 2023-11-06T01:30:22.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:30:22.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:30:22.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:22.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:22 smithi062 ceph-mon[103924]: osdmap e53: 8 total, 8 up, 8 in 2023-11-06T01:30:22.457 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:30:22.458 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:30:22.458 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:22.458 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:22 smithi163 ceph-mon[104416]: osdmap e53: 8 total, 8 up, 8 in 2023-11-06T01:30:22.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:30:22.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:30:22.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:22.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:22 smithi053 ceph-mon[99472]: osdmap e53: 8 total, 8 up, 8 in 2023-11-06T01:30:23.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:23 smithi062 ceph-mon[103924]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:23.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:23 smithi062 ceph-mon[103924]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:30:23.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:23 smithi062 ceph-mon[103924]: osdmap e54: 8 total, 8 up, 8 in 2023-11-06T01:30:23.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:23 smithi163 ceph-mon[104416]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:23.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:23 smithi163 ceph-mon[104416]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:30:23.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:23 smithi163 ceph-mon[104416]: osdmap e54: 8 total, 8 up, 8 in 2023-11-06T01:30:23.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:23 smithi053 ceph-mon[99472]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:23.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:23 smithi053 ceph-mon[99472]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:30:23.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:23 smithi053 ceph-mon[99472]: osdmap e54: 8 total, 8 up, 8 in 2023-11-06T01:30:24.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:24 smithi062 ceph-mon[103924]: from='client.? 172.21.15.163:0/2941407696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d1dc49-9a61-48da-9250-70a70c4a7fd8"}]: dispatch 2023-11-06T01:30:24.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:24 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d1dc49-9a61-48da-9250-70a70c4a7fd8"}]: dispatch 2023-11-06T01:30:24.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:24 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d1dc49-9a61-48da-9250-70a70c4a7fd8"}]': finished 2023-11-06T01:30:24.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:24 smithi062 ceph-mon[103924]: osdmap e55: 9 total, 8 up, 9 in 2023-11-06T01:30:24.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:24.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:24 smithi163 ceph-mon[104416]: from='client.? 172.21.15.163:0/2941407696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d1dc49-9a61-48da-9250-70a70c4a7fd8"}]: dispatch 2023-11-06T01:30:24.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:24 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d1dc49-9a61-48da-9250-70a70c4a7fd8"}]: dispatch 2023-11-06T01:30:24.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:24 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d1dc49-9a61-48da-9250-70a70c4a7fd8"}]': finished 2023-11-06T01:30:24.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:24 smithi163 ceph-mon[104416]: osdmap e55: 9 total, 8 up, 9 in 2023-11-06T01:30:24.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:24.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:24 smithi053 ceph-mon[99472]: from='client.? 172.21.15.163:0/2941407696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d1dc49-9a61-48da-9250-70a70c4a7fd8"}]: dispatch 2023-11-06T01:30:24.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:24 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d1dc49-9a61-48da-9250-70a70c4a7fd8"}]: dispatch 2023-11-06T01:30:24.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:24 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d1dc49-9a61-48da-9250-70a70c4a7fd8"}]': finished 2023-11-06T01:30:24.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:24 smithi053 ceph-mon[99472]: osdmap e55: 9 total, 8 up, 9 in 2023-11-06T01:30:24.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:25.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:25 smithi062 ceph-mon[103924]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:25.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:25 smithi062 ceph-mon[103924]: from='client.? 172.21.15.163:0/3312688792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:30:25.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:25 smithi163 ceph-mon[104416]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:25.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:25 smithi163 ceph-mon[104416]: from='client.? 172.21.15.163:0/3312688792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:30:25.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:25 smithi053 ceph-mon[99472]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:25.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:25 smithi053 ceph-mon[99472]: from='client.? 172.21.15.163:0/3312688792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:30:27.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:27 smithi062 ceph-mon[103924]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:27.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:27 smithi163 ceph-mon[104416]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:27.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:27 smithi053 ceph-mon[99472]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:29.426 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:29 smithi163 ceph-mon[104416]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:29.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:29 smithi053 ceph-mon[99472]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:29.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:29 smithi062 ceph-mon[103924]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:30.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-06T01:30:30.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:30 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:30.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-06T01:30:30.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:30 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:30.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-06T01:30:30.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:30 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:31.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:31 smithi163 ceph-mon[104416]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:31.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:31 smithi163 ceph-mon[104416]: Deploying daemon osd.8 on smithi163 2023-11-06T01:30:31.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:31 smithi053 ceph-mon[99472]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:31.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:31 smithi053 ceph-mon[99472]: Deploying daemon osd.8 on smithi163 2023-11-06T01:30:31.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:31 smithi062 ceph-mon[103924]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:31.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:31 smithi062 ceph-mon[103924]: Deploying daemon osd.8 on smithi163 2023-11-06T01:30:33.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:33 smithi163 ceph-mon[104416]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:33.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:33 smithi053 ceph-mon[99472]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:33.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:33 smithi062 ceph-mon[103924]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:35.458 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:35 smithi163 ceph-mon[104416]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:35.458 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:35.458 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:35.459 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:35.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:35 smithi053 ceph-mon[99472]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:35.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:35.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:35.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:35.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:35 smithi062 ceph-mon[103924]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:35.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:35.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:35.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:36.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.878+0000 7f08027fc700 1 -- 172.21.15.163:0/185029778 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f080c0611d0 con 0x7f07f80a0520 2023-11-06T01:30:36.880 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 8 on host 'smithi163' 2023-11-06T01:30:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.880+0000 7f0813523700 1 -- 172.21.15.163:0/185029778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f07f80a0520 msgr2=0x7f07f80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.880+0000 7f0813523700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f07f80a0520 0x7f07f80a29e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0808006010 tx=0x7f080800b260 comp rx=0 tx=0).stop 2023-11-06T01:30:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.880+0000 7f0813523700 1 -- 172.21.15.163:0/185029778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080c0ffd20 msgr2=0x7f080c196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.880+0000 7f0813523700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080c0ffd20 0x7f080c196ac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00d9b0 tx=0x7f07fc00dd70 comp rx=0 tx=0).stop 2023-11-06T01:30:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.880+0000 7f0813523700 1 -- 172.21.15.163:0/185029778 shutdown_connections 2023-11-06T01:30:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.880+0000 7f0813523700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f07f80a0520 0x7f07f80a29e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.880+0000 7f0813523700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f080c1018c0 0x7f080c19e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.880+0000 7f0813523700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080c0ffd20 0x7f080c196ac0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.880+0000 7f0813523700 1 --2- 172.21.15.163:0/185029778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c100f20 0x7f080c197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.880+0000 7f0813523700 1 -- 172.21.15.163:0/185029778 >> 172.21.15.163:0/185029778 conn(0x7f080c0fb2a0 msgr2=0x7f080c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:30:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.881+0000 7f0813523700 1 -- 172.21.15.163:0/185029778 shutdown_connections 2023-11-06T01:30:36.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:30:36.881+0000 7f0813523700 1 -- 172.21.15.163:0/185029778 wait complete. 2023-11-06T01:30:37.222 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:36 smithi163 ceph-mon[104416]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:37.222 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.223 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.223 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:37.223 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:37.223 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.223 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.224 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.224 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:37.341 DEBUG:teuthology.orchestra.run.smithi163:osd.8> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.8.service 2023-11-06T01:30:37.343 INFO:tasks.cephadm:Deploying osd.9 on smithi053 with /dev/vg_nvme/lv_1... 2023-11-06T01:30:37.343 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-06T01:30:37.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:36 smithi053 ceph-mon[99472]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:37.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:37.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:37.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:37.371 INFO:journalctl@ceph.osd.8.smithi163.stdout:-- Logs begin at Mon 2023-11-06 01:10:53 UTC. -- 2023-11-06T01:30:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:36 smithi062 ceph-mon[103924]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:37.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:37.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:37.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:37.920 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:30:38.490 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:30:38 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:30:38.298+0000 7f2f63fde640 -1 osd.8 0 log_to_monitors true 2023-11-06T01:30:39.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:38 smithi163 ceph-mon[104416]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:39.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:38 smithi163 ceph-mon[104416]: from='osd.8 [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-06T01:30:39.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:38 smithi163 ceph-mon[104416]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-06T01:30:39.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:38 smithi053 ceph-mon[99472]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:39.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:38 smithi053 ceph-mon[99472]: from='osd.8 [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-06T01:30:39.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:38 smithi053 ceph-mon[99472]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-06T01:30:39.370 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-06T01:30:39.370 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-06T01:30:39.370 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2023-11-06T01:30:39.371 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2023-11-06T01:30:39.371 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0081232 s, 1.3 GB/s 2023-11-06T01:30:39.371 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2023-11-06T01:30:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:38 smithi062 ceph-mon[103924]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:38 smithi062 ceph-mon[103924]: from='osd.8 [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-06T01:30:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:38 smithi062 ceph-mon[103924]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-06T01:30:39.769 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2023-11-06T01:30:40.239 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:30:39.972+0000 7f2f59655700 -1 osd.8 0 waiting for initial osdmap 2023-11-06T01:30:40.240 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:30:39.983+0000 7f2f54c49700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:30:40.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-06T01:30:40.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: osdmap e56: 9 total, 8 up, 9 in 2023-11-06T01:30:40.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:40.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='osd.8 [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:30:40.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:30:40.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:40.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:40.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:40.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:40.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:40.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:40.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:40.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:40.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:40.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-06T01:30:40.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: osdmap e56: 9 total, 8 up, 9 in 2023-11-06T01:30:40.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:40.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='osd.8 [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:30:40.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:30:40.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:40.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:40.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:40.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:40.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:40.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:40.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:40.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:40.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:40.348 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:30:40.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-06T01:30:40.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: osdmap e56: 9 total, 8 up, 9 in 2023-11-06T01:30:40.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:40.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='osd.8 [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:30:40.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:30:40.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:40.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:40.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:40.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:40.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:30:40.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:40.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:40.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:30:40.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:41.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:40 smithi053 ceph-mon[99472]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:41.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:40 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi163 2023-11-06T01:30:41.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:40 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi163 to 5722M 2023-11-06T01:30:41.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:40 smithi053 ceph-mon[99472]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:30:41.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:40 smithi053 ceph-mon[99472]: osdmap e57: 9 total, 8 up, 9 in 2023-11-06T01:30:41.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:41.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:41.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:40 smithi053 ceph-mon[99472]: osd.8 [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317] boot 2023-11-06T01:30:41.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:40 smithi053 ceph-mon[99472]: osdmap e58: 9 total, 9 up, 9 in 2023-11-06T01:30:41.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:41.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:40 smithi163 ceph-mon[104416]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:41.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:40 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi163 2023-11-06T01:30:41.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:40 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi163 to 5722M 2023-11-06T01:30:41.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:40 smithi163 ceph-mon[104416]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:30:41.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:40 smithi163 ceph-mon[104416]: osdmap e57: 9 total, 8 up, 9 in 2023-11-06T01:30:41.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:40 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:41.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:40 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:41.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:40 smithi163 ceph-mon[104416]: osd.8 [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317] boot 2023-11-06T01:30:41.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:40 smithi163 ceph-mon[104416]: osdmap e58: 9 total, 9 up, 9 in 2023-11-06T01:30:41.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:40 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:40 smithi062 ceph-mon[103924]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-11-06T01:30:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:40 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi163 2023-11-06T01:30:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:40 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi163 to 5722M 2023-11-06T01:30:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:40 smithi062 ceph-mon[103924]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:30:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:40 smithi062 ceph-mon[103924]: osdmap e57: 9 total, 8 up, 9 in 2023-11-06T01:30:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:40 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:40 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:40 smithi062 ceph-mon[103924]: osd.8 [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317] boot 2023-11-06T01:30:41.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:40 smithi062 ceph-mon[103924]: osdmap e58: 9 total, 9 up, 9 in 2023-11-06T01:30:41.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:40 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-06T01:30:41.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.559+0000 7f5335ba0700 1 -- 172.21.15.53:0/3220287878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53300ffb70 msgr2=0x7f53300fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:41.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.559+0000 7f5335ba0700 1 --2- 172.21.15.53:0/3220287878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53300ffb70 0x7f53300fff90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f53240099c0 tx=0x7f5324009cd0 comp rx=0 tx=0).stop 2023-11-06T01:30:41.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.559+0000 7f5335ba0700 1 -- 172.21.15.53:0/3220287878 shutdown_connections 2023-11-06T01:30:41.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.559+0000 7f5335ba0700 1 --2- 172.21.15.53:0/3220287878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5330101710 0x7f5330107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:41.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.559+0000 7f5335ba0700 1 --2- 172.21.15.53:0/3220287878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53300ffb70 0x7f53300fff90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:41.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.559+0000 7f5335ba0700 1 --2- 172.21.15.53:0/3220287878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5330100d70 0x7f53301011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:41.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.559+0000 7f5335ba0700 1 -- 172.21.15.53:0/3220287878 >> 172.21.15.53:0/3220287878 conn(0x7f53300fb130 msgr2=0x7f53300fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:30:41.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.559+0000 7f5335ba0700 1 -- 172.21.15.53:0/3220287878 shutdown_connections 2023-11-06T01:30:41.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.559+0000 7f5335ba0700 1 -- 172.21.15.53:0/3220287878 wait complete. 2023-11-06T01:30:41.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f5335ba0700 1 Processor -- start 2023-11-06T01:30:41.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f5335ba0700 1 -- start start 2023-11-06T01:30:41.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f5335ba0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53300ffb70 0x7f5330196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:30:41.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f5335ba0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5330100d70 0x7f5330196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:30:41.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f5335ba0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5330101710 0x7f533019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:30:41.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f5335ba0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533019e4e0 con 0x7f5330100d70 2023-11-06T01:30:41.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f5335ba0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533019e620 con 0x7f5330101710 2023-11-06T01:30:41.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f5335ba0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533019e7b0 con 0x7f53300ffb70 2023-11-06T01:30:41.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f532ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5330100d70 0x7f5330196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:30:41.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f532ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5330100d70 0x7f5330196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37392/0 (socket says 172.21.15.53:37392) 2023-11-06T01:30:41.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f532ffff700 1 -- 172.21.15.53:0/1513086857 learned_addr learned my addr 172.21.15.53:0/1513086857 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:30:41.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f5334b9e700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53300ffb70 0x7f5330196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:30:41.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f533539f700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5330101710 0x7f533019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:30:41.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f532ffff700 1 -- 172.21.15.53:0/1513086857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53300ffb70 msgr2=0x7f5330196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:41.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f532ffff700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53300ffb70 0x7f5330196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:41.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f532ffff700 1 -- 172.21.15.53:0/1513086857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5330101710 msgr2=0x7f533019df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:41.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f532ffff700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5330101710 0x7f533019df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:41.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.560+0000 7f532ffff700 1 -- 172.21.15.53:0/1513086857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53240095f0 con 0x7f5330100d70 2023-11-06T01:30:41.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.561+0000 7f532ffff700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5330100d70 0x7f5330196e90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f531c00c950 tx=0x7f531c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:30:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.561+0000 7f532dffb700 1 -- 172.21.15.53:0/1513086857 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c00cea0 con 0x7f5330100d70 2023-11-06T01:30:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.561+0000 7f5335ba0700 1 -- 172.21.15.53:0/1513086857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533019ea90 con 0x7f5330100d70 2023-11-06T01:30:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.561+0000 7f532dffb700 1 -- 172.21.15.53:0/1513086857 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531c004830 con 0x7f5330100d70 2023-11-06T01:30:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.561+0000 7f532dffb700 1 -- 172.21.15.53:0/1513086857 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0054e0 con 0x7f5330100d70 2023-11-06T01:30:41.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.561+0000 7f5335ba0700 1 -- 172.21.15.53:0/1513086857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533019ef90 con 0x7f5330100d70 2023-11-06T01:30:41.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.562+0000 7f5335ba0700 1 -- 172.21.15.53:0/1513086857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533010b370 con 0x7f5330100d70 2023-11-06T01:30:41.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.563+0000 7f532dffb700 1 -- 172.21.15.53:0/1513086857 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f531c0049a0 con 0x7f5330100d70 2023-11-06T01:30:41.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.563+0000 7f532dffb700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f53200a0520 0x7f53200a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:30:41.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.564+0000 7f5334b9e700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f53200a0520 0x7f53200a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:30:41.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.564+0000 7f532dffb700 1 -- 172.21.15.53:0/1513086857 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f531c0cac30 con 0x7f5330100d70 2023-11-06T01:30:41.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.564+0000 7f5334b9e700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f53200a0520 0x7f53200a29e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5324009f50 tx=0x7f532400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:30:41.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.568+0000 7f532dffb700 1 -- 172.21.15.53:0/1513086857 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f531c095160 con 0x7f5330100d70 2023-11-06T01:30:41.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:41.776+0000 7f5335ba0700 1 -- 172.21.15.53:0/1513086857 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f53300fc0c0 con 0x7f53200a0520 2023-11-06T01:30:42.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:41 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:30:42.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:41 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:30:42.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:30:42.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:30:42.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:42.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:41 smithi163 ceph-mon[104416]: osdmap e59: 9 total, 9 up, 9 in 2023-11-06T01:30:42.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:41 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:30:42.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:41 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:30:42.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:30:42.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:30:42.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:42.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:41 smithi053 ceph-mon[99472]: osdmap e59: 9 total, 9 up, 9 in 2023-11-06T01:30:42.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:41 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:30:42.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:41 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:30:42.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:30:42.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:30:42.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:42.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:41 smithi062 ceph-mon[103924]: osdmap e59: 9 total, 9 up, 9 in 2023-11-06T01:30:43.141 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:42 smithi053 ceph-mon[99472]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:30:43.142 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:42 smithi053 ceph-mon[99472]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:30:43.142 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:42 smithi053 ceph-mon[99472]: osdmap e60: 9 total, 9 up, 9 in 2023-11-06T01:30:43.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:42 smithi163 ceph-mon[104416]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:30:43.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:42 smithi163 ceph-mon[104416]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:30:43.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:42 smithi163 ceph-mon[104416]: osdmap e60: 9 total, 9 up, 9 in 2023-11-06T01:30:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:42 smithi062 ceph-mon[103924]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:30:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:42 smithi062 ceph-mon[103924]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:30:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:42 smithi062 ceph-mon[103924]: osdmap e60: 9 total, 9 up, 9 in 2023-11-06T01:30:45.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:44 smithi163 ceph-mon[104416]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:30:45.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:44 smithi163 ceph-mon[104416]: osdmap e61: 9 total, 9 up, 9 in 2023-11-06T01:30:45.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:44 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1129437567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23af8247-4e33-4d08-8467-452c339e1eee"}]: dispatch 2023-11-06T01:30:45.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:44 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1129437567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23af8247-4e33-4d08-8467-452c339e1eee"}]': finished 2023-11-06T01:30:45.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:44 smithi163 ceph-mon[104416]: osdmap e62: 10 total, 9 up, 10 in 2023-11-06T01:30:45.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:30:45.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:44 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3041155342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:30:45.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:44 smithi053 ceph-mon[99472]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:30:45.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:44 smithi053 ceph-mon[99472]: osdmap e61: 9 total, 9 up, 9 in 2023-11-06T01:30:45.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:44 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1129437567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23af8247-4e33-4d08-8467-452c339e1eee"}]: dispatch 2023-11-06T01:30:45.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:44 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1129437567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23af8247-4e33-4d08-8467-452c339e1eee"}]': finished 2023-11-06T01:30:45.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:44 smithi053 ceph-mon[99472]: osdmap e62: 10 total, 9 up, 10 in 2023-11-06T01:30:45.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:30:45.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:44 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3041155342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:30:45.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:44 smithi062 ceph-mon[103924]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:30:45.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:44 smithi062 ceph-mon[103924]: osdmap e61: 9 total, 9 up, 9 in 2023-11-06T01:30:45.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:44 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1129437567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23af8247-4e33-4d08-8467-452c339e1eee"}]: dispatch 2023-11-06T01:30:45.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:44 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1129437567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23af8247-4e33-4d08-8467-452c339e1eee"}]': finished 2023-11-06T01:30:45.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:44 smithi062 ceph-mon[103924]: osdmap e62: 10 total, 9 up, 10 in 2023-11-06T01:30:45.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:30:45.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:44 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3041155342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:30:47.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:46 smithi163 ceph-mon[104416]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-06T01:30:47.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:46 smithi053 ceph-mon[99472]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-06T01:30:47.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:46 smithi062 ceph-mon[103924]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-06T01:30:49.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:48 smithi163 ceph-mon[104416]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-11-06T01:30:49.252 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:48 smithi053 ceph-mon[99472]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-11-06T01:30:49.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:48 smithi062 ceph-mon[103924]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-11-06T01:30:51.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:50 smithi053 ceph-mon[99472]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-11-06T01:30:51.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-06T01:30:51.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:50 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:51.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:50 smithi062 ceph-mon[103924]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-11-06T01:30:51.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:50 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-06T01:30:51.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:50 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:51.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:50 smithi163 ceph-mon[104416]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-11-06T01:30:51.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:50 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-06T01:30:51.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:50 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:30:52.253 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:51 smithi053 ceph-mon[99472]: Deploying daemon osd.9 on smithi053 2023-11-06T01:30:52.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:51 smithi062 ceph-mon[103924]: Deploying daemon osd.9 on smithi053 2023-11-06T01:30:52.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:51 smithi163 ceph-mon[104416]: Deploying daemon osd.9 on smithi053 2023-11-06T01:30:53.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:52 smithi053 ceph-mon[99472]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-06T01:30:53.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:52 smithi062 ceph-mon[103924]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-06T01:30:53.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:52 smithi163 ceph-mon[104416]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-06T01:30:55.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:54 smithi053 ceph-mon[99472]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-11-06T01:30:55.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:54 smithi062 ceph-mon[103924]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-11-06T01:30:55.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:54 smithi163 ceph-mon[104416]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-11-06T01:30:56.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:56 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:56.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:56 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:56.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:56 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:56.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:56 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:56.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:56 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:56.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:56 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:56.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:56 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:56.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:56 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:56.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:56 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:30:57.335 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:57 smithi053 ceph-mon[99472]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-11-06T01:30:57.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:57 smithi062 ceph-mon[103924]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-11-06T01:30:57.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:57 smithi163 ceph-mon[104416]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-11-06T01:30:57.731 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 9 on host 'smithi053' 2023-11-06T01:30:57.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.730+0000 7f532dffb700 1 -- 172.21.15.53:0/1513086857 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f53300fc0c0 con 0x7f53200a0520 2023-11-06T01:30:57.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.732+0000 7f5335ba0700 1 -- 172.21.15.53:0/1513086857 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f53200a0520 msgr2=0x7f53200a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:57.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.732+0000 7f5335ba0700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f53200a0520 0x7f53200a29e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5324009f50 tx=0x7f532400b540 comp rx=0 tx=0).stop 2023-11-06T01:30:57.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.733+0000 7f5335ba0700 1 -- 172.21.15.53:0/1513086857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5330100d70 msgr2=0x7f5330196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:30:57.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.733+0000 7f5335ba0700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5330100d70 0x7f5330196e90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f531c00c950 tx=0x7f531c00cc60 comp rx=0 tx=0).stop 2023-11-06T01:30:57.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.733+0000 7f5335ba0700 1 -- 172.21.15.53:0/1513086857 shutdown_connections 2023-11-06T01:30:57.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.733+0000 7f5335ba0700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f53200a0520 0x7f53200a29e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:57.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.733+0000 7f5335ba0700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5330101710 0x7f533019df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:57.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.733+0000 7f5335ba0700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53300ffb70 0x7f5330196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:57.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.733+0000 7f5335ba0700 1 --2- 172.21.15.53:0/1513086857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5330100d70 0x7f5330196e90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:30:57.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.733+0000 7f5335ba0700 1 -- 172.21.15.53:0/1513086857 >> 172.21.15.53:0/1513086857 conn(0x7f53300fb130 msgr2=0x7f53300fb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:30:57.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.733+0000 7f5335ba0700 1 -- 172.21.15.53:0/1513086857 shutdown_connections 2023-11-06T01:30:57.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:30:57.733+0000 7f5335ba0700 1 -- 172.21.15.53:0/1513086857 wait complete. 2023-11-06T01:30:58.128 DEBUG:teuthology.orchestra.run.smithi053:osd.9> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.9.service 2023-11-06T01:30:58.131 INFO:tasks.cephadm:Deploying osd.10 on smithi062 with /dev/vg_nvme/lv_1... 2023-11-06T01:30:58.131 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-06T01:30:58.161 INFO:journalctl@ceph.osd.9.smithi053.stdout:-- Logs begin at Mon 2023-11-06 01:09:30 UTC. -- 2023-11-06T01:30:58.703 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:30:58.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:58 smithi163 ceph-mon[104416]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:30:58.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:58 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:58.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:58 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:58.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:58 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:58.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:58 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:59.005 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:58 smithi062 ceph-mon[103924]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:30:59.005 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:58 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:59.006 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:58 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:59.006 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:58 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:59.006 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:58 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:59.016 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:58 smithi053 ceph-mon[99472]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:30:59.017 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:59.017 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:59.017 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:59.017 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:30:59.344 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:30:59 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:30:59.014+0000 7f98959a4640 -1 osd.9 0 log_to_monitors true 2023-11-06T01:30:59.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:30:59 smithi053 ceph-mon[99472]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-06T01:30:59.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:30:59 smithi163 ceph-mon[104416]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-06T01:31:00.022 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:30:59 smithi062 ceph-mon[103924]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-06T01:31:00.722 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-06T01:31:00.722 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-06T01:31:00.722 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2023-11-06T01:31:00.723 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2023-11-06T01:31:00.723 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109555 s, 957 MB/s 2023-11-06T01:31:00.723 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2023-11-06T01:31:00.844 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:31:00 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:31:00.709+0000 7f988b01b700 -1 osd.9 0 waiting for initial osdmap 2023-11-06T01:31:00.844 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:31:00 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:31:00.723+0000 7f9884e0c700 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:31:00.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:00 smithi053 ceph-mon[99472]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:31:00.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:00 smithi053 ceph-mon[99472]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-06T01:31:00.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:00 smithi053 ceph-mon[99472]: osdmap e63: 10 total, 9 up, 10 in 2023-11-06T01:31:00.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:00.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:00 smithi053 ceph-mon[99472]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:31:00.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:00 smithi163 ceph-mon[104416]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:31:00.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:00 smithi163 ceph-mon[104416]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-06T01:31:00.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:00 smithi163 ceph-mon[104416]: osdmap e63: 10 total, 9 up, 10 in 2023-11-06T01:31:00.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:00 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:00.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:00 smithi163 ceph-mon[104416]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:31:01.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:00 smithi062 ceph-mon[103924]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:31:01.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:00 smithi062 ceph-mon[103924]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-06T01:31:01.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:00 smithi062 ceph-mon[103924]: osdmap e63: 10 total, 9 up, 10 in 2023-11-06T01:31:01.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:00 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:01.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:00 smithi062 ceph-mon[103924]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-06T01:31:01.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi062:vg_nvme/lv_1 2023-11-06T01:31:01.728 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:31:01.971 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:31:01.972 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: osdmap e64: 10 total, 9 up, 10 in 2023-11-06T01:31:01.972 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:01.972 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:01.973 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi053 2023-11-06T01:31:01.973 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:01.973 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:01.974 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:01.974 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:01.974 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:01.974 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:01.975 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi053 to 4293M 2023-11-06T01:31:01.975 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:01.975 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:01.976 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:01.976 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:01.976 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:01.976 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:01.977 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:01.977 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:01.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:31:01.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: osdmap e64: 10 total, 9 up, 10 in 2023-11-06T01:31:01.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:01.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:01.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi053 2023-11-06T01:31:01.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:01.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:01.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:01.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:01.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:01.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:01.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi053 to 4293M 2023-11-06T01:31:01.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:01.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:01.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:01.993 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:01.993 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:01.993 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:01.993 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:01.993 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:02.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-06T01:31:02.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: osdmap e64: 10 total, 9 up, 10 in 2023-11-06T01:31:02.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:02.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:02.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi053 2023-11-06T01:31:02.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:02.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:02.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:02.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:02.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:02.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:02.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi053 to 4293M 2023-11-06T01:31:02.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:02.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:02.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:02.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:02.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:02.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:02.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:02.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:02.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:02 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:31:02.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:02 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:31:02.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:02 smithi062 ceph-mon[103924]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:31:02.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:02 smithi062 ceph-mon[103924]: osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850] boot 2023-11-06T01:31:02.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:02 smithi062 ceph-mon[103924]: osdmap e65: 10 total, 10 up, 10 in 2023-11-06T01:31:02.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:02 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:02.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:02 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:31:02.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:02 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:31:02.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:02 smithi163 ceph-mon[104416]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:31:02.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:02 smithi163 ceph-mon[104416]: osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850] boot 2023-11-06T01:31:02.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:02 smithi163 ceph-mon[104416]: osdmap e65: 10 total, 10 up, 10 in 2023-11-06T01:31:02.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:02 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:03.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:02 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:31:03.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:02 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:31:03.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:02 smithi053 ceph-mon[99472]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-11-06T01:31:03.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:02 smithi053 ceph-mon[99472]: osd.9 [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850] boot 2023-11-06T01:31:03.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:02 smithi053 ceph-mon[99472]: osdmap e65: 10 total, 10 up, 10 in 2023-11-06T01:31:03.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:02 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-06T01:31:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.096+0000 7f0039489700 1 -- 172.21.15.62:0/611271801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00341006a0 msgr2=0x7f0034100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.096+0000 7f0039489700 1 --2- 172.21.15.62:0/611271801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00341006a0 0x7f0034100b00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f00280099c0 tx=0x7f0028009cd0 comp rx=0 tx=0).stop 2023-11-06T01:31:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.096+0000 7f0039489700 1 -- 172.21.15.62:0/611271801 shutdown_connections 2023-11-06T01:31:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.096+0000 7f0039489700 1 --2- 172.21.15.62:0/611271801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0034101040 0x7f00341078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.096+0000 7f0039489700 1 --2- 172.21.15.62:0/611271801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00340ff4a0 0x7f00340ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:03.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.096+0000 7f0039489700 1 --2- 172.21.15.62:0/611271801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00341006a0 0x7f0034100b00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:03.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.096+0000 7f0039489700 1 -- 172.21.15.62:0/611271801 >> 172.21.15.62:0/611271801 conn(0x7f00340faa40 msgr2=0x7f00340fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:03.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.096+0000 7f0039489700 1 -- 172.21.15.62:0/611271801 shutdown_connections 2023-11-06T01:31:03.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.096+0000 7f0039489700 1 -- 172.21.15.62:0/611271801 wait complete. 2023-11-06T01:31:03.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.096+0000 7f0039489700 1 Processor -- start 2023-11-06T01:31:03.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0039489700 1 -- start start 2023-11-06T01:31:03.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0039489700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340ff4a0 0x7f00340fce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:03.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0039489700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00341006a0 0x7f003419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:03.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0039489700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0034101040 0x7f003419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:03.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0039489700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003419e550 con 0x7f00341006a0 2023-11-06T01:31:03.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0039489700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003419e6c0 con 0x7f00340ff4a0 2023-11-06T01:31:03.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0039489700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003419e880 con 0x7f0034101040 2023-11-06T01:31:03.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0032ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340ff4a0 0x7f00340fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:03.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0032ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340ff4a0 0x7f00340fce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42692/0 (socket says 172.21.15.62:42692) 2023-11-06T01:31:03.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0032ffd700 1 -- 172.21.15.62:0/2980144486 learned_addr learned my addr 172.21.15.62:0/2980144486 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-11-06T01:31:03.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f00327fc700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00341006a0 0x7f003419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:03.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0032ffd700 1 -- 172.21.15.62:0/2980144486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0034101040 msgr2=0x7f003419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:03.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0032ffd700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0034101040 0x7f003419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:03.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0032ffd700 1 -- 172.21.15.62:0/2980144486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00341006a0 msgr2=0x7f003419b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:03.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.097+0000 7f0032ffd700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00341006a0 0x7f003419b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:03.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.098+0000 7f0032ffd700 1 -- 172.21.15.62:0/2980144486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00280095f0 con 0x7f00340ff4a0 2023-11-06T01:31:03.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.098+0000 7f0032ffd700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340ff4a0 0x7f00340fce60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f002400c970 tx=0x7f002400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:03.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.098+0000 7f001bfff700 1 -- 172.21.15.62:0/2980144486 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024004220 con 0x7f00340ff4a0 2023-11-06T01:31:03.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.098+0000 7f001bfff700 1 -- 172.21.15.62:0/2980144486 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0024007d60 con 0x7f00340ff4a0 2023-11-06T01:31:03.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.098+0000 7f0039489700 1 -- 172.21.15.62:0/2980144486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003419eb60 con 0x7f00340ff4a0 2023-11-06T01:31:03.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.098+0000 7f001bfff700 1 -- 172.21.15.62:0/2980144486 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024003c90 con 0x7f00340ff4a0 2023-11-06T01:31:03.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.098+0000 7f0039489700 1 -- 172.21.15.62:0/2980144486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003419f080 con 0x7f00340ff4a0 2023-11-06T01:31:03.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.099+0000 7f0039489700 1 -- 172.21.15.62:0/2980144486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0034107020 con 0x7f00340ff4a0 2023-11-06T01:31:03.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.100+0000 7f001bfff700 1 -- 172.21.15.62:0/2980144486 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f0024004380 con 0x7f00340ff4a0 2023-11-06T01:31:03.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.100+0000 7f001bfff700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f001c0a05a0 0x7f001c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:03.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.101+0000 7f001bfff700 1 -- 172.21.15.62:0/2980144486 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f00240cb010 con 0x7f00340ff4a0 2023-11-06T01:31:03.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.101+0000 7f00327fc700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f001c0a05a0 0x7f001c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:03.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.101+0000 7f00327fc700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f001c0a05a0 0x7f001c0a2a60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f002800b580 tx=0x7f0028000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:03.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.105+0000 7f001bfff700 1 -- 172.21.15.62:0/2980144486 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f00240953b0 con 0x7f00340ff4a0 2023-11-06T01:31:03.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:03.315+0000 7f0039489700 1 -- 172.21.15.62:0/2980144486 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f00340611d0 con 0x7f001c0a05a0 2023-11-06T01:31:03.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:03 smithi163 ceph-mon[104416]: osdmap e66: 10 total, 10 up, 10 in 2023-11-06T01:31:03.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:03 smithi163 ceph-mon[104416]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:31:03.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:31:03.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:31:03.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:04.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:03 smithi053 ceph-mon[99472]: osdmap e66: 10 total, 10 up, 10 in 2023-11-06T01:31:04.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:03 smithi053 ceph-mon[99472]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:31:04.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:03 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:31:04.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:03 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:31:04.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:03 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:04.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:03 smithi062 ceph-mon[103924]: osdmap e66: 10 total, 10 up, 10 in 2023-11-06T01:31:04.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:03 smithi062 ceph-mon[103924]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:31:04.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:03 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:31:04.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:03 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:31:04.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:03 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:04.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:04 smithi062 ceph-mon[103924]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:04.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:04 smithi163 ceph-mon[104416]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:05.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:04 smithi053 ceph-mon[99472]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:06.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:06 smithi163 ceph-mon[104416]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:06.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:06 smithi163 ceph-mon[104416]: from='client.? 172.21.15.62:0/997095588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e90a07b-b1b1-4120-a456-6c8106cc1b07"}]: dispatch 2023-11-06T01:31:06.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:06 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e90a07b-b1b1-4120-a456-6c8106cc1b07"}]: dispatch 2023-11-06T01:31:06.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:06 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e90a07b-b1b1-4120-a456-6c8106cc1b07"}]': finished 2023-11-06T01:31:06.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:06 smithi163 ceph-mon[104416]: osdmap e67: 11 total, 10 up, 11 in 2023-11-06T01:31:06.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:06 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:06.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:06 smithi163 ceph-mon[104416]: from='client.? 172.21.15.62:0/2469996753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:31:07.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:06 smithi053 ceph-mon[99472]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:07.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:06 smithi053 ceph-mon[99472]: from='client.? 172.21.15.62:0/997095588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e90a07b-b1b1-4120-a456-6c8106cc1b07"}]: dispatch 2023-11-06T01:31:07.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:06 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e90a07b-b1b1-4120-a456-6c8106cc1b07"}]: dispatch 2023-11-06T01:31:07.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:06 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e90a07b-b1b1-4120-a456-6c8106cc1b07"}]': finished 2023-11-06T01:31:07.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:06 smithi053 ceph-mon[99472]: osdmap e67: 11 total, 10 up, 11 in 2023-11-06T01:31:07.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:06 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:07.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:06 smithi053 ceph-mon[99472]: from='client.? 172.21.15.62:0/2469996753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:31:07.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:06 smithi062 ceph-mon[103924]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:07.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:06 smithi062 ceph-mon[103924]: from='client.? 172.21.15.62:0/997095588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e90a07b-b1b1-4120-a456-6c8106cc1b07"}]: dispatch 2023-11-06T01:31:07.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:06 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e90a07b-b1b1-4120-a456-6c8106cc1b07"}]: dispatch 2023-11-06T01:31:07.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:06 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e90a07b-b1b1-4120-a456-6c8106cc1b07"}]': finished 2023-11-06T01:31:07.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:06 smithi062 ceph-mon[103924]: osdmap e67: 11 total, 10 up, 11 in 2023-11-06T01:31:07.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:06 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:07.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:06 smithi062 ceph-mon[103924]: from='client.? 172.21.15.62:0/2469996753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:31:08.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:08 smithi163 ceph-mon[104416]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:09.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:08 smithi053 ceph-mon[99472]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:09.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:08 smithi062 ceph-mon[103924]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:10.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:10 smithi163 ceph-mon[104416]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:10.996 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:10 smithi062 ceph-mon[103924]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:11.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:10 smithi053 ceph-mon[99472]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:13.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:12 smithi062 ceph-mon[103924]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:13.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-06T01:31:13.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:12 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:13.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:12 smithi163 ceph-mon[104416]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:13.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:12 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-06T01:31:13.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:12 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:13.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:12 smithi053 ceph-mon[99472]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:13.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:12 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-06T01:31:13.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:12 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:14.032 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:13 smithi062 ceph-mon[103924]: Deploying daemon osd.10 on smithi062 2023-11-06T01:31:14.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:13 smithi163 ceph-mon[104416]: Deploying daemon osd.10 on smithi062 2023-11-06T01:31:14.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:13 smithi053 ceph-mon[99472]: Deploying daemon osd.10 on smithi062 2023-11-06T01:31:15.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:14 smithi163 ceph-mon[104416]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:15.328 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:14 smithi062 ceph-mon[103924]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:15.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:14 smithi053 ceph-mon[99472]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:17.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:16 smithi163 ceph-mon[104416]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:17.270 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:16 smithi062 ceph-mon[103924]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:17.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:16 smithi053 ceph-mon[99472]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:18.812 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:18 smithi062 ceph-mon[103924]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:18.812 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:18.813 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:18.813 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:18.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:18 smithi053 ceph-mon[99472]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:18.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:18.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:18.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:18.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:18 smithi163 ceph-mon[104416]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:18.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:18.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:18.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:20.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.684+0000 7f001bfff700 1 -- 172.21.15.62:0/2980144486 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f00340611d0 con 0x7f001c0a05a0 2023-11-06T01:31:20.686 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 10 on host 'smithi062' 2023-11-06T01:31:20.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.686+0000 7f0039489700 1 -- 172.21.15.62:0/2980144486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f001c0a05a0 msgr2=0x7f001c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:20.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.686+0000 7f0039489700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f001c0a05a0 0x7f001c0a2a60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f002800b580 tx=0x7f0028000bc0 comp rx=0 tx=0).stop 2023-11-06T01:31:20.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.686+0000 7f0039489700 1 -- 172.21.15.62:0/2980144486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340ff4a0 msgr2=0x7f00340fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:20.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.686+0000 7f0039489700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340ff4a0 0x7f00340fce60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f002400c970 tx=0x7f002400cc80 comp rx=0 tx=0).stop 2023-11-06T01:31:20.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.686+0000 7f0039489700 1 -- 172.21.15.62:0/2980144486 shutdown_connections 2023-11-06T01:31:20.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.686+0000 7f0039489700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f001c0a05a0 0x7f001c0a2a60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:20.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.686+0000 7f0039489700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340ff4a0 0x7f00340fce60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:20.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.686+0000 7f0039489700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0034101040 0x7f003419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:20.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.686+0000 7f0039489700 1 --2- 172.21.15.62:0/2980144486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00341006a0 0x7f003419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:20.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.686+0000 7f0039489700 1 -- 172.21.15.62:0/2980144486 >> 172.21.15.62:0/2980144486 conn(0x7f00340faa40 msgr2=0x7f0034104160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:20.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.687+0000 7f0039489700 1 -- 172.21.15.62:0/2980144486 shutdown_connections 2023-11-06T01:31:20.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-11-06T01:31:20.687+0000 7f0039489700 1 -- 172.21.15.62:0/2980144486 wait complete. 2023-11-06T01:31:20.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:20 smithi062 ceph-mon[103924]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:20.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:20 smithi163 ceph-mon[104416]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:21.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:20 smithi053 ceph-mon[99472]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:21.229 DEBUG:teuthology.orchestra.run.smithi062:osd.10> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.10.service 2023-11-06T01:31:21.232 INFO:tasks.cephadm:Deploying osd.11 on smithi163 with /dev/vg_nvme/lv_1... 2023-11-06T01:31:21.232 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-06T01:31:21.266 INFO:journalctl@ceph.osd.10.smithi062.stdout:-- Logs begin at Mon 2023-11-06 01:10:07 UTC. -- 2023-11-06T01:31:21.796 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:31:21.895 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:31:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:31:21.557+0000 7f22f5bc6640 -1 osd.10 0 log_to_monitors true 2023-11-06T01:31:21.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:21.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:21.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:21.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:21 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:21.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:21 smithi062 ceph-mon[103924]: from='osd.10 [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-06T01:31:21.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:21 smithi062 ceph-mon[103924]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-06T01:31:21.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:21.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:21.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:21.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:21 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:21.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:21 smithi163 ceph-mon[104416]: from='osd.10 [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-06T01:31:21.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:21 smithi163 ceph-mon[104416]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-06T01:31:22.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:22.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:22.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:22.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:21 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:22.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:21 smithi053 ceph-mon[99472]: from='osd.10 [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-06T01:31:22.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:21 smithi053 ceph-mon[99472]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-06T01:31:23.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:22 smithi163 ceph-mon[104416]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:23.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:22 smithi163 ceph-mon[104416]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-06T01:31:23.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:22 smithi163 ceph-mon[104416]: osdmap e68: 11 total, 10 up, 11 in 2023-11-06T01:31:23.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:23.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:22 smithi163 ceph-mon[104416]: from='osd.10 [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:31:23.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:22 smithi163 ceph-mon[104416]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:31:23.258 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-06T01:31:23.258 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-06T01:31:23.258 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2023-11-06T01:31:23.259 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2023-11-06T01:31:23.259 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112026 s, 936 MB/s 2023-11-06T01:31:23.259 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2023-11-06T01:31:23.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:22 smithi053 ceph-mon[99472]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:23.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:22 smithi053 ceph-mon[99472]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-06T01:31:23.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:22 smithi053 ceph-mon[99472]: osdmap e68: 11 total, 10 up, 11 in 2023-11-06T01:31:23.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:23.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:22 smithi053 ceph-mon[99472]: from='osd.10 [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:31:23.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:22 smithi053 ceph-mon[99472]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:31:23.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:22 smithi062 ceph-mon[103924]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:23.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:22 smithi062 ceph-mon[103924]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-06T01:31:23.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:22 smithi062 ceph-mon[103924]: osdmap e68: 11 total, 10 up, 11 in 2023-11-06T01:31:23.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:23.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:22 smithi062 ceph-mon[103924]: from='osd.10 [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:31:23.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:22 smithi062 ceph-mon[103924]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-11-06T01:31:23.398 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:31:22 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:31:22.984+0000 7f22eb23d700 -1 osd.10 0 waiting for initial osdmap 2023-11-06T01:31:23.398 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:31:23 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:31:23.002+0000 7f22e502e700 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:31:23.639 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi163:vg_nvme/lv_1 2023-11-06T01:31:24.191 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:31:24.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:23 smithi163 ceph-mon[104416]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:31:24.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:23 smithi163 ceph-mon[104416]: osdmap e69: 11 total, 10 up, 11 in 2023-11-06T01:31:24.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:23 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:24.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:23 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:24.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:23 smithi053 ceph-mon[99472]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:31:24.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:23 smithi053 ceph-mon[99472]: osdmap e69: 11 total, 10 up, 11 in 2023-11-06T01:31:24.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:23 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:24.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:23 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:24.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:23 smithi062 ceph-mon[103924]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-11-06T01:31:24.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:23 smithi062 ceph-mon[103924]: osdmap e69: 11 total, 10 up, 11 in 2023-11-06T01:31:24.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:23 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:24.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:23 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:25.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:31:25.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:31:25.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:25.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:25.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: osd.10 [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861] boot 2023-11-06T01:31:25.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: osdmap e70: 11 total, 11 up, 11 in 2023-11-06T01:31:25.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:25.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi062 2023-11-06T01:31:25.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.242 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi062 to 4292M 2023-11-06T01:31:25.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:25.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:25.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:25.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:25.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:25.244 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:24 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:31:25.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:31:25.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:25.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:25.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: osd.10 [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861] boot 2023-11-06T01:31:25.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: osdmap e70: 11 total, 11 up, 11 in 2023-11-06T01:31:25.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:25.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi062 2023-11-06T01:31:25.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi062 to 4292M 2023-11-06T01:31:25.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:25.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:25.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:25.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:25.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:25.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:24 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:31:25.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:31:25.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-11-06T01:31:25.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:25.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: osd.10 [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861] boot 2023-11-06T01:31:25.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: osdmap e70: 11 total, 11 up, 11 in 2023-11-06T01:31:25.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-06T01:31:25.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi062 2023-11-06T01:31:25.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:25.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi062 to 4292M 2023-11-06T01:31:25.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:25.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:25.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:25.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:25.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:25.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:24 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:25.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.413+0000 7fca03fa0700 1 -- 172.21.15.163:0/4201734197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9fc1018c0 msgr2=0x7fc9fc108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:25.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.413+0000 7fca03fa0700 1 --2- 172.21.15.163:0/4201734197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9fc1018c0 0x7fc9fc108170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400b0a0 tx=0x7fc9f400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:31:25.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.413+0000 7fca03fa0700 1 -- 172.21.15.163:0/4201734197 shutdown_connections 2023-11-06T01:31:25.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.413+0000 7fca03fa0700 1 --2- 172.21.15.163:0/4201734197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9fc0ffd20 0x7fc9fc100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:25.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.413+0000 7fca03fa0700 1 --2- 172.21.15.163:0/4201734197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9fc1018c0 0x7fc9fc108170 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:25.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.413+0000 7fca03fa0700 1 --2- 172.21.15.163:0/4201734197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc100f20 0x7fc9fc101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:25.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.413+0000 7fca03fa0700 1 -- 172.21.15.163:0/4201734197 >> 172.21.15.163:0/4201734197 conn(0x7fc9fc0fb2a0 msgr2=0x7fc9fc0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:25.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.413+0000 7fca03fa0700 1 -- 172.21.15.163:0/4201734197 shutdown_connections 2023-11-06T01:31:25.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.413+0000 7fca03fa0700 1 -- 172.21.15.163:0/4201734197 wait complete. 2023-11-06T01:31:25.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.413+0000 7fca03fa0700 1 Processor -- start 2023-11-06T01:31:25.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.413+0000 7fca03fa0700 1 -- start start 2023-11-06T01:31:25.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca03fa0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9fc0ffd20 0x7fc9fc069c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:25.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca03fa0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc100f20 0x7fc9fc19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:25.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca03fa0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9fc1018c0 0x7fc9fc19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:25.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca03fa0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc19e6d0 con 0x7fc9fc100f20 2023-11-06T01:31:25.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca03fa0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc19e840 con 0x7fc9fc0ffd20 2023-11-06T01:31:25.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca03fa0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc19ea00 con 0x7fc9fc1018c0 2023-11-06T01:31:25.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca0253d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9fc1018c0 0x7fc9fc19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:25.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca0253d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9fc1018c0 0x7fc9fc19df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47034/0 (socket says 172.21.15.163:47034) 2023-11-06T01:31:25.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca0253d700 1 -- 172.21.15.163:0/1411394207 learned_addr learned my addr 172.21.15.163:0/1411394207 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-11-06T01:31:25.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca0253d700 1 -- 172.21.15.163:0/1411394207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9fc0ffd20 msgr2=0x7fc9fc069c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:25.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca0153b700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc100f20 0x7fc9fc19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:25.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca0253d700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9fc0ffd20 0x7fc9fc069c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:25.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca0253d700 1 -- 172.21.15.163:0/1411394207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc100f20 msgr2=0x7fc9fc19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:25.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca0253d700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc100f20 0x7fc9fc19b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:25.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca0253d700 1 -- 172.21.15.163:0/1411394207 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f4009d00 con 0x7fc9fc1018c0 2023-11-06T01:31:25.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca0253d700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9fc1018c0 0x7fc9fc19df30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400b070 tx=0x7fc9f4004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:25.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fc9f2ffd700 1 -- 172.21.15.163:0/1411394207 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f400e040 con 0x7fc9fc1018c0 2023-11-06T01:31:25.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fc9f2ffd700 1 -- 172.21.15.163:0/1411394207 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9f401bd20 con 0x7fc9fc1018c0 2023-11-06T01:31:25.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca03fa0700 1 -- 172.21.15.163:0/1411394207 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9fc19ec80 con 0x7fc9fc1018c0 2023-11-06T01:31:25.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fc9f2ffd700 1 -- 172.21.15.163:0/1411394207 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f40058e0 con 0x7fc9fc1018c0 2023-11-06T01:31:25.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.414+0000 7fca03fa0700 1 -- 172.21.15.163:0/1411394207 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9fc19f1c0 con 0x7fc9fc1018c0 2023-11-06T01:31:25.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.415+0000 7fca03fa0700 1 -- 172.21.15.163:0/1411394207 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9fc106b70 con 0x7fc9fc1018c0 2023-11-06T01:31:25.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.416+0000 7fc9f2ffd700 1 -- 172.21.15.163:0/1411394207 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fc9f4005040 con 0x7fc9fc1018c0 2023-11-06T01:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.417+0000 7fc9f2ffd700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9e80a05a0 0x7fc9e80a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.417+0000 7fc9f2ffd700 1 -- 172.21.15.163:0/1411394207 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fc9f40cd7e0 con 0x7fc9fc1018c0 2023-11-06T01:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.417+0000 7fca01d3c700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9e80a05a0 0x7fc9e80a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.418+0000 7fca01d3c700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9e80a05a0 0x7fc9e80a2a60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc100d80 tx=0x7fc9ec005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:25.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.420+0000 7fc9f2ffd700 1 -- 172.21.15.163:0/1411394207 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc9f4097a40 con 0x7fc9fc1018c0 2023-11-06T01:31:25.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:25.617+0000 7fca03fa0700 1 -- 172.21.15.163:0/1411394207 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9fc0611d0 con 0x7fc9e80a05a0 2023-11-06T01:31:26.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:25 smithi053 ceph-mon[99472]: osdmap e71: 11 total, 11 up, 11 in 2023-11-06T01:31:26.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:31:26.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:31:26.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:25 smithi062 ceph-mon[103924]: osdmap e71: 11 total, 11 up, 11 in 2023-11-06T01:31:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:31:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:31:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:26.396 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:26 smithi163 ceph-mon[104416]: osdmap e71: 11 total, 11 up, 11 in 2023-11-06T01:31:26.397 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-06T01:31:26.397 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-06T01:31:26.397 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:27.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:26 smithi053 ceph-mon[99472]: pgmap v211: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:27.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:26 smithi053 ceph-mon[99472]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:31:27.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:26 smithi053 ceph-mon[99472]: osdmap e72: 11 total, 11 up, 11 in 2023-11-06T01:31:27.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:26 smithi062 ceph-mon[103924]: pgmap v211: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:27.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:26 smithi062 ceph-mon[103924]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:31:27.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:26 smithi062 ceph-mon[103924]: osdmap e72: 11 total, 11 up, 11 in 2023-11-06T01:31:27.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:26 smithi163 ceph-mon[104416]: pgmap v211: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:27.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:26 smithi163 ceph-mon[104416]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:31:27.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:26 smithi163 ceph-mon[104416]: osdmap e72: 11 total, 11 up, 11 in 2023-11-06T01:31:28.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:28 smithi053 ceph-mon[99472]: osdmap e73: 11 total, 11 up, 11 in 2023-11-06T01:31:28.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:28 smithi053 ceph-mon[99472]: from='client.? 172.21.15.163:0/845535166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "758da1b3-af80-435f-bc06-3c9a3af6e501"}]: dispatch 2023-11-06T01:31:28.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:28 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "758da1b3-af80-435f-bc06-3c9a3af6e501"}]: dispatch 2023-11-06T01:31:28.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:28 smithi053 ceph-mon[99472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "758da1b3-af80-435f-bc06-3c9a3af6e501"}]': finished 2023-11-06T01:31:28.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:28 smithi053 ceph-mon[99472]: osdmap e74: 12 total, 11 up, 12 in 2023-11-06T01:31:28.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:28.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:28 smithi062 ceph-mon[103924]: osdmap e73: 11 total, 11 up, 11 in 2023-11-06T01:31:28.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:28 smithi062 ceph-mon[103924]: from='client.? 172.21.15.163:0/845535166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "758da1b3-af80-435f-bc06-3c9a3af6e501"}]: dispatch 2023-11-06T01:31:28.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:28 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "758da1b3-af80-435f-bc06-3c9a3af6e501"}]: dispatch 2023-11-06T01:31:28.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:28 smithi062 ceph-mon[103924]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "758da1b3-af80-435f-bc06-3c9a3af6e501"}]': finished 2023-11-06T01:31:28.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:28 smithi062 ceph-mon[103924]: osdmap e74: 12 total, 11 up, 12 in 2023-11-06T01:31:28.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:28.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:28 smithi163 ceph-mon[104416]: osdmap e73: 11 total, 11 up, 11 in 2023-11-06T01:31:28.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:28 smithi163 ceph-mon[104416]: from='client.? 172.21.15.163:0/845535166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "758da1b3-af80-435f-bc06-3c9a3af6e501"}]: dispatch 2023-11-06T01:31:28.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:28 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "758da1b3-af80-435f-bc06-3c9a3af6e501"}]: dispatch 2023-11-06T01:31:28.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:28 smithi163 ceph-mon[104416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "758da1b3-af80-435f-bc06-3c9a3af6e501"}]': finished 2023-11-06T01:31:28.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:28 smithi163 ceph-mon[104416]: osdmap e74: 12 total, 11 up, 12 in 2023-11-06T01:31:28.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:29.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:29 smithi053 ceph-mon[99472]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:29.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:29 smithi053 ceph-mon[99472]: from='client.? 172.21.15.163:0/353485632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:31:29.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:29 smithi062 ceph-mon[103924]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:29.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:29 smithi062 ceph-mon[103924]: from='client.? 172.21.15.163:0/353485632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:31:29.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:29 smithi163 ceph-mon[104416]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:29.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:29 smithi163 ceph-mon[104416]: from='client.? 172.21.15.163:0/353485632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-06T01:31:31.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:31 smithi053 ceph-mon[99472]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:31.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:31 smithi062 ceph-mon[103924]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:31.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:31 smithi163 ceph-mon[104416]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:33.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:33 smithi053 ceph-mon[99472]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-06T01:31:33.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:33 smithi062 ceph-mon[103924]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-06T01:31:33.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:33 smithi163 ceph-mon[104416]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-06T01:31:35.306 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:35 smithi163 ceph-mon[104416]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-11-06T01:31:35.306 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-06T01:31:35.306 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:35 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:35.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:35 smithi053 ceph-mon[99472]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-11-06T01:31:35.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-06T01:31:35.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:35 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:35.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:35 smithi062 ceph-mon[103924]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-11-06T01:31:35.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-06T01:31:35.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:35 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:36.219 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:36 smithi163 ceph-mon[104416]: Deploying daemon osd.11 on smithi163 2023-11-06T01:31:36.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:36 smithi053 ceph-mon[99472]: Deploying daemon osd.11 on smithi163 2023-11-06T01:31:36.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:36 smithi062 ceph-mon[103924]: Deploying daemon osd.11 on smithi163 2023-11-06T01:31:37.240 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:37 smithi163 ceph-mon[104416]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-11-06T01:31:37.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:37 smithi053 ceph-mon[99472]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-11-06T01:31:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:37 smithi062 ceph-mon[103924]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-11-06T01:31:39.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:39 smithi053 ceph-mon[99472]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-06T01:31:39.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:39.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:39.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:39 smithi062 ceph-mon[103924]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-06T01:31:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:39.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:39 smithi163 ceph-mon[104416]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-06T01:31:39.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:39.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:39.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:41.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:41 smithi053 ceph-mon[99472]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-11-06T01:31:41.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:41.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:41.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:41.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:41.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:41 smithi062 ceph-mon[103924]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-11-06T01:31:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:41.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:41.434 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:41 smithi163 ceph-mon[104416]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-11-06T01:31:41.435 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:41.435 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:41.435 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:41.435 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:41.436 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:41.436 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 11 on host 'smithi163' 2023-11-06T01:31:41.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.435+0000 7fc9f2ffd700 1 -- 172.21.15.163:0/1411394207 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc9fc0611d0 con 0x7fc9e80a05a0 2023-11-06T01:31:41.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.437+0000 7fca03fa0700 1 -- 172.21.15.163:0/1411394207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9e80a05a0 msgr2=0x7fc9e80a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:41.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.437+0000 7fca03fa0700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9e80a05a0 0x7fc9e80a2a60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc100d80 tx=0x7fc9ec005d20 comp rx=0 tx=0).stop 2023-11-06T01:31:41.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.437+0000 7fca03fa0700 1 -- 172.21.15.163:0/1411394207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9fc1018c0 msgr2=0x7fc9fc19df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:41.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.437+0000 7fca03fa0700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9fc1018c0 0x7fc9fc19df30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400b070 tx=0x7fc9f4004db0 comp rx=0 tx=0).stop 2023-11-06T01:31:41.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.437+0000 7fca03fa0700 1 -- 172.21.15.163:0/1411394207 shutdown_connections 2023-11-06T01:31:41.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.437+0000 7fca03fa0700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9e80a05a0 0x7fc9e80a2a60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:41.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.437+0000 7fca03fa0700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9fc0ffd20 0x7fc9fc069c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:41.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.437+0000 7fca03fa0700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9fc1018c0 0x7fc9fc19df30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:41.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.437+0000 7fca03fa0700 1 --2- 172.21.15.163:0/1411394207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc100f20 0x7fc9fc19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:41.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.437+0000 7fca03fa0700 1 -- 172.21.15.163:0/1411394207 >> 172.21.15.163:0/1411394207 conn(0x7fc9fc0fb2a0 msgr2=0x7fc9fc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:41.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.438+0000 7fca03fa0700 1 -- 172.21.15.163:0/1411394207 shutdown_connections 2023-11-06T01:31:41.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:41.438+0000 7fca03fa0700 1 -- 172.21.15.163:0/1411394207 wait complete. 2023-11-06T01:31:41.876 DEBUG:teuthology.orchestra.run.smithi163:osd.11> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.11.service 2023-11-06T01:31:41.878 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-11-06T01:31:41.879 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd stat -f json 2023-11-06T01:31:41.913 INFO:journalctl@ceph.osd.11.smithi163.stdout:-- Logs begin at Mon 2023-11-06 01:10:53 UTC. -- 2023-11-06T01:31:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:31:42.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:42 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:42.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:42 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:42.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:42 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:42.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:42 smithi062 ceph-mon[103924]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-06T01:31:42.653 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:42 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:42.654 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:42 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:42.654 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:42 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:42.654 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:42 smithi053 ceph-mon[99472]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-06T01:31:42.663 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:42 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:42.664 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:42 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:42.664 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:42 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:42.664 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:42 smithi163 ceph-mon[104416]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-06T01:31:42.932 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:31:42 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:31:42.693+0000 7f1ce1bb1640 -1 osd.11 0 log_to_monitors true 2023-11-06T01:31:43.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:43 smithi062 ceph-mon[103924]: from='osd.11 [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-06T01:31:43.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:43 smithi062 ceph-mon[103924]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-06T01:31:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.644+0000 7f693b17e700 1 -- 172.21.15.53:0/1599313674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934104260 msgr2=0x7f6934106650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.644+0000 7f693b17e700 1 --2- 172.21.15.53:0/1599313674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934104260 0x7f6934106650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6920009990 tx=0x7f6920009ca0 comp rx=0 tx=0).stop 2023-11-06T01:31:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.644+0000 7f693b17e700 1 -- 172.21.15.53:0/1599313674 shutdown_connections 2023-11-06T01:31:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.644+0000 7f693b17e700 1 --2- 172.21.15.53:0/1599313674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934106b90 0x7f6934108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.644+0000 7f693b17e700 1 --2- 172.21.15.53:0/1599313674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6934101930 0x7f6934103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.644+0000 7f693b17e700 1 --2- 172.21.15.53:0/1599313674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934104260 0x7f6934106650 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.644+0000 7f693b17e700 1 -- 172.21.15.53:0/1599313674 >> 172.21.15.53:0/1599313674 conn(0x7f69340fb2a0 msgr2=0x7f69340fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.644+0000 7f693b17e700 1 -- 172.21.15.53:0/1599313674 shutdown_connections 2023-11-06T01:31:43.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.644+0000 7f693b17e700 1 -- 172.21.15.53:0/1599313674 wait complete. 2023-11-06T01:31:43.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.644+0000 7f693b17e700 1 Processor -- start 2023-11-06T01:31:43.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.644+0000 7f693b17e700 1 -- start start 2023-11-06T01:31:43.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f693b17e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934101930 0x7f6934101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:43.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f693b17e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934104260 0x7f69340ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:43.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f693b17e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6934106b90 0x7f69340ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:43.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f693b17e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341005a0 con 0x7f6934101930 2023-11-06T01:31:43.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f693b17e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934100710 con 0x7f6934104260 2023-11-06T01:31:43.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f693b17e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341a0bc0 con 0x7f6934106b90 2023-11-06T01:31:43.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6938f1a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934101930 0x7f6934101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:43.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6938f1a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934101930 0x7f6934101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37560/0 (socket says 172.21.15.53:37560) 2023-11-06T01:31:43.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6938f1a700 1 -- 172.21.15.53:0/334289610 learned_addr learned my addr 172.21.15.53:0/334289610 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:31:43.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f693971b700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6934106b90 0x7f69340ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:43.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6933fff700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934104260 0x7f69340ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:43.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6938f1a700 1 -- 172.21.15.53:0/334289610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6934106b90 msgr2=0x7f69340ffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:43.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6938f1a700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6934106b90 0x7f69340ffe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:43.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6938f1a700 1 -- 172.21.15.53:0/334289610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934104260 msgr2=0x7f69340ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:43.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6938f1a700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934104260 0x7f69340ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:43.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6938f1a700 1 -- 172.21.15.53:0/334289610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200095f0 con 0x7f6934101930 2023-11-06T01:31:43.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6938f1a700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934101930 0x7f6934101290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f692800d980 tx=0x7f692800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:43.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6931ffb700 1 -- 172.21.15.53:0/334289610 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928004220 con 0x7f6934101930 2023-11-06T01:31:43.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6931ffb700 1 -- 172.21.15.53:0/334289610 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6928007640 con 0x7f6934101930 2023-11-06T01:31:43.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f693b17e700 1 -- 172.21.15.53:0/334289610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69341a0ea0 con 0x7f6934101930 2023-11-06T01:31:43.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.645+0000 7f6931ffb700 1 -- 172.21.15.53:0/334289610 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928005610 con 0x7f6934101930 2023-11-06T01:31:43.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.646+0000 7f693b17e700 1 -- 172.21.15.53:0/334289610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69341a12b0 con 0x7f6934101930 2023-11-06T01:31:43.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.647+0000 7f693b17e700 1 -- 172.21.15.53:0/334289610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69340fce20 con 0x7f6934101930 2023-11-06T01:31:43.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.648+0000 7f6931ffb700 1 -- 172.21.15.53:0/334289610 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f6928004750 con 0x7f6934101930 2023-11-06T01:31:43.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.648+0000 7f6931ffb700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69240a0520 0x7f69240a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:43.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.648+0000 7f6933fff700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69240a0520 0x7f69240a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:43.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.648+0000 7f6931ffb700 1 -- 172.21.15.53:0/334289610 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f69280cb2a0 con 0x7f6934101930 2023-11-06T01:31:43.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.649+0000 7f6933fff700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69240a0520 0x7f69240a29e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f692000b5c0 tx=0x7f6920005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:43.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.653+0000 7f6931ffb700 1 -- 172.21.15.53:0/334289610 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6928095410 con 0x7f6934101930 2023-11-06T01:31:43.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:43 smithi163 ceph-mon[104416]: from='osd.11 [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-06T01:31:43.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:43 smithi163 ceph-mon[104416]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-06T01:31:43.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:43 smithi053 ceph-mon[99472]: from='osd.11 [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-06T01:31:43.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:43 smithi053 ceph-mon[99472]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-06T01:31:43.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.852+0000 7f693b17e700 1 -- 172.21.15.53:0/334289610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f693402cc70 con 0x7f6934101930 2023-11-06T01:31:43.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.853+0000 7f6931ffb700 1 -- 172.21.15.53:0/334289610 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f6928098a30 con 0x7f6934101930 2023-11-06T01:31:43.854 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:31:43.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 -- 172.21.15.53:0/334289610 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69240a0520 msgr2=0x7f69240a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:43.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69240a0520 0x7f69240a29e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f692000b5c0 tx=0x7f6920005fb0 comp rx=0 tx=0).stop 2023-11-06T01:31:43.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 -- 172.21.15.53:0/334289610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934101930 msgr2=0x7f6934101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:43.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934101930 0x7f6934101290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f692800d980 tx=0x7f692800dc90 comp rx=0 tx=0).stop 2023-11-06T01:31:43.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 -- 172.21.15.53:0/334289610 shutdown_connections 2023-11-06T01:31:43.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69240a0520 0x7f69240a29e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:43.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6934106b90 0x7f69340ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:43.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934101930 0x7f6934101290 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:43.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 --2- 172.21.15.53:0/334289610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934104260 0x7f69340ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:43.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 -- 172.21.15.53:0/334289610 >> 172.21.15.53:0/334289610 conn(0x7f69340fb2a0 msgr2=0x7f6934102620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:43.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 -- 172.21.15.53:0/334289610 shutdown_connections 2023-11-06T01:31:43.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:43.855+0000 7f693b17e700 1 -- 172.21.15.53:0/334289610 wait complete. 2023-11-06T01:31:44.213 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1699234283,"num_in_osds":12,"osd_in_since":1699234287,"num_remapped_pgs":0} 2023-11-06T01:31:44.489 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:31:44.406+0000 7f1cd7228700 -1 osd.11 0 waiting for initial osdmap 2023-11-06T01:31:44.490 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:31:44.421+0000 7f1cd181a700 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T01:31:44.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-06T01:31:44.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: osdmap e75: 12 total, 11 up, 12 in 2023-11-06T01:31:44.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:44.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='osd.11 [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:31:44.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:31:44.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:44.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/334289610' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-06T01:31:44.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: Detected new or changed devices on smithi163 2023-11-06T01:31:44.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:44.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:44.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi163 to 4292M 2023-11-06T01:31:44.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:44.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:44.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:44.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:44.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-06T01:31:44.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: osdmap e75: 12 total, 11 up, 12 in 2023-11-06T01:31:44.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:44.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='osd.11 [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:31:44.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:31:44.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:44.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/334289610' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-06T01:31:44.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: Detected new or changed devices on smithi163 2023-11-06T01:31:44.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:44.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:44.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi163 to 4292M 2023-11-06T01:31:44.598 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:44.598 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:44.598 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:44.598 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:44.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-06T01:31:44.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: osdmap e75: 12 total, 11 up, 12 in 2023-11-06T01:31:44.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:44.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='osd.11 [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:31:44.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2023-11-06T01:31:44.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:44.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/334289610' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-06T01:31:44.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: Detected new or changed devices on smithi163 2023-11-06T01:31:44.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:44.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:44.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:31:44.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi163 to 4292M 2023-11-06T01:31:44.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:44.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:44.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:44.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:45.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd stat -f json 2023-11-06T01:31:45.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:45 smithi053 ceph-mon[99472]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:31:45.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:45 smithi053 ceph-mon[99472]: osdmap e76: 12 total, 11 up, 12 in 2023-11-06T01:31:45.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:45.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:45.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:45 smithi163 ceph-mon[104416]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:31:45.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:45 smithi163 ceph-mon[104416]: osdmap e76: 12 total, 11 up, 12 in 2023-11-06T01:31:45.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:45 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:45.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:45 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:45.788 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:31:45.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:45 smithi062 ceph-mon[103924]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2023-11-06T01:31:45.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:45 smithi062 ceph-mon[103924]: osdmap e76: 12 total, 11 up, 12 in 2023-11-06T01:31:45.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:45 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:45.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:45 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:46.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:46 smithi053 ceph-mon[99472]: purged_snaps scrub starts 2023-11-06T01:31:46.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:46 smithi053 ceph-mon[99472]: purged_snaps scrub ok 2023-11-06T01:31:46.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:46 smithi053 ceph-mon[99472]: osd.11 [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705] boot 2023-11-06T01:31:46.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:46 smithi053 ceph-mon[99472]: osdmap e77: 12 total, 12 up, 12 in 2023-11-06T01:31:46.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:46.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:46 smithi053 ceph-mon[99472]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:46.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:46 smithi163 ceph-mon[104416]: purged_snaps scrub starts 2023-11-06T01:31:46.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:46 smithi163 ceph-mon[104416]: purged_snaps scrub ok 2023-11-06T01:31:46.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:46 smithi163 ceph-mon[104416]: osd.11 [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705] boot 2023-11-06T01:31:46.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:46 smithi163 ceph-mon[104416]: osdmap e77: 12 total, 12 up, 12 in 2023-11-06T01:31:46.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:46.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:46 smithi163 ceph-mon[104416]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:46.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:46 smithi062 ceph-mon[103924]: purged_snaps scrub starts 2023-11-06T01:31:46.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:46 smithi062 ceph-mon[103924]: purged_snaps scrub ok 2023-11-06T01:31:46.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:46 smithi062 ceph-mon[103924]: osd.11 [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705] boot 2023-11-06T01:31:46.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:46 smithi062 ceph-mon[103924]: osdmap e77: 12 total, 12 up, 12 in 2023-11-06T01:31:46.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-06T01:31:46.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:46 smithi062 ceph-mon[103924]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-11-06T01:31:46.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.973+0000 7fd007a0c700 1 -- 172.21.15.53:0/1831252440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0000ffd20 msgr2=0x7fd000100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:46.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.973+0000 7fd007a0c700 1 --2- 172.21.15.53:0/1831252440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0000ffd20 0x7fd000100140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcff00099c0 tx=0x7fcff0009cd0 comp rx=0 tx=0).stop 2023-11-06T01:31:46.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.973+0000 7fd007a0c700 1 -- 172.21.15.53:0/1831252440 shutdown_connections 2023-11-06T01:31:46.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.973+0000 7fd007a0c700 1 --2- 172.21.15.53:0/1831252440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0001018c0 0x7fd000108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:46.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.973+0000 7fd007a0c700 1 --2- 172.21.15.53:0/1831252440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0000ffd20 0x7fd000100140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:46.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.973+0000 7fd007a0c700 1 --2- 172.21.15.53:0/1831252440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000100f20 0x7fd000101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:46.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.973+0000 7fd007a0c700 1 -- 172.21.15.53:0/1831252440 >> 172.21.15.53:0/1831252440 conn(0x7fd0000fb2a0 msgr2=0x7fd0000fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:46.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.973+0000 7fd007a0c700 1 -- 172.21.15.53:0/1831252440 shutdown_connections 2023-11-06T01:31:46.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.973+0000 7fd007a0c700 1 -- 172.21.15.53:0/1831252440 wait complete. 2023-11-06T01:31:46.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd007a0c700 1 Processor -- start 2023-11-06T01:31:46.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd007a0c700 1 -- start start 2023-11-06T01:31:46.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd007a0c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0000ffd20 0x7fd000195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:46.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd007a0c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000100f20 0x7fd000196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:46.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd007a0c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0001018c0 0x7fd00019d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:46.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd007a0c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00019d750 con 0x7fd0001018c0 2023-11-06T01:31:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd007a0c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00019d890 con 0x7fd0000ffd20 2023-11-06T01:31:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd007a0c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00019da20 con 0x7fd000100f20 2023-11-06T01:31:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd005fa9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0001018c0 0x7fd00019d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd005fa9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0001018c0 0x7fd00019d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37580/0 (socket says 172.21.15.53:37580) 2023-11-06T01:31:46.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd005fa9700 1 -- 172.21.15.53:0/886929698 learned_addr learned my addr 172.21.15.53:0/886929698 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:31:46.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd0057a8700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0000ffd20 0x7fd000195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:46.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd004fa7700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000100f20 0x7fd000196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:46.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd005fa9700 1 -- 172.21.15.53:0/886929698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000100f20 msgr2=0x7fd000196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:46.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd005fa9700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000100f20 0x7fd000196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:46.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd005fa9700 1 -- 172.21.15.53:0/886929698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0000ffd20 msgr2=0x7fd000195bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:46.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd005fa9700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0000ffd20 0x7fd000195bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:46.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.974+0000 7fd005fa9700 1 -- 172.21.15.53:0/886929698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff00095f0 con 0x7fd0001018c0 2023-11-06T01:31:46.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.975+0000 7fd005fa9700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0001018c0 0x7fd00019d180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcff800b9a0 tx=0x7fcff800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:46.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.975+0000 7fcff67fc700 1 -- 172.21.15.53:0/886929698 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff800c930 con 0x7fd0001018c0 2023-11-06T01:31:46.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.975+0000 7fcff67fc700 1 -- 172.21.15.53:0/886929698 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcff8014440 con 0x7fd0001018c0 2023-11-06T01:31:46.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.975+0000 7fd007a0c700 1 -- 172.21.15.53:0/886929698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00019dd00 con 0x7fd0001018c0 2023-11-06T01:31:46.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.975+0000 7fcff67fc700 1 -- 172.21.15.53:0/886929698 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff80075d0 con 0x7fd0001018c0 2023-11-06T01:31:46.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.975+0000 7fd007a0c700 1 -- 172.21.15.53:0/886929698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00019e1b0 con 0x7fd0001018c0 2023-11-06T01:31:46.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.976+0000 7fd007a0c700 1 -- 172.21.15.53:0/886929698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0001073c0 con 0x7fd0001018c0 2023-11-06T01:31:46.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.977+0000 7fcff67fc700 1 -- 172.21.15.53:0/886929698 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fcff80145b0 con 0x7fd0001018c0 2023-11-06T01:31:46.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.977+0000 7fcff67fc700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcfec0a05a0 0x7fcfec0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:46.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.978+0000 7fcff67fc700 1 -- 172.21.15.53:0/886929698 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcff8006e40 con 0x7fd0001018c0 2023-11-06T01:31:46.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.978+0000 7fd0057a8700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcfec0a05a0 0x7fcfec0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:46.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.978+0000 7fd0057a8700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcfec0a05a0 0x7fcfec0a2a60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcff000b580 tx=0x7fcff0009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:46.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:46.982+0000 7fcff67fc700 1 -- 172.21.15.53:0/886929698 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcff8094b40 con 0x7fd0001018c0 2023-11-06T01:31:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.181+0000 7fd007a0c700 1 -- 172.21.15.53:0/886929698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fd00002ce70 con 0x7fd0001018c0 2023-11-06T01:31:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.181+0000 7fcff67fc700 1 -- 172.21.15.53:0/886929698 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7fcff8098160 con 0x7fd0001018c0 2023-11-06T01:31:47.183 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:31:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.183+0000 7fd007a0c700 1 -- 172.21.15.53:0/886929698 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcfec0a05a0 msgr2=0x7fcfec0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.183+0000 7fd007a0c700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcfec0a05a0 0x7fcfec0a2a60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcff000b580 tx=0x7fcff0009f10 comp rx=0 tx=0).stop 2023-11-06T01:31:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.183+0000 7fd007a0c700 1 -- 172.21.15.53:0/886929698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0001018c0 msgr2=0x7fd00019d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.183+0000 7fd007a0c700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0001018c0 0x7fd00019d180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcff800b9a0 tx=0x7fcff800bcb0 comp rx=0 tx=0).stop 2023-11-06T01:31:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.183+0000 7fd007a0c700 1 -- 172.21.15.53:0/886929698 shutdown_connections 2023-11-06T01:31:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.183+0000 7fd007a0c700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcfec0a05a0 0x7fcfec0a2a60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.183+0000 7fd007a0c700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0000ffd20 0x7fd000195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.183+0000 7fd007a0c700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0001018c0 0x7fd00019d180 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.183+0000 7fd007a0c700 1 --2- 172.21.15.53:0/886929698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000100f20 0x7fd000196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:47.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.183+0000 7fd007a0c700 1 -- 172.21.15.53:0/886929698 >> 172.21.15.53:0/886929698 conn(0x7fd0000fb2a0 msgr2=0x7fd000104500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:47.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.184+0000 7fd007a0c700 1 -- 172.21.15.53:0/886929698 shutdown_connections 2023-11-06T01:31:47.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:47.184+0000 7fd007a0c700 1 -- 172.21.15.53:0/886929698 wait complete. 2023-11-06T01:31:47.526 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:47 smithi053 ceph-mon[99472]: osdmap e78: 12 total, 12 up, 12 in 2023-11-06T01:31:47.526 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:47 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/886929698' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-06T01:31:47.536 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1699234305,"num_in_osds":12,"osd_in_since":1699234287,"num_remapped_pgs":0} 2023-11-06T01:31:47.536 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd dump --format=json 2023-11-06T01:31:47.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:47 smithi163 ceph-mon[104416]: osdmap e78: 12 total, 12 up, 12 in 2023-11-06T01:31:47.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:47 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/886929698' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-06T01:31:47.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:47 smithi062 ceph-mon[103924]: osdmap e78: 12 total, 12 up, 12 in 2023-11-06T01:31:47.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:47 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/886929698' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-06T01:31:48.098 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:31:48.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:48 smithi053 ceph-mon[99472]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:48.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:48 smithi163 ceph-mon[104416]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:48.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:48 smithi062 ceph-mon[103924]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:49.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.282+0000 7f6098dec700 1 -- 172.21.15.53:0/998198122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6094103160 msgr2=0x7f609410dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:49.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.282+0000 7f6098dec700 1 --2- 172.21.15.53:0/998198122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6094103160 0x7f609410dd80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f608c00b0d0 tx=0x7f608c00b3e0 comp rx=0 tx=0).stop 2023-11-06T01:31:49.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.282+0000 7f6098dec700 1 -- 172.21.15.53:0/998198122 shutdown_connections 2023-11-06T01:31:49.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.282+0000 7f6098dec700 1 --2- 172.21.15.53:0/998198122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6094103160 0x7f609410dd80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:49.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.282+0000 7f6098dec700 1 --2- 172.21.15.53:0/998198122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60940694a0 0x7f6094069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:49.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.282+0000 7f6098dec700 1 --2- 172.21.15.53:0/998198122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6094069dc0 0x7f6094102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:49.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.282+0000 7f6098dec700 1 -- 172.21.15.53:0/998198122 >> 172.21.15.53:0/998198122 conn(0x7f6094076d80 msgr2=0x7f6094077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:49.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.282+0000 7f6098dec700 1 -- 172.21.15.53:0/998198122 shutdown_connections 2023-11-06T01:31:49.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.282+0000 7f6098dec700 1 -- 172.21.15.53:0/998198122 wait complete. 2023-11-06T01:31:49.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6098dec700 1 Processor -- start 2023-11-06T01:31:49.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6098dec700 1 -- start start 2023-11-06T01:31:49.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6098dec700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60940694a0 0x7f60941972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:49.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6098dec700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6094069dc0 0x7f6094197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:49.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6098dec700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6094103160 0x7f609419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:49.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6098dec700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609419c280 con 0x7f6094103160 2023-11-06T01:31:49.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6098dec700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609419c3f0 con 0x7f60940694a0 2023-11-06T01:31:49.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6098dec700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609419c5b0 con 0x7f6094069dc0 2023-11-06T01:31:49.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6092d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6094103160 0x7f609419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:49.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6092d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6094103160 0x7f609419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37602/0 (socket says 172.21.15.53:37602) 2023-11-06T01:31:49.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6092d9d700 1 -- 172.21.15.53:0/1691871778 learned_addr learned my addr 172.21.15.53:0/1691871778 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:31:49.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f609259c700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60940694a0 0x7f60941972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:49.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.283+0000 7f6091d9b700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6094069dc0 0x7f6094197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:49.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f6092d9d700 1 -- 172.21.15.53:0/1691871778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6094069dc0 msgr2=0x7f6094197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:49.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f6092d9d700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6094069dc0 0x7f6094197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:49.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f6092d9d700 1 -- 172.21.15.53:0/1691871778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60940694a0 msgr2=0x7f60941972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:49.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f6092d9d700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60940694a0 0x7f60941972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:49.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f6092d9d700 1 -- 172.21.15.53:0/1691871778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c009d00 con 0x7f6094103160 2023-11-06T01:31:49.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f6091d9b700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6094069dc0 0x7f6094197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:31:49.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f609259c700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60940694a0 0x7f60941972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:31:49.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f6092d9d700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6094103160 0x7f609419bc70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f608c007930 tx=0x7f608c007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:49.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f608b7fe700 1 -- 172.21.15.53:0/1691871778 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c00e040 con 0x7f6094103160 2023-11-06T01:31:49.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f608b7fe700 1 -- 172.21.15.53:0/1691871778 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f608c003e10 con 0x7f6094103160 2023-11-06T01:31:49.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f6098dec700 1 -- 172.21.15.53:0/1691871778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609419c830 con 0x7f6094103160 2023-11-06T01:31:49.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f608b7fe700 1 -- 172.21.15.53:0/1691871778 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c0056c0 con 0x7f6094103160 2023-11-06T01:31:49.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.284+0000 7f6098dec700 1 -- 172.21.15.53:0/1691871778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60941a40f0 con 0x7f6094103160 2023-11-06T01:31:49.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.285+0000 7f6098dec700 1 -- 172.21.15.53:0/1691871778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609404ea90 con 0x7f6094103160 2023-11-06T01:31:49.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.286+0000 7f608b7fe700 1 -- 172.21.15.53:0/1691871778 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f608c005820 con 0x7f6094103160 2023-11-06T01:31:49.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.287+0000 7f608b7fe700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f60800a05a0 0x7f60800a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:49.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.287+0000 7f608b7fe700 1 -- 172.21.15.53:0/1691871778 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f608c0ccc00 con 0x7f6094103160 2023-11-06T01:31:49.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.287+0000 7f609259c700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f60800a05a0 0x7f60800a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:49.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.287+0000 7f609259c700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f60800a05a0 0x7f60800a2a60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f607c0099d0 tx=0x7f607c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:49.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.291+0000 7f608b7fe700 1 -- 172.21.15.53:0/1691871778 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f608c0d3080 con 0x7f6094103160 2023-11-06T01:31:49.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.490+0000 7f6098dec700 1 -- 172.21.15.53:0/1691871778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60941a4280 con 0x7f6094103160 2023-11-06T01:31:49.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.491+0000 7f608b7fe700 1 -- 172.21.15.53:0/1691871778 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f608c096cd0 con 0x7f6094103160 2023-11-06T01:31:49.493 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:31:49.494 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:31:46.410396+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:31:49.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 -- 172.21.15.53:0/1691871778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f60800a05a0 msgr2=0x7f60800a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:49.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f60800a05a0 0x7f60800a2a60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f607c0099d0 tx=0x7f607c008040 comp rx=0 tx=0).stop 2023-11-06T01:31:49.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 -- 172.21.15.53:0/1691871778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6094103160 msgr2=0x7f609419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:49.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6094103160 0x7f609419bc70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f608c007930 tx=0x7f608c007a10 comp rx=0 tx=0).stop 2023-11-06T01:31:49.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 -- 172.21.15.53:0/1691871778 shutdown_connections 2023-11-06T01:31:49.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f60800a05a0 0x7f60800a2a60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:49.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60940694a0 0x7f60941972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:49.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6094103160 0x7f609419bc70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:49.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 --2- 172.21.15.53:0/1691871778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6094069dc0 0x7f6094197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:49.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 -- 172.21.15.53:0/1691871778 >> 172.21.15.53:0/1691871778 conn(0x7f6094076d80 msgr2=0x7f60940fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:49.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 -- 172.21.15.53:0/1691871778 shutdown_connections 2023-11-06T01:31:49.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:49.494+0000 7f6098dec700 1 -- 172.21.15.53:0/1691871778 wait complete. 2023-11-06T01:31:49.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:49 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1691871778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:31:49.855 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:31:49.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-11-06T01:31:49.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:49 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1691871778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:31:49.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:49 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1691871778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:31:50.415 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:31:50.819 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:50 smithi053 ceph-mon[99472]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:50.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:50 smithi062 ceph-mon[103924]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:50.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:50 smithi163 ceph-mon[104416]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:51.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.632+0000 7fd7297aa700 1 -- 172.21.15.53:0/878796242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72410b390 msgr2=0x7fd72410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:51.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.632+0000 7fd7297aa700 1 --2- 172.21.15.53:0/878796242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72410b390 0x7fd72410d850 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00b0a0 tx=0x7fd71c00b3b0 comp rx=0 tx=0).stop 2023-11-06T01:31:51.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.632+0000 7fd7297aa700 1 -- 172.21.15.53:0/878796242 shutdown_connections 2023-11-06T01:31:51.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.632+0000 7fd7297aa700 1 --2- 172.21.15.53:0/878796242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd724101f60 0x7fd724102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:51.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.632+0000 7fd7297aa700 1 --2- 172.21.15.53:0/878796242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72410b390 0x7fd72410d850 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:51.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.632+0000 7fd7297aa700 1 --2- 172.21.15.53:0/878796242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd724102880 0x7fd72410ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:51.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.632+0000 7fd7297aa700 1 -- 172.21.15.53:0/878796242 >> 172.21.15.53:0/878796242 conn(0x7fd7240fb790 msgr2=0x7fd7240fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:51.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.632+0000 7fd7297aa700 1 -- 172.21.15.53:0/878796242 shutdown_connections 2023-11-06T01:31:51.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.632+0000 7fd7297aa700 1 -- 172.21.15.53:0/878796242 wait complete. 2023-11-06T01:31:51.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.633+0000 7fd7297aa700 1 Processor -- start 2023-11-06T01:31:51.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.633+0000 7fd7297aa700 1 -- start start 2023-11-06T01:31:51.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.633+0000 7fd7297aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd724101f60 0x7fd7241991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:51.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.633+0000 7fd7297aa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd724102880 0x7fd7241996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:51.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.633+0000 7fd7297aa700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72410b390 0x7fd72419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:51.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.633+0000 7fd7297aa700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72419e160 con 0x7fd72410b390 2023-11-06T01:31:51.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.633+0000 7fd7297aa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72419e2d0 con 0x7fd724101f60 2023-11-06T01:31:51.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.633+0000 7fd7297aa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72419e490 con 0x7fd724102880 2023-11-06T01:31:51.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7237fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72410b390 0x7fd72419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:51.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7237fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72410b390 0x7fd72419db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37614/0 (socket says 172.21.15.53:37614) 2023-11-06T01:31:51.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7237fe700 1 -- 172.21.15.53:0/3870125848 learned_addr learned my addr 172.21.15.53:0/3870125848 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:31:51.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7227fc700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd724102880 0x7fd7241996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:51.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd722ffd700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd724101f60 0x7fd7241991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:51.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7237fe700 1 -- 172.21.15.53:0/3870125848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd724102880 msgr2=0x7fd7241996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:51.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7237fe700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd724102880 0x7fd7241996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:51.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7237fe700 1 -- 172.21.15.53:0/3870125848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd724101f60 msgr2=0x7fd7241991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:51.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7237fe700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd724101f60 0x7fd7241991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:51.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7237fe700 1 -- 172.21.15.53:0/3870125848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c009d00 con 0x7fd72410b390 2023-11-06T01:31:51.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7227fc700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd724102880 0x7fd7241996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:31:51.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7237fe700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72410b390 0x7fd72419db50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00b830 tx=0x7fd71c007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:51.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd70bfff700 1 -- 172.21.15.53:0/3870125848 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c00e040 con 0x7fd72410b390 2023-11-06T01:31:51.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd70bfff700 1 -- 172.21.15.53:0/3870125848 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd71c004e10 con 0x7fd72410b390 2023-11-06T01:31:51.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7297aa700 1 -- 172.21.15.53:0/3870125848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72419e710 con 0x7fd72410b390 2023-11-06T01:31:51.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd70bfff700 1 -- 172.21.15.53:0/3870125848 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c01d480 con 0x7fd72410b390 2023-11-06T01:31:51.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.634+0000 7fd7297aa700 1 -- 172.21.15.53:0/3870125848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7241a5fd0 con 0x7fd72410b390 2023-11-06T01:31:51.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.635+0000 7fd7297aa700 1 -- 172.21.15.53:0/3870125848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72404ea90 con 0x7fd72410b390 2023-11-06T01:31:51.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.636+0000 7fd70bfff700 1 -- 172.21.15.53:0/3870125848 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fd71c003a00 con 0x7fd72410b390 2023-11-06T01:31:51.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.637+0000 7fd70bfff700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd70c0a0520 0x7fd70c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:51.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.637+0000 7fd722ffd700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd70c0a0520 0x7fd70c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:51.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.637+0000 7fd70bfff700 1 -- 172.21.15.53:0/3870125848 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd71c0cd390 con 0x7fd72410b390 2023-11-06T01:31:51.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.638+0000 7fd722ffd700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd70c0a0520 0x7fd70c0a29e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd718005950 tx=0x7fd7180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:51.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.641+0000 7fd70bfff700 1 -- 172.21.15.53:0/3870125848 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd71c097460 con 0x7fd72410b390 2023-11-06T01:31:51.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.836+0000 7fd7297aa700 1 -- 172.21.15.53:0/3870125848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7241a6200 con 0x7fd72410b390 2023-11-06T01:31:51.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.837+0000 7fd70bfff700 1 -- 172.21.15.53:0/3870125848 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd71c09aa80 con 0x7fd72410b390 2023-11-06T01:31:51.839 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:31:51.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 -- 172.21.15.53:0/3870125848 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd70c0a0520 msgr2=0x7fd70c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:51.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd70c0a0520 0x7fd70c0a29e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd718005950 tx=0x7fd7180058e0 comp rx=0 tx=0).stop 2023-11-06T01:31:51.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 -- 172.21.15.53:0/3870125848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72410b390 msgr2=0x7fd72419db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:51.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72410b390 0x7fd72419db50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00b830 tx=0x7fd71c007a60 comp rx=0 tx=0).stop 2023-11-06T01:31:51.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 -- 172.21.15.53:0/3870125848 shutdown_connections 2023-11-06T01:31:51.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd70c0a0520 0x7fd70c0a29e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:51.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd724101f60 0x7fd7241991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:51.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72410b390 0x7fd72419db50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:51.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 --2- 172.21.15.53:0/3870125848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd724102880 0x7fd7241996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:51.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 -- 172.21.15.53:0/3870125848 >> 172.21.15.53:0/3870125848 conn(0x7fd7240fb790 msgr2=0x7fd7240fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:51.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 -- 172.21.15.53:0/3870125848 shutdown_connections 2023-11-06T01:31:51.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:51.839+0000 7fd7297aa700 1 -- 172.21.15.53:0/3870125848 wait complete. 2023-11-06T01:31:52.182 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:51 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3870125848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:31:52.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:51 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3870125848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:31:52.275 INFO:tasks.cephadm:Adding mds.a on smithi053 2023-11-06T01:31:52.276 INFO:tasks.cephadm:Adding mds.d on smithi053 2023-11-06T01:31:52.276 INFO:tasks.cephadm:Adding mds.g on smithi053 2023-11-06T01:31:52.276 INFO:tasks.cephadm:Adding mds.j on smithi053 2023-11-06T01:31:52.277 INFO:tasks.cephadm:Adding mds.b on smithi062 2023-11-06T01:31:52.277 INFO:tasks.cephadm:Adding mds.e on smithi062 2023-11-06T01:31:52.277 INFO:tasks.cephadm:Adding mds.h on smithi062 2023-11-06T01:31:52.278 INFO:tasks.cephadm:Adding mds.k on smithi062 2023-11-06T01:31:52.278 INFO:tasks.cephadm:Adding mds.c on smithi163 2023-11-06T01:31:52.278 INFO:tasks.cephadm:Adding mds.f on smithi163 2023-11-06T01:31:52.278 INFO:tasks.cephadm:Adding mds.i on smithi163 2023-11-06T01:31:52.279 INFO:tasks.cephadm:Adding mds.l on smithi163 2023-11-06T01:31:52.279 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph orch apply mds all '12;smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l' 2023-11-06T01:31:52.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:51 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3870125848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:31:52.847 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:31:53.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:52 smithi163 ceph-mon[104416]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:53.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:52 smithi053 ceph-mon[99472]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:53.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:52 smithi062 ceph-mon[103924]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.023+0000 7f123cb74700 1 -- 172.21.15.163:0/4283389399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12381018c0 msgr2=0x7f1238108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.023+0000 7f123cb74700 1 --2- 172.21.15.163:0/4283389399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12381018c0 0x7f1238108170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f123000b0a0 tx=0x7f123000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:31:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.023+0000 7f123cb74700 1 -- 172.21.15.163:0/4283389399 shutdown_connections 2023-11-06T01:31:54.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.023+0000 7f123cb74700 1 --2- 172.21.15.163:0/4283389399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12380ffd20 0x7f1238100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:54.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.023+0000 7f123cb74700 1 --2- 172.21.15.163:0/4283389399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12381018c0 0x7f1238108170 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:54.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.024+0000 7f123cb74700 1 --2- 172.21.15.163:0/4283389399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238100f20 0x7f1238101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:54.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.024+0000 7f123cb74700 1 -- 172.21.15.163:0/4283389399 >> 172.21.15.163:0/4283389399 conn(0x7f12380fb2a0 msgr2=0x7f12380fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:54.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.024+0000 7f123cb74700 1 -- 172.21.15.163:0/4283389399 shutdown_connections 2023-11-06T01:31:54.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.024+0000 7f123cb74700 1 -- 172.21.15.163:0/4283389399 wait complete. 2023-11-06T01:31:54.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.024+0000 7f123cb74700 1 Processor -- start 2023-11-06T01:31:54.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f123cb74700 1 -- start start 2023-11-06T01:31:54.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f123cb74700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12380ffd20 0x7f1238069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:54.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f123cb74700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1238100f20 0x7f123819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:54.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f123cb74700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12381018c0 0x7f123819e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:54.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f123cb74700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123819e890 con 0x7f12380ffd20 2023-11-06T01:31:54.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f123cb74700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123819ea00 con 0x7f1238100f20 2023-11-06T01:31:54.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f123cb74700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123819ebc0 con 0x7f12381018c0 2023-11-06T01:31:54.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f1236d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12381018c0 0x7f123819e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:54.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f1236d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12381018c0 0x7f123819e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47158/0 (socket says 172.21.15.163:47158) 2023-11-06T01:31:54.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f1236d9d700 1 -- 172.21.15.163:0/1254355741 learned_addr learned my addr 172.21.15.163:0/1254355741 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-11-06T01:31:54.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f123659c700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12380ffd20 0x7f1238069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:54.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f1235d9b700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1238100f20 0x7f123819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:54.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f1236d9d700 1 -- 172.21.15.163:0/1254355741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1238100f20 msgr2=0x7f123819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:54.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f1236d9d700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1238100f20 0x7f123819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:54.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f1236d9d700 1 -- 172.21.15.163:0/1254355741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12380ffd20 msgr2=0x7f1238069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:54.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f1236d9d700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12380ffd20 0x7f1238069410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:54.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f1236d9d700 1 -- 172.21.15.163:0/1254355741 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1230009d00 con 0x7f12381018c0 2023-11-06T01:31:54.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f1236d9d700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12381018c0 0x7f123819e0f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f123000b070 tx=0x7f1230004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:54.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.025+0000 7f12277fe700 1 -- 172.21.15.163:0/1254355741 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123000e040 con 0x7f12381018c0 2023-11-06T01:31:54.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.026+0000 7f123cb74700 1 -- 172.21.15.163:0/1254355741 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123819ee40 con 0x7f12381018c0 2023-11-06T01:31:54.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.026+0000 7f12277fe700 1 -- 172.21.15.163:0/1254355741 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f123001bd20 con 0x7f12381018c0 2023-11-06T01:31:54.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.026+0000 7f12277fe700 1 -- 172.21.15.163:0/1254355741 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12300058e0 con 0x7f12381018c0 2023-11-06T01:31:54.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.026+0000 7f123cb74700 1 -- 172.21.15.163:0/1254355741 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123819f2d0 con 0x7f12381018c0 2023-11-06T01:31:54.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.026+0000 7f123cb74700 1 -- 172.21.15.163:0/1254355741 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1238107340 con 0x7f12381018c0 2023-11-06T01:31:54.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.028+0000 7f12277fe700 1 -- 172.21.15.163:0/1254355741 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f1230005040 con 0x7f12381018c0 2023-11-06T01:31:54.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.029+0000 7f12277fe700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a0520 0x7f12200a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:54.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.029+0000 7f12277fe700 1 -- 172.21.15.163:0/1254355741 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f12300cd7e0 con 0x7f12381018c0 2023-11-06T01:31:54.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.029+0000 7f123659c700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a0520 0x7f12200a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:54.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.029+0000 7f123659c700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a0520 0x7f12200a29e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f122c006fd0 tx=0x7f122c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:54.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.032+0000 7f12277fe700 1 -- 172.21.15.163:0/1254355741 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f12300978b0 con 0x7f12381018c0 2023-11-06T01:31:54.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.230+0000 7f123cb74700 1 -- 172.21.15.163:0/1254355741 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f1238061390 con 0x7f12200a0520 2023-11-06T01:31:54.240 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled mds.all update... 2023-11-06T01:31:54.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.239+0000 7f12277fe700 1 -- 172.21.15.163:0/1254355741 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f1238061390 con 0x7f12200a0520 2023-11-06T01:31:54.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 -- 172.21.15.163:0/1254355741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a0520 msgr2=0x7f12200a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:54.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a0520 0x7f12200a29e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f122c006fd0 tx=0x7f122c008040 comp rx=0 tx=0).stop 2023-11-06T01:31:54.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 -- 172.21.15.163:0/1254355741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12381018c0 msgr2=0x7f123819e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:54.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12381018c0 0x7f123819e0f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f123000b070 tx=0x7f1230004db0 comp rx=0 tx=0).stop 2023-11-06T01:31:54.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 -- 172.21.15.163:0/1254355741 shutdown_connections 2023-11-06T01:31:54.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a0520 0x7f12200a29e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:54.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12381018c0 0x7f123819e0f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:54.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12380ffd20 0x7f1238069410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:54.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 --2- 172.21.15.163:0/1254355741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1238100f20 0x7f123819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:54.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 -- 172.21.15.163:0/1254355741 >> 172.21.15.163:0/1254355741 conn(0x7f12380fb2a0 msgr2=0x7f1238104480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:54.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 -- 172.21.15.163:0/1254355741 shutdown_connections 2023-11-06T01:31:54.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-11-06T01:31:54.241+0000 7f123cb74700 1 -- 172.21.15.163:0/1254355741 wait complete. 2023-11-06T01:31:54.604 DEBUG:teuthology.orchestra.run.smithi053:mds.a> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.a.service 2023-11-06T01:31:54.608 DEBUG:teuthology.orchestra.run.smithi053:mds.d> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.d.service 2023-11-06T01:31:54.645 INFO:journalctl@ceph.mds.a.smithi053.stdout:-- Logs begin at Mon 2023-11-06 01:09:30 UTC. -- 2023-11-06T01:31:54.646 DEBUG:teuthology.orchestra.run.smithi053:mds.g> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.g.service 2023-11-06T01:31:54.676 INFO:journalctl@ceph.mds.d.smithi053.stdout:-- Logs begin at Mon 2023-11-06 01:09:30 UTC. -- 2023-11-06T01:31:54.677 DEBUG:teuthology.orchestra.run.smithi053:mds.j> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.j.service 2023-11-06T01:31:54.706 INFO:journalctl@ceph.mds.g.smithi053.stdout:-- Logs begin at Mon 2023-11-06 01:09:30 UTC. -- 2023-11-06T01:31:54.707 DEBUG:teuthology.orchestra.run.smithi062:mds.b> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.b.service 2023-11-06T01:31:54.710 DEBUG:teuthology.orchestra.run.smithi062:mds.e> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.e.service 2023-11-06T01:31:54.734 INFO:journalctl@ceph.mds.j.smithi053.stdout:-- Logs begin at Mon 2023-11-06 01:09:30 UTC. -- 2023-11-06T01:31:54.749 INFO:journalctl@ceph.mds.b.smithi062.stdout:-- Logs begin at Mon 2023-11-06 01:10:07 UTC. -- 2023-11-06T01:31:54.750 DEBUG:teuthology.orchestra.run.smithi062:mds.h> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.h.service 2023-11-06T01:31:54.780 INFO:journalctl@ceph.mds.e.smithi062.stdout:-- Logs begin at Mon 2023-11-06 01:10:07 UTC. -- 2023-11-06T01:31:54.781 DEBUG:teuthology.orchestra.run.smithi062:mds.k> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.k.service 2023-11-06T01:31:54.809 INFO:journalctl@ceph.mds.h.smithi062.stdout:-- Logs begin at Mon 2023-11-06 01:10:07 UTC. -- 2023-11-06T01:31:54.810 DEBUG:teuthology.orchestra.run.smithi163:mds.c> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.c.service 2023-11-06T01:31:54.813 DEBUG:teuthology.orchestra.run.smithi163:mds.f> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.f.service 2023-11-06T01:31:54.838 INFO:journalctl@ceph.mds.k.smithi062.stdout:-- Logs begin at Mon 2023-11-06 01:10:07 UTC. -- 2023-11-06T01:31:54.841 INFO:journalctl@ceph.mds.c.smithi163.stdout:-- Logs begin at Mon 2023-11-06 01:10:53 UTC. -- 2023-11-06T01:31:54.842 DEBUG:teuthology.orchestra.run.smithi163:mds.i> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.i.service 2023-11-06T01:31:54.868 INFO:journalctl@ceph.mds.f.smithi163.stdout:-- Logs begin at Mon 2023-11-06 01:10:53 UTC. -- 2023-11-06T01:31:54.869 DEBUG:teuthology.orchestra.run.smithi163:mds.l> sudo journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.l.service 2023-11-06T01:31:54.895 INFO:journalctl@ceph.mds.i.smithi163.stdout:-- Logs begin at Mon 2023-11-06 01:10:53 UTC. -- 2023-11-06T01:31:54.896 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-11-06T01:31:54.897 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:31:54.921 INFO:journalctl@ceph.mds.l.smithi163.stdout:-- Logs begin at Mon 2023-11-06 01:10:53 UTC. -- 2023-11-06T01:31:55.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.039+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3864342062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf0101f80 msgr2=0x7f7bf01023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.039+0000 7f7bf7eb0700 1 --2- 172.21.15.53:0/3864342062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf0101f80 0x7f7bf01023e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7be00099c0 tx=0x7f7be0009cd0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.039+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3864342062 shutdown_connections 2023-11-06T01:31:55.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.039+0000 7f7bf7eb0700 1 --2- 172.21.15.53:0/3864342062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf0100d90 0x7f7bf01011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.039+0000 7f7bf7eb0700 1 --2- 172.21.15.53:0/3864342062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf0102920 0x7f7bf01091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.039+0000 7f7bf7eb0700 1 --2- 172.21.15.53:0/3864342062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf0101f80 0x7f7bf01023e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.039+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3864342062 >> 172.21.15.53:0/3864342062 conn(0x7f7bf00fc2e0 msgr2=0x7f7bf00fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:55.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.039+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3864342062 shutdown_connections 2023-11-06T01:31:55.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.039+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3864342062 wait complete. 2023-11-06T01:31:55.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf7eb0700 1 Processor -- start 2023-11-06T01:31:55.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf7eb0700 1 -- start start 2023-11-06T01:31:55.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf7eb0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf0100d90 0x7f7bf01969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf7eb0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf0101f80 0x7f7bf0196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf7eb0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf0102920 0x7f7bf019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf7eb0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf019e570 con 0x7f7bf0100d90 2023-11-06T01:31:55.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf7eb0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf019e6b0 con 0x7f7bf0101f80 2023-11-06T01:31:55.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf7eb0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf019e840 con 0x7f7bf0102920 2023-11-06T01:31:55.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf5c4c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf0100d90 0x7f7bf01969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:55.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf5c4c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf0100d90 0x7f7bf01969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37622/0 (socket says 172.21.15.53:37622) 2023-11-06T01:31:55.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf5c4c700 1 -- 172.21.15.53:0/3211023283 learned_addr learned my addr 172.21.15.53:0/3211023283 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:31:55.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf644d700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf0102920 0x7f7bf019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:55.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.040+0000 7f7bf544b700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf0101f80 0x7f7bf0196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:55.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.041+0000 7f7bf5c4c700 1 -- 172.21.15.53:0/3211023283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf0102920 msgr2=0x7f7bf019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.041+0000 7f7bf5c4c700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf0102920 0x7f7bf019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.041+0000 7f7bf5c4c700 1 -- 172.21.15.53:0/3211023283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf0101f80 msgr2=0x7f7bf0196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.041+0000 7f7bf5c4c700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf0101f80 0x7f7bf0196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.041+0000 7f7bf5c4c700 1 -- 172.21.15.53:0/3211023283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be00095f0 con 0x7f7bf0100d90 2023-11-06T01:31:55.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.041+0000 7f7bf5c4c700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf0100d90 0x7f7bf01969f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7be800efc0 tx=0x7f7be800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:55.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.041+0000 7f7be6ffd700 1 -- 172.21.15.53:0/3211023283 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8018070 con 0x7f7bf0100d90 2023-11-06T01:31:55.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.041+0000 7f7be6ffd700 1 -- 172.21.15.53:0/3211023283 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7be800f040 con 0x7f7bf0100d90 2023-11-06T01:31:55.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.041+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3211023283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf019eac0 con 0x7f7bf0100d90 2023-11-06T01:31:55.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.041+0000 7f7be6ffd700 1 -- 172.21.15.53:0/3211023283 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8004280 con 0x7f7bf0100d90 2023-11-06T01:31:55.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.041+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3211023283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf019efd0 con 0x7f7bf0100d90 2023-11-06T01:31:55.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.042+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3211023283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf01082e0 con 0x7f7bf0100d90 2023-11-06T01:31:55.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.043+0000 7f7be6ffd700 1 -- 172.21.15.53:0/3211023283 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f7be8003a00 con 0x7f7bf0100d90 2023-11-06T01:31:55.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.044+0000 7f7be6ffd700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bdc0a0570 0x7f7bdc0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.044+0000 7f7bf544b700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bdc0a0570 0x7f7bdc0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:55.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.044+0000 7f7be6ffd700 1 -- 172.21.15.53:0/3211023283 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7be80cb490 con 0x7f7bf0100d90 2023-11-06T01:31:55.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.044+0000 7f7bf544b700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bdc0a0570 0x7f7bdc0a2a30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7be000b580 tx=0x7f7be0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:55.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.048+0000 7f7be6ffd700 1 -- 172.21.15.53:0/3211023283 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7be80954c0 con 0x7f7bf0100d90 2023-11-06T01:31:55.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.253+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3211023283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bf00619a0 con 0x7f7bf0100d90 2023-11-06T01:31:55.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.253+0000 7f7be6ffd700 1 -- 172.21.15.53:0/3211023283 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f7be8098ae0 con 0x7f7bf0100d90 2023-11-06T01:31:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:31:55.256 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:31:46.410396+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:31:55.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3211023283 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bdc0a0570 msgr2=0x7f7bdc0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bdc0a0570 0x7f7bdc0a2a30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7be000b580 tx=0x7f7be0005fb0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3211023283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf0100d90 msgr2=0x7f7bf01969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf0100d90 0x7f7bf01969f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7be800efc0 tx=0x7f7be800c3a0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3211023283 shutdown_connections 2023-11-06T01:31:55.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bdc0a0570 0x7f7bdc0a2a30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf0102920 0x7f7bf019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf0100d90 0x7f7bf01969f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 --2- 172.21.15.53:0/3211023283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf0101f80 0x7f7bf0196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3211023283 >> 172.21.15.53:0/3211023283 conn(0x7f7bf00fc2e0 msgr2=0x7f7bf0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:55.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3211023283 shutdown_connections 2023-11-06T01:31:55.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.256+0000 7f7bf7eb0700 1 -- 172.21.15.53:0/3211023283 wait complete. 2023-11-06T01:31:55.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:31:55.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T01:31:55.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.400+0000 7fc73cf9a700 1 -- 172.21.15.53:0/1384790673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380ffcb0 msgr2=0x7fc7381000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.400+0000 7fc73cf9a700 1 --2- 172.21.15.53:0/1384790673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380ffcb0 0x7fc7381000d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc7280099c0 tx=0x7fc728009cd0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.400+0000 7fc73cf9a700 1 -- 172.21.15.53:0/1384790673 shutdown_connections 2023-11-06T01:31:55.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.400+0000 7fc73cf9a700 1 --2- 172.21.15.53:0/1384790673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc738101840 0x7fc7381080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.400+0000 7fc73cf9a700 1 --2- 172.21.15.53:0/1384790673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380ffcb0 0x7fc7381000d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.400+0000 7fc73cf9a700 1 --2- 172.21.15.53:0/1384790673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc738100ea0 0x7fc738101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.400+0000 7fc73cf9a700 1 -- 172.21.15.53:0/1384790673 >> 172.21.15.53:0/1384790673 conn(0x7fc7380fb220 msgr2=0x7fc7380fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:55.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.400+0000 7fc73cf9a700 1 -- 172.21.15.53:0/1384790673 shutdown_connections 2023-11-06T01:31:55.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.400+0000 7fc73cf9a700 1 -- 172.21.15.53:0/1384790673 wait complete. 2023-11-06T01:31:55.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.400+0000 7fc73cf9a700 1 Processor -- start 2023-11-06T01:31:55.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.400+0000 7fc73cf9a700 1 -- start start 2023-11-06T01:31:55.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc73cf9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7380ffcb0 0x7fc738195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc73cf9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc738100ea0 0x7fc738195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc73cf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc738101840 0x7fc73819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc73cf9a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73819d4f0 con 0x7fc738100ea0 2023-11-06T01:31:55.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc73cf9a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73819d630 con 0x7fc738101840 2023-11-06T01:31:55.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc73cf9a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73819d7c0 con 0x7fc7380ffcb0 2023-11-06T01:31:55.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc735d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc738100ea0 0x7fc738195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:55.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc735d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc738100ea0 0x7fc738195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37638/0 (socket says 172.21.15.53:37638) 2023-11-06T01:31:55.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc735d9b700 1 -- 172.21.15.53:0/2959288163 learned_addr learned my addr 172.21.15.53:0/2959288163 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:31:55.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc73659c700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7380ffcb0 0x7fc738195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:55.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc735d9b700 1 -- 172.21.15.53:0/2959288163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7380ffcb0 msgr2=0x7fc738195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc735d9b700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7380ffcb0 0x7fc738195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc735d9b700 1 -- 172.21.15.53:0/2959288163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc738101840 msgr2=0x7fc73819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc735d9b700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc738101840 0x7fc73819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc735d9b700 1 -- 172.21.15.53:0/2959288163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7280095f0 con 0x7fc738100ea0 2023-11-06T01:31:55.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc735d9b700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc738100ea0 0x7fc738195eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00d950 tx=0x7fc72c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:55.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc7277fe700 1 -- 172.21.15.53:0/2959288163 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c009b40 con 0x7fc738100ea0 2023-11-06T01:31:55.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc73cf9a700 1 -- 172.21.15.53:0/2959288163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73819daa0 con 0x7fc738100ea0 2023-11-06T01:31:55.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc7277fe700 1 -- 172.21.15.53:0/2959288163 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc72c009ca0 con 0x7fc738100ea0 2023-11-06T01:31:55.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc7277fe700 1 -- 172.21.15.53:0/2959288163 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00f620 con 0x7fc738100ea0 2023-11-06T01:31:55.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.401+0000 7fc73cf9a700 1 -- 172.21.15.53:0/2959288163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73819dfb0 con 0x7fc738100ea0 2023-11-06T01:31:55.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.402+0000 7fc73cf9a700 1 -- 172.21.15.53:0/2959288163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc738106be0 con 0x7fc738100ea0 2023-11-06T01:31:55.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.404+0000 7fc7277fe700 1 -- 172.21.15.53:0/2959288163 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fc72c010460 con 0x7fc738100ea0 2023-11-06T01:31:55.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.404+0000 7fc7277fe700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7200a0520 0x7fc7200a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.405+0000 7fc73659c700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7200a0520 0x7fc7200a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:55.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.405+0000 7fc7277fe700 1 -- 172.21.15.53:0/2959288163 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc72c0cb890 con 0x7fc738100ea0 2023-11-06T01:31:55.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.405+0000 7fc73659c700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7200a0520 0x7fc7200a29e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc72800b580 tx=0x7fc728005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:55.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.408+0000 7fc7277fe700 1 -- 172.21.15.53:0/2959288163 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc72c0958d0 con 0x7fc738100ea0 2023-11-06T01:31:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:31:55.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: Saving service mds.all spec with placement smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2023-11-06T01:31:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:55.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:55.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:31:55.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:31:55.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:55.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:55 smithi163 ceph-mon[104416]: Deploying daemon mds.l on smithi163 2023-11-06T01:31:55.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:55.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:31:55.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: Saving service mds.all spec with placement smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2023-11-06T01:31:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:55.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:55.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:31:55.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:31:55.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:55.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:55 smithi053 ceph-mon[99472]: Deploying daemon mds.l on smithi163 2023-11-06T01:31:55.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.608+0000 7fc73cf9a700 1 -- 172.21.15.53:0/2959288163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc73802cea0 con 0x7fc738100ea0 2023-11-06T01:31:55.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.609+0000 7fc7277fe700 1 -- 172.21.15.53:0/2959288163 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc72c098ef0 con 0x7fc738100ea0 2023-11-06T01:31:55.611 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:31:55.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 -- 172.21.15.53:0/2959288163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7200a0520 msgr2=0x7fc7200a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7200a0520 0x7fc7200a29e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc72800b580 tx=0x7fc728005fb0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 -- 172.21.15.53:0/2959288163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc738100ea0 msgr2=0x7fc738195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc738100ea0 0x7fc738195eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00d950 tx=0x7fc72c00dd10 comp rx=0 tx=0).stop 2023-11-06T01:31:55.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 -- 172.21.15.53:0/2959288163 shutdown_connections 2023-11-06T01:31:55.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7200a0520 0x7fc7200a29e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc738101840 0x7fc73819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7380ffcb0 0x7fc738195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 --2- 172.21.15.53:0/2959288163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc738100ea0 0x7fc738195eb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 -- 172.21.15.53:0/2959288163 >> 172.21.15.53:0/2959288163 conn(0x7fc7380fb220 msgr2=0x7fc7380fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:55.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 -- 172.21.15.53:0/2959288163 shutdown_connections 2023-11-06T01:31:55.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.611+0000 7fc73cf9a700 1 -- 172.21.15.53:0/2959288163 wait complete. 2023-11-06T01:31:55.625 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:31:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:31:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: Saving service mds.all spec with placement smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2023-11-06T01:31:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:31:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:31:55.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:55.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:31:55.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:31:55.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:55.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:55 smithi062 ceph-mon[103924]: Deploying daemon mds.l on smithi163 2023-11-06T01:31:55.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.749+0000 7fc0c25c4700 1 -- 172.21.15.53:0/196724606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0bc101800 msgr2=0x7fc0bc1080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.749+0000 7fc0c25c4700 1 --2- 172.21.15.53:0/196724606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0bc101800 0x7fc0bc1080b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400b0d0 tx=0x7fc0b400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.749+0000 7fc0c25c4700 1 -- 172.21.15.53:0/196724606 shutdown_connections 2023-11-06T01:31:55.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.749+0000 7fc0c25c4700 1 --2- 172.21.15.53:0/196724606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0bc101800 0x7fc0bc1080b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.749+0000 7fc0c25c4700 1 --2- 172.21.15.53:0/196724606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0bc0ffc70 0x7fc0bc100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.749+0000 7fc0c25c4700 1 --2- 172.21.15.53:0/196724606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc100e60 0x7fc0bc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.749+0000 7fc0c25c4700 1 -- 172.21.15.53:0/196724606 >> 172.21.15.53:0/196724606 conn(0x7fc0bc0fb1c0 msgr2=0x7fc0bc0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:55.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.749+0000 7fc0c25c4700 1 -- 172.21.15.53:0/196724606 shutdown_connections 2023-11-06T01:31:55.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.749+0000 7fc0c25c4700 1 -- 172.21.15.53:0/196724606 wait complete. 2023-11-06T01:31:55.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0c25c4700 1 Processor -- start 2023-11-06T01:31:55.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0c25c4700 1 -- start start 2023-11-06T01:31:55.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0c25c4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc0ffc70 0x7fc0bc068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0c25c4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0bc100e60 0x7fc0bc19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0c25c4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0bc101800 0x7fc0bc1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0c25c4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc1a0840 con 0x7fc0bc100e60 2023-11-06T01:31:55.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0c25c4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc1a09b0 con 0x7fc0bc0ffc70 2023-11-06T01:31:55.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0c25c4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc1a0b70 con 0x7fc0bc101800 2023-11-06T01:31:55.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0bb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0bc100e60 0x7fc0bc19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:55.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0bb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0bc100e60 0x7fc0bc19dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37652/0 (socket says 172.21.15.53:37652) 2023-11-06T01:31:55.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0bb7fe700 1 -- 172.21.15.53:0/2219435102 learned_addr learned my addr 172.21.15.53:0/2219435102 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:31:55.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0c0b61700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0bc101800 0x7fc0bc1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:55.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0bbfff700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc0ffc70 0x7fc0bc068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:55.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0bb7fe700 1 -- 172.21.15.53:0/2219435102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0bc101800 msgr2=0x7fc0bc1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0bb7fe700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0bc101800 0x7fc0bc1a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0bb7fe700 1 -- 172.21.15.53:0/2219435102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc0ffc70 msgr2=0x7fc0bc068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:55.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0bb7fe700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc0ffc70 0x7fc0bc068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:55.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0bb7fe700 1 -- 172.21.15.53:0/2219435102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b4009d00 con 0x7fc0bc100e60 2023-11-06T01:31:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0c0b61700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0bc101800 0x7fc0bc1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:31:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.750+0000 7fc0bbfff700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc0ffc70 0x7fc0bc068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:31:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.751+0000 7fc0bb7fe700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0bc100e60 0x7fc0bc19dc40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000d980 tx=0x7fc0b000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.751+0000 7fc0b97fa700 1 -- 172.21.15.53:0/2219435102 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000bb60 con 0x7fc0bc100e60 2023-11-06T01:31:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.751+0000 7fc0b97fa700 1 -- 172.21.15.53:0/2219435102 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b0007640 con 0x7fc0bc100e60 2023-11-06T01:31:55.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.751+0000 7fc0c25c4700 1 -- 172.21.15.53:0/2219435102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0bc1a0e50 con 0x7fc0bc100e60 2023-11-06T01:31:55.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.751+0000 7fc0b97fa700 1 -- 172.21.15.53:0/2219435102 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0004e90 con 0x7fc0bc100e60 2023-11-06T01:31:55.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.751+0000 7fc0c25c4700 1 -- 172.21.15.53:0/2219435102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc1a1320 con 0x7fc0bc100e60 2023-11-06T01:31:55.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.752+0000 7fc0c25c4700 1 -- 172.21.15.53:0/2219435102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0bc106b50 con 0x7fc0bc100e60 2023-11-06T01:31:55.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.753+0000 7fc0b97fa700 1 -- 172.21.15.53:0/2219435102 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fc0b000bcc0 con 0x7fc0bc100e60 2023-11-06T01:31:55.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.753+0000 7fc0b97fa700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0a40a05f0 0x7fc0a40a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.753+0000 7fc0bbfff700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0a40a05f0 0x7fc0a40a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.753+0000 7fc0b97fa700 1 -- 172.21.15.53:0/2219435102 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc0b00cb320 con 0x7fc0bc100e60 2023-11-06T01:31:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.754+0000 7fc0bbfff700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0a40a05f0 0x7fc0a40a2ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc100cc0 tx=0x7fc0ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:55.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:55.758+0000 7fc0b97fa700 1 -- 172.21.15.53:0/2219435102 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc0b0095360 con 0x7fc0bc100e60 2023-11-06T01:31:55.791 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:31:55 smithi163 systemd[1]: Starting Ceph mds.l for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:31:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.006+0000 7fc0c25c4700 1 -- 172.21.15.53:0/2219435102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0bc02cc70 con 0x7fc0bc100e60 2023-11-06T01:31:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.007+0000 7fc0b97fa700 1 -- 172.21.15.53:0/2219435102 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fc0b0098980 con 0x7fc0bc100e60 2023-11-06T01:31:56.009 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:31:56.010 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-11-06T01:31:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 -- 172.21.15.53:0/2219435102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0a40a05f0 msgr2=0x7fc0a40a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0a40a05f0 0x7fc0a40a2ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc100cc0 tx=0x7fc0ac009260 comp rx=0 tx=0).stop 2023-11-06T01:31:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 -- 172.21.15.53:0/2219435102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0bc100e60 msgr2=0x7fc0bc19dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0bc100e60 0x7fc0bc19dc40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000d980 tx=0x7fc0b000dd40 comp rx=0 tx=0).stop 2023-11-06T01:31:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 -- 172.21.15.53:0/2219435102 shutdown_connections 2023-11-06T01:31:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0a40a05f0 0x7fc0a40a2ab0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc0ffc70 0x7fc0bc068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0bc101800 0x7fc0bc1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 --2- 172.21.15.53:0/2219435102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0bc100e60 0x7fc0bc19dc40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 -- 172.21.15.53:0/2219435102 >> 172.21.15.53:0/2219435102 conn(0x7fc0bc0fb1c0 msgr2=0x7fc0bc0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 -- 172.21.15.53:0/2219435102 shutdown_connections 2023-11-06T01:31:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:56.009+0000 7fc0c25c4700 1 -- 172.21.15.53:0/2219435102 wait complete. 2023-11-06T01:31:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 1 2023-11-06T01:31:56.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:56 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3211023283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:31:56.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:56 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2959288163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:31:56.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:56 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2219435102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:31:56.379 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:56 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3211023283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:31:56.380 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:56 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2959288163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:31:56.380 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:56 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2219435102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:31:56.380 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:31:56 smithi163 podman[139689]: 2023-11-06T01:31:56.380 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:31:56 smithi163 bash[139689]: 3991491d8cbf9ad85ae5e2de45a11b5440ae6a3477a19e848c2127fdc73d190c 2023-11-06T01:31:56.381 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:31:56 smithi163 ceph-mds[139729]: -- 172.21.15.163:0/2569103591 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b346ff51e0 con 0x55b346fcb000 2023-11-06T01:31:56.381 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:31:56 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-l[139705]: starting mds.l at 2023-11-06T01:31:56.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:56 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3211023283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:31:56.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:56 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2959288163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:31:56.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:56 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2219435102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:31:56.739 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:31:56 smithi163 systemd[1]: Started Ceph mds.l for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:31:57.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:57 smithi062 ceph-mon[103924]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:57.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:57 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.163:6832/731722016,v1:172.21.15.163:6833/731722016] up:boot 2023-11-06T01:31:57.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:57 smithi062 ceph-mon[103924]: fsmap 1 up:standby 2023-11-06T01:31:57.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-06T01:31:57.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:57.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:57.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:57.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:31:57.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:31:57.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:57.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:57 smithi053 ceph-mon[99472]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:57.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:57 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.163:6832/731722016,v1:172.21.15.163:6833/731722016] up:boot 2023-11-06T01:31:57.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:57 smithi053 ceph-mon[99472]: fsmap 1 up:standby 2023-11-06T01:31:57.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-06T01:31:57.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:57.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:57.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:57.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:31:57.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:31:57.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:57.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:57 smithi163 ceph-mon[104416]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:57.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:57 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.163:6832/731722016,v1:172.21.15.163:6833/731722016] up:boot 2023-11-06T01:31:57.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:57 smithi163 ceph-mon[104416]: fsmap 1 up:standby 2023-11-06T01:31:57.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-06T01:31:57.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:57.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:57.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:57.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:31:57.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:31:57.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:58.024 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:31:58.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.163+0000 7fbd419d6700 1 -- 172.21.15.53:0/3974058264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd3c101800 msgr2=0x7fbd3c1080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:58.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.163+0000 7fbd419d6700 1 --2- 172.21.15.53:0/3974058264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd3c101800 0x7fbd3c1080b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400b0d0 tx=0x7fbd3400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.163+0000 7fbd419d6700 1 -- 172.21.15.53:0/3974058264 shutdown_connections 2023-11-06T01:31:58.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.163+0000 7fbd419d6700 1 --2- 172.21.15.53:0/3974058264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd3c101800 0x7fbd3c1080b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.163+0000 7fbd419d6700 1 --2- 172.21.15.53:0/3974058264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd3c0ffc70 0x7fbd3c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.163+0000 7fbd419d6700 1 --2- 172.21.15.53:0/3974058264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c100e60 0x7fbd3c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.163+0000 7fbd419d6700 1 -- 172.21.15.53:0/3974058264 >> 172.21.15.53:0/3974058264 conn(0x7fbd3c0fb1c0 msgr2=0x7fbd3c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:58.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.163+0000 7fbd419d6700 1 -- 172.21.15.53:0/3974058264 shutdown_connections 2023-11-06T01:31:58.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.163+0000 7fbd419d6700 1 -- 172.21.15.53:0/3974058264 wait complete. 2023-11-06T01:31:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd419d6700 1 Processor -- start 2023-11-06T01:31:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd419d6700 1 -- start start 2023-11-06T01:31:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd419d6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c0ffc70 0x7fbd3c103dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd419d6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd3c100e60 0x7fbd3c102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd419d6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd3c101800 0x7fbd3c102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd419d6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c1030f0 con 0x7fbd3c100e60 2023-11-06T01:31:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd419d6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c103260 con 0x7fbd3c0ffc70 2023-11-06T01:31:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd419d6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c1a0b10 con 0x7fbd3c101800 2023-11-06T01:31:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd3a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd3c100e60 0x7fbd3c102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd3a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd3c100e60 0x7fbd3c102410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37666/0 (socket says 172.21.15.53:37666) 2023-11-06T01:31:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd3a7fc700 1 -- 172.21.15.53:0/3607548926 learned_addr learned my addr 172.21.15.53:0/3607548926 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:31:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd3affd700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c0ffc70 0x7fbd3c103dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd3b7fe700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd3c101800 0x7fbd3c102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd3a7fc700 1 -- 172.21.15.53:0/3607548926 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd3c101800 msgr2=0x7fbd3c102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:58.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd3a7fc700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd3c101800 0x7fbd3c102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd3a7fc700 1 -- 172.21.15.53:0/3607548926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c0ffc70 msgr2=0x7fbd3c103dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd3a7fc700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c0ffc70 0x7fbd3c103dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.164+0000 7fbd3a7fc700 1 -- 172.21.15.53:0/3607548926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd34009d00 con 0x7fbd3c100e60 2023-11-06T01:31:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.165+0000 7fbd3b7fe700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd3c101800 0x7fbd3c102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:31:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.165+0000 7fbd3a7fc700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd3c100e60 0x7fbd3c102410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400e9a0 tx=0x7fbd2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.165+0000 7fbd409d4700 1 -- 172.21.15.53:0/3607548926 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24009790 con 0x7fbd3c100e60 2023-11-06T01:31:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.165+0000 7fbd419d6700 1 -- 172.21.15.53:0/3607548926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c1a0df0 con 0x7fbd3c100e60 2023-11-06T01:31:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.165+0000 7fbd409d4700 1 -- 172.21.15.53:0/3607548926 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd2400cbd0 con 0x7fbd3c100e60 2023-11-06T01:31:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.165+0000 7fbd409d4700 1 -- 172.21.15.53:0/3607548926 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24018680 con 0x7fbd3c100e60 2023-11-06T01:31:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.165+0000 7fbd419d6700 1 -- 172.21.15.53:0/3607548926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c1a1270 con 0x7fbd3c100e60 2023-11-06T01:31:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.166+0000 7fbd419d6700 1 -- 172.21.15.53:0/3607548926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3c107c10 con 0x7fbd3c100e60 2023-11-06T01:31:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.167+0000 7fbd409d4700 1 -- 172.21.15.53:0/3607548926 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fbd2400cd40 con 0x7fbd3c100e60 2023-11-06T01:31:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.168+0000 7fbd409d4700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbd280a0570 0x7fbd280a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:31:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.168+0000 7fbd3affd700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbd280a0570 0x7fbd280a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:31:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.168+0000 7fbd409d4700 1 -- 172.21.15.53:0/3607548926 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbd24014070 con 0x7fbd3c100e60 2023-11-06T01:31:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.168+0000 7fbd3affd700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbd280a0570 0x7fbd280a2a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c100cc0 tx=0x7fbd2c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:31:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.171+0000 7fbd409d4700 1 -- 172.21.15.53:0/3607548926 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbd24095ad0 con 0x7fbd3c100e60 2023-11-06T01:31:58.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:58 smithi062 ceph-mon[103924]: Deploying daemon mds.k on smithi062 2023-11-06T01:31:58.396 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:31:58 smithi062 systemd[1]: Starting Ceph mds.k for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:31:58.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.421+0000 7fbd419d6700 1 -- 172.21.15.53:0/3607548926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd3c02cee0 con 0x7fbd3c100e60 2023-11-06T01:31:58.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.421+0000 7fbd409d4700 1 -- 172.21.15.53:0/3607548926 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7fbd240990f0 con 0x7fbd3c100e60 2023-11-06T01:31:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:31:58.426 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-06T01:31:58.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 -- 172.21.15.53:0/3607548926 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbd280a0570 msgr2=0x7fbd280a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:58.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbd280a0570 0x7fbd280a2a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c100cc0 tx=0x7fbd2c00a3b0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 -- 172.21.15.53:0/3607548926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd3c100e60 msgr2=0x7fbd3c102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:31:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd3c100e60 0x7fbd3c102410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400e9a0 tx=0x7fbd2400ecb0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 -- 172.21.15.53:0/3607548926 shutdown_connections 2023-11-06T01:31:58.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbd280a0570 0x7fbd280a2a30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c0ffc70 0x7fbd3c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd3c101800 0x7fbd3c102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 --2- 172.21.15.53:0/3607548926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd3c100e60 0x7fbd3c102410 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:31:58.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 -- 172.21.15.53:0/3607548926 >> 172.21.15.53:0/3607548926 conn(0x7fbd3c0fb1c0 msgr2=0x7fbd3c0fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:31:58.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 -- 172.21.15.53:0/3607548926 shutdown_connections 2023-11-06T01:31:58.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:31:58.424+0000 7fbd419d6700 1 -- 172.21.15.53:0/3607548926 wait complete. 2023-11-06T01:31:58.432 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 2 2023-11-06T01:31:58.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:58 smithi053 ceph-mon[99472]: Deploying daemon mds.k on smithi062 2023-11-06T01:31:58.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:58 smithi163 ceph-mon[104416]: Deploying daemon mds.k on smithi062 2023-11-06T01:31:58.864 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:31:58 smithi062 podman[138966]: 2023-11-06T01:31:59.145 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:31:58 smithi062 bash[138966]: 13ea49f7a8bb36ea3f8218fabdff40c27b09c8043a1660aa351da01852a6efe1 2023-11-06T01:31:59.145 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:31:58 smithi062 ceph-mds[139006]: -- 172.21.15.62:0/2277493219 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5575cce431e0 con 0x5575cce18800 2023-11-06T01:31:59.145 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:31:58 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-k[138982]: starting mds.k at 2023-11-06T01:31:59.145 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:31:58 smithi062 systemd[1]: Started Ceph mds.k for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:31:59.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:59 smithi053 ceph-mon[99472]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:59.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:59 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3607548926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:31:59.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:59.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:59.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:59.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:31:59.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:31:59.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:31:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:59.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:59 smithi062 ceph-mon[103924]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:59.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:59 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3607548926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:31:59.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:59.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:59.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:59.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:31:59.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:31:59.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:31:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:31:59.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:59 smithi163 ceph-mon[104416]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:31:59.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:59 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3607548926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:31:59.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:59.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:59.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:31:59.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:31:59.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:31:59.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:31:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:00.388 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:00 smithi053 ceph-mon[99472]: Deploying daemon mds.j on smithi053 2023-11-06T01:32:00.388 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:00 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] up:boot 2023-11-06T01:32:00.388 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:00 smithi053 ceph-mon[99472]: fsmap 2 up:standby 2023-11-06T01:32:00.389 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-06T01:32:00.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:32:00.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.601+0000 7f5904940700 1 -- 172.21.15.53:0/1451122805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59001041d0 msgr2=0x7f59001065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:00.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.601+0000 7f5904940700 1 --2- 172.21.15.53:0/1451122805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59001041d0 0x7f59001065c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009990 tx=0x7f58f4009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:00.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.601+0000 7f5904940700 1 -- 172.21.15.53:0/1451122805 shutdown_connections 2023-11-06T01:32:00.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.601+0000 7f5904940700 1 --2- 172.21.15.53:0/1451122805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59001018a0 0x7f5900103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:00.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.601+0000 7f5904940700 1 --2- 172.21.15.53:0/1451122805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5900106b00 0x7f5900108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:00.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.601+0000 7f5904940700 1 --2- 172.21.15.53:0/1451122805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59001041d0 0x7f59001065c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:00.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.601+0000 7f5904940700 1 -- 172.21.15.53:0/1451122805 >> 172.21.15.53:0/1451122805 conn(0x7f59000fb1f0 msgr2=0x7f59000fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:00.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.601+0000 7f5904940700 1 -- 172.21.15.53:0/1451122805 shutdown_connections 2023-11-06T01:32:00.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.601+0000 7f5904940700 1 -- 172.21.15.53:0/1451122805 wait complete. 2023-11-06T01:32:00.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.602+0000 7f5904940700 1 Processor -- start 2023-11-06T01:32:00.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.603+0000 7f5904940700 1 -- start start 2023-11-06T01:32:00.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.603+0000 7f5904940700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59001018a0 0x7f5900196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:00.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.603+0000 7f5904940700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59001041d0 0x7f5900196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:00.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.603+0000 7f5904940700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5900106b00 0x7f590019e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:00.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.603+0000 7f5904940700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590019e610 con 0x7f5900106b00 2023-11-06T01:32:00.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.603+0000 7f5904940700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590019e750 con 0x7f59001018a0 2023-11-06T01:32:00.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.603+0000 7f5904940700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590019e8e0 con 0x7f59001041d0 2023-11-06T01:32:00.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.603+0000 7f58fed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5900106b00 0x7f590019e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:00.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58fed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5900106b00 0x7f590019e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37680/0 (socket says 172.21.15.53:37680) 2023-11-06T01:32:00.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58fed9d700 1 -- 172.21.15.53:0/2373774169 learned_addr learned my addr 172.21.15.53:0/2373774169 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:00.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58fe59c700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59001018a0 0x7f5900196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:00.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58fdd9b700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59001041d0 0x7f5900196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:00.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58fed9d700 1 -- 172.21.15.53:0/2373774169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59001041d0 msgr2=0x7f5900196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:00.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58fed9d700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59001041d0 0x7f5900196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:00.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58fed9d700 1 -- 172.21.15.53:0/2373774169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59001018a0 msgr2=0x7f5900196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:00.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58fed9d700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59001018a0 0x7f5900196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:00.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58fed9d700 1 -- 172.21.15.53:0/2373774169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f40095f0 con 0x7f5900106b00 2023-11-06T01:32:00.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58fdd9b700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59001041d0 0x7f5900196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:32:00.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58fed9d700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5900106b00 0x7f590019e040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f58f800b970 tx=0x7f58f800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:00.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58ef7fe700 1 -- 172.21.15.53:0/2373774169 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f800c930 con 0x7f5900106b00 2023-11-06T01:32:00.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58ef7fe700 1 -- 172.21.15.53:0/2373774169 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f8014440 con 0x7f5900106b00 2023-11-06T01:32:00.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f5904940700 1 -- 172.21.15.53:0/2373774169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590019ebc0 con 0x7f5900106b00 2023-11-06T01:32:00.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f58ef7fe700 1 -- 172.21.15.53:0/2373774169 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f80075d0 con 0x7f5900106b00 2023-11-06T01:32:00.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.604+0000 7f5904940700 1 -- 172.21.15.53:0/2373774169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590019f0d0 con 0x7f5900106b00 2023-11-06T01:32:00.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.605+0000 7f5904940700 1 -- 172.21.15.53:0/2373774169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59000fcd70 con 0x7f5900106b00 2023-11-06T01:32:00.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.606+0000 7f58ef7fe700 1 -- 172.21.15.53:0/2373774169 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f58f80145b0 con 0x7f5900106b00 2023-11-06T01:32:00.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.607+0000 7f58ef7fe700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f58e80a0570 0x7f58e80a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:00.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.607+0000 7f58ef7fe700 1 -- 172.21.15.53:0/2373774169 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f58f8006e40 con 0x7f5900106b00 2023-11-06T01:32:00.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.607+0000 7f58fe59c700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f58e80a0570 0x7f58e80a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:00.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.607+0000 7f58fe59c700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f58e80a0570 0x7f58e80a2a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58f0005950 tx=0x7f58f000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:00.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.611+0000 7f58ef7fe700 1 -- 172.21.15.53:0/2373774169 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f58f8094b70 con 0x7f5900106b00 2023-11-06T01:32:00.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:00 smithi062 ceph-mon[103924]: Deploying daemon mds.j on smithi053 2023-11-06T01:32:00.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:00 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] up:boot 2023-11-06T01:32:00.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:00 smithi062 ceph-mon[103924]: fsmap 2 up:standby 2023-11-06T01:32:00.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:00 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-06T01:32:00.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:00 smithi163 ceph-mon[104416]: Deploying daemon mds.j on smithi053 2023-11-06T01:32:00.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:00 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] up:boot 2023-11-06T01:32:00.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:00 smithi163 ceph-mon[104416]: fsmap 2 up:standby 2023-11-06T01:32:00.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:00 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-06T01:32:00.791 INFO:journalctl@ceph.mds.j.smithi053.stdout:Nov 06 01:32:00 smithi053 systemd[1]: Starting Ceph mds.j for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:32:00.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.869+0000 7f5904940700 1 -- 172.21.15.53:0/2373774169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f590002cde0 con 0x7f5900106b00 2023-11-06T01:32:00.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.870+0000 7f58ef7fe700 1 -- 172.21.15.53:0/2373774169 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2011 (secure 0 0 0) 0x7f58f8098190 con 0x7f5900106b00 2023-11-06T01:32:00.872 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:00.873 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-06T01:32:00.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.872+0000 7f5904940700 1 -- 172.21.15.53:0/2373774169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f58e80a0570 msgr2=0x7f58e80a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:00.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.872+0000 7f5904940700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f58e80a0570 0x7f58e80a2a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58f0005950 tx=0x7f58f000a300 comp rx=0 tx=0).stop 2023-11-06T01:32:00.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.872+0000 7f5904940700 1 -- 172.21.15.53:0/2373774169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5900106b00 msgr2=0x7f590019e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:00.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.872+0000 7f5904940700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5900106b00 0x7f590019e040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f58f800b970 tx=0x7f58f800bc80 comp rx=0 tx=0).stop 2023-11-06T01:32:00.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.872+0000 7f5904940700 1 -- 172.21.15.53:0/2373774169 shutdown_connections 2023-11-06T01:32:00.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.872+0000 7f5904940700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f58e80a0570 0x7f58e80a2a30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:00.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.872+0000 7f5904940700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59001018a0 0x7f5900196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:00.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.872+0000 7f5904940700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5900106b00 0x7f590019e040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:00.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.872+0000 7f5904940700 1 --2- 172.21.15.53:0/2373774169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59001041d0 0x7f5900196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:00.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.872+0000 7f5904940700 1 -- 172.21.15.53:0/2373774169 >> 172.21.15.53:0/2373774169 conn(0x7f59000fb1f0 msgr2=0x7f5900102510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:00.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.874+0000 7f5904940700 1 -- 172.21.15.53:0/2373774169 shutdown_connections 2023-11-06T01:32:00.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:00.874+0000 7f5904940700 1 -- 172.21.15.53:0/2373774169 wait complete. 2023-11-06T01:32:00.877 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 3 2023-11-06T01:32:01.088 INFO:journalctl@ceph.mds.j.smithi053.stdout:Nov 06 01:32:00 smithi053 podman[151242]: 2023-11-06T01:32:01.088 INFO:journalctl@ceph.mds.j.smithi053.stdout:Nov 06 01:32:01 smithi053 bash[151242]: add7cb9622f10c6f4a98a91713b5ea67ede1c58e927c3d21d6f5e4d24fbfa5ff 2023-11-06T01:32:01.088 INFO:journalctl@ceph.mds.j.smithi053.stdout:Nov 06 01:32:01 smithi053 ceph-mds[151284]: -- 172.21.15.53:0/3351495650 <== mon.0 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fa464d31e0 con 0x55fa464a8c00 2023-11-06T01:32:01.088 INFO:journalctl@ceph.mds.j.smithi053.stdout:Nov 06 01:32:01 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-j[151258]: starting mds.j at 2023-11-06T01:32:01.338 INFO:journalctl@ceph.mds.j.smithi053.stdout:Nov 06 01:32:01 smithi053 systemd[1]: Started Ceph mds.j for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:32:01.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:01 smithi053 ceph-mon[99472]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:01.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:01 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2373774169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:01.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:01.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:01.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:01.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:01.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:01.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:01.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:01 smithi062 ceph-mon[103924]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:01.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:01 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2373774169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:01.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:01.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:01.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:01.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:01.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:01.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:01.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:01 smithi163 ceph-mon[104416]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:01.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:01 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2373774169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:01.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:01.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:01.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:01.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:01.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:01.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:02.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:02 smithi062 ceph-mon[103924]: Deploying daemon mds.d on smithi053 2023-11-06T01:32:02.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:02 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.53:6834/275572743,v1:172.21.15.53:6835/275572743] up:boot 2023-11-06T01:32:02.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:02 smithi062 ceph-mon[103924]: fsmap 3 up:standby 2023-11-06T01:32:02.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:02 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-06T01:32:02.658 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:02 smithi053 ceph-mon[99472]: Deploying daemon mds.d on smithi053 2023-11-06T01:32:02.658 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:02 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.53:6834/275572743,v1:172.21.15.53:6835/275572743] up:boot 2023-11-06T01:32:02.659 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:02 smithi053 ceph-mon[99472]: fsmap 3 up:standby 2023-11-06T01:32:02.659 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:02 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-06T01:32:02.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:02 smithi163 ceph-mon[104416]: Deploying daemon mds.d on smithi053 2023-11-06T01:32:02.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:02 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.53:6834/275572743,v1:172.21.15.53:6835/275572743] up:boot 2023-11-06T01:32:02.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:02 smithi163 ceph-mon[104416]: fsmap 3 up:standby 2023-11-06T01:32:02.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:02 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-06T01:32:02.891 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:32:02.920 INFO:journalctl@ceph.mds.d.smithi053.stdout:Nov 06 01:32:02 smithi053 systemd[1]: Starting Ceph mds.d for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:32:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.031+0000 7f14f2c34700 1 -- 172.21.15.53:0/3707357927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14ec100e80 msgr2=0x7f14ec1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.031+0000 7f14f2c34700 1 --2- 172.21.15.53:0/3707357927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14ec100e80 0x7f14ec1012e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f14d80099c0 tx=0x7f14d8009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.031+0000 7f14f2c34700 1 -- 172.21.15.53:0/3707357927 shutdown_connections 2023-11-06T01:32:03.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.031+0000 7f14f2c34700 1 --2- 172.21.15.53:0/3707357927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ec101820 0x7f14ec1080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.031+0000 7f14f2c34700 1 --2- 172.21.15.53:0/3707357927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14ec0ffc90 0x7f14ec1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.031+0000 7f14f2c34700 1 --2- 172.21.15.53:0/3707357927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14ec100e80 0x7f14ec1012e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.031+0000 7f14f2c34700 1 -- 172.21.15.53:0/3707357927 >> 172.21.15.53:0/3707357927 conn(0x7f14ec0fb1c0 msgr2=0x7f14ec0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:03.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.031+0000 7f14f2c34700 1 -- 172.21.15.53:0/3707357927 shutdown_connections 2023-11-06T01:32:03.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.031+0000 7f14f2c34700 1 -- 172.21.15.53:0/3707357927 wait complete. 2023-11-06T01:32:03.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f2c34700 1 Processor -- start 2023-11-06T01:32:03.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f2c34700 1 -- start start 2023-11-06T01:32:03.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f2c34700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14ec0ffc90 0x7f14ec069a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:03.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f2c34700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ec100e80 0x7f14ec19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:03.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f2c34700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14ec101820 0x7f14ec19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:03.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f2c34700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec19e590 con 0x7f14ec101820 2023-11-06T01:32:03.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f2c34700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec19e700 con 0x7f14ec100e80 2023-11-06T01:32:03.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f2c34700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec19e8c0 con 0x7f14ec0ffc90 2023-11-06T01:32:03.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f11d1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14ec101820 0x7f14ec19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:03.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f11d1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14ec101820 0x7f14ec19ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37702/0 (socket says 172.21.15.53:37702) 2023-11-06T01:32:03.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f11d1700 1 -- 172.21.15.53:0/2159202292 learned_addr learned my addr 172.21.15.53:0/2159202292 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:03.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14ebfff700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ec100e80 0x7f14ec19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:03.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f09d0700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14ec0ffc90 0x7f14ec069a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:03.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f11d1700 1 -- 172.21.15.53:0/2159202292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14ec0ffc90 msgr2=0x7f14ec069a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:03.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f11d1700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14ec0ffc90 0x7f14ec069a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.032+0000 7f14f11d1700 1 -- 172.21.15.53:0/2159202292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ec100e80 msgr2=0x7f14ec19b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:03.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.033+0000 7f14f11d1700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ec100e80 0x7f14ec19b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.033+0000 7f14f11d1700 1 -- 172.21.15.53:0/2159202292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d80095f0 con 0x7f14ec101820 2023-11-06T01:32:03.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.033+0000 7f14f11d1700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14ec101820 0x7f14ec19ddc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14e400ef30 tx=0x7f14e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:03.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.033+0000 7f14e9ffb700 1 -- 172.21.15.53:0/2159202292 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4018070 con 0x7f14ec101820 2023-11-06T01:32:03.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.033+0000 7f14e9ffb700 1 -- 172.21.15.53:0/2159202292 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14e400f040 con 0x7f14ec101820 2023-11-06T01:32:03.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.033+0000 7f14f2c34700 1 -- 172.21.15.53:0/2159202292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec19eb40 con 0x7f14ec101820 2023-11-06T01:32:03.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.033+0000 7f14e9ffb700 1 -- 172.21.15.53:0/2159202292 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e400cdb0 con 0x7f14ec101820 2023-11-06T01:32:03.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.033+0000 7f14f2c34700 1 -- 172.21.15.53:0/2159202292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec19efd0 con 0x7f14ec101820 2023-11-06T01:32:03.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.034+0000 7f14f2c34700 1 -- 172.21.15.53:0/2159202292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec10b4f0 con 0x7f14ec101820 2023-11-06T01:32:03.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.036+0000 7f14e9ffb700 1 -- 172.21.15.53:0/2159202292 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f14e4003a00 con 0x7f14ec101820 2023-11-06T01:32:03.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.036+0000 7f14e9ffb700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14dc0a0570 0x7f14dc0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:03.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.036+0000 7f14e9ffb700 1 -- 172.21.15.53:0/2159202292 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f14e40cb0d0 con 0x7f14ec101820 2023-11-06T01:32:03.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.036+0000 7f14f09d0700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14dc0a0570 0x7f14dc0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:03.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.037+0000 7f14f09d0700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14dc0a0570 0x7f14dc0a2a30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005950 tx=0x7f14e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:03.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.041+0000 7f14e9ffb700 1 -- 172.21.15.53:0/2159202292 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f14e40951a0 con 0x7f14ec101820 2023-11-06T01:32:03.182 INFO:journalctl@ceph.mds.d.smithi053.stdout:Nov 06 01:32:02 smithi053 podman[151584]: 2023-11-06T01:32:03.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.308+0000 7f14f2c34700 1 -- 172.21.15.53:0/2159202292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14ec106b70 con 0x7f14ec101820 2023-11-06T01:32:03.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.309+0000 7f14e9ffb700 1 -- 172.21.15.53:0/2159202292 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2765 (secure 0 0 0) 0x7f14e40987c0 con 0x7f14ec101820 2023-11-06T01:32:03.311 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:03.311 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-06T01:32:03.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 -- 172.21.15.53:0/2159202292 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14dc0a0570 msgr2=0x7f14dc0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:03.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14dc0a0570 0x7f14dc0a2a30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005950 tx=0x7f14e00058e0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 -- 172.21.15.53:0/2159202292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14ec101820 msgr2=0x7f14ec19ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:03.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14ec101820 0x7f14ec19ddc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14e400ef30 tx=0x7f14e400c3a0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 -- 172.21.15.53:0/2159202292 shutdown_connections 2023-11-06T01:32:03.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14dc0a0570 0x7f14dc0a2a30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14ec101820 0x7f14ec19ddc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14ec0ffc90 0x7f14ec069a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 --2- 172.21.15.53:0/2159202292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ec100e80 0x7f14ec19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:03.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 -- 172.21.15.53:0/2159202292 >> 172.21.15.53:0/2159202292 conn(0x7f14ec0fb1c0 msgr2=0x7f14ec0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:03.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 -- 172.21.15.53:0/2159202292 shutdown_connections 2023-11-06T01:32:03.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:03.311+0000 7f14f2c34700 1 -- 172.21.15.53:0/2159202292 wait complete. 2023-11-06T01:32:03.315 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 4 2023-11-06T01:32:03.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:03 smithi053 ceph-mon[99472]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:03.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:03 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:03.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:03 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:03.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:03 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2159202292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:03.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:03 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:03.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:03 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:03.596 INFO:journalctl@ceph.mds.d.smithi053.stdout:Nov 06 01:32:03 smithi053 bash[151584]: a74c678039c5b10af7cd688204521715bdb8f1c7c1c02c9718fd1f94ddd1e777 2023-11-06T01:32:03.596 INFO:journalctl@ceph.mds.d.smithi053.stdout:Nov 06 01:32:03 smithi053 ceph-mds[151668]: -- 172.21.15.53:0/3929137796 <== mon.0 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b5a30131e0 con 0x55b5a2fe8800 2023-11-06T01:32:03.596 INFO:journalctl@ceph.mds.d.smithi053.stdout:Nov 06 01:32:03 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-d[151623]: starting mds.d at 2023-11-06T01:32:03.596 INFO:journalctl@ceph.mds.d.smithi053.stdout:Nov 06 01:32:03 smithi053 systemd[1]: Started Ceph mds.d for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:32:03.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:03 smithi062 ceph-mon[103924]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:03.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:03 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:03.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:03 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:03.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:03 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2159202292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:03.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:03 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:03.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:03 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:03.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:03 smithi163 ceph-mon[104416]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:03.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:03.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:03.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:03 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2159202292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:03.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:03.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:03 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:04.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:04 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:04.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:04 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:04.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:04 smithi062 ceph-mon[103924]: Deploying daemon mds.e on smithi062 2023-11-06T01:32:04.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:04 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.53:6836/4211608225,v1:172.21.15.53:6837/4211608225] up:boot 2023-11-06T01:32:04.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:04 smithi062 ceph-mon[103924]: fsmap 4 up:standby 2023-11-06T01:32:04.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:04 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-06T01:32:04.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:04 smithi062 ceph-mon[103924]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:04.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:04 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:04.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:04 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:04.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:04 smithi163 ceph-mon[104416]: Deploying daemon mds.e on smithi062 2023-11-06T01:32:04.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:04 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.53:6836/4211608225,v1:172.21.15.53:6837/4211608225] up:boot 2023-11-06T01:32:04.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:04 smithi163 ceph-mon[104416]: fsmap 4 up:standby 2023-11-06T01:32:04.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:04 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-06T01:32:04.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:04 smithi163 ceph-mon[104416]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:04.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:04 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:04.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:04 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:04.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:04 smithi053 ceph-mon[99472]: Deploying daemon mds.e on smithi062 2023-11-06T01:32:04.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:04 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.53:6836/4211608225,v1:172.21.15.53:6837/4211608225] up:boot 2023-11-06T01:32:04.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:04 smithi053 ceph-mon[99472]: fsmap 4 up:standby 2023-11-06T01:32:04.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:04 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-06T01:32:04.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:04 smithi053 ceph-mon[99472]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:05.293 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:32:05 smithi062 systemd[1]: Starting Ceph mds.e for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:32:05.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:32:05.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.466+0000 7f67a8744700 1 -- 172.21.15.53:0/765874090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a00ff0a0 msgr2=0x7f67a00ff4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:05.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.466+0000 7f67a8744700 1 --2- 172.21.15.53:0/765874090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a00ff0a0 0x7f67a00ff4c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f679c009990 tx=0x7f679c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:05.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.466+0000 7f67a8744700 1 -- 172.21.15.53:0/765874090 shutdown_connections 2023-11-06T01:32:05.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.466+0000 7f67a8744700 1 --2- 172.21.15.53:0/765874090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a0102ab0 0x7f67a0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:05.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.466+0000 7f67a8744700 1 --2- 172.21.15.53:0/765874090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a00ff0a0 0x7f67a00ff4c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:05.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.466+0000 7f67a8744700 1 --2- 172.21.15.53:0/765874090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a0102110 0x7f67a0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:05.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.466+0000 7f67a8744700 1 -- 172.21.15.53:0/765874090 >> 172.21.15.53:0/765874090 conn(0x7f67a00fa9b0 msgr2=0x7f67a00fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:05.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.466+0000 7f67a8744700 1 -- 172.21.15.53:0/765874090 shutdown_connections 2023-11-06T01:32:05.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.466+0000 7f67a8744700 1 -- 172.21.15.53:0/765874090 wait complete. 2023-11-06T01:32:05.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.466+0000 7f67a8744700 1 Processor -- start 2023-11-06T01:32:05.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.466+0000 7f67a8744700 1 -- start start 2023-11-06T01:32:05.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a8744700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a00ff0a0 0x7f67a0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:05.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a8744700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a0102110 0x7f67a01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:05.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a8744700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a0102ab0 0x7f67a01a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:05.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a8744700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a01a07b0 con 0x7f67a0102ab0 2023-11-06T01:32:05.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a8744700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a01a0920 con 0x7f67a00ff0a0 2023-11-06T01:32:05.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a8744700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a01a0ae0 con 0x7f67a0102110 2023-11-06T01:32:05.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a6ce1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a0102ab0 0x7f67a01a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:05.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a64e0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a00ff0a0 0x7f67a0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:05.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a6ce1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a0102ab0 0x7f67a01a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37730/0 (socket says 172.21.15.53:37730) 2023-11-06T01:32:05.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a6ce1700 1 -- 172.21.15.53:0/2627702514 learned_addr learned my addr 172.21.15.53:0/2627702514 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:05.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a5cdf700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a0102110 0x7f67a01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:05.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a6ce1700 1 -- 172.21.15.53:0/2627702514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a0102110 msgr2=0x7f67a01991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:05.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a6ce1700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a0102110 0x7f67a01991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:05.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a6ce1700 1 -- 172.21.15.53:0/2627702514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a00ff0a0 msgr2=0x7f67a0198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:05.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a6ce1700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a00ff0a0 0x7f67a0198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:05.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a6ce1700 1 -- 172.21.15.53:0/2627702514 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c0095f0 con 0x7f67a0102ab0 2023-11-06T01:32:05.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a64e0700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a00ff0a0 0x7f67a0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:32:05.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67a6ce1700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a0102ab0 0x7f67a01a01e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f679800b970 tx=0x7f679800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:05.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.467+0000 7f67977fe700 1 -- 172.21.15.53:0/2627702514 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800d860 con 0x7f67a0102ab0 2023-11-06T01:32:05.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.468+0000 7f67a8744700 1 -- 172.21.15.53:0/2627702514 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a01a0dc0 con 0x7f67a0102ab0 2023-11-06T01:32:05.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.468+0000 7f67977fe700 1 -- 172.21.15.53:0/2627702514 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f679800de80 con 0x7f67a0102ab0 2023-11-06T01:32:05.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.468+0000 7f67977fe700 1 -- 172.21.15.53:0/2627702514 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798017400 con 0x7f67a0102ab0 2023-11-06T01:32:05.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.468+0000 7f67a8744700 1 -- 172.21.15.53:0/2627702514 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a01a1240 con 0x7f67a0102ab0 2023-11-06T01:32:05.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.469+0000 7f67a8744700 1 -- 172.21.15.53:0/2627702514 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a010c630 con 0x7f67a0102ab0 2023-11-06T01:32:05.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.470+0000 7f67977fe700 1 -- 172.21.15.53:0/2627702514 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f679800e630 con 0x7f67a0102ab0 2023-11-06T01:32:05.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.470+0000 7f67977fe700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f678c0a0570 0x7f678c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:05.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.471+0000 7f67a64e0700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f678c0a0570 0x7f678c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:05.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.471+0000 7f67977fe700 1 -- 172.21.15.53:0/2627702514 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6798006e40 con 0x7f67a0102ab0 2023-11-06T01:32:05.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.471+0000 7f67a64e0700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f678c0a0570 0x7f678c0a2a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f679c000c00 tx=0x7f679c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:05.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.474+0000 7f67977fe700 1 -- 172.21.15.53:0/2627702514 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6798095330 con 0x7f67a0102ab0 2023-11-06T01:32:05.637 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:32:05 smithi062 podman[139307]: 2023-11-06T01:32:05.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.725+0000 7f67a8744700 1 -- 172.21.15.53:0/2627702514 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67a010c7f0 con 0x7f67a0102ab0 2023-11-06T01:32:05.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.726+0000 7f67977fe700 1 -- 172.21.15.53:0/2627702514 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3522 (secure 0 0 0) 0x7f6798098950 con 0x7f67a0102ab0 2023-11-06T01:32:05.728 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:05.728 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-06T01:32:05.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.728+0000 7f67a8744700 1 -- 172.21.15.53:0/2627702514 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f678c0a0570 msgr2=0x7f678c0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:05.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.728+0000 7f67a8744700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f678c0a0570 0x7f678c0a2a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f679c000c00 tx=0x7f679c005fb0 comp rx=0 tx=0).stop 2023-11-06T01:32:05.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.728+0000 7f67a8744700 1 -- 172.21.15.53:0/2627702514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a0102ab0 msgr2=0x7f67a01a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:05.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.728+0000 7f67a8744700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a0102ab0 0x7f67a01a01e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f679800b970 tx=0x7f679800bc80 comp rx=0 tx=0).stop 2023-11-06T01:32:05.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.728+0000 7f67a8744700 1 -- 172.21.15.53:0/2627702514 shutdown_connections 2023-11-06T01:32:05.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.728+0000 7f67a8744700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f678c0a0570 0x7f678c0a2a30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:05.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.728+0000 7f67a8744700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a00ff0a0 0x7f67a0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:05.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.728+0000 7f67a8744700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a0102ab0 0x7f67a01a01e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:05.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.728+0000 7f67a8744700 1 --2- 172.21.15.53:0/2627702514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a0102110 0x7f67a01991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:05.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.728+0000 7f67a8744700 1 -- 172.21.15.53:0/2627702514 >> 172.21.15.53:0/2627702514 conn(0x7f67a00fa9b0 msgr2=0x7f67a00fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:05.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.728+0000 7f67a8744700 1 -- 172.21.15.53:0/2627702514 shutdown_connections 2023-11-06T01:32:05.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:05.729+0000 7f67a8744700 1 -- 172.21.15.53:0/2627702514 wait complete. 2023-11-06T01:32:05.734 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 5 2023-11-06T01:32:05.895 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:32:05 smithi062 ceph-mds[139348]: -- 172.21.15.62:0/2184281067 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555cb8a4d1e0 con 0x555cb8a22800 2023-11-06T01:32:05.895 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:32:05 smithi062 bash[139307]: c9a617deb6b7db3f681f1829ca396cdc1ce34dd5be256b5f10a432f793043f06 2023-11-06T01:32:05.896 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:32:05 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-e[139323]: starting mds.e at 2023-11-06T01:32:05.896 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:32:05 smithi062 systemd[1]: Started Ceph mds.e for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:32:06.734 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:06 smithi163 ceph-mon[104416]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:06.734 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:06 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2627702514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:06.734 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:06 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:06.735 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:06 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:06.735 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:06 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:06.735 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:06 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:06.735 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:06 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:06.735 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:06 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:06.736 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:06 smithi163 ceph-mon[104416]: Deploying daemon mds.f on smithi163 2023-11-06T01:32:06.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:06 smithi053 ceph-mon[99472]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:06.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:06 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2627702514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:06.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:06 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:06.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:06 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:06.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:06 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:06.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:06 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:06.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:06 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:06.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:06 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:06.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:06 smithi053 ceph-mon[99472]: Deploying daemon mds.f on smithi163 2023-11-06T01:32:06.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:06 smithi062 ceph-mon[103924]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:06.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:06 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2627702514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:06.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:06 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:06.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:06 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:06.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:06 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:06.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:06 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:06.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:06 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:06.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:06 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:06.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:06 smithi062 ceph-mon[103924]: Deploying daemon mds.f on smithi163 2023-11-06T01:32:07.490 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:32:07 smithi163 systemd[1]: Starting Ceph mds.f for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:32:07.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:32:07.790 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:32:07 smithi163 podman[140038]: 2023-11-06T01:32:07.791 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:07 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] up:boot 2023-11-06T01:32:07.791 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:07 smithi163 ceph-mon[104416]: fsmap 5 up:standby 2023-11-06T01:32:07.791 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:07 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-06T01:32:07.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:07 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] up:boot 2023-11-06T01:32:07.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:07 smithi053 ceph-mon[99472]: fsmap 5 up:standby 2023-11-06T01:32:07.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:07 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-06T01:32:07.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.882+0000 7efd39276700 1 -- 172.21.15.53:0/329672736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd340ff7d0 msgr2=0x7efd341093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:07.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.882+0000 7efd39276700 1 --2- 172.21.15.53:0/329672736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd340ff7d0 0x7efd341093a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00b0a0 tx=0x7efd2c00b3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:07.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.882+0000 7efd39276700 1 -- 172.21.15.53:0/329672736 shutdown_connections 2023-11-06T01:32:07.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.882+0000 7efd39276700 1 --2- 172.21.15.53:0/329672736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd340ff7d0 0x7efd341093a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:07.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.882+0000 7efd39276700 1 --2- 172.21.15.53:0/329672736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd341023c0 0x7efd341027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:07.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.882+0000 7efd39276700 1 --2- 172.21.15.53:0/329672736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd340fee30 0x7efd340ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:07.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.882+0000 7efd39276700 1 -- 172.21.15.53:0/329672736 >> 172.21.15.53:0/329672736 conn(0x7efd340fa9d0 msgr2=0x7efd340fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:07.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.882+0000 7efd39276700 1 -- 172.21.15.53:0/329672736 shutdown_connections 2023-11-06T01:32:07.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.882+0000 7efd39276700 1 -- 172.21.15.53:0/329672736 wait complete. 2023-11-06T01:32:07.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd39276700 1 Processor -- start 2023-11-06T01:32:07.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd39276700 1 -- start start 2023-11-06T01:32:07.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd39276700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd340fee30 0x7efd34196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:07.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd39276700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd340ff7d0 0x7efd34197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:07.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd39276700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd341023c0 0x7efd3419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:07.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd39276700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3419e6c0 con 0x7efd340ff7d0 2023-11-06T01:32:07.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd39276700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3419e800 con 0x7efd341023c0 2023-11-06T01:32:07.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd39276700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3419e990 con 0x7efd340fee30 2023-11-06T01:32:07.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd327fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd340ff7d0 0x7efd34197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:07.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd327fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd340ff7d0 0x7efd34197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37742/0 (socket says 172.21.15.53:37742) 2023-11-06T01:32:07.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd327fc700 1 -- 172.21.15.53:0/699486146 learned_addr learned my addr 172.21.15.53:0/699486146 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:07.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd32ffd700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd340fee30 0x7efd34196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:07.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd337fe700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd341023c0 0x7efd3419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:07.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd327fc700 1 -- 172.21.15.53:0/699486146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd340fee30 msgr2=0x7efd34196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:07.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd327fc700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd340fee30 0x7efd34196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:07.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd327fc700 1 -- 172.21.15.53:0/699486146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd341023c0 msgr2=0x7efd3419e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:07.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.883+0000 7efd327fc700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd341023c0 0x7efd3419e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:07.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.884+0000 7efd327fc700 1 -- 172.21.15.53:0/699486146 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c009d00 con 0x7efd340ff7d0 2023-11-06T01:32:07.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.884+0000 7efd337fe700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd341023c0 0x7efd3419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:32:07.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.884+0000 7efd327fc700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd340ff7d0 0x7efd34197080 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efd2800d950 tx=0x7efd2800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:07.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.884+0000 7efd1bfff700 1 -- 172.21.15.53:0/699486146 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28009b40 con 0x7efd340ff7d0 2023-11-06T01:32:07.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.884+0000 7efd39276700 1 -- 172.21.15.53:0/699486146 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3419ec70 con 0x7efd340ff7d0 2023-11-06T01:32:07.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.884+0000 7efd1bfff700 1 -- 172.21.15.53:0/699486146 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd28009ca0 con 0x7efd340ff7d0 2023-11-06T01:32:07.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.884+0000 7efd1bfff700 1 -- 172.21.15.53:0/699486146 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2800f620 con 0x7efd340ff7d0 2023-11-06T01:32:07.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.884+0000 7efd39276700 1 -- 172.21.15.53:0/699486146 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3419f1d0 con 0x7efd340ff7d0 2023-11-06T01:32:07.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.885+0000 7efd39276700 1 -- 172.21.15.53:0/699486146 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd34108400 con 0x7efd340ff7d0 2023-11-06T01:32:07.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.886+0000 7efd1bfff700 1 -- 172.21.15.53:0/699486146 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7efd28010460 con 0x7efd340ff7d0 2023-11-06T01:32:07.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.886+0000 7efd1bfff700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efd1c0a0570 0x7efd1c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:07.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.887+0000 7efd1bfff700 1 -- 172.21.15.53:0/699486146 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efd280cb870 con 0x7efd340ff7d0 2023-11-06T01:32:07.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.887+0000 7efd32ffd700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efd1c0a0570 0x7efd1c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:07.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.887+0000 7efd32ffd700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efd1c0a0570 0x7efd1c0a2a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efd34068eb0 tx=0x7efd2400bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:07.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:07 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] up:boot 2023-11-06T01:32:07.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:07 smithi062 ceph-mon[103924]: fsmap 5 up:standby 2023-11-06T01:32:07.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:07 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-06T01:32:07.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:07.891+0000 7efd1bfff700 1 -- 172.21.15.53:0/699486146 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efd280958b0 con 0x7efd340ff7d0 2023-11-06T01:32:08.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.140+0000 7efd39276700 1 -- 172.21.15.53:0/699486146 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd3402ce00 con 0x7efd340ff7d0 2023-11-06T01:32:08.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.141+0000 7efd1bfff700 1 -- 172.21.15.53:0/699486146 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4279 (secure 0 0 0) 0x7efd28098ed0 con 0x7efd340ff7d0 2023-11-06T01:32:08.143 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:08.143 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-06T01:32:08.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 -- 172.21.15.53:0/699486146 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efd1c0a0570 msgr2=0x7efd1c0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:08.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efd1c0a0570 0x7efd1c0a2a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efd34068eb0 tx=0x7efd2400bef0 comp rx=0 tx=0).stop 2023-11-06T01:32:08.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 -- 172.21.15.53:0/699486146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd340ff7d0 msgr2=0x7efd34197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:08.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd340ff7d0 0x7efd34197080 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efd2800d950 tx=0x7efd2800dd10 comp rx=0 tx=0).stop 2023-11-06T01:32:08.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 -- 172.21.15.53:0/699486146 shutdown_connections 2023-11-06T01:32:08.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efd1c0a0570 0x7efd1c0a2a30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:08.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd341023c0 0x7efd3419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:08.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd340fee30 0x7efd34196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:08.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 --2- 172.21.15.53:0/699486146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd340ff7d0 0x7efd34197080 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:08.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 -- 172.21.15.53:0/699486146 >> 172.21.15.53:0/699486146 conn(0x7efd340fa9d0 msgr2=0x7efd34105540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:08.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 -- 172.21.15.53:0/699486146 shutdown_connections 2023-11-06T01:32:08.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:08.143+0000 7efd39276700 1 -- 172.21.15.53:0/699486146 wait complete. 2023-11-06T01:32:08.147 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 6 2023-11-06T01:32:08.239 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:32:07 smithi163 bash[140038]: 3258437e66a8ab5f9de3f6ada04873165a1d6511687c0c5cee74086d8de86429 2023-11-06T01:32:08.240 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:32:07 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-f[140054]: starting mds.f at 2023-11-06T01:32:08.240 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:32:07 smithi163 systemd[1]: Started Ceph mds.f for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:32:09.141 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:09.141 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:09.142 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:09.142 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:09.142 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:09.142 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:09.142 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:09.143 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: Deploying daemon mds.c on smithi163 2023-11-06T01:32:09.143 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/699486146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:09.143 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.163:6834/392449784,v1:172.21.15.163:6835/392449784] up:boot 2023-11-06T01:32:09.143 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: fsmap 6 up:standby 2023-11-06T01:32:09.143 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:08 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-06T01:32:09.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:09.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:09.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:09.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:09.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:09.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:09.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:09.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: Deploying daemon mds.c on smithi163 2023-11-06T01:32:09.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/699486146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:09.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.163:6834/392449784,v1:172.21.15.163:6835/392449784] up:boot 2023-11-06T01:32:09.147 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: fsmap 6 up:standby 2023-11-06T01:32:09.147 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:08 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-06T01:32:09.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:09.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:09.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:09.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:09.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:09.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:09.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:09.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: Deploying daemon mds.c on smithi163 2023-11-06T01:32:09.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/699486146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:09.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.163:6834/392449784,v1:172.21.15.163:6835/392449784] up:boot 2023-11-06T01:32:09.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: fsmap 6 up:standby 2023-11-06T01:32:09.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:08 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-06T01:32:09.490 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:32:09 smithi163 systemd[1]: Starting Ceph mds.c for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:32:09.932 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:32:09 smithi163 podman[140378]: 2023-11-06T01:32:09.932 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:32:09 smithi163 ceph-mds[140418]: -- 172.21.15.163:0/1557492198 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dfb97a51e0 con 0x55dfb977ac00 2023-11-06T01:32:09.932 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:32:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-c[140394]: starting mds.c at 2023-11-06T01:32:10.158 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:32:10.239 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:32:09 smithi163 bash[140378]: 90aa7c916ad93d74f9231990fbecf3df64d2d166e783adfe478b71fc0f68c1a3 2023-11-06T01:32:10.240 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:32:09 smithi163 systemd[1]: Started Ceph mds.c for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:32:10.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.297+0000 7f4fa1853700 1 -- 172.21.15.53:0/1653034177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f9c1009b0 msgr2=0x7f4f9c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:10.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.297+0000 7f4fa1853700 1 --2- 172.21.15.53:0/1653034177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f9c1009b0 0x7f4f9c100e30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400b0a0 tx=0x7f4f9400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:10.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.297+0000 7f4fa1853700 1 -- 172.21.15.53:0/1653034177 shutdown_connections 2023-11-06T01:32:10.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.297+0000 7f4fa1853700 1 --2- 172.21.15.53:0/1653034177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f9c1009b0 0x7f4f9c100e30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:10.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.297+0000 7f4fa1853700 1 --2- 172.21.15.53:0/1653034177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f9c0ff690 0x7f4f9c0ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:10.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.297+0000 7f4fa1853700 1 --2- 172.21.15.53:0/1653034177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9c0ffff0 0x7f4f9c100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:10.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.297+0000 7f4fa1853700 1 -- 172.21.15.53:0/1653034177 >> 172.21.15.53:0/1653034177 conn(0x7f4f9c0fb1f0 msgr2=0x7f4f9c0fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:10.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.297+0000 7f4fa1853700 1 -- 172.21.15.53:0/1653034177 shutdown_connections 2023-11-06T01:32:10.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.297+0000 7f4fa1853700 1 -- 172.21.15.53:0/1653034177 wait complete. 2023-11-06T01:32:10.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4fa1853700 1 Processor -- start 2023-11-06T01:32:10.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4fa1853700 1 -- start start 2023-11-06T01:32:10.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4fa1853700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f9c0ff690 0x7f4f9c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:10.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4fa1853700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f9c0ffff0 0x7f4f9c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:10.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4fa1853700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9c1009b0 0x7f4f9c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:10.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4fa1853700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1a09b0 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4fa1853700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1a0af0 con 0x7f4f9c1009b0 2023-11-06T01:32:10.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4fa1853700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1a0c80 con 0x7f4f9c0ff690 2023-11-06T01:32:10.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f9c0ffff0 0x7f4f9c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:10.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f9c0ffff0 0x7f4f9c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37756/0 (socket says 172.21.15.53:37756) 2023-11-06T01:32:10.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9a7fc700 1 -- 172.21.15.53:0/2218959688 learned_addr learned my addr 172.21.15.53:0/2218959688 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:10.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9affd700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f9c0ff690 0x7f4f9c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:10.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9b7fe700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9c1009b0 0x7f4f9c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:10.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9a7fc700 1 -- 172.21.15.53:0/2218959688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f9c0ff690 msgr2=0x7f4f9c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:10.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9a7fc700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f9c0ff690 0x7f4f9c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:10.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9a7fc700 1 -- 172.21.15.53:0/2218959688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9c1009b0 msgr2=0x7f4f9c1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:10.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9a7fc700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9c1009b0 0x7f4f9c1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9a7fc700 1 -- 172.21.15.53:0/2218959688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94009d00 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9b7fe700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9c1009b0 0x7f4f9c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.298+0000 7f4f9a7fc700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f9c0ffff0 0x7f4f9c199370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400b530 tx=0x7f4f8400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.299+0000 7f4fa0851700 1 -- 172.21.15.53:0/2218959688 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f84011830 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.299+0000 7f4fa0851700 1 -- 172.21.15.53:0/2218959688 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f84011e50 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.299+0000 7f4fa0851700 1 -- 172.21.15.53:0/2218959688 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8400f2e0 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.299+0000 7f4fa1853700 1 -- 172.21.15.53:0/2218959688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c1a0f60 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.299+0000 7f4fa1853700 1 -- 172.21.15.53:0/2218959688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c1a1410 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.300+0000 7f4fa1853700 1 -- 172.21.15.53:0/2218959688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9c10a470 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.301+0000 7f4fa0851700 1 -- 172.21.15.53:0/2218959688 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4f84010380 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.301+0000 7f4fa0851700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4f880a0570 0x7f4f880a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:10.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.302+0000 7f4fa0851700 1 -- 172.21.15.53:0/2218959688 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4f840cb060 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.302+0000 7f4f9affd700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4f880a0570 0x7f4f880a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:10.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.302+0000 7f4f9affd700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4f880a0570 0x7f4f880a2a30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009a10 tx=0x7f4f8c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:10.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.306+0000 7f4fa0851700 1 -- 172.21.15.53:0/2218959688 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4f840950a0 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.554+0000 7f4fa1853700 1 -- 172.21.15.53:0/2218959688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f9c02cc70 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.555+0000 7f4fa0851700 1 -- 172.21.15.53:0/2218959688 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5036 (secure 0 0 0) 0x7f4f840986c0 con 0x7f4f9c0ffff0 2023-11-06T01:32:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:10.557 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-11-06T01:32:10.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 -- 172.21.15.53:0/2218959688 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4f880a0570 msgr2=0x7f4f880a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:10.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4f880a0570 0x7f4f880a2a30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009a10 tx=0x7f4f8c009190 comp rx=0 tx=0).stop 2023-11-06T01:32:10.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 -- 172.21.15.53:0/2218959688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f9c0ffff0 msgr2=0x7f4f9c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:10.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f9c0ffff0 0x7f4f9c199370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400b530 tx=0x7f4f8400b840 comp rx=0 tx=0).stop 2023-11-06T01:32:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 -- 172.21.15.53:0/2218959688 shutdown_connections 2023-11-06T01:32:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4f880a0570 0x7f4f880a2a30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9c1009b0 0x7f4f9c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f9c0ff690 0x7f4f9c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 --2- 172.21.15.53:0/2218959688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f9c0ffff0 0x7f4f9c199370 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 -- 172.21.15.53:0/2218959688 >> 172.21.15.53:0/2218959688 conn(0x7f4f9c0fb1f0 msgr2=0x7f4f9c0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 -- 172.21.15.53:0/2218959688 shutdown_connections 2023-11-06T01:32:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:10.557+0000 7f4fa1853700 1 -- 172.21.15.53:0/2218959688 wait complete. 2023-11-06T01:32:10.562 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 7 2023-11-06T01:32:11.296 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:11.296 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:11.296 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:11.296 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:11.297 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:11.297 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:11.297 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:11.297 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: Deploying daemon mds.h on smithi062 2023-11-06T01:32:11.298 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2218959688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:11.298 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] up:boot 2023-11-06T01:32:11.298 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: fsmap 7 up:standby 2023-11-06T01:32:11.298 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-06T01:32:11.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:11.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:11.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:11.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:11.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:11.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:11.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:11.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: Deploying daemon mds.h on smithi062 2023-11-06T01:32:11.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2218959688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:11.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] up:boot 2023-11-06T01:32:11.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: fsmap 7 up:standby 2023-11-06T01:32:11.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-06T01:32:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:11.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:11.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:11.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: Deploying daemon mds.h on smithi062 2023-11-06T01:32:11.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2218959688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:11.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] up:boot 2023-11-06T01:32:11.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: fsmap 7 up:standby 2023-11-06T01:32:11.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-06T01:32:11.625 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:32:11 smithi062 systemd[1]: Starting Ceph mds.h for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:32:12.237 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:32:11 smithi062 podman[139644]: 2023-11-06T01:32:12.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:32:12.645 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:32:12 smithi062 bash[139644]: b00858d5de9b042173f873c17b3de53d37fefa817e95d808cb7c9aa5805f5cfb 2023-11-06T01:32:12.645 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:32:12 smithi062 ceph-mds[139684]: -- 172.21.15.62:0/1784495968 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56285a7ef1e0 con 0x56285a7c4800 2023-11-06T01:32:12.645 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:32:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-h[139660]: starting mds.h at 2023-11-06T01:32:12.646 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:32:12 smithi062 systemd[1]: Started Ceph mds.h for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:32:12.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.710+0000 7f591d326700 1 -- 172.21.15.53:0/537347400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5918100880 msgr2=0x7f5918108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:12.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.710+0000 7f591d326700 1 --2- 172.21.15.53:0/537347400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5918100880 0x7f5918108120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f591000b0a0 tx=0x7f591000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:12.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.710+0000 7f591d326700 1 -- 172.21.15.53:0/537347400 shutdown_connections 2023-11-06T01:32:12.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.710+0000 7f591d326700 1 --2- 172.21.15.53:0/537347400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5918100880 0x7f5918108120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:12.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.711+0000 7f591d326700 1 --2- 172.21.15.53:0/537347400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5918101270 0x7f5918101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:12.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.711+0000 7f591d326700 1 --2- 172.21.15.53:0/537347400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59180ffec0 0x7f5918100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:12.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.711+0000 7f591d326700 1 -- 172.21.15.53:0/537347400 >> 172.21.15.53:0/537347400 conn(0x7f59180fa990 msgr2=0x7f59180fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:12.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.711+0000 7f591d326700 1 -- 172.21.15.53:0/537347400 shutdown_connections 2023-11-06T01:32:12.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.711+0000 7f591d326700 1 -- 172.21.15.53:0/537347400 wait complete. 2023-11-06T01:32:12.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.711+0000 7f591d326700 1 Processor -- start 2023-11-06T01:32:12.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.711+0000 7f591d326700 1 -- start start 2023-11-06T01:32:12.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f591d326700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180ffec0 0x7f591810a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:12.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f591d326700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5918100880 0x7f5918108c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:12.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f591d326700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918101270 0x7f5918109150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:12.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f591d326700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59181098f0 con 0x7f59180ffec0 2023-11-06T01:32:12.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f591d326700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5918109a60 con 0x7f5918101270 2023-11-06T01:32:12.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f591d326700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59181a0bb0 con 0x7f5918100880 2023-11-06T01:32:12.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f5916ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180ffec0 0x7f591810a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:12.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f5916ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180ffec0 0x7f591810a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37768/0 (socket says 172.21.15.53:37768) 2023-11-06T01:32:12.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f5916ffd700 1 -- 172.21.15.53:0/1565921221 learned_addr learned my addr 172.21.15.53:0/1565921221 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:12.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f59167fc700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5918100880 0x7f5918108c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:12.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f59177fe700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918101270 0x7f5918109150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:12.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f5916ffd700 1 -- 172.21.15.53:0/1565921221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5918100880 msgr2=0x7f5918108c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:12.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f5916ffd700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5918100880 0x7f5918108c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:12.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f5916ffd700 1 -- 172.21.15.53:0/1565921221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918101270 msgr2=0x7f5918109150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:12.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f5916ffd700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918101270 0x7f5918109150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:12.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f5916ffd700 1 -- 172.21.15.53:0/1565921221 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5910009d00 con 0x7f59180ffec0 2023-11-06T01:32:12.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f59177fe700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918101270 0x7f5918109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:12.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f59167fc700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5918100880 0x7f5918108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:12.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f5916ffd700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180ffec0 0x7f591810a5d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f590800d950 tx=0x7f590800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:12.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f58fffff700 1 -- 172.21.15.53:0/1565921221 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908009b40 con 0x7f59180ffec0 2023-11-06T01:32:12.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f58fffff700 1 -- 172.21.15.53:0/1565921221 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5908009ca0 con 0x7f59180ffec0 2023-11-06T01:32:12.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f591d326700 1 -- 172.21.15.53:0/1565921221 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59181a0e90 con 0x7f59180ffec0 2023-11-06T01:32:12.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.712+0000 7f58fffff700 1 -- 172.21.15.53:0/1565921221 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590800f620 con 0x7f59180ffec0 2023-11-06T01:32:12.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.713+0000 7f591d326700 1 -- 172.21.15.53:0/1565921221 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59181a12b0 con 0x7f59180ffec0 2023-11-06T01:32:12.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.713+0000 7f591d326700 1 -- 172.21.15.53:0/1565921221 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591810c680 con 0x7f59180ffec0 2023-11-06T01:32:12.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.715+0000 7f58fffff700 1 -- 172.21.15.53:0/1565921221 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5908010460 con 0x7f59180ffec0 2023-11-06T01:32:12.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.715+0000 7f58fffff700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59000a0570 0x7f59000a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:12.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.716+0000 7f59167fc700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59000a0570 0x7f59000a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:12.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.716+0000 7f58fffff700 1 -- 172.21.15.53:0/1565921221 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f59080cb8d0 con 0x7f59180ffec0 2023-11-06T01:32:12.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.716+0000 7f59167fc700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59000a0570 0x7f59000a2a30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f590c005950 tx=0x7f590c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:12.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.719+0000 7f58fffff700 1 -- 172.21.15.53:0/1565921221 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5908095910 con 0x7f59180ffec0 2023-11-06T01:32:12.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.971+0000 7f591d326700 1 -- 172.21.15.53:0/1565921221 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5918106af0 con 0x7f59180ffec0 2023-11-06T01:32:12.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.972+0000 7f58fffff700 1 -- 172.21.15.53:0/1565921221 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5796 (secure 0 0 0) 0x7f5908098f30 con 0x7f59180ffec0 2023-11-06T01:32:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:12.974 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-11-06T01:32:12.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 -- 172.21.15.53:0/1565921221 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59000a0570 msgr2=0x7f59000a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:12.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59000a0570 0x7f59000a2a30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f590c005950 tx=0x7f590c005f50 comp rx=0 tx=0).stop 2023-11-06T01:32:12.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 -- 172.21.15.53:0/1565921221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180ffec0 msgr2=0x7f591810a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:12.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180ffec0 0x7f591810a5d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f590800d950 tx=0x7f590800dd10 comp rx=0 tx=0).stop 2023-11-06T01:32:12.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 -- 172.21.15.53:0/1565921221 shutdown_connections 2023-11-06T01:32:12.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59000a0570 0x7f59000a2a30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:12.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918101270 0x7f5918109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:12.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180ffec0 0x7f591810a5d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:12.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 --2- 172.21.15.53:0/1565921221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5918100880 0x7f5918108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:12.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 -- 172.21.15.53:0/1565921221 >> 172.21.15.53:0/1565921221 conn(0x7f59180fa990 msgr2=0x7f59180fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:12.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 -- 172.21.15.53:0/1565921221 shutdown_connections 2023-11-06T01:32:12.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:12.974+0000 7f591d326700 1 -- 172.21.15.53:0/1565921221 wait complete. 2023-11-06T01:32:12.979 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 8 2023-11-06T01:32:13.333 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:13 smithi163 ceph-mon[104416]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:13.333 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:13.333 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:13.334 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:13.334 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:13.334 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:13.334 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:13 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:13.334 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:13 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1565921221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:13.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:13 smithi053 ceph-mon[99472]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:13.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:13.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:13.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:13.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:13.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:13.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:13 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:13.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:13 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1565921221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:13.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:13 smithi062 ceph-mon[103924]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:13.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:13.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:13.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:13.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:13.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:13.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:13 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:13.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:13 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1565921221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:13.990 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:32:13 smithi163 systemd[1]: Starting Ceph mds.i for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:32:14.301 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:32:14 smithi163 podman[140720]: 2023-11-06T01:32:14.302 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:14 smithi163 ceph-mon[104416]: Deploying daemon mds.i on smithi163 2023-11-06T01:32:14.302 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:14 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.62:6836/644941927,v1:172.21.15.62:6837/644941927] up:boot 2023-11-06T01:32:14.302 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:14 smithi163 ceph-mon[104416]: fsmap 8 up:standby 2023-11-06T01:32:14.302 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:14 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-06T01:32:14.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:14 smithi053 ceph-mon[99472]: Deploying daemon mds.i on smithi163 2023-11-06T01:32:14.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:14 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.62:6836/644941927,v1:172.21.15.62:6837/644941927] up:boot 2023-11-06T01:32:14.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:14 smithi053 ceph-mon[99472]: fsmap 8 up:standby 2023-11-06T01:32:14.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:14 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-06T01:32:14.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:14 smithi062 ceph-mon[103924]: Deploying daemon mds.i on smithi163 2023-11-06T01:32:14.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:14 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.62:6836/644941927,v1:172.21.15.62:6837/644941927] up:boot 2023-11-06T01:32:14.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:14 smithi062 ceph-mon[103924]: fsmap 8 up:standby 2023-11-06T01:32:14.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:14 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-06T01:32:14.739 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:32:14 smithi163 ceph-mds[140760]: -- 172.21.15.163:0/2454115427 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c8dead91e0 con 0x55c8deaae800 2023-11-06T01:32:14.740 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:32:14 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-i[140736]: starting mds.i at 2023-11-06T01:32:14.740 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:32:14 smithi163 bash[140720]: aea540823cede738beb7ef0f81712a7ac845171a0ff3c7b4174e0d74c85757a2 2023-11-06T01:32:14.740 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:32:14 smithi163 systemd[1]: Started Ceph mds.i for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:32:14.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:32:15.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.123+0000 7ff851436700 1 -- 172.21.15.53:0/4089858138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff84c101800 msgr2=0x7ff84c1080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:15.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.123+0000 7ff851436700 1 --2- 172.21.15.53:0/4089858138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff84c101800 0x7ff84c1080b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff84400b0d0 tx=0x7ff84400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:32:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.123+0000 7ff851436700 1 -- 172.21.15.53:0/4089858138 shutdown_connections 2023-11-06T01:32:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.123+0000 7ff851436700 1 --2- 172.21.15.53:0/4089858138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff84c0ffc70 0x7ff84c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.123+0000 7ff851436700 1 --2- 172.21.15.53:0/4089858138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff84c101800 0x7ff84c1080b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.123+0000 7ff851436700 1 --2- 172.21.15.53:0/4089858138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff84c100e60 0x7ff84c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:15.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.123+0000 7ff851436700 1 -- 172.21.15.53:0/4089858138 >> 172.21.15.53:0/4089858138 conn(0x7ff84c0fb1c0 msgr2=0x7ff84c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:15.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.123+0000 7ff851436700 1 -- 172.21.15.53:0/4089858138 shutdown_connections 2023-11-06T01:32:15.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.123+0000 7ff851436700 1 -- 172.21.15.53:0/4089858138 wait complete. 2023-11-06T01:32:15.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff851436700 1 Processor -- start 2023-11-06T01:32:15.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff851436700 1 -- start start 2023-11-06T01:32:15.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff851436700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff84c0ffc70 0x7ff84c194950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:15.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff851436700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff84c100e60 0x7ff84c194e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:15.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff851436700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff84c101800 0x7ff84c19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:15.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff851436700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c19c4d0 con 0x7ff84c101800 2023-11-06T01:32:15.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff851436700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c19c610 con 0x7ff84c100e60 2023-11-06T01:32:15.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff851436700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c19c7a0 con 0x7ff84c0ffc70 2023-11-06T01:32:15.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff84b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff84c101800 0x7ff84c19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:15.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff84b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff84c101800 0x7ff84c19bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37786/0 (socket says 172.21.15.53:37786) 2023-11-06T01:32:15.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff84b7fe700 1 -- 172.21.15.53:0/2208053002 learned_addr learned my addr 172.21.15.53:0/2208053002 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:15.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff84affd700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff84c0ffc70 0x7ff84c194950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:15.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff84b7fe700 1 -- 172.21.15.53:0/2208053002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff84c0ffc70 msgr2=0x7ff84c194950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:15.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff84a7fc700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff84c100e60 0x7ff84c194e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:15.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff84b7fe700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff84c0ffc70 0x7ff84c194950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:15.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.124+0000 7ff84b7fe700 1 -- 172.21.15.53:0/2208053002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff84c100e60 msgr2=0x7ff84c194e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:15.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.125+0000 7ff84b7fe700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff84c100e60 0x7ff84c194e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:15.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.125+0000 7ff84b7fe700 1 -- 172.21.15.53:0/2208053002 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff844009d00 con 0x7ff84c101800 2023-11-06T01:32:15.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.125+0000 7ff84affd700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff84c0ffc70 0x7ff84c194950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:32:15.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.125+0000 7ff84b7fe700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff84c101800 0x7ff84c19bf00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff844007a00 tx=0x7ff844009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:15.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.125+0000 7ff843fff700 1 -- 172.21.15.53:0/2208053002 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84400e040 con 0x7ff84c101800 2023-11-06T01:32:15.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.125+0000 7ff851436700 1 -- 172.21.15.53:0/2208053002 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84c19ca20 con 0x7ff84c101800 2023-11-06T01:32:15.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.125+0000 7ff843fff700 1 -- 172.21.15.53:0/2208053002 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff844003e10 con 0x7ff84c101800 2023-11-06T01:32:15.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.125+0000 7ff843fff700 1 -- 172.21.15.53:0/2208053002 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844005750 con 0x7ff84c101800 2023-11-06T01:32:15.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.125+0000 7ff851436700 1 -- 172.21.15.53:0/2208053002 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84c19cf20 con 0x7ff84c101800 2023-11-06T01:32:15.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.126+0000 7ff851436700 1 -- 172.21.15.53:0/2208053002 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84c107170 con 0x7ff84c101800 2023-11-06T01:32:15.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.127+0000 7ff843fff700 1 -- 172.21.15.53:0/2208053002 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff844012cb0 con 0x7ff84c101800 2023-11-06T01:32:15.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.128+0000 7ff843fff700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8380a2780 0x7ff8380a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:15.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.128+0000 7ff84affd700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8380a2780 0x7ff8380a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:15.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.128+0000 7ff843fff700 1 -- 172.21.15.53:0/2208053002 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff8440cd5f0 con 0x7ff84c101800 2023-11-06T01:32:15.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.128+0000 7ff84affd700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8380a2780 0x7ff8380a4c40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff84c100cc0 tx=0x7ff834008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:15.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.131+0000 7ff843fff700 1 -- 172.21.15.53:0/2208053002 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff844097620 con 0x7ff84c101800 2023-11-06T01:32:15.281 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:15 smithi053 ceph-mon[99472]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:15.281 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:15 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:15.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:15 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:15.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:15 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:15.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:15 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:15.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:15 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:15.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:15 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:15.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.391+0000 7ff851436700 1 -- 172.21.15.53:0/2208053002 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff84c02cc70 con 0x7ff84c101800 2023-11-06T01:32:15.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.391+0000 7ff843fff700 1 -- 172.21.15.53:0/2208053002 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7312 (secure 0 0 0) 0x7ff84409ac40 con 0x7ff84c101800 2023-11-06T01:32:15.393 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:15.394 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-11-06T01:32:15.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:15 smithi062 ceph-mon[103924]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:15.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:15 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:15.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:15 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:15.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:15 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:15.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:15 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:15.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:15 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:15.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:15 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:15.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.393+0000 7ff851436700 1 -- 172.21.15.53:0/2208053002 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8380a2780 msgr2=0x7ff8380a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:15.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.393+0000 7ff851436700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8380a2780 0x7ff8380a4c40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff84c100cc0 tx=0x7ff834008040 comp rx=0 tx=0).stop 2023-11-06T01:32:15.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.393+0000 7ff851436700 1 -- 172.21.15.53:0/2208053002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff84c101800 msgr2=0x7ff84c19bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:15.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.393+0000 7ff851436700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff84c101800 0x7ff84c19bf00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff844007a00 tx=0x7ff844009570 comp rx=0 tx=0).stop 2023-11-06T01:32:15.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.393+0000 7ff851436700 1 -- 172.21.15.53:0/2208053002 shutdown_connections 2023-11-06T01:32:15.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.393+0000 7ff851436700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8380a2780 0x7ff8380a4c40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:15.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.393+0000 7ff851436700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff84c101800 0x7ff84c19bf00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.393+0000 7ff851436700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff84c0ffc70 0x7ff84c194950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.393+0000 7ff851436700 1 --2- 172.21.15.53:0/2208053002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff84c100e60 0x7ff84c194e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.393+0000 7ff851436700 1 -- 172.21.15.53:0/2208053002 >> 172.21.15.53:0/2208053002 conn(0x7ff84c0fb1c0 msgr2=0x7ff84c1042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.394+0000 7ff851436700 1 -- 172.21.15.53:0/2208053002 shutdown_connections 2023-11-06T01:32:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:15.394+0000 7ff851436700 1 -- 172.21.15.53:0/2208053002 wait complete. 2023-11-06T01:32:15.400 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 10 2023-11-06T01:32:15.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:15 smithi163 ceph-mon[104416]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:15.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:15 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:15.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:15 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:15.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:15 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:15.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:15 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:15.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:15 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:15.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:15 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:16.222 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:16 smithi053 ceph-mon[99472]: Deploying daemon mds.g on smithi053 2023-11-06T01:32:16.222 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:16 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] up:boot 2023-11-06T01:32:16.223 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:16 smithi053 ceph-mon[99472]: fsmap 9 up:standby 2023-11-06T01:32:16.223 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:16 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-06T01:32:16.223 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:16 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2208053002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:16.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:16 smithi062 ceph-mon[103924]: Deploying daemon mds.g on smithi053 2023-11-06T01:32:16.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:16 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] up:boot 2023-11-06T01:32:16.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:16 smithi062 ceph-mon[103924]: fsmap 9 up:standby 2023-11-06T01:32:16.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:16 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-06T01:32:16.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:16 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2208053002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:16.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:16 smithi163 ceph-mon[104416]: Deploying daemon mds.g on smithi053 2023-11-06T01:32:16.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:16 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] up:boot 2023-11-06T01:32:16.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:16 smithi163 ceph-mon[104416]: fsmap 9 up:standby 2023-11-06T01:32:16.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:16 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-06T01:32:16.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:16 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2208053002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:16.799 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:32:16 smithi053 systemd[1]: Starting Ceph mds.g for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:32:16.800 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:32:16 smithi053 podman[152196]: 2023-11-06T01:32:17.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:17 smithi053 ceph-mon[99472]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:17.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:17.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:17.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:17.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:17.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:17.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:17 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:17.096 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:32:16 smithi053 ceph-mds[152236]: -- 172.21.15.53:0/1148546001 <== mon.0 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562b1e8931e0 con 0x562b1e868c00 2023-11-06T01:32:17.097 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:32:16 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-g[152212]: starting mds.g at 2023-11-06T01:32:17.097 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:32:16 smithi053 bash[152196]: c377542377e7c5a91d5cb8f0d4b594a65c02c0a7883e459d4bc0d09f0f8f7771 2023-11-06T01:32:17.097 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:32:16 smithi053 systemd[1]: Started Ceph mds.g for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:32:17.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:17 smithi062 ceph-mon[103924]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:17.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:17 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:17.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:17 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:17.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:17 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:17.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:17 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:17.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:17 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:17.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:17 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:17.409 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:32:17.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:17 smithi163 ceph-mon[104416]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:17.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:17.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:17.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:17.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:17.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:17.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:17 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:17.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.543+0000 7fe04995f700 1 -- 172.21.15.53:0/4160443604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe044100d90 msgr2=0x7fe0441011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:17.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.543+0000 7fe04995f700 1 --2- 172.21.15.53:0/4160443604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe044100d90 0x7fe0441011b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe0340099c0 tx=0x7fe034009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:17.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.543+0000 7fe04995f700 1 -- 172.21.15.53:0/4160443604 shutdown_connections 2023-11-06T01:32:17.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.543+0000 7fe04995f700 1 --2- 172.21.15.53:0/4160443604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe044102920 0x7fe0441091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:17.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.543+0000 7fe04995f700 1 --2- 172.21.15.53:0/4160443604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe044100d90 0x7fe0441011b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:17.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.543+0000 7fe04995f700 1 --2- 172.21.15.53:0/4160443604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe044101f80 0x7fe0441023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:17.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.543+0000 7fe04995f700 1 -- 172.21.15.53:0/4160443604 >> 172.21.15.53:0/4160443604 conn(0x7fe0440fc2c0 msgr2=0x7fe0440fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:17.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.543+0000 7fe04995f700 1 -- 172.21.15.53:0/4160443604 shutdown_connections 2023-11-06T01:32:17.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.543+0000 7fe04995f700 1 -- 172.21.15.53:0/4160443604 wait complete. 2023-11-06T01:32:17.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe04995f700 1 Processor -- start 2023-11-06T01:32:17.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe04995f700 1 -- start start 2023-11-06T01:32:17.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe04995f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe044100d90 0x7fe044074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:17.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe04995f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe044101f80 0x7fe044073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:17.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe04995f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe044102920 0x7fe0440736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe04995f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe044073e60 con 0x7fe044100d90 2023-11-06T01:32:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe04995f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe044073fd0 con 0x7fe044102920 2023-11-06T01:32:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe04995f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0441a0ab0 con 0x7fe044101f80 2023-11-06T01:32:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe0437fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe044102920 0x7fe0440736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe0437fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe044102920 0x7fe0440736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:49252/0 (socket says 172.21.15.53:49252) 2023-11-06T01:32:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe0437fe700 1 -- 172.21.15.53:0/1305389307 learned_addr learned my addr 172.21.15.53:0/1305389307 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.544+0000 7fe0427fc700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe044101f80 0x7fe044073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe042ffd700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe044100d90 0x7fe044074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe0437fe700 1 -- 172.21.15.53:0/1305389307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe044101f80 msgr2=0x7fe044073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe0437fe700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe044101f80 0x7fe044073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe0437fe700 1 -- 172.21.15.53:0/1305389307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe044100d90 msgr2=0x7fe044074b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe0437fe700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe044100d90 0x7fe044074b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe0437fe700 1 -- 172.21.15.53:0/1305389307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c009c40 con 0x7fe044102920 2023-11-06T01:32:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe0437fe700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe044102920 0x7fe0440736c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00c370 tx=0x7fe03c00c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe04895d700 1 -- 172.21.15.53:0/1305389307 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c017070 con 0x7fe044102920 2023-11-06T01:32:17.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe04995f700 1 -- 172.21.15.53:0/1305389307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0340095f0 con 0x7fe044102920 2023-11-06T01:32:17.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe04895d700 1 -- 172.21.15.53:0/1305389307 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe03c00f040 con 0x7fe044102920 2023-11-06T01:32:17.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe04895d700 1 -- 172.21.15.53:0/1305389307 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c00e050 con 0x7fe044102920 2023-11-06T01:32:17.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.545+0000 7fe04995f700 1 -- 172.21.15.53:0/1305389307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0441a1100 con 0x7fe044102920 2023-11-06T01:32:17.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.547+0000 7fe04995f700 1 -- 172.21.15.53:0/1305389307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04410c5a0 con 0x7fe044102920 2023-11-06T01:32:17.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.548+0000 7fe04895d700 1 -- 172.21.15.53:0/1305389307 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe03c00cb60 con 0x7fe044102920 2023-11-06T01:32:17.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.548+0000 7fe04895d700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe02c0a0570 0x7fe02c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:17.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.548+0000 7fe04895d700 1 -- 172.21.15.53:0/1305389307 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe03c0cadc0 con 0x7fe044102920 2023-11-06T01:32:17.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.548+0000 7fe042ffd700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe02c0a0570 0x7fe02c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:17.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.549+0000 7fe042ffd700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe02c0a0570 0x7fe02c0a2a30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe034009fd0 tx=0x7fe034005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:17.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.554+0000 7fe04895d700 1 -- 172.21.15.53:0/1305389307 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe03c094e00 con 0x7fe044102920 2023-11-06T01:32:17.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.811+0000 7fe04995f700 1 -- 172.21.15.53:0/1305389307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe044107c90 con 0x7fe044102920 2023-11-06T01:32:17.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.812+0000 7fe04895d700 1 -- 172.21.15.53:0/1305389307 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8070 (secure 0 0 0) 0x7fe03c098420 con 0x7fe044102920 2023-11-06T01:32:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:17.814 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-11-06T01:32:17.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.814+0000 7fe04995f700 1 -- 172.21.15.53:0/1305389307 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe02c0a0570 msgr2=0x7fe02c0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:17.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.814+0000 7fe04995f700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe02c0a0570 0x7fe02c0a2a30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe034009fd0 tx=0x7fe034005fb0 comp rx=0 tx=0).stop 2023-11-06T01:32:17.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.814+0000 7fe04995f700 1 -- 172.21.15.53:0/1305389307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe044102920 msgr2=0x7fe0440736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:17.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.814+0000 7fe04995f700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe044102920 0x7fe0440736c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00c370 tx=0x7fe03c00c680 comp rx=0 tx=0).stop 2023-11-06T01:32:17.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.814+0000 7fe04995f700 1 -- 172.21.15.53:0/1305389307 shutdown_connections 2023-11-06T01:32:17.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.814+0000 7fe04995f700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe02c0a0570 0x7fe02c0a2a30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:17.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.814+0000 7fe04995f700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe044102920 0x7fe0440736c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:17.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.814+0000 7fe04995f700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe044100d90 0x7fe044074b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:17.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.814+0000 7fe04995f700 1 --2- 172.21.15.53:0/1305389307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe044101f80 0x7fe044073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:17.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.814+0000 7fe04995f700 1 -- 172.21.15.53:0/1305389307 >> 172.21.15.53:0/1305389307 conn(0x7fe0440fc2c0 msgr2=0x7fe0440fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:17.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.815+0000 7fe04995f700 1 -- 172.21.15.53:0/1305389307 shutdown_connections 2023-11-06T01:32:17.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:17.815+0000 7fe04995f700 1 -- 172.21.15.53:0/1305389307 wait complete. 2023-11-06T01:32:17.820 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 11 2023-11-06T01:32:18.319 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:18 smithi053 ceph-mon[99472]: Deploying daemon mds.a on smithi053 2023-11-06T01:32:18.319 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:18 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.53:6838/1353368498,v1:172.21.15.53:6839/1353368498] up:boot 2023-11-06T01:32:18.320 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:18 smithi053 ceph-mon[99472]: fsmap 10 up:standby 2023-11-06T01:32:18.320 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:18 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-06T01:32:18.320 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:18 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1305389307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:18.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:18 smithi062 ceph-mon[103924]: Deploying daemon mds.a on smithi053 2023-11-06T01:32:18.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:18 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.53:6838/1353368498,v1:172.21.15.53:6839/1353368498] up:boot 2023-11-06T01:32:18.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:18 smithi062 ceph-mon[103924]: fsmap 10 up:standby 2023-11-06T01:32:18.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:18 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-06T01:32:18.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:18 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1305389307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:18.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:18 smithi163 ceph-mon[104416]: Deploying daemon mds.a on smithi053 2023-11-06T01:32:18.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:18 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.53:6838/1353368498,v1:172.21.15.53:6839/1353368498] up:boot 2023-11-06T01:32:18.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:18 smithi163 ceph-mon[104416]: fsmap 10 up:standby 2023-11-06T01:32:18.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:18 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-06T01:32:18.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:18 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1305389307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:18.594 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:32:18 smithi053 systemd[1]: Starting Ceph mds.a for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:32:19.022 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:32:18 smithi053 podman[152583]: 2023-11-06T01:32:19.022 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:32:18 smithi053 bash[152583]: e80f60241caff8f2d5141d988653ec8672b23e9f27a1adcff3fce8ebf2fe8e78 2023-11-06T01:32:19.023 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:32:18 smithi053 ceph-mds[152623]: -- 172.21.15.53:0/3311103545 <== mon.0 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555fffc2f1e0 con 0x555fffc05000 2023-11-06T01:32:19.023 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:32:18 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-a[152599]: starting mds.a at 2023-11-06T01:32:19.344 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:32:19 smithi053 systemd[1]: Started Ceph mds.a for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:32:19.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:19 smithi053 ceph-mon[99472]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:19.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:19 smithi062 ceph-mon[103924]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:19.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:19 smithi163 ceph-mon[104416]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:19.833 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:32:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.978+0000 7f5252e9e700 1 -- 172.21.15.53:0/764841033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524c102920 msgr2=0x7f524c1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.978+0000 7f5252e9e700 1 --2- 172.21.15.53:0/764841033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524c102920 0x7f524c1091d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f524400b0d0 tx=0x7f524400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:32:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.978+0000 7f5252e9e700 1 -- 172.21.15.53:0/764841033 shutdown_connections 2023-11-06T01:32:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.978+0000 7f5252e9e700 1 --2- 172.21.15.53:0/764841033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524c102920 0x7f524c1091d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.978+0000 7f5252e9e700 1 --2- 172.21.15.53:0/764841033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f524c100d90 0x7f524c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.978+0000 7f5252e9e700 1 --2- 172.21.15.53:0/764841033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f524c101f80 0x7f524c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.978+0000 7f5252e9e700 1 -- 172.21.15.53:0/764841033 >> 172.21.15.53:0/764841033 conn(0x7f524c0fc2e0 msgr2=0x7f524c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.978+0000 7f5252e9e700 1 -- 172.21.15.53:0/764841033 shutdown_connections 2023-11-06T01:32:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.978+0000 7f5252e9e700 1 -- 172.21.15.53:0/764841033 wait complete. 2023-11-06T01:32:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f5252e9e700 1 Processor -- start 2023-11-06T01:32:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f5252e9e700 1 -- start start 2023-11-06T01:32:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f5252e9e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f524c100d90 0x7f524c198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f5252e9e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f524c101f80 0x7f524c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f5252e9e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524c102920 0x7f524c1a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f5252e9e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c1a0820 con 0x7f524c102920 2023-11-06T01:32:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f5252e9e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c1a0990 con 0x7f524c101f80 2023-11-06T01:32:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f5252e9e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c1a0b50 con 0x7f524c100d90 2023-11-06T01:32:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f525143b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524c102920 0x7f524c1a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f525143b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524c102920 0x7f524c1a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37840/0 (socket says 172.21.15.53:37840) 2023-11-06T01:32:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f525143b700 1 -- 172.21.15.53:0/1790716199 learned_addr learned my addr 172.21.15.53:0/1790716199 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.979+0000 7f524bfff700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f524c101f80 0x7f524c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f5250c3a700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f524c100d90 0x7f524c198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f525143b700 1 -- 172.21.15.53:0/1790716199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f524c100d90 msgr2=0x7f524c198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f525143b700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f524c100d90 0x7f524c198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f525143b700 1 -- 172.21.15.53:0/1790716199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f524c101f80 msgr2=0x7f524c1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f525143b700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f524c101f80 0x7f524c1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f525143b700 1 -- 172.21.15.53:0/1790716199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5244009d00 con 0x7f524c102920 2023-11-06T01:32:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f525143b700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524c102920 0x7f524c1a0250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5244008f10 tx=0x7f5244008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:19.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f5249ffb700 1 -- 172.21.15.53:0/1790716199 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524400e040 con 0x7f524c102920 2023-11-06T01:32:19.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f5252e9e700 1 -- 172.21.15.53:0/1790716199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c1a0dd0 con 0x7f524c102920 2023-11-06T01:32:19.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f5249ffb700 1 -- 172.21.15.53:0/1790716199 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f524400be60 con 0x7f524c102920 2023-11-06T01:32:19.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f5249ffb700 1 -- 172.21.15.53:0/1790716199 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524401b6a0 con 0x7f524c102920 2023-11-06T01:32:19.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.980+0000 7f5252e9e700 1 -- 172.21.15.53:0/1790716199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c1a11a0 con 0x7f524c102920 2023-11-06T01:32:19.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.981+0000 7f5252e9e700 1 -- 172.21.15.53:0/1790716199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c10c5b0 con 0x7f524c102920 2023-11-06T01:32:19.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.982+0000 7f5249ffb700 1 -- 172.21.15.53:0/1790716199 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5244024400 con 0x7f524c102920 2023-11-06T01:32:19.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.983+0000 7f5249ffb700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f523c0a0570 0x7f523c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:19.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.983+0000 7f5250c3a700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f523c0a0570 0x7f523c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:19.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.983+0000 7f5249ffb700 1 -- 172.21.15.53:0/1790716199 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f52440cc830 con 0x7f524c102920 2023-11-06T01:32:19.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.984+0000 7f5250c3a700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f523c0a0570 0x7f523c0a2a30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f524c199b80 tx=0x7f5240005d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:19.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:19.988+0000 7f5249ffb700 1 -- 172.21.15.53:0/1790716199 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5244096900 con 0x7f524c102920 2023-11-06T01:32:20.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.239+0000 7f5252e9e700 1 -- 172.21.15.53:0/1790716199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f524c1a1750 con 0x7f524c102920 2023-11-06T01:32:20.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.240+0000 7f5249ffb700 1 -- 172.21.15.53:0/1790716199 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8828 (secure 0 0 0) 0x7f5244099f20 con 0x7f524c102920 2023-11-06T01:32:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:20.242 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-11-06T01:32:20.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 -- 172.21.15.53:0/1790716199 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f523c0a0570 msgr2=0x7f523c0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:20.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f523c0a0570 0x7f523c0a2a30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f524c199b80 tx=0x7f5240005d10 comp rx=0 tx=0).stop 2023-11-06T01:32:20.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 -- 172.21.15.53:0/1790716199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524c102920 msgr2=0x7f524c1a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:20.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524c102920 0x7f524c1a0250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5244008f10 tx=0x7f5244008f40 comp rx=0 tx=0).stop 2023-11-06T01:32:20.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 -- 172.21.15.53:0/1790716199 shutdown_connections 2023-11-06T01:32:20.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f523c0a0570 0x7f523c0a2a30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:20.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f524c102920 0x7f524c1a0250 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:20.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f524c100d90 0x7f524c198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:20.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 --2- 172.21.15.53:0/1790716199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f524c101f80 0x7f524c1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:20.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 -- 172.21.15.53:0/1790716199 >> 172.21.15.53:0/1790716199 conn(0x7f524c0fc2e0 msgr2=0x7f524c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:20.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 -- 172.21.15.53:0/1790716199 shutdown_connections 2023-11-06T01:32:20.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:20.242+0000 7f5252e9e700 1 -- 172.21.15.53:0/1790716199 wait complete. 2023-11-06T01:32:20.246 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 12 2023-11-06T01:32:20.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:20 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.53:6840/1972292821,v1:172.21.15.53:6841/1972292821] up:boot 2023-11-06T01:32:20.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:20 smithi053 ceph-mon[99472]: fsmap 11 up:standby 2023-11-06T01:32:20.255 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-06T01:32:20.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:20.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:20.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:20.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:20.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:20.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:20 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:20.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:20 smithi053 ceph-mon[99472]: Deploying daemon mds.b on smithi062 2023-11-06T01:32:20.320 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:20 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.53:6840/1972292821,v1:172.21.15.53:6841/1972292821] up:boot 2023-11-06T01:32:20.321 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:20 smithi062 ceph-mon[103924]: fsmap 11 up:standby 2023-11-06T01:32:20.321 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-06T01:32:20.321 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:20.321 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:20.322 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:20.322 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:20.322 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:20.322 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:20 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:20.322 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:20 smithi062 ceph-mon[103924]: Deploying daemon mds.b on smithi062 2023-11-06T01:32:20.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:20 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.53:6840/1972292821,v1:172.21.15.53:6841/1972292821] up:boot 2023-11-06T01:32:20.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:20 smithi163 ceph-mon[104416]: fsmap 11 up:standby 2023-11-06T01:32:20.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-06T01:32:20.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:20.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:20.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:20.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-06T01:32:20.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-06T01:32:20.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:20 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:20.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:20 smithi163 ceph-mon[104416]: Deploying daemon mds.b on smithi062 2023-11-06T01:32:20.895 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:32:20 smithi062 systemd[1]: Starting Ceph mds.b for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:32:21.267 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:32:21 smithi062 podman[139997]: 2023-11-06T01:32:21.267 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:21 smithi062 ceph-mon[103924]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:21.268 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:21 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1790716199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:21.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:21 smithi163 ceph-mon[104416]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:21.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:21 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1790716199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:21 smithi053 ceph-mon[99472]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:21 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1790716199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:21.645 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:32:21 smithi062 ceph-mds[140037]: -- 172.21.15.62:0/2996448911 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558b37c5f1e0 con 0x558b37c34c00 2023-11-06T01:32:21.645 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:32:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-b[140013]: starting mds.b at 2023-11-06T01:32:21.645 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:32:21 smithi062 bash[139997]: 7c14addd1cc109894b2046a56f6a58e07fc0b2a0fdd6bbea9f02d6f2075d51de 2023-11-06T01:32:21.645 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:32:21 smithi062 systemd[1]: Started Ceph mds.b for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7. 2023-11-06T01:32:22.259 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:32:22.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.392+0000 7f3d98b2e700 1 -- 172.21.15.53:0/2972958070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94101800 msgr2=0x7f3d94067ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:22.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.392+0000 7f3d98b2e700 1 --2- 172.21.15.53:0/2972958070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94101800 0x7f3d94067ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00b0a0 tx=0x7f3d8c00b3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.392+0000 7f3d98b2e700 1 -- 172.21.15.53:0/2972958070 shutdown_connections 2023-11-06T01:32:22.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.392+0000 7f3d98b2e700 1 --2- 172.21.15.53:0/2972958070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d940ffc70 0x7f3d94100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.392+0000 7f3d98b2e700 1 --2- 172.21.15.53:0/2972958070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94101800 0x7f3d94067ef0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.392+0000 7f3d98b2e700 1 --2- 172.21.15.53:0/2972958070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d94100e60 0x7f3d941012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.392+0000 7f3d98b2e700 1 -- 172.21.15.53:0/2972958070 >> 172.21.15.53:0/2972958070 conn(0x7f3d940fb1c0 msgr2=0x7f3d940fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:22.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.392+0000 7f3d98b2e700 1 -- 172.21.15.53:0/2972958070 shutdown_connections 2023-11-06T01:32:22.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.392+0000 7f3d98b2e700 1 -- 172.21.15.53:0/2972958070 wait complete. 2023-11-06T01:32:22.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d98b2e700 1 Processor -- start 2023-11-06T01:32:22.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d98b2e700 1 -- start start 2023-11-06T01:32:22.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d98b2e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d940ffc70 0x7f3d94198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:22.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d98b2e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d94100e60 0x7f3d94199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:22.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d98b2e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d94101800 0x7f3d941a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:22.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d98b2e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d941a0890 con 0x7f3d940ffc70 2023-11-06T01:32:22.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d98b2e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d941a0a00 con 0x7f3d94100e60 2023-11-06T01:32:22.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d98b2e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d941a0bc0 con 0x7f3d94101800 2023-11-06T01:32:22.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d9259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d940ffc70 0x7f3d94198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:22.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d9259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d940ffc70 0x7f3d94198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37848/0 (socket says 172.21.15.53:37848) 2023-11-06T01:32:22.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d9259c700 1 -- 172.21.15.53:0/1811324093 learned_addr learned my addr 172.21.15.53:0/1811324093 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:22.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.393+0000 7f3d91d9b700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d94100e60 0x7f3d94199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:22.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d92d9d700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d94101800 0x7f3d941a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:22.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d9259c700 1 -- 172.21.15.53:0/1811324093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d94101800 msgr2=0x7f3d941a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:22.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d9259c700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d94101800 0x7f3d941a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d9259c700 1 -- 172.21.15.53:0/1811324093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d94100e60 msgr2=0x7f3d94199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:22.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d9259c700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d94100e60 0x7f3d94199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d9259c700 1 -- 172.21.15.53:0/1811324093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c009d00 con 0x7f3d940ffc70 2023-11-06T01:32:22.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d92d9d700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d94101800 0x7f3d941a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:32:22.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d9259c700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d940ffc70 0x7f3d94198d50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400b550 tx=0x7f3d8400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:22.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d837fe700 1 -- 172.21.15.53:0/1811324093 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84011830 con 0x7f3d940ffc70 2023-11-06T01:32:22.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d837fe700 1 -- 172.21.15.53:0/1811324093 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d84011e50 con 0x7f3d940ffc70 2023-11-06T01:32:22.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d98b2e700 1 -- 172.21.15.53:0/1811324093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d941a0ea0 con 0x7f3d940ffc70 2023-11-06T01:32:22.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d837fe700 1 -- 172.21.15.53:0/1811324093 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400f2e0 con 0x7f3d940ffc70 2023-11-06T01:32:22.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.394+0000 7f3d98b2e700 1 -- 172.21.15.53:0/1811324093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d941a12f0 con 0x7f3d940ffc70 2023-11-06T01:32:22.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.395+0000 7f3d98b2e700 1 -- 172.21.15.53:0/1811324093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d94108a50 con 0x7f3d940ffc70 2023-11-06T01:32:22.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.396+0000 7f3d837fe700 1 -- 172.21.15.53:0/1811324093 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f3d840105a0 con 0x7f3d940ffc70 2023-11-06T01:32:22.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.397+0000 7f3d837fe700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3d7c0a0570 0x7f3d7c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:22.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.397+0000 7f3d91d9b700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3d7c0a0570 0x7f3d7c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:22.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.397+0000 7f3d837fe700 1 -- 172.21.15.53:0/1811324093 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3d840caf80 con 0x7f3d940ffc70 2023-11-06T01:32:22.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.397+0000 7f3d91d9b700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3d7c0a0570 0x7f3d7c0a2a30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3d88006fd0 tx=0x7f3d88009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:22.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.402+0000 7f3d837fe700 1 -- 172.21.15.53:0/1811324093 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3d840fe0e0 con 0x7f3d940ffc70 2023-11-06T01:32:22.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:32:22.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:22 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] up:boot 2023-11-06T01:32:22.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:22 smithi053 ceph-mon[99472]: fsmap 12 up:standby 2023-11-06T01:32:22.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:22 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-06T01:32:22.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:32:22.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:22 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] up:boot 2023-11-06T01:32:22.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:22 smithi062 ceph-mon[103924]: fsmap 12 up:standby 2023-11-06T01:32:22.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:22 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-06T01:32:22.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.659+0000 7f3d98b2e700 1 -- 172.21.15.53:0/1811324093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d9402cc70 con 0x7f3d940ffc70 2023-11-06T01:32:22.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.660+0000 7f3d837fe700 1 -- 172.21.15.53:0/1811324093 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9586 (secure 0 0 0) 0x7f3d840b6e60 con 0x7f3d940ffc70 2023-11-06T01:32:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:22.662 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-11-06T01:32:22.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 -- 172.21.15.53:0/1811324093 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3d7c0a0570 msgr2=0x7f3d7c0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:22.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3d7c0a0570 0x7f3d7c0a2a30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3d88006fd0 tx=0x7f3d88009190 comp rx=0 tx=0).stop 2023-11-06T01:32:22.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 -- 172.21.15.53:0/1811324093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d940ffc70 msgr2=0x7f3d94198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:22.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d940ffc70 0x7f3d94198d50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400b550 tx=0x7f3d8400b860 comp rx=0 tx=0).stop 2023-11-06T01:32:22.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 -- 172.21.15.53:0/1811324093 shutdown_connections 2023-11-06T01:32:22.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3d7c0a0570 0x7f3d7c0a2a30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d94101800 0x7f3d941a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d940ffc70 0x7f3d94198d50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 --2- 172.21.15.53:0/1811324093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d94100e60 0x7f3d94199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 -- 172.21.15.53:0/1811324093 >> 172.21.15.53:0/1811324093 conn(0x7f3d940fb1c0 msgr2=0x7f3d940fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:22.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 -- 172.21.15.53:0/1811324093 shutdown_connections 2023-11-06T01:32:22.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.662+0000 7f3d98b2e700 1 -- 172.21.15.53:0/1811324093 wait complete. 2023-11-06T01:32:22.666 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 13 2023-11-06T01:32:22.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:32:22.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:22.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:32:22.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:22 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] up:boot 2023-11-06T01:32:22.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:22 smithi163 ceph-mon[104416]: fsmap 12 up:standby 2023-11-06T01:32:22.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:22 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-06T01:32:22.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.807+0000 7f9e717e4700 1 -- 172.21.15.53:0/3332439051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c102920 msgr2=0x7f9e6c1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:22.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.807+0000 7f9e717e4700 1 --2- 172.21.15.53:0/3332439051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c102920 0x7f9e6c1091d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400b0d0 tx=0x7f9e6400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.807+0000 7f9e717e4700 1 -- 172.21.15.53:0/3332439051 shutdown_connections 2023-11-06T01:32:22.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.807+0000 7f9e717e4700 1 --2- 172.21.15.53:0/3332439051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c100d90 0x7f9e6c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.807+0000 7f9e717e4700 1 --2- 172.21.15.53:0/3332439051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c102920 0x7f9e6c1091d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.807+0000 7f9e717e4700 1 --2- 172.21.15.53:0/3332439051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e6c101f80 0x7f9e6c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.807+0000 7f9e717e4700 1 -- 172.21.15.53:0/3332439051 >> 172.21.15.53:0/3332439051 conn(0x7f9e6c0fc2c0 msgr2=0x7f9e6c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:22.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.807+0000 7f9e717e4700 1 -- 172.21.15.53:0/3332439051 shutdown_connections 2023-11-06T01:32:22.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.807+0000 7f9e717e4700 1 -- 172.21.15.53:0/3332439051 wait complete. 2023-11-06T01:32:22.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.807+0000 7f9e717e4700 1 Processor -- start 2023-11-06T01:32:22.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e717e4700 1 -- start start 2023-11-06T01:32:22.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e717e4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c100d90 0x7f9e6c196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:22.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e717e4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c101f80 0x7f9e6c196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:22.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e717e4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e6c102920 0x7f9e6c19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:22.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e717e4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c19e630 con 0x7f9e6c101f80 2023-11-06T01:32:22.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e717e4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c19e770 con 0x7f9e6c100d90 2023-11-06T01:32:22.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e717e4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c19e900 con 0x7f9e6c102920 2023-11-06T01:32:22.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e6a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c101f80 0x7f9e6c196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:22.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e6a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c101f80 0x7f9e6c196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37864/0 (socket says 172.21.15.53:37864) 2023-11-06T01:32:22.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e6a7fc700 1 -- 172.21.15.53:0/1581580076 learned_addr learned my addr 172.21.15.53:0/1581580076 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:22.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e6affd700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c100d90 0x7f9e6c196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:22.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e6b7fe700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e6c102920 0x7f9e6c19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:22.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e6a7fc700 1 -- 172.21.15.53:0/1581580076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e6c102920 msgr2=0x7f9e6c19e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:22.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e6a7fc700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e6c102920 0x7f9e6c19e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e6a7fc700 1 -- 172.21.15.53:0/1581580076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c100d90 msgr2=0x7f9e6c196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:22.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e6a7fc700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c100d90 0x7f9e6c196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:22.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e6a7fc700 1 -- 172.21.15.53:0/1581580076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c009520 con 0x7f9e6c101f80 2023-11-06T01:32:22.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e6a7fc700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c101f80 0x7f9e6c196ff0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000e9a0 tx=0x7f9e6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:22.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.808+0000 7f9e53fff700 1 -- 172.21.15.53:0/1581580076 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60009790 con 0x7f9e6c101f80 2023-11-06T01:32:22.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.809+0000 7f9e717e4700 1 -- 172.21.15.53:0/1581580076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e64009d00 con 0x7f9e6c101f80 2023-11-06T01:32:22.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.809+0000 7f9e53fff700 1 -- 172.21.15.53:0/1581580076 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e60010430 con 0x7f9e6c101f80 2023-11-06T01:32:22.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.809+0000 7f9e53fff700 1 -- 172.21.15.53:0/1581580076 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60018680 con 0x7f9e6c101f80 2023-11-06T01:32:22.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.809+0000 7f9e717e4700 1 -- 172.21.15.53:0/1581580076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c19ef60 con 0x7f9e6c101f80 2023-11-06T01:32:22.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.810+0000 7f9e717e4700 1 -- 172.21.15.53:0/1581580076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6c10c5b0 con 0x7f9e6c101f80 2023-11-06T01:32:22.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.811+0000 7f9e53fff700 1 -- 172.21.15.53:0/1581580076 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f9e600105a0 con 0x7f9e6c101f80 2023-11-06T01:32:22.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.811+0000 7f9e53fff700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e540a0570 0x7f9e540a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:22.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.811+0000 7f9e53fff700 1 -- 172.21.15.53:0/1581580076 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9e60014070 con 0x7f9e6c101f80 2023-11-06T01:32:22.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.811+0000 7f9e6affd700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e540a0570 0x7f9e540a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:22.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.812+0000 7f9e6affd700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e540a0570 0x7f9e540a2a30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c101de0 tx=0x7f9e5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:22.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:22.816+0000 7f9e53fff700 1 -- 172.21.15.53:0/1581580076 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9e600520e0 con 0x7f9e6c101f80 2023-11-06T01:32:23.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.043+0000 7f9e717e4700 1 -- 172.21.15.53:0/1581580076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e6c107bc0 con 0x7f9e6c101f80 2023-11-06T01:32:23.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.044+0000 7f9e53fff700 1 -- 172.21.15.53:0/1581580076 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f9e600956f0 con 0x7f9e6c101f80 2023-11-06T01:32:23.046 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:23.046 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:31:46.410396+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:32:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 -- 172.21.15.53:0/1581580076 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e540a0570 msgr2=0x7f9e540a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e540a0570 0x7f9e540a2a30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c101de0 tx=0x7f9e5c009260 comp rx=0 tx=0).stop 2023-11-06T01:32:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 -- 172.21.15.53:0/1581580076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c101f80 msgr2=0x7f9e6c196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c101f80 0x7f9e6c196ff0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000e9a0 tx=0x7f9e6000ecb0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 -- 172.21.15.53:0/1581580076 shutdown_connections 2023-11-06T01:32:23.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e540a0570 0x7f9e540a2a30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c100d90 0x7f9e6c196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e6c102920 0x7f9e6c19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 --2- 172.21.15.53:0/1581580076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e6c101f80 0x7f9e6c196ff0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 -- 172.21.15.53:0/1581580076 >> 172.21.15.53:0/1581580076 conn(0x7f9e6c0fc2c0 msgr2=0x7f9e6c0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:23.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 -- 172.21.15.53:0/1581580076 shutdown_connections 2023-11-06T01:32:23.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.046+0000 7f9e717e4700 1 -- 172.21.15.53:0/1581580076 wait complete. 2023-11-06T01:32:23.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:32:23.061 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T01:32:23.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.198+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/3901696198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9a40ff080 msgr2=0x7fb9a40ff4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.198+0000 7fb9ab9ba700 1 --2- 172.21.15.53:0/3901696198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9a40ff080 0x7fb9a40ff4a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009990 tx=0x7fb9a0009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.200+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/3901696198 shutdown_connections 2023-11-06T01:32:23.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.200+0000 7fb9ab9ba700 1 --2- 172.21.15.53:0/3901696198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9a41029f0 0x7fb9a41092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.200+0000 7fb9ab9ba700 1 --2- 172.21.15.53:0/3901696198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9a40ff080 0x7fb9a40ff4a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.200+0000 7fb9ab9ba700 1 --2- 172.21.15.53:0/3901696198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a4102050 0x7fb9a41024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.200+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/3901696198 >> 172.21.15.53:0/3901696198 conn(0x7fb9a40fa9b0 msgr2=0x7fb9a40fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:23.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.200+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/3901696198 shutdown_connections 2023-11-06T01:32:23.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.200+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/3901696198 wait complete. 2023-11-06T01:32:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9ab9ba700 1 Processor -- start 2023-11-06T01:32:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9ab9ba700 1 -- start start 2023-11-06T01:32:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9ab9ba700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a40ff080 0x7fb9a4196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9ab9ba700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9a4102050 0x7fb9a4196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9ab9ba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9a41029f0 0x7fb9a419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9ab9ba700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a419e610 con 0x7fb9a4102050 2023-11-06T01:32:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9ab9ba700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a419e750 con 0x7fb9a41029f0 2023-11-06T01:32:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9ab9ba700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a419e8e0 con 0x7fb9a40ff080 2023-11-06T01:32:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9a8f55700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9a4102050 0x7fb9a4196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9a8f55700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9a4102050 0x7fb9a4196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37880/0 (socket says 172.21.15.53:37880) 2023-11-06T01:32:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9a8f55700 1 -- 172.21.15.53:0/1042104273 learned_addr learned my addr 172.21.15.53:0/1042104273 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9a9756700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a40ff080 0x7fb9a4196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9a9f57700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9a41029f0 0x7fb9a419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9a8f55700 1 -- 172.21.15.53:0/1042104273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a40ff080 msgr2=0x7fb9a4196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9a8f55700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a40ff080 0x7fb9a4196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9a8f55700 1 -- 172.21.15.53:0/1042104273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9a41029f0 msgr2=0x7fb9a419e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9a8f55700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9a41029f0 0x7fb9a419e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9a8f55700 1 -- 172.21.15.53:0/1042104273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a00095f0 con 0x7fb9a4102050 2023-11-06T01:32:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.201+0000 7fb9a9f57700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9a41029f0 0x7fb9a419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.202+0000 7fb9a8f55700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9a4102050 0x7fb9a4196fd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb99400e9a0 tx=0x7fb99400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.202+0000 7fb99a7fc700 1 -- 172.21.15.53:0/1042104273 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994009790 con 0x7fb9a4102050 2023-11-06T01:32:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.202+0000 7fb99a7fc700 1 -- 172.21.15.53:0/1042104273 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb99400cbd0 con 0x7fb9a4102050 2023-11-06T01:32:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.202+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/1042104273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a419ebc0 con 0x7fb9a4102050 2023-11-06T01:32:23.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.202+0000 7fb99a7fc700 1 -- 172.21.15.53:0/1042104273 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994018680 con 0x7fb9a4102050 2023-11-06T01:32:23.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.202+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/1042104273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a419f120 con 0x7fb9a4102050 2023-11-06T01:32:23.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.203+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/1042104273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a4107e10 con 0x7fb9a4102050 2023-11-06T01:32:23.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.205+0000 7fb99a7fc700 1 -- 172.21.15.53:0/1042104273 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fb99400cd40 con 0x7fb9a4102050 2023-11-06T01:32:23.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.205+0000 7fb99a7fc700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9900a05a0 0x7fb9900a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.205+0000 7fb99a7fc700 1 -- 172.21.15.53:0/1042104273 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb994014070 con 0x7fb9a4102050 2023-11-06T01:32:23.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.205+0000 7fb9a9756700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9900a05a0 0x7fb9900a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.206+0000 7fb9a9756700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9900a05a0 0x7fb9900a2a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b2e0 tx=0x7fb9a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:23.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.209+0000 7fb99a7fc700 1 -- 172.21.15.53:0/1042104273 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb9940959a0 con 0x7fb9a4102050 2023-11-06T01:32:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.416+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/1042104273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9a402d100 con 0x7fb9a4102050 2023-11-06T01:32:23.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.417+0000 7fb99a7fc700 1 -- 172.21.15.53:0/1042104273 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb994098fc0 con 0x7fb9a4102050 2023-11-06T01:32:23.419 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:32:23.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/1042104273 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9900a05a0 msgr2=0x7fb9900a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9900a05a0 0x7fb9900a2a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b2e0 tx=0x7fb9a0005fb0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/1042104273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9a4102050 msgr2=0x7fb9a4196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9a4102050 0x7fb9a4196fd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb99400e9a0 tx=0x7fb99400ecb0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/1042104273 shutdown_connections 2023-11-06T01:32:23.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9900a05a0 0x7fb9900a2a60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9a41029f0 0x7fb9a419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a40ff080 0x7fb9a4196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 --2- 172.21.15.53:0/1042104273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9a4102050 0x7fb9a4196fd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/1042104273 >> 172.21.15.53:0/1042104273 conn(0x7fb9a40fa9b0 msgr2=0x7fb9a40fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/1042104273 shutdown_connections 2023-11-06T01:32:23.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.419+0000 7fb9ab9ba700 1 -- 172.21.15.53:0/1042104273 wait complete. 2023-11-06T01:32:23.433 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T01:32:23.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.569+0000 7f97653a3700 1 -- 172.21.15.53:0/3457115601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9760106af0 msgr2=0x7f9760108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.569+0000 7f97653a3700 1 --2- 172.21.15.53:0/3457115601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9760106af0 0x7f9760108ee0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f975800b0a0 tx=0x7f975800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.569+0000 7f97653a3700 1 -- 172.21.15.53:0/3457115601 shutdown_connections 2023-11-06T01:32:23.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.569+0000 7f97653a3700 1 --2- 172.21.15.53:0/3457115601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9760106af0 0x7f9760108ee0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.569+0000 7f97653a3700 1 --2- 172.21.15.53:0/3457115601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9760101890 0x7f9760103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.569+0000 7f97653a3700 1 --2- 172.21.15.53:0/3457115601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97601041c0 0x7f97601065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.569+0000 7f97653a3700 1 -- 172.21.15.53:0/3457115601 >> 172.21.15.53:0/3457115601 conn(0x7f97600fb1c0 msgr2=0x7f97600fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:23.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.569+0000 7f97653a3700 1 -- 172.21.15.53:0/3457115601 shutdown_connections 2023-11-06T01:32:23.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.569+0000 7f97653a3700 1 -- 172.21.15.53:0/3457115601 wait complete. 2023-11-06T01:32:23.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f97653a3700 1 Processor -- start 2023-11-06T01:32:23.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f97653a3700 1 -- start start 2023-11-06T01:32:23.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f97653a3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9760101890 0x7f9760101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f97653a3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97601041c0 0x7f97600ff840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f97653a3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9760106af0 0x7f97600ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f97653a3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9760100520 con 0x7f97601041c0 2023-11-06T01:32:23.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f97653a3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9760100690 con 0x7f9760101890 2023-11-06T01:32:23.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f97653a3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97601a0be0 con 0x7f9760106af0 2023-11-06T01:32:23.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f975e7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97601041c0 0x7f97600ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f975effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9760101890 0x7f9760101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f975effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9760101890 0x7f9760101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:49328/0 (socket says 172.21.15.53:49328) 2023-11-06T01:32:23.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f975effd700 1 -- 172.21.15.53:0/4253319910 learned_addr learned my addr 172.21.15.53:0/4253319910 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f975e7fc700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97601041c0 0x7f97600ff840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37892/0 (socket says 172.21.15.53:37892) 2023-11-06T01:32:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.570+0000 7f975f7fe700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9760106af0 0x7f97600ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f975e7fc700 1 -- 172.21.15.53:0/4253319910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9760106af0 msgr2=0x7f97600ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f975e7fc700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9760106af0 0x7f97600ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f975e7fc700 1 -- 172.21.15.53:0/4253319910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9760101890 msgr2=0x7f9760101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f975e7fc700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9760101890 0x7f9760101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f975e7fc700 1 -- 172.21.15.53:0/4253319910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9758009d00 con 0x7f97601041c0 2023-11-06T01:32:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f975effd700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9760101890 0x7f9760101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:32:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f975e7fc700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97601041c0 0x7f97600ff840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f975400b450 tx=0x7f975400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f9747fff700 1 -- 172.21.15.53:0/4253319910 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754011830 con 0x7f97601041c0 2023-11-06T01:32:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f9747fff700 1 -- 172.21.15.53:0/4253319910 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f975400be60 con 0x7f97601041c0 2023-11-06T01:32:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f97653a3700 1 -- 172.21.15.53:0/4253319910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97601a0ec0 con 0x7f97601041c0 2023-11-06T01:32:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f9747fff700 1 -- 172.21.15.53:0/4253319910 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754010d20 con 0x7f97601041c0 2023-11-06T01:32:23.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.571+0000 7f97653a3700 1 -- 172.21.15.53:0/4253319910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97601a1370 con 0x7f97601041c0 2023-11-06T01:32:23.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.573+0000 7f97653a3700 1 -- 172.21.15.53:0/4253319910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97600fcd40 con 0x7f97601041c0 2023-11-06T01:32:23.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.573+0000 7f9747fff700 1 -- 172.21.15.53:0/4253319910 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f9754010380 con 0x7f97601041c0 2023-11-06T01:32:23.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.574+0000 7f9747fff700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f97480a0570 0x7f97480a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.574+0000 7f9747fff700 1 -- 172.21.15.53:0/4253319910 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f97540cae50 con 0x7f97601041c0 2023-11-06T01:32:23.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.574+0000 7f975effd700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f97480a0570 0x7f97480a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.574+0000 7f975effd700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f97480a0570 0x7f97480a2a30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9750005950 tx=0x7f975000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:23.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.578+0000 7f9747fff700 1 -- 172.21.15.53:0/4253319910 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9754094e90 con 0x7f97601041c0 2023-11-06T01:32:23.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:23 smithi062 ceph-mon[103924]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:23.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:23 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1811324093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:23.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:23 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1581580076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:23.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:23 smithi163 ceph-mon[104416]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:23.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:23 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1811324093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:23.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:23 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1581580076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:23.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.793+0000 7f97653a3700 1 -- 172.21.15.53:0/4253319910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f976002ce70 con 0x7f97601041c0 2023-11-06T01:32:23.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.794+0000 7f9747fff700 1 -- 172.21.15.53:0/4253319910 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7f97540984b0 con 0x7f97601041c0 2023-11-06T01:32:23.796 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:23.796 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-11-06T01:32:23.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:32:23.797 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-11-06T01:32:23.797 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-11-06T01:32:23.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:32:23.797 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-11-06T01:32:23.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 -- 172.21.15.53:0/4253319910 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f97480a0570 msgr2=0x7f97480a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f97480a0570 0x7f97480a2a30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9750005950 tx=0x7f975000b500 comp rx=0 tx=0).stop 2023-11-06T01:32:23.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 -- 172.21.15.53:0/4253319910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97601041c0 msgr2=0x7f97600ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97601041c0 0x7f97600ff840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f975400b450 tx=0x7f975400b760 comp rx=0 tx=0).stop 2023-11-06T01:32:23.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 -- 172.21.15.53:0/4253319910 shutdown_connections 2023-11-06T01:32:23.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f97480a0570 0x7f97480a2a30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9760101890 0x7f9760101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9760106af0 0x7f97600ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 --2- 172.21.15.53:0/4253319910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97601041c0 0x7f97600ff840 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 -- 172.21.15.53:0/4253319910 >> 172.21.15.53:0/4253319910 conn(0x7f97600fb1c0 msgr2=0x7f9760102580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:23.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 -- 172.21.15.53:0/4253319910 shutdown_connections 2023-11-06T01:32:23.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.796+0000 7f97653a3700 1 -- 172.21.15.53:0/4253319910 wait complete. 2023-11-06T01:32:23.811 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:23 smithi053 ceph-mon[99472]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:23.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:23 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1811324093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:23.812 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:23 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1581580076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:23.813 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T01:32:23.813 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T01:32:23.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.944+0000 7f4bc5468700 1 -- 172.21.15.53:0/3144411005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc0100ee0 msgr2=0x7f4bc0101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.944+0000 7f4bc5468700 1 --2- 172.21.15.53:0/3144411005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc0100ee0 0x7f4bc0101340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40099c0 tx=0x7f4bb4009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.945+0000 7f4bc5468700 1 -- 172.21.15.53:0/3144411005 shutdown_connections 2023-11-06T01:32:23.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.945+0000 7f4bc5468700 1 --2- 172.21.15.53:0/3144411005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc0101880 0x7f4bc0108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.945+0000 7f4bc5468700 1 --2- 172.21.15.53:0/3144411005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc00ffcf0 0x7f4bc0100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.945+0000 7f4bc5468700 1 --2- 172.21.15.53:0/3144411005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc0100ee0 0x7f4bc0101340 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.945+0000 7f4bc5468700 1 -- 172.21.15.53:0/3144411005 >> 172.21.15.53:0/3144411005 conn(0x7f4bc00fb260 msgr2=0x7f4bc00fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:23.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.945+0000 7f4bc5468700 1 -- 172.21.15.53:0/3144411005 shutdown_connections 2023-11-06T01:32:23.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.945+0000 7f4bc5468700 1 -- 172.21.15.53:0/3144411005 wait complete. 2023-11-06T01:32:23.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.945+0000 7f4bc5468700 1 Processor -- start 2023-11-06T01:32:23.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.945+0000 7f4bc5468700 1 -- start start 2023-11-06T01:32:23.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bc5468700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc00ffcf0 0x7f4bc0069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bc5468700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc0100ee0 0x7f4bc019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bc5468700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc0101880 0x7f4bc019de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bc5468700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc019e5b0 con 0x7f4bc0101880 2023-11-06T01:32:23.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bc5468700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc019e720 con 0x7f4bc00ffcf0 2023-11-06T01:32:23.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bc5468700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc019e8e0 con 0x7f4bc0100ee0 2023-11-06T01:32:23.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbf7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc0101880 0x7f4bc019de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbf7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc0101880 0x7f4bc019de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37908/0 (socket says 172.21.15.53:37908) 2023-11-06T01:32:23.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbf7fe700 1 -- 172.21.15.53:0/2913572979 learned_addr learned my addr 172.21.15.53:0/2913572979 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:23.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbe7fc700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc0100ee0 0x7f4bc019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbeffd700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc00ffcf0 0x7f4bc0069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbf7fe700 1 -- 172.21.15.53:0/2913572979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc0100ee0 msgr2=0x7f4bc019b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbf7fe700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc0100ee0 0x7f4bc019b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbf7fe700 1 -- 172.21.15.53:0/2913572979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc00ffcf0 msgr2=0x7f4bc0069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:23.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbf7fe700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc00ffcf0 0x7f4bc0069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:23.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbf7fe700 1 -- 172.21.15.53:0/2913572979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb40095f0 con 0x7f4bc0101880 2023-11-06T01:32:23.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbe7fc700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc0100ee0 0x7f4bc019b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:32:23.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bbf7fe700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc0101880 0x7f4bc019de10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007ee0 tx=0x7f4bb800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:23.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4ba7fff700 1 -- 172.21.15.53:0/2913572979 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8017070 con 0x7f4bc0101880 2023-11-06T01:32:23.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4ba7fff700 1 -- 172.21.15.53:0/2913572979 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bb800f040 con 0x7f4bc0101880 2023-11-06T01:32:23.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4bc5468700 1 -- 172.21.15.53:0/2913572979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc019ebc0 con 0x7f4bc0101880 2023-11-06T01:32:23.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.946+0000 7f4ba7fff700 1 -- 172.21.15.53:0/2913572979 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8005740 con 0x7f4bc0101880 2023-11-06T01:32:23.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.947+0000 7f4bc5468700 1 -- 172.21.15.53:0/2913572979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc019f080 con 0x7f4bc0101880 2023-11-06T01:32:23.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.948+0000 7f4bc5468700 1 -- 172.21.15.53:0/2913572979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc010b4e0 con 0x7f4bc0101880 2023-11-06T01:32:23.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.949+0000 7f4ba7fff700 1 -- 172.21.15.53:0/2913572979 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4bb800dc60 con 0x7f4bc0101880 2023-11-06T01:32:23.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.950+0000 7f4ba7fff700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4ba80a0520 0x7f4ba80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:23.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.950+0000 7f4ba7fff700 1 -- 172.21.15.53:0/2913572979 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4bb80cb320 con 0x7f4bc0101880 2023-11-06T01:32:23.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.950+0000 7f4bbeffd700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4ba80a0520 0x7f4ba80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:23.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.950+0000 7f4bbeffd700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4ba80a0520 0x7f4ba80a29e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0100d40 tx=0x7f4bb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:23.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:23.953+0000 7f4ba7fff700 1 -- 172.21.15.53:0/2913572979 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4bb8095360 con 0x7f4bc0101880 2023-11-06T01:32:24.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.165+0000 7f4bc5468700 1 -- 172.21.15.53:0/2913572979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4bc0106c30 con 0x7f4bc0101880 2023-11-06T01:32:24.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.657+0000 7f4ba7fff700 1 -- 172.21.15.53:0/2913572979 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4bb8098980 con 0x7f4bc0101880 2023-11-06T01:32:24.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.663+0000 7f4bc5468700 1 -- 172.21.15.53:0/2913572979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4ba80a0520 msgr2=0x7f4ba80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:24.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.663+0000 7f4bc5468700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4ba80a0520 0x7f4ba80a29e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0100d40 tx=0x7f4bb0009260 comp rx=0 tx=0).stop 2023-11-06T01:32:24.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.663+0000 7f4bc5468700 1 -- 172.21.15.53:0/2913572979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc0101880 msgr2=0x7f4bc019de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:24.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.663+0000 7f4bc5468700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc0101880 0x7f4bc019de10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007ee0 tx=0x7f4bb800d3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:24.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.664+0000 7f4bc5468700 1 -- 172.21.15.53:0/2913572979 shutdown_connections 2023-11-06T01:32:24.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.664+0000 7f4bc5468700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4ba80a0520 0x7f4ba80a29e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:24.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.664+0000 7f4bc5468700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc00ffcf0 0x7f4bc0069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:24.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.664+0000 7f4bc5468700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc0101880 0x7f4bc019de10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:24.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.664+0000 7f4bc5468700 1 --2- 172.21.15.53:0/2913572979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc0100ee0 0x7f4bc019b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:24.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.664+0000 7f4bc5468700 1 -- 172.21.15.53:0/2913572979 >> 172.21.15.53:0/2913572979 conn(0x7f4bc00fb260 msgr2=0x7f4bc00fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:24.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.665+0000 7f4bc5468700 1 -- 172.21.15.53:0/2913572979 shutdown_connections 2023-11-06T01:32:24.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.665+0000 7f4bc5468700 1 -- 172.21.15.53:0/2913572979 wait complete. 2023-11-06T01:32:24.668 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-11-06T01:32:24.680 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T01:32:24.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.812+0000 7f7a66897700 1 -- 172.21.15.53:0/3842329932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a60100e70 msgr2=0x7f7a601012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:24.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.812+0000 7f7a66897700 1 --2- 172.21.15.53:0/3842329932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a60100e70 0x7f7a601012d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7a480099c0 tx=0x7f7a48009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:24.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.812+0000 7f7a66897700 1 -- 172.21.15.53:0/3842329932 shutdown_connections 2023-11-06T01:32:24.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.812+0000 7f7a66897700 1 --2- 172.21.15.53:0/3842329932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a60101810 0x7f7a601080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:24.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.812+0000 7f7a66897700 1 --2- 172.21.15.53:0/3842329932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a600ffc80 0x7f7a601000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:24.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.812+0000 7f7a66897700 1 --2- 172.21.15.53:0/3842329932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a60100e70 0x7f7a601012d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:24.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.812+0000 7f7a66897700 1 -- 172.21.15.53:0/3842329932 >> 172.21.15.53:0/3842329932 conn(0x7f7a600fb1f0 msgr2=0x7f7a600fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:24.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.812+0000 7f7a66897700 1 -- 172.21.15.53:0/3842329932 shutdown_connections 2023-11-06T01:32:24.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.812+0000 7f7a66897700 1 -- 172.21.15.53:0/3842329932 wait complete. 2023-11-06T01:32:24.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.812+0000 7f7a66897700 1 Processor -- start 2023-11-06T01:32:24.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.812+0000 7f7a66897700 1 -- start start 2023-11-06T01:32:24.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a66897700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a600ffc80 0x7f7a60196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:24.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a66897700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a60100e70 0x7f7a60196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:24.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a66897700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a60101810 0x7f7a6019e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:24.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a66897700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6019e600 con 0x7f7a600ffc80 2023-11-06T01:32:24.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a66897700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6019e740 con 0x7f7a60100e70 2023-11-06T01:32:24.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a66897700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6019e8d0 con 0x7f7a60101810 2023-11-06T01:32:24.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a5ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a600ffc80 0x7f7a60196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:24.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a5ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a600ffc80 0x7f7a60196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37918/0 (socket says 172.21.15.53:37918) 2023-11-06T01:32:24.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a5ffff700 1 -- 172.21.15.53:0/2657056621 learned_addr learned my addr 172.21.15.53:0/2657056621 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:24.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a5f7fe700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a60100e70 0x7f7a60196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:24.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a64e34700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a60101810 0x7f7a6019e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:24.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a5ffff700 1 -- 172.21.15.53:0/2657056621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a60101810 msgr2=0x7f7a6019e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:24.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a5ffff700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a60101810 0x7f7a6019e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:24.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a5ffff700 1 -- 172.21.15.53:0/2657056621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a60100e70 msgr2=0x7f7a60196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:24.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a5ffff700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a60100e70 0x7f7a60196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:24.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a5ffff700 1 -- 172.21.15.53:0/2657056621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a480095f0 con 0x7f7a600ffc80 2023-11-06T01:32:24.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a5ffff700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a600ffc80 0x7f7a60196a80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d980 tx=0x7f7a5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:24.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.813+0000 7f7a5d7fa700 1 -- 172.21.15.53:0/2657056621 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50004020 con 0x7f7a600ffc80 2023-11-06T01:32:24.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.814+0000 7f7a5d7fa700 1 -- 172.21.15.53:0/2657056621 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a5000f690 con 0x7f7a600ffc80 2023-11-06T01:32:24.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.814+0000 7f7a66897700 1 -- 172.21.15.53:0/2657056621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6019ebb0 con 0x7f7a600ffc80 2023-11-06T01:32:24.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.814+0000 7f7a5d7fa700 1 -- 172.21.15.53:0/2657056621 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50003a60 con 0x7f7a600ffc80 2023-11-06T01:32:24.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.814+0000 7f7a66897700 1 -- 172.21.15.53:0/2657056621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6019f0c0 con 0x7f7a600ffc80 2023-11-06T01:32:24.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.814+0000 7f7a66897700 1 -- 172.21.15.53:0/2657056621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a60107e00 con 0x7f7a600ffc80 2023-11-06T01:32:24.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.816+0000 7f7a5d7fa700 1 -- 172.21.15.53:0/2657056621 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f7a5000fb00 con 0x7f7a600ffc80 2023-11-06T01:32:24.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.816+0000 7f7a5d7fa700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7a4c0a0520 0x7f7a4c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:24.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.816+0000 7f7a5d7fa700 1 -- 172.21.15.53:0/2657056621 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f7a500cb570 con 0x7f7a600ffc80 2023-11-06T01:32:24.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.816+0000 7f7a5f7fe700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7a4c0a0520 0x7f7a4c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:24.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.817+0000 7f7a5f7fe700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7a4c0a0520 0x7f7a4c0a29e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800b580 tx=0x7f7a48005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:24.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:24.820+0000 7f7a5d7fa700 1 -- 172.21.15.53:0/2657056621 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7a50095440 con 0x7f7a600ffc80 2023-11-06T01:32:24.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:24 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1042104273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:32:24.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:24 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/4253319910' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-06T01:32:24.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:24 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1042104273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:32:24.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:24 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/4253319910' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-06T01:32:25.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.023+0000 7f7a66897700 1 -- 172.21.15.53:0/2657056621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7a6002cc70 con 0x7f7a600ffc80 2023-11-06T01:32:25.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:24 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1042104273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:32:25.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:24 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/4253319910' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-06T01:32:25.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.660+0000 7f7a5d7fa700 1 -- 172.21.15.53:0/2657056621 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7a50098a60 con 0x7f7a600ffc80 2023-11-06T01:32:25.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.665+0000 7f7a66897700 1 -- 172.21.15.53:0/2657056621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7a4c0a0520 msgr2=0x7f7a4c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:25.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.665+0000 7f7a66897700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7a4c0a0520 0x7f7a4c0a29e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800b580 tx=0x7f7a48005bc0 comp rx=0 tx=0).stop 2023-11-06T01:32:25.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.665+0000 7f7a66897700 1 -- 172.21.15.53:0/2657056621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a600ffc80 msgr2=0x7f7a60196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:25.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.665+0000 7f7a66897700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a600ffc80 0x7f7a60196a80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d980 tx=0x7f7a5000dc90 comp rx=0 tx=0).stop 2023-11-06T01:32:25.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.666+0000 7f7a66897700 1 -- 172.21.15.53:0/2657056621 shutdown_connections 2023-11-06T01:32:25.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.666+0000 7f7a66897700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7a4c0a0520 0x7f7a4c0a29e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:25.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.666+0000 7f7a66897700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a60101810 0x7f7a6019e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:25.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.666+0000 7f7a66897700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a600ffc80 0x7f7a60196a80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:25.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.666+0000 7f7a66897700 1 --2- 172.21.15.53:0/2657056621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a60100e70 0x7f7a60196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:25.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.666+0000 7f7a66897700 1 -- 172.21.15.53:0/2657056621 >> 172.21.15.53:0/2657056621 conn(0x7f7a600fb1f0 msgr2=0x7f7a600fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:25.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.668+0000 7f7a66897700 1 -- 172.21.15.53:0/2657056621 shutdown_connections 2023-11-06T01:32:25.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.670+0000 7f7a66897700 1 -- 172.21.15.53:0/2657056621 wait complete. 2023-11-06T01:32:25.683 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-11-06T01:32:25.693 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T01:32:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.822+0000 7f14cbc2f700 1 -- 172.21.15.53:0/457714564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c40fede0 msgr2=0x7f14c40ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.822+0000 7f14cbc2f700 1 --2- 172.21.15.53:0/457714564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c40fede0 0x7f14c40ff240 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f14b80099c0 tx=0x7f14b8009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.822+0000 7f14cbc2f700 1 -- 172.21.15.53:0/457714564 shutdown_connections 2023-11-06T01:32:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.822+0000 7f14cbc2f700 1 --2- 172.21.15.53:0/457714564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14c40ff780 0x7f14c4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.822+0000 7f14cbc2f700 1 --2- 172.21.15.53:0/457714564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14c4102370 0x7f14c4102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.822+0000 7f14cbc2f700 1 --2- 172.21.15.53:0/457714564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c40fede0 0x7f14c40ff240 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.822+0000 7f14cbc2f700 1 -- 172.21.15.53:0/457714564 >> 172.21.15.53:0/457714564 conn(0x7f14c40fa960 msgr2=0x7f14c40fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.822+0000 7f14cbc2f700 1 -- 172.21.15.53:0/457714564 shutdown_connections 2023-11-06T01:32:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.822+0000 7f14cbc2f700 1 -- 172.21.15.53:0/457714564 wait complete. 2023-11-06T01:32:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14cbc2f700 1 Processor -- start 2023-11-06T01:32:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14cbc2f700 1 -- start start 2023-11-06T01:32:25.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14cbc2f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c40fede0 0x7f14c4196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:25.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14cbc2f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14c40ff780 0x7f14c4197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:25.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14cbc2f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14c4102370 0x7f14c419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:25.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14cbc2f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c419e680 con 0x7f14c40fede0 2023-11-06T01:32:25.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14cbc2f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c419e7c0 con 0x7f14c4102370 2023-11-06T01:32:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14cbc2f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c419e950 con 0x7f14c40ff780 2023-11-06T01:32:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14c99cb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c40fede0 0x7f14c4196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14c99cb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c40fede0 0x7f14c4196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37940/0 (socket says 172.21.15.53:37940) 2023-11-06T01:32:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14c99cb700 1 -- 172.21.15.53:0/2251692313 learned_addr learned my addr 172.21.15.53:0/2251692313 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14ca1cc700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14c4102370 0x7f14c419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14c91ca700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14c40ff780 0x7f14c4197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14c99cb700 1 -- 172.21.15.53:0/2251692313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14c40ff780 msgr2=0x7f14c4197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14c99cb700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14c40ff780 0x7f14c4197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14c99cb700 1 -- 172.21.15.53:0/2251692313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14c4102370 msgr2=0x7f14c419e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14c99cb700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14c4102370 0x7f14c419e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.823+0000 7f14c99cb700 1 -- 172.21.15.53:0/2251692313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b80095f0 con 0x7f14c40fede0 2023-11-06T01:32:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.824+0000 7f14c99cb700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c40fede0 0x7f14c4196b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f14c000d980 tx=0x7f14c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.824+0000 7f14b6ffd700 1 -- 172.21.15.53:0/2251692313 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0004020 con 0x7f14c40fede0 2023-11-06T01:32:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.824+0000 7f14b6ffd700 1 -- 172.21.15.53:0/2251692313 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14c000f690 con 0x7f14c40fede0 2023-11-06T01:32:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.824+0000 7f14cbc2f700 1 -- 172.21.15.53:0/2251692313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c419ec30 con 0x7f14c40fede0 2023-11-06T01:32:25.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.824+0000 7f14b6ffd700 1 -- 172.21.15.53:0/2251692313 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0003a60 con 0x7f14c40fede0 2023-11-06T01:32:25.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.824+0000 7f14cbc2f700 1 -- 172.21.15.53:0/2251692313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c419f140 con 0x7f14c40fede0 2023-11-06T01:32:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.825+0000 7f14cbc2f700 1 -- 172.21.15.53:0/2251692313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c410c5e0 con 0x7f14c40fede0 2023-11-06T01:32:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.827+0000 7f14b6ffd700 1 -- 172.21.15.53:0/2251692313 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f14c000fb00 con 0x7f14c40fede0 2023-11-06T01:32:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.827+0000 7f14b6ffd700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14b00a0520 0x7f14b00a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.827+0000 7f14b6ffd700 1 -- 172.21.15.53:0/2251692313 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f14c00cb6f0 con 0x7f14c40fede0 2023-11-06T01:32:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.827+0000 7f14c91ca700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14b00a0520 0x7f14b00a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.828+0000 7f14c91ca700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14b00a0520 0x7f14b00a29e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f14b8000c00 tx=0x7f14b8009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:25.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:25.830+0000 7f14b6ffd700 1 -- 172.21.15.53:0/2251692313 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f14c0095440 con 0x7f14c40fede0 2023-11-06T01:32:25.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:25.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2913572979' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-06T01:32:25.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2913572979' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-06T01:32:25.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: osdmap e79: 12 total, 12 up, 12 in 2023-11-06T01:32:25.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2657056621' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-06T01:32:25.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:25.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:32:25.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:25 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:25.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2913572979' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-06T01:32:25.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2913572979' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-06T01:32:25.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: osdmap e79: 12 total, 12 up, 12 in 2023-11-06T01:32:25.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2657056621' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-06T01:32:25.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:25.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:32:25.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:25.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:25 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:26.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.091+0000 7f14cbc2f700 1 -- 172.21.15.53:0/2251692313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f14c4107e80 con 0x7f14c40fede0 2023-11-06T01:32:26.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:26.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2913572979' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-06T01:32:26.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:26.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:26.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:26.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:26.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:26.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:26.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2913572979' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-06T01:32:26.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: osdmap e79: 12 total, 12 up, 12 in 2023-11-06T01:32:26.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2657056621' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-06T01:32:26.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:26.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:32:26.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:26.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:25 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:26.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.097+0000 7f14b6ffd700 1 -- 172.21.15.53:0/2251692313 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f14c0098a60 con 0x7f14c40fede0 2023-11-06T01:32:26.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.101+0000 7f14cbc2f700 1 -- 172.21.15.53:0/2251692313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14b00a0520 msgr2=0x7f14b00a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.101+0000 7f14cbc2f700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14b00a0520 0x7f14b00a29e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f14b8000c00 tx=0x7f14b8009f10 comp rx=0 tx=0).stop 2023-11-06T01:32:26.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.101+0000 7f14cbc2f700 1 -- 172.21.15.53:0/2251692313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c40fede0 msgr2=0x7f14c4196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.101+0000 7f14cbc2f700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c40fede0 0x7f14c4196b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f14c000d980 tx=0x7f14c000dc90 comp rx=0 tx=0).stop 2023-11-06T01:32:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.103+0000 7f14cbc2f700 1 -- 172.21.15.53:0/2251692313 shutdown_connections 2023-11-06T01:32:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.103+0000 7f14cbc2f700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14b00a0520 0x7f14b00a29e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.103+0000 7f14cbc2f700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14c4102370 0x7f14c419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.103+0000 7f14cbc2f700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c40fede0 0x7f14c4196b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.103+0000 7f14cbc2f700 1 --2- 172.21.15.53:0/2251692313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14c40ff780 0x7f14c4197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.103+0000 7f14cbc2f700 1 -- 172.21.15.53:0/2251692313 >> 172.21.15.53:0/2251692313 conn(0x7f14c40fa960 msgr2=0x7f14c40fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:26.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.106+0000 7f14cbc2f700 1 -- 172.21.15.53:0/2251692313 shutdown_connections 2023-11-06T01:32:26.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.107+0000 7f14cbc2f700 1 -- 172.21.15.53:0/2251692313 wait complete. 2023-11-06T01:32:26.121 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 2 and data pool 3 2023-11-06T01:32:26.129 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:32:26.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.257+0000 7f4614f03700 1 -- 172.21.15.53:0/912590004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4610100ea0 msgr2=0x7f4610101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.257+0000 7f4614f03700 1 --2- 172.21.15.53:0/912590004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4610100ea0 0x7f4610101300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f46040099c0 tx=0x7f4604009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.257+0000 7f4614f03700 1 -- 172.21.15.53:0/912590004 shutdown_connections 2023-11-06T01:32:26.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.257+0000 7f4614f03700 1 --2- 172.21.15.53:0/912590004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610101840 0x7f46101080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.257+0000 7f4614f03700 1 --2- 172.21.15.53:0/912590004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46100ffcb0 0x7f46101000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.257+0000 7f4614f03700 1 --2- 172.21.15.53:0/912590004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4610100ea0 0x7f4610101300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.257+0000 7f4614f03700 1 -- 172.21.15.53:0/912590004 >> 172.21.15.53:0/912590004 conn(0x7f46100fb220 msgr2=0x7f46100fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:26.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.257+0000 7f4614f03700 1 -- 172.21.15.53:0/912590004 shutdown_connections 2023-11-06T01:32:26.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.257+0000 7f4614f03700 1 -- 172.21.15.53:0/912590004 wait complete. 2023-11-06T01:32:26.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f4614f03700 1 Processor -- start 2023-11-06T01:32:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f4614f03700 1 -- start start 2023-11-06T01:32:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f4614f03700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46100ffcb0 0x7f4610198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f4614f03700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4610100ea0 0x7f46101991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f4614f03700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610101840 0x7f46101a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f4614f03700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46101a0810 con 0x7f4610100ea0 2023-11-06T01:32:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f4614f03700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46101a0980 con 0x7f4610101840 2023-11-06T01:32:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f4614f03700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46101a0b40 con 0x7f46100ffcb0 2023-11-06T01:32:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f460dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4610100ea0 0x7f46101991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f460dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4610100ea0 0x7f46101991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37956/0 (socket says 172.21.15.53:37956) 2023-11-06T01:32:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f460dd9b700 1 -- 172.21.15.53:0/1010938087 learned_addr learned my addr 172.21.15.53:0/1010938087 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f460e59c700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46100ffcb0 0x7f4610198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f460ed9d700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610101840 0x7f46101a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f460dd9b700 1 -- 172.21.15.53:0/1010938087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46100ffcb0 msgr2=0x7f4610198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f460dd9b700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46100ffcb0 0x7f4610198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f460dd9b700 1 -- 172.21.15.53:0/1010938087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610101840 msgr2=0x7f46101a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f460dd9b700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610101840 0x7f46101a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.258+0000 7f460dd9b700 1 -- 172.21.15.53:0/1010938087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46040095f0 con 0x7f4610100ea0 2023-11-06T01:32:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.259+0000 7f460dd9b700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4610100ea0 0x7f46101991e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f460400c010 tx=0x7f4604009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.259+0000 7f45ff7fe700 1 -- 172.21.15.53:0/1010938087 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460401d070 con 0x7f4610100ea0 2023-11-06T01:32:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.259+0000 7f45ff7fe700 1 -- 172.21.15.53:0/1010938087 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46040040d0 con 0x7f4610100ea0 2023-11-06T01:32:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.259+0000 7f4614f03700 1 -- 172.21.15.53:0/1010938087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46101a0dc0 con 0x7f4610100ea0 2023-11-06T01:32:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.259+0000 7f45ff7fe700 1 -- 172.21.15.53:0/1010938087 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604005290 con 0x7f4610100ea0 2023-11-06T01:32:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.259+0000 7f4614f03700 1 -- 172.21.15.53:0/1010938087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46101a1190 con 0x7f4610100ea0 2023-11-06T01:32:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.260+0000 7f4614f03700 1 -- 172.21.15.53:0/1010938087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461010b590 con 0x7f4610100ea0 2023-11-06T01:32:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.261+0000 7f45ff7fe700 1 -- 172.21.15.53:0/1010938087 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4604004810 con 0x7f4610100ea0 2023-11-06T01:32:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.262+0000 7f45ff7fe700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f45f80a0570 0x7f45f80a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.262+0000 7f460e59c700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f45f80a0570 0x7f45f80a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.262+0000 7f45ff7fe700 1 -- 172.21.15.53:0/1010938087 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f46040ccc50 con 0x7f4610100ea0 2023-11-06T01:32:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.262+0000 7f460e59c700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f45f80a0570 0x7f45f80a2a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4610199b70 tx=0x7f460000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:26.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.266+0000 7f45ff7fe700 1 -- 172.21.15.53:0/1010938087 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f46040969d0 con 0x7f4610100ea0 2023-11-06T01:32:26.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.493+0000 7f4614f03700 1 -- 172.21.15.53:0/1010938087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46101a17a0 con 0x7f4610100ea0 2023-11-06T01:32:26.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.496+0000 7f45ff7fe700 1 -- 172.21.15.53:0/1010938087 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22084 (secure 0 0 0) 0x7f4604099ff0 con 0x7f4610100ea0 2023-11-06T01:32:26.497 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:26.498 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":81,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:32:26.092815+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:32:26.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 -- 172.21.15.53:0/1010938087 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f45f80a0570 msgr2=0x7f45f80a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f45f80a0570 0x7f45f80a2a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4610199b70 tx=0x7f460000b410 comp rx=0 tx=0).stop 2023-11-06T01:32:26.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 -- 172.21.15.53:0/1010938087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4610100ea0 msgr2=0x7f46101991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4610100ea0 0x7f46101991e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f460400c010 tx=0x7f4604009580 comp rx=0 tx=0).stop 2023-11-06T01:32:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 -- 172.21.15.53:0/1010938087 shutdown_connections 2023-11-06T01:32:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f45f80a0570 0x7f45f80a2a30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610101840 0x7f46101a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46100ffcb0 0x7f4610198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 --2- 172.21.15.53:0/1010938087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4610100ea0 0x7f46101991e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 -- 172.21.15.53:0/1010938087 >> 172.21.15.53:0/1010938087 conn(0x7f46100fb220 msgr2=0x7f4610068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:26.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 -- 172.21.15.53:0/1010938087 shutdown_connections 2023-11-06T01:32:26.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.498+0000 7f4614f03700 1 -- 172.21.15.53:0/1010938087 wait complete. 2023-11-06T01:32:26.511 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:32:26.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.640+0000 7feb822ab700 1 -- 172.21.15.53:0/2107370331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c100e60 msgr2=0x7feb7c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.640+0000 7feb822ab700 1 --2- 172.21.15.53:0/2107370331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c100e60 0x7feb7c1012c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb70009920 tx=0x7feb70009c30 comp rx=0 tx=0).stop 2023-11-06T01:32:26.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.640+0000 7feb822ab700 1 -- 172.21.15.53:0/2107370331 shutdown_connections 2023-11-06T01:32:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.640+0000 7feb822ab700 1 --2- 172.21.15.53:0/2107370331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c101800 0x7feb7c1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.640+0000 7feb822ab700 1 --2- 172.21.15.53:0/2107370331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb7c0ffc70 0x7feb7c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.640+0000 7feb822ab700 1 --2- 172.21.15.53:0/2107370331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c100e60 0x7feb7c1012c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.640+0000 7feb822ab700 1 -- 172.21.15.53:0/2107370331 >> 172.21.15.53:0/2107370331 conn(0x7feb7c0fb1c0 msgr2=0x7feb7c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.640+0000 7feb822ab700 1 -- 172.21.15.53:0/2107370331 shutdown_connections 2023-11-06T01:32:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.640+0000 7feb822ab700 1 -- 172.21.15.53:0/2107370331 wait complete. 2023-11-06T01:32:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.640+0000 7feb822ab700 1 Processor -- start 2023-11-06T01:32:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb822ab700 1 -- start start 2023-11-06T01:32:26.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb822ab700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0ffc70 0x7feb7c103ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:26.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb822ab700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c100e60 0x7feb7c102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:26.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb822ab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb7c101800 0x7feb7c102a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb822ab700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c103210 con 0x7feb7c0ffc70 2023-11-06T01:32:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb822ab700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c103380 con 0x7feb7c100e60 2023-11-06T01:32:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb822ab700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c1a0ad0 con 0x7feb7c101800 2023-11-06T01:32:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb7bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0ffc70 0x7feb7c103ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb7bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0ffc70 0x7feb7c103ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37968/0 (socket says 172.21.15.53:37968) 2023-11-06T01:32:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb7bfff700 1 -- 172.21.15.53:0/2494134537 learned_addr learned my addr 172.21.15.53:0/2494134537 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb7b7fe700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c100e60 0x7feb7c102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb80848700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb7c101800 0x7feb7c102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb7b7fe700 1 -- 172.21.15.53:0/2494134537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb7c101800 msgr2=0x7feb7c102a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb7b7fe700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb7c101800 0x7feb7c102a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb7b7fe700 1 -- 172.21.15.53:0/2494134537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0ffc70 msgr2=0x7feb7c103ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb7b7fe700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0ffc70 0x7feb7c103ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb7b7fe700 1 -- 172.21.15.53:0/2494134537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb70009520 con 0x7feb7c100e60 2023-11-06T01:32:26.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb7bfff700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0ffc70 0x7feb7c103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:32:26.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.641+0000 7feb7b7fe700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c100e60 0x7feb7c102530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb70000c00 tx=0x7feb7000b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:26.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.642+0000 7feb797fa700 1 -- 172.21.15.53:0/2494134537 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7001c070 con 0x7feb7c100e60 2023-11-06T01:32:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.642+0000 7feb822ab700 1 -- 172.21.15.53:0/2494134537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7c1a0d50 con 0x7feb7c100e60 2023-11-06T01:32:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.642+0000 7feb797fa700 1 -- 172.21.15.53:0/2494134537 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb7000f7d0 con 0x7feb7c100e60 2023-11-06T01:32:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.642+0000 7feb797fa700 1 -- 172.21.15.53:0/2494134537 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb700038e0 con 0x7feb7c100e60 2023-11-06T01:32:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.642+0000 7feb822ab700 1 -- 172.21.15.53:0/2494134537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c1a1170 con 0x7feb7c100e60 2023-11-06T01:32:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.643+0000 7feb822ab700 1 -- 172.21.15.53:0/2494134537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7c107d30 con 0x7feb7c100e60 2023-11-06T01:32:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.645+0000 7feb797fa700 1 -- 172.21.15.53:0/2494134537 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7feb70020030 con 0x7feb7c100e60 2023-11-06T01:32:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.645+0000 7feb797fa700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb640a0570 0x7feb640a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.645+0000 7feb7bfff700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb640a0570 0x7feb640a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.645+0000 7feb797fa700 1 -- 172.21.15.53:0/2494134537 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7feb700cc8c0 con 0x7feb7c100e60 2023-11-06T01:32:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.645+0000 7feb7bfff700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb640a0570 0x7feb640a2a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feb6c005950 tx=0x7feb6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.649+0000 7feb797fa700 1 -- 172.21.15.53:0/2494134537 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7feb700966d0 con 0x7feb7c100e60 2023-11-06T01:32:26.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: pgmap v249: 33 pgs: 11 creating+peering, 21 unknown, 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:26.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:32:26.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2657056621' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-06T01:32:26.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: osdmap e80: 12 total, 12 up, 12 in 2023-11-06T01:32:26.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2251692313' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-06T01:32:26.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-11-06T01:32:26.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: osdmap e81: 12 total, 12 up, 12 in 2023-11-06T01:32:26.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2251692313' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-06T01:32:26.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-11-06T01:32:26.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-11-06T01:32:26.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-11-06T01:32:26.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:26 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1010938087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:26.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.855+0000 7feb822ab700 1 -- 172.21.15.53:0/2494134537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb7c10b4f0 con 0x7feb7c100e60 2023-11-06T01:32:26.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.857+0000 7feb797fa700 1 -- 172.21.15.53:0/2494134537 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22084 (secure 0 0 0) 0x7feb70099cf0 con 0x7feb7c100e60 2023-11-06T01:32:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:26.859 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":81,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:32:26.092815+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:32:26.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.859+0000 7feb822ab700 1 -- 172.21.15.53:0/2494134537 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb640a0570 msgr2=0x7feb640a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.859+0000 7feb822ab700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb640a0570 0x7feb640a2a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feb6c005950 tx=0x7feb6c009260 comp rx=0 tx=0).stop 2023-11-06T01:32:26.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.859+0000 7feb822ab700 1 -- 172.21.15.53:0/2494134537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c100e60 msgr2=0x7feb7c102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:26.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.859+0000 7feb822ab700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c100e60 0x7feb7c102530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb70000c00 tx=0x7feb7000b790 comp rx=0 tx=0).stop 2023-11-06T01:32:26.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.859+0000 7feb822ab700 1 -- 172.21.15.53:0/2494134537 shutdown_connections 2023-11-06T01:32:26.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.859+0000 7feb822ab700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb640a0570 0x7feb640a2a30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.859+0000 7feb822ab700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb7c101800 0x7feb7c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.859+0000 7feb822ab700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0ffc70 0x7feb7c103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.859+0000 7feb822ab700 1 --2- 172.21.15.53:0/2494134537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7c100e60 0x7feb7c102530 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:26.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.859+0000 7feb822ab700 1 -- 172.21.15.53:0/2494134537 >> 172.21.15.53:0/2494134537 conn(0x7feb7c0fb1c0 msgr2=0x7feb7c0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:26.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.860+0000 7feb822ab700 1 -- 172.21.15.53:0/2494134537 shutdown_connections 2023-11-06T01:32:26.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:26.860+0000 7feb822ab700 1 -- 172.21.15.53:0/2494134537 wait complete. 2023-11-06T01:32:26.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T01:32:26.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: pgmap v249: 33 pgs: 11 creating+peering, 21 unknown, 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:26.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:32:26.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2657056621' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-06T01:32:26.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: osdmap e80: 12 total, 12 up, 12 in 2023-11-06T01:32:26.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2251692313' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-06T01:32:26.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-11-06T01:32:26.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: osdmap e81: 12 total, 12 up, 12 in 2023-11-06T01:32:26.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2251692313' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-06T01:32:26.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-11-06T01:32:26.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-11-06T01:32:26.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-11-06T01:32:26.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:26 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1010938087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:26.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: pgmap v249: 33 pgs: 11 creating+peering, 21 unknown, 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:32:26.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:32:26.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2657056621' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-06T01:32:26.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: osdmap e80: 12 total, 12 up, 12 in 2023-11-06T01:32:26.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2251692313' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-06T01:32:26.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-11-06T01:32:26.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: osdmap e81: 12 total, 12 up, 12 in 2023-11-06T01:32:26.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2251692313' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-06T01:32:26.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-11-06T01:32:26.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-11-06T01:32:26.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-11-06T01:32:26.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:26 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1010938087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:27.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.004+0000 7f841acd6700 1 -- 172.21.15.53:0/2227531535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84141041f0 msgr2=0x7f84141065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:27.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.004+0000 7f841acd6700 1 --2- 172.21.15.53:0/2227531535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84141041f0 0x7f84141065e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8400009990 tx=0x7f8400009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:27.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.004+0000 7f841acd6700 1 -- 172.21.15.53:0/2227531535 shutdown_connections 2023-11-06T01:32:27.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.004+0000 7f841acd6700 1 --2- 172.21.15.53:0/2227531535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84141018c0 0x7f8414103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:27.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.004+0000 7f841acd6700 1 --2- 172.21.15.53:0/2227531535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8414106b20 0x7f8414108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:27.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.004+0000 7f841acd6700 1 --2- 172.21.15.53:0/2227531535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84141041f0 0x7f84141065e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:27.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.004+0000 7f841acd6700 1 -- 172.21.15.53:0/2227531535 >> 172.21.15.53:0/2227531535 conn(0x7f84140fb1f0 msgr2=0x7f84140fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:27.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.004+0000 7f841acd6700 1 -- 172.21.15.53:0/2227531535 shutdown_connections 2023-11-06T01:32:27.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.004+0000 7f841acd6700 1 -- 172.21.15.53:0/2227531535 wait complete. 2023-11-06T01:32:27.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f841acd6700 1 Processor -- start 2023-11-06T01:32:27.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f841acd6700 1 -- start start 2023-11-06T01:32:27.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f841acd6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84141018c0 0x7f8414196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:27.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f841acd6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84141041f0 0x7f8414196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:27.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f841acd6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8414106b20 0x7f841419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:27.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f841acd6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841419e5d0 con 0x7f84141041f0 2023-11-06T01:32:27.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f841acd6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841419e710 con 0x7f8414106b20 2023-11-06T01:32:27.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f841acd6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841419e8a0 con 0x7f84141018c0 2023-11-06T01:32:27.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8413fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84141041f0 0x7f8414196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:27.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8418a72700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84141018c0 0x7f8414196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:27.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8413fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84141041f0 0x7f8414196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37982/0 (socket says 172.21.15.53:37982) 2023-11-06T01:32:27.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8413fff700 1 -- 172.21.15.53:0/4158507530 learned_addr learned my addr 172.21.15.53:0/4158507530 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:27.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8419273700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8414106b20 0x7f841419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:27.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8418a72700 1 -- 172.21.15.53:0/4158507530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8414106b20 msgr2=0x7f841419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:27.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8418a72700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8414106b20 0x7f841419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:27.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8418a72700 1 -- 172.21.15.53:0/4158507530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84141041f0 msgr2=0x7f8414196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:27.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8418a72700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84141041f0 0x7f8414196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:27.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8418a72700 1 -- 172.21.15.53:0/4158507530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84000095f0 con 0x7f84141018c0 2023-11-06T01:32:27.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8413fff700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84141041f0 0x7f8414196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:32:27.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.005+0000 7f8418a72700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84141018c0 0x7f8414196a50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f840800e9f0 tx=0x7f840800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:27.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.006+0000 7f8411ffb700 1 -- 172.21.15.53:0/4158507530 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800c9c0 con 0x7f84141018c0 2023-11-06T01:32:27.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.006+0000 7f841acd6700 1 -- 172.21.15.53:0/4158507530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841419eb80 con 0x7f84141018c0 2023-11-06T01:32:27.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.006+0000 7f8411ffb700 1 -- 172.21.15.53:0/4158507530 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8408010430 con 0x7f84141018c0 2023-11-06T01:32:27.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.006+0000 7f8411ffb700 1 -- 172.21.15.53:0/4158507530 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408018680 con 0x7f84141018c0 2023-11-06T01:32:27.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.006+0000 7f841acd6700 1 -- 172.21.15.53:0/4158507530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841419f090 con 0x7f84141018c0 2023-11-06T01:32:27.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.007+0000 7f841acd6700 1 -- 172.21.15.53:0/4158507530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84140fcd70 con 0x7f84141018c0 2023-11-06T01:32:27.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.008+0000 7f8411ffb700 1 -- 172.21.15.53:0/4158507530 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f8408010860 con 0x7f84141018c0 2023-11-06T01:32:27.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.009+0000 7f8411ffb700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f84040a0520 0x7f84040a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:27.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.009+0000 7f8413fff700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f84040a0520 0x7f84040a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:27.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.009+0000 7f8411ffb700 1 -- 172.21.15.53:0/4158507530 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8408014070 con 0x7f84141018c0 2023-11-06T01:32:27.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.009+0000 7f8413fff700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f84040a0520 0x7f84040a29e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f840000b2e0 tx=0x7f8400005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:27.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.012+0000 7f8411ffb700 1 -- 172.21.15.53:0/4158507530 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8408095830 con 0x7f84141018c0 2023-11-06T01:32:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:27.275+0000 7f841acd6700 1 -- 172.21.15.53:0/4158507530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f841402d010 con 0x7f84141018c0 2023-11-06T01:32:27.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:27 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2494134537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:27.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:27 smithi062 ceph-mon[103924]: osdmap e82: 12 total, 12 up, 12 in 2023-11-06T01:32:27.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:27 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] up:active 2023-11-06T01:32:27.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:27 smithi062 ceph-mon[103924]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:27.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:27 smithi062 ceph-mon[103924]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:27.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:27 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/4158507530' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-06T01:32:27.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:27 smithi062 ceph-mon[103924]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-06T01:32:27.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:27 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2494134537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:27.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:27 smithi163 ceph-mon[104416]: osdmap e82: 12 total, 12 up, 12 in 2023-11-06T01:32:27.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:27 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] up:active 2023-11-06T01:32:27.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:27 smithi163 ceph-mon[104416]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:27.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:27 smithi163 ceph-mon[104416]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:27.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:27 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/4158507530' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-06T01:32:27.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:27 smithi163 ceph-mon[104416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-06T01:32:28.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:27 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2494134537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:28.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:27 smithi053 ceph-mon[99472]: osdmap e82: 12 total, 12 up, 12 in 2023-11-06T01:32:28.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:27 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] up:active 2023-11-06T01:32:28.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:27 smithi053 ceph-mon[99472]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:28.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:27 smithi053 ceph-mon[99472]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:28.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:27 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/4158507530' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-06T01:32:28.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:27 smithi053 ceph-mon[99472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-06T01:32:28.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.115+0000 7f8411ffb700 1 -- 172.21.15.53:0/4158507530 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8408098e50 con 0x7f84141018c0 2023-11-06T01:32:28.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 -- 172.21.15.53:0/4158507530 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f84040a0520 msgr2=0x7f84040a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:28.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f84040a0520 0x7f84040a29e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f840000b2e0 tx=0x7f8400005960 comp rx=0 tx=0).stop 2023-11-06T01:32:28.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 -- 172.21.15.53:0/4158507530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84141018c0 msgr2=0x7f8414196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:28.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84141018c0 0x7f8414196a50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f840800e9f0 tx=0x7f840800ed00 comp rx=0 tx=0).stop 2023-11-06T01:32:28.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 -- 172.21.15.53:0/4158507530 shutdown_connections 2023-11-06T01:32:28.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f84040a0520 0x7f84040a29e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:28.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8414106b20 0x7f841419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:28.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84141018c0 0x7f8414196a50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:28.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 --2- 172.21.15.53:0/4158507530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84141041f0 0x7f8414196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:28.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 -- 172.21.15.53:0/4158507530 >> 172.21.15.53:0/4158507530 conn(0x7f84140fb1f0 msgr2=0x7f8414102530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:28.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 -- 172.21.15.53:0/4158507530 shutdown_connections 2023-11-06T01:32:28.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.119+0000 7f841acd6700 1 -- 172.21.15.53:0/4158507530 wait complete. 2023-11-06T01:32:28.135 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-11-06T01:32:28.135 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-11-06T01:32:28.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.268+0000 7fca290ed700 1 -- 172.21.15.53:0/620343264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca24100e70 msgr2=0x7fca241012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:28.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.268+0000 7fca290ed700 1 --2- 172.21.15.53:0/620343264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca24100e70 0x7fca241012d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fca180099c0 tx=0x7fca18009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:28.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.268+0000 7fca290ed700 1 -- 172.21.15.53:0/620343264 shutdown_connections 2023-11-06T01:32:28.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.268+0000 7fca290ed700 1 --2- 172.21.15.53:0/620343264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca24101810 0x7fca241080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:28.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.268+0000 7fca290ed700 1 --2- 172.21.15.53:0/620343264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca240ffc80 0x7fca241000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:28.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.268+0000 7fca290ed700 1 --2- 172.21.15.53:0/620343264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca24100e70 0x7fca241012d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:28.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.268+0000 7fca290ed700 1 -- 172.21.15.53:0/620343264 >> 172.21.15.53:0/620343264 conn(0x7fca240fb1f0 msgr2=0x7fca240fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:28.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.268+0000 7fca290ed700 1 -- 172.21.15.53:0/620343264 shutdown_connections 2023-11-06T01:32:28.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.268+0000 7fca290ed700 1 -- 172.21.15.53:0/620343264 wait complete. 2023-11-06T01:32:28.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.269+0000 7fca290ed700 1 Processor -- start 2023-11-06T01:32:28.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.269+0000 7fca290ed700 1 -- start start 2023-11-06T01:32:28.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.269+0000 7fca290ed700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca240ffc80 0x7fca24068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:28.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.269+0000 7fca290ed700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca24100e70 0x7fca2419b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:28.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.269+0000 7fca290ed700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca24101810 0x7fca2419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:28.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.269+0000 7fca290ed700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2419e620 con 0x7fca240ffc80 2023-11-06T01:32:28.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca290ed700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2419e790 con 0x7fca24101810 2023-11-06T01:32:28.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca290ed700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2419e950 con 0x7fca24100e70 2023-11-06T01:32:28.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca22d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca240ffc80 0x7fca24068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:28.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca22d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca240ffc80 0x7fca24068a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37994/0 (socket says 172.21.15.53:37994) 2023-11-06T01:32:28.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca22d9d700 1 -- 172.21.15.53:0/2336917368 learned_addr learned my addr 172.21.15.53:0/2336917368 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:28.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca2359e700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca24101810 0x7fca2419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:28.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca2259c700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca24100e70 0x7fca2419b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:28.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca22d9d700 1 -- 172.21.15.53:0/2336917368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca24100e70 msgr2=0x7fca2419b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:28.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca22d9d700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca24100e70 0x7fca2419b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:28.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca22d9d700 1 -- 172.21.15.53:0/2336917368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca24101810 msgr2=0x7fca2419de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:28.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca22d9d700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca24101810 0x7fca2419de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:28.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca22d9d700 1 -- 172.21.15.53:0/2336917368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca180095f0 con 0x7fca240ffc80 2023-11-06T01:32:28.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca2359e700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca24101810 0x7fca2419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:32:28.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca2259c700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca24100e70 0x7fca2419b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:28.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca22d9d700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca240ffc80 0x7fca24068a80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fca1400d980 tx=0x7fca1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:28.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca0bfff700 1 -- 172.21.15.53:0/2336917368 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14004020 con 0x7fca240ffc80 2023-11-06T01:32:28.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca290ed700 1 -- 172.21.15.53:0/2336917368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2419ec30 con 0x7fca240ffc80 2023-11-06T01:32:28.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca0bfff700 1 -- 172.21.15.53:0/2336917368 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca14009730 con 0x7fca240ffc80 2023-11-06T01:32:28.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca0bfff700 1 -- 172.21.15.53:0/2336917368 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14003a30 con 0x7fca240ffc80 2023-11-06T01:32:28.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.270+0000 7fca290ed700 1 -- 172.21.15.53:0/2336917368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2419f160 con 0x7fca240ffc80 2023-11-06T01:32:28.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.272+0000 7fca290ed700 1 -- 172.21.15.53:0/2336917368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca24107290 con 0x7fca240ffc80 2023-11-06T01:32:28.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.273+0000 7fca0bfff700 1 -- 172.21.15.53:0/2336917368 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fca140098a0 con 0x7fca240ffc80 2023-11-06T01:32:28.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.274+0000 7fca0bfff700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fca0c0a0520 0x7fca0c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:28.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.274+0000 7fca2259c700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fca0c0a0520 0x7fca0c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:28.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.274+0000 7fca0bfff700 1 -- 172.21.15.53:0/2336917368 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fca140cb890 con 0x7fca240ffc80 2023-11-06T01:32:28.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.274+0000 7fca2259c700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fca0c0a0520 0x7fca0c0a29e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fca1800b580 tx=0x7fca1801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:28.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.277+0000 7fca0bfff700 1 -- 172.21.15.53:0/2336917368 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fca14095580 con 0x7fca240ffc80 2023-11-06T01:32:28.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:28.540+0000 7fca290ed700 1 -- 172.21.15.53:0/2336917368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fca2402ce00 con 0x7fca240ffc80 2023-11-06T01:32:28.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: pgmap v253: 97 pgs: 15 creating+peering, 55 unknown, 27 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-06T01:32:28.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-11-06T01:32:28.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: Cluster is now healthy 2023-11-06T01:32:28.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-06T01:32:28.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:28.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:28.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:32:28.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:28 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2336917368' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-11-06T01:32:28.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: pgmap v253: 97 pgs: 15 creating+peering, 55 unknown, 27 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-06T01:32:28.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-11-06T01:32:28.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: Cluster is now healthy 2023-11-06T01:32:28.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-06T01:32:28.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:28.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:32:28.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:28.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:28 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2336917368' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-11-06T01:32:29.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: pgmap v253: 97 pgs: 15 creating+peering, 55 unknown, 27 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-06T01:32:29.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-11-06T01:32:29.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: Cluster is now healthy 2023-11-06T01:32:29.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-06T01:32:29.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:29.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:29.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:29.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:29.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:29.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:29.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:29.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:32:29.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:32:29.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:32:29.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:28 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2336917368' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-11-06T01:32:29.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.434+0000 7fca0bfff700 1 -- 172.21.15.53:0/2336917368 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7fca14098ba0 con 0x7fca240ffc80 2023-11-06T01:32:29.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.438+0000 7fca290ed700 1 -- 172.21.15.53:0/2336917368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fca0c0a0520 msgr2=0x7fca0c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:29.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.438+0000 7fca290ed700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fca0c0a0520 0x7fca0c0a29e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fca1800b580 tx=0x7fca1801a040 comp rx=0 tx=0).stop 2023-11-06T01:32:29.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.438+0000 7fca290ed700 1 -- 172.21.15.53:0/2336917368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca240ffc80 msgr2=0x7fca24068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:29.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.438+0000 7fca290ed700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca240ffc80 0x7fca24068a80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fca1400d980 tx=0x7fca1400dc90 comp rx=0 tx=0).stop 2023-11-06T01:32:29.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.438+0000 7fca290ed700 1 -- 172.21.15.53:0/2336917368 shutdown_connections 2023-11-06T01:32:29.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.438+0000 7fca290ed700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fca0c0a0520 0x7fca0c0a29e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:29.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.438+0000 7fca290ed700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca24101810 0x7fca2419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:29.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.438+0000 7fca290ed700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca240ffc80 0x7fca24068a80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:29.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.438+0000 7fca290ed700 1 --2- 172.21.15.53:0/2336917368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca24100e70 0x7fca2419b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:29.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.438+0000 7fca290ed700 1 -- 172.21.15.53:0/2336917368 >> 172.21.15.53:0/2336917368 conn(0x7fca240fb1f0 msgr2=0x7fca241043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:29.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.439+0000 7fca290ed700 1 -- 172.21.15.53:0/2336917368 shutdown_connections 2023-11-06T01:32:29.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.439+0000 7fca290ed700 1 -- 172.21.15.53:0/2336917368 wait complete. 2023-11-06T01:32:29.457 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-11-06T01:32:29.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.593+0000 7f90940da700 1 -- 172.21.15.53:0/1696203154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908c106b00 msgr2=0x7f908c108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:29.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.593+0000 7f90940da700 1 --2- 172.21.15.53:0/1696203154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908c106b00 0x7f908c108ef0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f908400b0a0 tx=0x7f908400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:29.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.593+0000 7f90940da700 1 -- 172.21.15.53:0/1696203154 shutdown_connections 2023-11-06T01:32:29.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.593+0000 7f90940da700 1 --2- 172.21.15.53:0/1696203154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908c106b00 0x7f908c108ef0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:29.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.593+0000 7f90940da700 1 --2- 172.21.15.53:0/1696203154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f908c1018a0 0x7f908c103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:29.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.593+0000 7f90940da700 1 --2- 172.21.15.53:0/1696203154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f908c1041d0 0x7f908c1065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:29.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.593+0000 7f90940da700 1 -- 172.21.15.53:0/1696203154 >> 172.21.15.53:0/1696203154 conn(0x7f908c0fb1f0 msgr2=0x7f908c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:29.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.593+0000 7f90940da700 1 -- 172.21.15.53:0/1696203154 shutdown_connections 2023-11-06T01:32:29.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.593+0000 7f90940da700 1 -- 172.21.15.53:0/1696203154 wait complete. 2023-11-06T01:32:29.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f90940da700 1 Processor -- start 2023-11-06T01:32:29.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f90940da700 1 -- start start 2023-11-06T01:32:29.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f90940da700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f908c1018a0 0x7f908c196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:29.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f90940da700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908c1041d0 0x7f908c197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:29.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f90940da700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f908c106b00 0x7f908c19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:29.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f90940da700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c19e690 con 0x7f908c1041d0 2023-11-06T01:32:29.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f90940da700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c19e7d0 con 0x7f908c106b00 2023-11-06T01:32:29.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f90940da700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c19e960 con 0x7f908c1018a0 2023-11-06T01:32:29.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f9091675700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908c1041d0 0x7f908c197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:29.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f9091675700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908c1041d0 0x7f908c197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38010/0 (socket says 172.21.15.53:38010) 2023-11-06T01:32:29.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f9091675700 1 -- 172.21.15.53:0/2379255237 learned_addr learned my addr 172.21.15.53:0/2379255237 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:29.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f9091e76700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f908c1018a0 0x7f908c196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:29.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f9092677700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f908c106b00 0x7f908c19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:29.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f9091675700 1 -- 172.21.15.53:0/2379255237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f908c1018a0 msgr2=0x7f908c196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:29.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f9091675700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f908c1018a0 0x7f908c196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:29.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f9091675700 1 -- 172.21.15.53:0/2379255237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f908c106b00 msgr2=0x7f908c19e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:29.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f9091675700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f908c106b00 0x7f908c19e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:29.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f9091675700 1 -- 172.21.15.53:0/2379255237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9084009d00 con 0x7f908c1041d0 2023-11-06T01:32:29.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.594+0000 7f9091675700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908c1041d0 0x7f908c197050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f908800e9a0 tx=0x7f908800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:29.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.595+0000 7f9082ffd700 1 -- 172.21.15.53:0/2379255237 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088009790 con 0x7f908c1041d0 2023-11-06T01:32:29.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.595+0000 7f90940da700 1 -- 172.21.15.53:0/2379255237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908c19ec40 con 0x7f908c1041d0 2023-11-06T01:32:29.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.595+0000 7f9082ffd700 1 -- 172.21.15.53:0/2379255237 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908800cbd0 con 0x7f908c1041d0 2023-11-06T01:32:29.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.595+0000 7f9082ffd700 1 -- 172.21.15.53:0/2379255237 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088018680 con 0x7f908c1041d0 2023-11-06T01:32:29.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.595+0000 7f90940da700 1 -- 172.21.15.53:0/2379255237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c19f1a0 con 0x7f908c1041d0 2023-11-06T01:32:29.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.596+0000 7f90940da700 1 -- 172.21.15.53:0/2379255237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908c0fcd70 con 0x7f908c1041d0 2023-11-06T01:32:29.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.597+0000 7f9082ffd700 1 -- 172.21.15.53:0/2379255237 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f908800cd40 con 0x7f908c1041d0 2023-11-06T01:32:29.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.597+0000 7f9082ffd700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90780a0520 0x7f90780a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:29.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.597+0000 7f9091e76700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90780a0520 0x7f90780a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:29.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.597+0000 7f9082ffd700 1 -- 172.21.15.53:0/2379255237 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9088014070 con 0x7f908c1041d0 2023-11-06T01:32:29.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.598+0000 7f9091e76700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90780a0520 0x7f90780a29e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f907c005950 tx=0x7f907c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:29.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.601+0000 7f9082ffd700 1 -- 172.21.15.53:0/2379255237 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9088095ad0 con 0x7f908c1041d0 2023-11-06T01:32:29.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:29 smithi053 ceph-mon[99472]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-06T01:32:29.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:29 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2336917368' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-11-06T01:32:29.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:29 smithi053 ceph-mon[99472]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-06T01:32:29.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:29 smithi053 ceph-mon[99472]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:29.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:29 smithi053 ceph-mon[99472]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2023-11-06T01:32:29.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:29 smithi053 ceph-mon[99472]: daemon mds.c is now active in filesystem cephfs as rank 1 2023-11-06T01:32:29.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:29.855+0000 7f90940da700 1 -- 172.21.15.53:0/2379255237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f908c02cc70 con 0x7f908c1041d0 2023-11-06T01:32:29.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:29 smithi062 ceph-mon[103924]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-06T01:32:29.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:29 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2336917368' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-11-06T01:32:29.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:29 smithi062 ceph-mon[103924]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-06T01:32:29.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:29 smithi062 ceph-mon[103924]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:29.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:29 smithi062 ceph-mon[103924]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2023-11-06T01:32:29.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:29 smithi062 ceph-mon[103924]: daemon mds.c is now active in filesystem cephfs as rank 1 2023-11-06T01:32:29.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:29 smithi163 ceph-mon[104416]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-06T01:32:29.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:29 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2336917368' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-11-06T01:32:29.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:29 smithi163 ceph-mon[104416]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-06T01:32:29.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:29 smithi163 ceph-mon[104416]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-11-06T01:32:29.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:29 smithi163 ceph-mon[104416]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2023-11-06T01:32:29.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:29 smithi163 ceph-mon[104416]: daemon mds.c is now active in filesystem cephfs as rank 1 2023-11-06T01:32:30.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.443+0000 7f9082ffd700 1 -- 172.21.15.53:0/2379255237 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f90880990f0 con 0x7f908c1041d0 2023-11-06T01:32:30.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.447+0000 7f90940da700 1 -- 172.21.15.53:0/2379255237 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90780a0520 msgr2=0x7f90780a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:30.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.447+0000 7f90940da700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90780a0520 0x7f90780a29e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f907c005950 tx=0x7f907c00b410 comp rx=0 tx=0).stop 2023-11-06T01:32:30.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.447+0000 7f90940da700 1 -- 172.21.15.53:0/2379255237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908c1041d0 msgr2=0x7f908c197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:30.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.447+0000 7f90940da700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908c1041d0 0x7f908c197050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f908800e9a0 tx=0x7f908800ecb0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.448+0000 7f90940da700 1 -- 172.21.15.53:0/2379255237 shutdown_connections 2023-11-06T01:32:30.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.448+0000 7f90940da700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f90780a0520 0x7f90780a29e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.448+0000 7f90940da700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f908c106b00 0x7f908c19e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.448+0000 7f90940da700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f908c1018a0 0x7f908c196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.448+0000 7f90940da700 1 --2- 172.21.15.53:0/2379255237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f908c1041d0 0x7f908c197050 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.448+0000 7f90940da700 1 -- 172.21.15.53:0/2379255237 >> 172.21.15.53:0/2379255237 conn(0x7f908c0fb1f0 msgr2=0x7f908c102510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:30.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.448+0000 7f90940da700 1 -- 172.21.15.53:0/2379255237 shutdown_connections 2023-11-06T01:32:30.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.448+0000 7f90940da700 1 -- 172.21.15.53:0/2379255237 wait complete. 2023-11-06T01:32:30.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-11-06T01:32:30.489 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:32:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-i[140736]: 2023-11-06T01:32:30.024+0000 7fcf223c8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T01:32:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.603+0000 7fb0b158d700 1 -- 172.21.15.53:0/1494362250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0ac0ffc80 msgr2=0x7fb0ac1000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.603+0000 7fb0b158d700 1 --2- 172.21.15.53:0/1494362250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0ac0ffc80 0x7fb0ac1000a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb0940099c0 tx=0x7fb094009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.603+0000 7fb0b158d700 1 -- 172.21.15.53:0/1494362250 shutdown_connections 2023-11-06T01:32:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.603+0000 7fb0b158d700 1 --2- 172.21.15.53:0/1494362250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac101810 0x7fb0ac1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.603+0000 7fb0b158d700 1 --2- 172.21.15.53:0/1494362250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0ac0ffc80 0x7fb0ac1000a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.603+0000 7fb0b158d700 1 --2- 172.21.15.53:0/1494362250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0ac100e70 0x7fb0ac1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.603+0000 7fb0b158d700 1 -- 172.21.15.53:0/1494362250 >> 172.21.15.53:0/1494362250 conn(0x7fb0ac0fb1f0 msgr2=0x7fb0ac0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.603+0000 7fb0b158d700 1 -- 172.21.15.53:0/1494362250 shutdown_connections 2023-11-06T01:32:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.603+0000 7fb0b158d700 1 -- 172.21.15.53:0/1494362250 wait complete. 2023-11-06T01:32:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.603+0000 7fb0b158d700 1 Processor -- start 2023-11-06T01:32:30.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0b158d700 1 -- start start 2023-11-06T01:32:30.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0b158d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac0ffc80 0x7fb0ac068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:30.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0b158d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0ac100e70 0x7fb0ac19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:30.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0b158d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0ac101810 0x7fb0ac19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:30.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0b158d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac19e5a0 con 0x7fb0ac100e70 2023-11-06T01:32:30.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0b158d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac19e710 con 0x7fb0ac0ffc80 2023-11-06T01:32:30.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0b158d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac19e8d0 con 0x7fb0ac101810 2023-11-06T01:32:30.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0aa7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0ac100e70 0x7fb0ac19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:30.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0aa7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0ac100e70 0x7fb0ac19b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38024/0 (socket says 172.21.15.53:38024) 2023-11-06T01:32:30.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0aa7fc700 1 -- 172.21.15.53:0/58479561 learned_addr learned my addr 172.21.15.53:0/58479561 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:30.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0ab7fe700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0ac101810 0x7fb0ac19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:30.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0aaffd700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac0ffc80 0x7fb0ac068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:30.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0aa7fc700 1 -- 172.21.15.53:0/58479561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0ac101810 msgr2=0x7fb0ac19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:30.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0aa7fc700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0ac101810 0x7fb0ac19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0aa7fc700 1 -- 172.21.15.53:0/58479561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac0ffc80 msgr2=0x7fb0ac068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:30.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0aa7fc700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac0ffc80 0x7fb0ac068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0aa7fc700 1 -- 172.21.15.53:0/58479561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0940095f0 con 0x7fb0ac100e70 2023-11-06T01:32:30.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0aa7fc700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0ac100e70 0x7fb0ac19b7c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00d950 tx=0x7fb09c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:30.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.604+0000 7fb0a3fff700 1 -- 172.21.15.53:0/58479561 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c009940 con 0x7fb0ac100e70 2023-11-06T01:32:30.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.605+0000 7fb0a3fff700 1 -- 172.21.15.53:0/58479561 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb09c010460 con 0x7fb0ac100e70 2023-11-06T01:32:30.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.605+0000 7fb0b158d700 1 -- 172.21.15.53:0/58479561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ac19ebb0 con 0x7fb0ac100e70 2023-11-06T01:32:30.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.605+0000 7fb0a3fff700 1 -- 172.21.15.53:0/58479561 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c00f620 con 0x7fb0ac100e70 2023-11-06T01:32:30.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.605+0000 7fb0b158d700 1 -- 172.21.15.53:0/58479561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ac19f0c0 con 0x7fb0ac100e70 2023-11-06T01:32:30.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.605+0000 7fb0b158d700 1 -- 172.21.15.53:0/58479561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ac106bb0 con 0x7fb0ac100e70 2023-11-06T01:32:30.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.607+0000 7fb0a3fff700 1 -- 172.21.15.53:0/58479561 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fb09c0108d0 con 0x7fb0ac100e70 2023-11-06T01:32:30.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.607+0000 7fb0a3fff700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb0980a0520 0x7fb0980a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:30.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.608+0000 7fb0aaffd700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb0980a0520 0x7fb0980a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:30.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.608+0000 7fb0a3fff700 1 -- 172.21.15.53:0/58479561 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb09c0cb660 con 0x7fb0ac100e70 2023-11-06T01:32:30.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.608+0000 7fb0aaffd700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb0980a0520 0x7fb0980a29e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb09400b580 tx=0x7fb094005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:30.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.611+0000 7fb0a3fff700 1 -- 172.21.15.53:0/58479561 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb09c095350 con 0x7fb0ac100e70 2023-11-06T01:32:30.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:30 smithi053 ceph-mon[99472]: pgmap v254: 97 pgs: 15 creating+peering, 24 unknown, 58 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 637 B/s wr, 3 op/s 2023-11-06T01:32:30.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:30 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2379255237' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-06T01:32:30.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:30 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] up:active 2023-11-06T01:32:30.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:30 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2379255237' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-06T01:32:30.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:30 smithi053 ceph-mon[99472]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-06T01:32:30.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:30 smithi053 ceph-mon[99472]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2023-11-06T01:32:30.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:30 smithi053 ceph-mon[99472]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2023-11-06T01:32:30.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:30 smithi053 ceph-mon[99472]: daemon mds.b is now active in filesystem cephfs as rank 2 2023-11-06T01:32:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.862+0000 7fb0b158d700 1 -- 172.21.15.53:0/58479561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fb0ac02ccd0 con 0x7fb0ac100e70 2023-11-06T01:32:30.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:30 smithi062 ceph-mon[103924]: pgmap v254: 97 pgs: 15 creating+peering, 24 unknown, 58 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 637 B/s wr, 3 op/s 2023-11-06T01:32:30.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:30 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2379255237' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-06T01:32:30.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:30 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] up:active 2023-11-06T01:32:30.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:30 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2379255237' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-06T01:32:30.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:30 smithi062 ceph-mon[103924]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-06T01:32:30.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:30 smithi062 ceph-mon[103924]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2023-11-06T01:32:30.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:30 smithi062 ceph-mon[103924]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2023-11-06T01:32:30.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:30 smithi062 ceph-mon[103924]: daemon mds.b is now active in filesystem cephfs as rank 2 2023-11-06T01:32:30.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.989+0000 7fb0a3fff700 1 -- 172.21.15.53:0/58479561 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb09c0160b0 con 0x7fb0ac100e70 2023-11-06T01:32:30.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:30 smithi163 ceph-mon[104416]: pgmap v254: 97 pgs: 15 creating+peering, 24 unknown, 58 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 637 B/s wr, 3 op/s 2023-11-06T01:32:30.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:30 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2379255237' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-06T01:32:30.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:30 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] up:active 2023-11-06T01:32:30.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:30 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2379255237' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-06T01:32:30.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:30 smithi163 ceph-mon[104416]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-06T01:32:30.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:30 smithi163 ceph-mon[104416]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2023-11-06T01:32:30.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:30 smithi163 ceph-mon[104416]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2023-11-06T01:32:30.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:30 smithi163 ceph-mon[104416]: daemon mds.b is now active in filesystem cephfs as rank 2 2023-11-06T01:32:30.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 -- 172.21.15.53:0/58479561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb0980a0520 msgr2=0x7fb0980a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:30.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb0980a0520 0x7fb0980a29e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb09400b580 tx=0x7fb094005fb0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 -- 172.21.15.53:0/58479561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0ac100e70 msgr2=0x7fb0ac19b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:30.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0ac100e70 0x7fb0ac19b7c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00d950 tx=0x7fb09c00dc60 comp rx=0 tx=0).stop 2023-11-06T01:32:30.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 -- 172.21.15.53:0/58479561 shutdown_connections 2023-11-06T01:32:30.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb0980a0520 0x7fb0980a29e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac0ffc80 0x7fb0ac068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0ac101810 0x7fb0ac19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 --2- 172.21.15.53:0/58479561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0ac100e70 0x7fb0ac19b7c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:30.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 -- 172.21.15.53:0/58479561 >> 172.21.15.53:0/58479561 conn(0x7fb0ac0fb1f0 msgr2=0x7fb0ac0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:30.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 -- 172.21.15.53:0/58479561 shutdown_connections 2023-11-06T01:32:30.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:30.992+0000 7fb0b158d700 1 -- 172.21.15.53:0/58479561 wait complete. 2023-11-06T01:32:31.008 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-11-06T01:32:31.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-11-06T01:32:31.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.138+0000 7f0eb745f700 1 -- 172.21.15.53:0/2388722082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0eb0100db0 msgr2=0x7f0eb01011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:31.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.138+0000 7f0eb745f700 1 --2- 172.21.15.53:0/2388722082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0eb0100db0 0x7f0eb01011d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00b0d0 tx=0x7f0eac00b3e0 comp rx=0 tx=0).stop 2023-11-06T01:32:31.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.138+0000 7f0eb745f700 1 -- 172.21.15.53:0/2388722082 shutdown_connections 2023-11-06T01:32:31.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.138+0000 7f0eb745f700 1 --2- 172.21.15.53:0/2388722082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb0102940 0x7f0eb01091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:31.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.138+0000 7f0eb745f700 1 --2- 172.21.15.53:0/2388722082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0eb0100db0 0x7f0eb01011d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:31.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.138+0000 7f0eb745f700 1 --2- 172.21.15.53:0/2388722082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eb0101fa0 0x7f0eb0102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:31.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.138+0000 7f0eb745f700 1 -- 172.21.15.53:0/2388722082 >> 172.21.15.53:0/2388722082 conn(0x7f0eb00fc300 msgr2=0x7f0eb00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:31.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.138+0000 7f0eb745f700 1 -- 172.21.15.53:0/2388722082 shutdown_connections 2023-11-06T01:32:31.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.138+0000 7f0eb745f700 1 -- 172.21.15.53:0/2388722082 wait complete. 2023-11-06T01:32:31.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb745f700 1 Processor -- start 2023-11-06T01:32:31.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb745f700 1 -- start start 2023-11-06T01:32:31.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb745f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0eb0100db0 0x7f0eb0196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:31.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb745f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eb0101fa0 0x7f0eb0197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:31.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb745f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb0102940 0x7f0eb019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:31.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb745f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb019e6b0 con 0x7f0eb0100db0 2023-11-06T01:32:31.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb745f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb019e7f0 con 0x7f0eb0101fa0 2023-11-06T01:32:31.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb745f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb019e980 con 0x7f0eb0102940 2023-11-06T01:32:31.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb51fb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0eb0100db0 0x7f0eb0196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:31.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb51fb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0eb0100db0 0x7f0eb0196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38036/0 (socket says 172.21.15.53:38036) 2023-11-06T01:32:31.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb51fb700 1 -- 172.21.15.53:0/1917251799 learned_addr learned my addr 172.21.15.53:0/1917251799 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:31.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.139+0000 7f0eb49fa700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eb0101fa0 0x7f0eb0197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:31.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0eb59fc700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb0102940 0x7f0eb019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:31.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0eb51fb700 1 -- 172.21.15.53:0/1917251799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb0102940 msgr2=0x7f0eb019e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:31.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0eb51fb700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb0102940 0x7f0eb019e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:31.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0eb51fb700 1 -- 172.21.15.53:0/1917251799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eb0101fa0 msgr2=0x7f0eb0197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:31.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0eb51fb700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eb0101fa0 0x7f0eb0197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:31.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0eb51fb700 1 -- 172.21.15.53:0/1917251799 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac009d00 con 0x7f0eb0100db0 2023-11-06T01:32:31.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0eb59fc700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb0102940 0x7f0eb019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:32:31.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0eb51fb700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0eb0100db0 0x7f0eb0196b30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0eac000f80 tx=0x7f0eac009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:31.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0ea67fc700 1 -- 172.21.15.53:0/1917251799 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00e040 con 0x7f0eb0100db0 2023-11-06T01:32:31.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0ea67fc700 1 -- 172.21.15.53:0/1917251799 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eac01d480 con 0x7f0eb0100db0 2023-11-06T01:32:31.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0eb745f700 1 -- 172.21.15.53:0/1917251799 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb019ec00 con 0x7f0eb0100db0 2023-11-06T01:32:31.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0ea67fc700 1 -- 172.21.15.53:0/1917251799 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac01b6b0 con 0x7f0eb0100db0 2023-11-06T01:32:31.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.140+0000 7f0eb745f700 1 -- 172.21.15.53:0/1917251799 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb019f080 con 0x7f0eb0100db0 2023-11-06T01:32:31.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.141+0000 7f0eb745f700 1 -- 172.21.15.53:0/1917251799 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb0107c40 con 0x7f0eb0100db0 2023-11-06T01:32:31.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.142+0000 7f0ea67fc700 1 -- 172.21.15.53:0/1917251799 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f0eac01d5f0 con 0x7f0eb0100db0 2023-11-06T01:32:31.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.143+0000 7f0ea67fc700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e9c0a0520 0x7f0e9c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:31.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.143+0000 7f0eb49fa700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e9c0a0520 0x7f0e9c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:31.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.143+0000 7f0ea67fc700 1 -- 172.21.15.53:0/1917251799 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0eac0cdb40 con 0x7f0eb0100db0 2023-11-06T01:32:31.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.143+0000 7f0eb49fa700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e9c0a0520 0x7f0e9c0a29e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005ea0 tx=0x7f0ea0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:31.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.146+0000 7f0ea67fc700 1 -- 172.21.15.53:0/1917251799 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0eac097820 con 0x7f0eb0100db0 2023-11-06T01:32:31.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:31.384+0000 7f0eb745f700 1 -- 172.21.15.53:0/1917251799 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f0eb00611d0 con 0x7f0e9c0a0520 2023-11-06T01:32:31.844 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:32:31.390+0000 7fb01cf8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T01:32:31.844 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:32:31.390+0000 7fb01cf8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T01:32:31.844 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:32:31.390+0000 7fb01cf8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T01:32:31.845 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:32:31.390+0000 7fb01cf8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T01:32:31.845 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:32:31.390+0000 7fb01cf8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T01:32:31.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/58479561' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-06T01:32:31.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] up:active 2023-11-06T01:32:31.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/58479561' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-06T01:32:31.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-mon[99472]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-11-06T01:32:31.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-mon[99472]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2023-11-06T01:32:31.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-mon[99472]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2023-11-06T01:32:31.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:31 smithi053 ceph-mon[99472]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-11-06T01:32:31.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:31 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/58479561' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-06T01:32:31.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:31 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] up:active 2023-11-06T01:32:31.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:31 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/58479561' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-06T01:32:31.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:31 smithi062 ceph-mon[103924]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-11-06T01:32:31.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:31 smithi062 ceph-mon[103924]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2023-11-06T01:32:31.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:31 smithi062 ceph-mon[103924]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2023-11-06T01:32:31.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:31 smithi062 ceph-mon[103924]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-11-06T01:32:31.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:31 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/58479561' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-06T01:32:31.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:31 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] up:active 2023-11-06T01:32:31.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:31 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/58479561' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-06T01:32:31.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:31 smithi163 ceph-mon[104416]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-11-06T01:32:31.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:31 smithi163 ceph-mon[104416]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2023-11-06T01:32:31.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:31 smithi163 ceph-mon[104416]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2023-11-06T01:32:31.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:31 smithi163 ceph-mon[104416]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-11-06T01:32:32.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.039+0000 7f0ea67fc700 1 -- 172.21.15.53:0/1917251799 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0eb00611d0 con 0x7f0e9c0a0520 2023-11-06T01:32:32.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 -- 172.21.15.53:0/1917251799 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e9c0a0520 msgr2=0x7f0e9c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e9c0a0520 0x7f0e9c0a29e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005ea0 tx=0x7f0ea0005e30 comp rx=0 tx=0).stop 2023-11-06T01:32:32.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 -- 172.21.15.53:0/1917251799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0eb0100db0 msgr2=0x7f0eb0196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0eb0100db0 0x7f0eb0196b30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0eac000f80 tx=0x7f0eac009570 comp rx=0 tx=0).stop 2023-11-06T01:32:32.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 -- 172.21.15.53:0/1917251799 shutdown_connections 2023-11-06T01:32:32.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e9c0a0520 0x7f0e9c0a29e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb0102940 0x7f0eb019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0eb0100db0 0x7f0eb0196b30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 --2- 172.21.15.53:0/1917251799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eb0101fa0 0x7f0eb0197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 -- 172.21.15.53:0/1917251799 >> 172.21.15.53:0/1917251799 conn(0x7f0eb00fc300 msgr2=0x7f0eb00fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:32.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 -- 172.21.15.53:0/1917251799 shutdown_connections 2023-11-06T01:32:32.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.042+0000 7f0eb745f700 1 -- 172.21.15.53:0/1917251799 wait complete. 2023-11-06T01:32:32.057 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-11-06T01:32:32.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.191+0000 7f5783941700 1 -- 172.21.15.53:0/1860476565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f577c0ffcb0 msgr2=0x7f577c1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.191+0000 7f5783941700 1 --2- 172.21.15.53:0/1860476565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f577c0ffcb0 0x7f577c1000d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f57700099c0 tx=0x7f5770009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.191+0000 7f5783941700 1 -- 172.21.15.53:0/1860476565 shutdown_connections 2023-11-06T01:32:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.191+0000 7f5783941700 1 --2- 172.21.15.53:0/1860476565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f577c101840 0x7f577c1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.191+0000 7f5783941700 1 --2- 172.21.15.53:0/1860476565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f577c0ffcb0 0x7f577c1000d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.191+0000 7f5783941700 1 --2- 172.21.15.53:0/1860476565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c100ea0 0x7f577c101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.191+0000 7f5783941700 1 -- 172.21.15.53:0/1860476565 >> 172.21.15.53:0/1860476565 conn(0x7f577c0fb220 msgr2=0x7f577c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.191+0000 7f5783941700 1 -- 172.21.15.53:0/1860476565 shutdown_connections 2023-11-06T01:32:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.191+0000 7f5783941700 1 -- 172.21.15.53:0/1860476565 wait complete. 2023-11-06T01:32:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.191+0000 7f5783941700 1 Processor -- start 2023-11-06T01:32:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.191+0000 7f5783941700 1 -- start start 2023-11-06T01:32:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5783941700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c0ffcb0 0x7f577c069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5783941700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f577c100ea0 0x7f577c19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5783941700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f577c101840 0x7f577c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5783941700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c19e540 con 0x7f577c101840 2023-11-06T01:32:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5783941700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c19e6b0 con 0x7f577c0ffcb0 2023-11-06T01:32:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5783941700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c19e870 con 0x7f577c100ea0 2023-11-06T01:32:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5781ede700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f577c101840 0x7f577c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5781ede700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f577c101840 0x7f577c19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38072/0 (socket says 172.21.15.53:38072) 2023-11-06T01:32:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5781ede700 1 -- 172.21.15.53:0/3387716363 learned_addr learned my addr 172.21.15.53:0/3387716363 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f57816dd700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c0ffcb0 0x7f577c069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5780edc700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f577c100ea0 0x7f577c19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5781ede700 1 -- 172.21.15.53:0/3387716363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f577c100ea0 msgr2=0x7f577c19b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5781ede700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f577c100ea0 0x7f577c19b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5781ede700 1 -- 172.21.15.53:0/3387716363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c0ffcb0 msgr2=0x7f577c069a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5781ede700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c0ffcb0 0x7f577c069a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5781ede700 1 -- 172.21.15.53:0/3387716363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700095f0 con 0x7f577c101840 2023-11-06T01:32:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f57816dd700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c0ffcb0 0x7f577c069a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5781ede700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f577c101840 0x7f577c19dda0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f577400ada0 tx=0x7f577400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f576e7fc700 1 -- 172.21.15.53:0/3387716363 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577400f070 con 0x7f577c101840 2023-11-06T01:32:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f576e7fc700 1 -- 172.21.15.53:0/3387716363 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5774007b30 con 0x7f577c101840 2023-11-06T01:32:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5783941700 1 -- 172.21.15.53:0/3387716363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c19eb50 con 0x7f577c101840 2023-11-06T01:32:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f576e7fc700 1 -- 172.21.15.53:0/3387716363 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774013cb0 con 0x7f577c101840 2023-11-06T01:32:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.192+0000 7f5783941700 1 -- 172.21.15.53:0/3387716363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c19f0b0 con 0x7f577c101840 2023-11-06T01:32:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.193+0000 7f5783941700 1 -- 172.21.15.53:0/3387716363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577c10b4c0 con 0x7f577c101840 2023-11-06T01:32:32.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.194+0000 7f576e7fc700 1 -- 172.21.15.53:0/3387716363 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f577400c980 con 0x7f577c101840 2023-11-06T01:32:32.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.195+0000 7f576e7fc700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57680a05a0 0x7f57680a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:32.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.195+0000 7f57816dd700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57680a05a0 0x7f57680a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:32.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.195+0000 7f576e7fc700 1 -- 172.21.15.53:0/3387716363 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f57740cba80 con 0x7f577c101840 2023-11-06T01:32:32.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.195+0000 7f57816dd700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57680a05a0 0x7f57680a2a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f577000b580 tx=0x7f5770005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:32.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.198+0000 7f576e7fc700 1 -- 172.21.15.53:0/3387716363 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5774095760 con 0x7f577c101840 2023-11-06T01:32:32.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.437+0000 7f5783941700 1 -- 172.21.15.53:0/3387716363 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f577c0611d0 con 0x7f57680a05a0 2023-11-06T01:32:32.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.442+0000 7f576e7fc700 1 -- 172.21.15.53:0/3387716363 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f577c0611d0 con 0x7f57680a05a0 2023-11-06T01:32:32.442 INFO:teuthology.orchestra.run.smithi053.stdout:/volumes/_nogroup/sv_0/89ffd47c-d02c-4f3f-9461-a88952de3238 2023-11-06T01:32:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 -- 172.21.15.53:0/3387716363 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57680a05a0 msgr2=0x7f57680a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57680a05a0 0x7f57680a2a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f577000b580 tx=0x7f5770005fb0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 -- 172.21.15.53:0/3387716363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f577c101840 msgr2=0x7f577c19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f577c101840 0x7f577c19dda0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f577400ada0 tx=0x7f577400c330 comp rx=0 tx=0).stop 2023-11-06T01:32:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 -- 172.21.15.53:0/3387716363 shutdown_connections 2023-11-06T01:32:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57680a05a0 0x7f57680a2a60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c0ffcb0 0x7f577c069a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f577c101840 0x7f577c19dda0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 --2- 172.21.15.53:0/3387716363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f577c100ea0 0x7f577c19b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 -- 172.21.15.53:0/3387716363 >> 172.21.15.53:0/3387716363 conn(0x7f577c0fb220 msgr2=0x7f577c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:32.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 -- 172.21.15.53:0/3387716363 shutdown_connections 2023-11-06T01:32:32.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.444+0000 7f5783941700 1 -- 172.21.15.53:0/3387716363 wait complete. 2023-11-06T01:32:32.457 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-11-06T01:32:32.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.586+0000 7f8164c14700 1 -- 172.21.15.53:0/1540997629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8160102910 msgr2=0x7f81601091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.586+0000 7f8164c14700 1 --2- 172.21.15.53:0/1540997629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8160102910 0x7f81601091c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f815800b0d0 tx=0x7f815800b3e0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.586+0000 7f8164c14700 1 -- 172.21.15.53:0/1540997629 shutdown_connections 2023-11-06T01:32:32.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.586+0000 7f8164c14700 1 --2- 172.21.15.53:0/1540997629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8160102910 0x7f81601091c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.586+0000 7f8164c14700 1 --2- 172.21.15.53:0/1540997629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8160100d80 0x7f81601011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.586+0000 7f8164c14700 1 --2- 172.21.15.53:0/1540997629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160101f70 0x7f81601023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.586+0000 7f8164c14700 1 -- 172.21.15.53:0/1540997629 >> 172.21.15.53:0/1540997629 conn(0x7f81600fc310 msgr2=0x7f81600fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.586+0000 7f8164c14700 1 -- 172.21.15.53:0/1540997629 shutdown_connections 2023-11-06T01:32:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.586+0000 7f8164c14700 1 -- 172.21.15.53:0/1540997629 wait complete. 2023-11-06T01:32:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.586+0000 7f8164c14700 1 Processor -- start 2023-11-06T01:32:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f8164c14700 1 -- start start 2023-11-06T01:32:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f8164c14700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8160100d80 0x7f8160196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f8164c14700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8160101f70 0x7f8160197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f8164c14700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160102910 0x7f816019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f8164c14700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816019e6d0 con 0x7f8160101f70 2023-11-06T01:32:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f8164c14700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816019e810 con 0x7f8160102910 2023-11-06T01:32:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f8164c14700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816019e9a0 con 0x7f8160100d80 2023-11-06T01:32:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f815dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8160101f70 0x7f8160197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f815dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8160101f70 0x7f8160197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38084/0 (socket says 172.21.15.53:38084) 2023-11-06T01:32:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f815dd9b700 1 -- 172.21.15.53:0/715886216 learned_addr learned my addr 172.21.15.53:0/715886216 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f815e59c700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8160100d80 0x7f8160196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f815ed9d700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160102910 0x7f816019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f815dd9b700 1 -- 172.21.15.53:0/715886216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8160100d80 msgr2=0x7f8160196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f815dd9b700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8160100d80 0x7f8160196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f815dd9b700 1 -- 172.21.15.53:0/715886216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160102910 msgr2=0x7f816019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f815dd9b700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160102910 0x7f816019e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f815dd9b700 1 -- 172.21.15.53:0/715886216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8158009d00 con 0x7f8160101f70 2023-11-06T01:32:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f815dd9b700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8160101f70 0x7f8160197090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f815000d980 tx=0x7f815000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.587+0000 7f814f7fe700 1 -- 172.21.15.53:0/715886216 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150004220 con 0x7f8160101f70 2023-11-06T01:32:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.588+0000 7f814f7fe700 1 -- 172.21.15.53:0/715886216 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8150007640 con 0x7f8160101f70 2023-11-06T01:32:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.588+0000 7f8164c14700 1 -- 172.21.15.53:0/715886216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816019ec80 con 0x7f8160101f70 2023-11-06T01:32:32.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.588+0000 7f814f7fe700 1 -- 172.21.15.53:0/715886216 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150005690 con 0x7f8160101f70 2023-11-06T01:32:32.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.588+0000 7f8164c14700 1 -- 172.21.15.53:0/715886216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816019f190 con 0x7f8160101f70 2023-11-06T01:32:32.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.588+0000 7f8164c14700 1 -- 172.21.15.53:0/715886216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816010c5a0 con 0x7f8160101f70 2023-11-06T01:32:32.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.590+0000 7f814f7fe700 1 -- 172.21.15.53:0/715886216 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f8150004750 con 0x7f8160101f70 2023-11-06T01:32:32.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.590+0000 7f814f7fe700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f81480a0520 0x7f81480a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:32.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.590+0000 7f815e59c700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f81480a0520 0x7f81480a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:32.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.590+0000 7f814f7fe700 1 -- 172.21.15.53:0/715886216 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f81500cbab0 con 0x7f8160101f70 2023-11-06T01:32:32.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.591+0000 7f815e59c700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f81480a0520 0x7f81480a29e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8160101dd0 tx=0x7f815400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:32.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.594+0000 7f814f7fe700 1 -- 172.21.15.53:0/715886216 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f81500957a0 con 0x7f8160101f70 2023-11-06T01:32:32.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.830+0000 7f8164c14700 1 -- 172.21.15.53:0/715886216 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f81600611d0 con 0x7f81480a0520 2023-11-06T01:32:32.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.854+0000 7f814f7fe700 1 -- 172.21.15.53:0/715886216 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f81600611d0 con 0x7f81480a0520 2023-11-06T01:32:32.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 -- 172.21.15.53:0/715886216 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f81480a0520 msgr2=0x7f81480a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f81480a0520 0x7f81480a29e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8160101dd0 tx=0x7f815400b410 comp rx=0 tx=0).stop 2023-11-06T01:32:32.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 -- 172.21.15.53:0/715886216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8160101f70 msgr2=0x7f8160197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8160101f70 0x7f8160197090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f815000d980 tx=0x7f815000dc90 comp rx=0 tx=0).stop 2023-11-06T01:32:32.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 -- 172.21.15.53:0/715886216 shutdown_connections 2023-11-06T01:32:32.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f81480a0520 0x7f81480a29e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160102910 0x7f816019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8160100d80 0x7f8160196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 --2- 172.21.15.53:0/715886216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8160101f70 0x7f8160197090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 -- 172.21.15.53:0/715886216 >> 172.21.15.53:0/715886216 conn(0x7f81600fc310 msgr2=0x7f81600fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:32.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 -- 172.21.15.53:0/715886216 shutdown_connections 2023-11-06T01:32:32.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.856+0000 7f8164c14700 1 -- 172.21.15.53:0/715886216 wait complete. 2023-11-06T01:32:32.869 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-11-06T01:32:32.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.998+0000 7f303e84c700 1 -- 172.21.15.53:0/2888416582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380ffc80 msgr2=0x7f30381000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:32.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.998+0000 7f303e84c700 1 --2- 172.21.15.53:0/2888416582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380ffc80 0x7f30381000a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f30280099c0 tx=0x7f3028009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:32.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.998+0000 7f303e84c700 1 -- 172.21.15.53:0/2888416582 shutdown_connections 2023-11-06T01:32:32.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.998+0000 7f303e84c700 1 --2- 172.21.15.53:0/2888416582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3038101810 0x7f30381080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.998+0000 7f303e84c700 1 --2- 172.21.15.53:0/2888416582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380ffc80 0x7f30381000a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.998+0000 7f303e84c700 1 --2- 172.21.15.53:0/2888416582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038100e70 0x7f30381012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.998+0000 7f303e84c700 1 -- 172.21.15.53:0/2888416582 >> 172.21.15.53:0/2888416582 conn(0x7f30380fb1f0 msgr2=0x7f30380fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:33.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.998+0000 7f303e84c700 1 -- 172.21.15.53:0/2888416582 shutdown_connections 2023-11-06T01:32:33.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.998+0000 7f303e84c700 1 -- 172.21.15.53:0/2888416582 wait complete. 2023-11-06T01:32:33.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.999+0000 7f303e84c700 1 Processor -- start 2023-11-06T01:32:33.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.999+0000 7f303e84c700 1 -- start start 2023-11-06T01:32:33.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.999+0000 7f303e84c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380ffc80 0x7f3038198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.999+0000 7f303e84c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3038100e70 0x7f3038199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.999+0000 7f303e84c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038101810 0x7f30381a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.999+0000 7f303e84c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30381a0860 con 0x7f30380ffc80 2023-11-06T01:32:33.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.999+0000 7f303e84c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30381a09d0 con 0x7f3038101810 2023-11-06T01:32:33.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.999+0000 7f303e84c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30381a0b90 con 0x7f3038100e70 2023-11-06T01:32:33.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.999+0000 7f3037fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380ffc80 0x7f3038198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.999+0000 7f3037fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380ffc80 0x7f3038198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38098/0 (socket says 172.21.15.53:38098) 2023-11-06T01:32:33.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:32.999+0000 7f3037fff700 1 -- 172.21.15.53:0/3215023761 learned_addr learned my addr 172.21.15.53:0/3215023761 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:33.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.000+0000 7f303cde9700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038101810 0x7f30381a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.000+0000 7f30377fe700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3038100e70 0x7f3038199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.000+0000 7f3037fff700 1 -- 172.21.15.53:0/3215023761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3038100e70 msgr2=0x7f3038199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.000+0000 7f3037fff700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3038100e70 0x7f3038199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.000+0000 7f3037fff700 1 -- 172.21.15.53:0/3215023761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038101810 msgr2=0x7f30381a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.000+0000 7f3037fff700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038101810 0x7f30381a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.000+0000 7f3037fff700 1 -- 172.21.15.53:0/3215023761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30280095f0 con 0x7f30380ffc80 2023-11-06T01:32:33.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.000+0000 7f3037fff700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380ffc80 0x7f3038198d20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3028009990 tx=0x7f302800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:33.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.001+0000 7f30357fa700 1 -- 172.21.15.53:0/3215023761 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302801d070 con 0x7f30380ffc80 2023-11-06T01:32:33.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.001+0000 7f30357fa700 1 -- 172.21.15.53:0/3215023761 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3028005030 con 0x7f30380ffc80 2023-11-06T01:32:33.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.001+0000 7f30357fa700 1 -- 172.21.15.53:0/3215023761 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3028004a70 con 0x7f30380ffc80 2023-11-06T01:32:33.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.001+0000 7f303e84c700 1 -- 172.21.15.53:0/3215023761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30381a0e10 con 0x7f30380ffc80 2023-11-06T01:32:33.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.002+0000 7f303e84c700 1 -- 172.21.15.53:0/3215023761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30381a1180 con 0x7f30380ffc80 2023-11-06T01:32:33.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.003+0000 7f303e84c700 1 -- 172.21.15.53:0/3215023761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3038106c30 con 0x7f30380ffc80 2023-11-06T01:32:33.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.004+0000 7f30357fa700 1 -- 172.21.15.53:0/3215023761 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f3028005410 con 0x7f30380ffc80 2023-11-06T01:32:33.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.005+0000 7f30357fa700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30200a05a0 0x7f30200a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.005+0000 7f30377fe700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30200a05a0 0x7f30200a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.005+0000 7f30357fa700 1 -- 172.21.15.53:0/3215023761 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f30280cc920 con 0x7f30380ffc80 2023-11-06T01:32:33.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.005+0000 7f30377fe700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30200a05a0 0x7f30200a2a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f302c009c60 tx=0x7f302c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:33.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.011+0000 7f30357fa700 1 -- 172.21.15.53:0/3215023761 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f30280966a0 con 0x7f30380ffc80 2023-11-06T01:32:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.254+0000 7f303e84c700 1 -- 172.21.15.53:0/3215023761 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f30380611d0 con 0x7f30200a05a0 2023-11-06T01:32:33.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.259+0000 7f30357fa700 1 -- 172.21.15.53:0/3215023761 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f30380611d0 con 0x7f30200a05a0 2023-11-06T01:32:33.259 INFO:teuthology.orchestra.run.smithi053.stdout:/volumes/_nogroup/sv_1/9b213191-048f-4ff8-846e-a578c140694d 2023-11-06T01:32:33.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 -- 172.21.15.53:0/3215023761 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30200a05a0 msgr2=0x7f30200a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30200a05a0 0x7f30200a2a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f302c009c60 tx=0x7f302c009260 comp rx=0 tx=0).stop 2023-11-06T01:32:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 -- 172.21.15.53:0/3215023761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380ffc80 msgr2=0x7f3038198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380ffc80 0x7f3038198d20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3028009990 tx=0x7f302800f690 comp rx=0 tx=0).stop 2023-11-06T01:32:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 -- 172.21.15.53:0/3215023761 shutdown_connections 2023-11-06T01:32:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f30200a05a0 0x7f30200a2a60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038101810 0x7f30381a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380ffc80 0x7f3038198d20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 --2- 172.21.15.53:0/3215023761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3038100e70 0x7f3038199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 -- 172.21.15.53:0/3215023761 >> 172.21.15.53:0/3215023761 conn(0x7f30380fb1f0 msgr2=0x7f30380fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 -- 172.21.15.53:0/3215023761 shutdown_connections 2023-11-06T01:32:33.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.261+0000 7f303e84c700 1 -- 172.21.15.53:0/3215023761 wait complete. 2023-11-06T01:32:33.272 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:32 smithi053 ceph-mon[99472]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:33.273 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:32 smithi053 ceph-mon[99472]: pgmap v255: 97 pgs: 97 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s wr, 12 op/s 2023-11-06T01:32:33.273 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:32 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] up:active 2023-11-06T01:32:33.273 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:32 smithi053 ceph-mon[99472]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-11-06T01:32:33.273 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:32 smithi053 ceph-mon[99472]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-06T01:32:33.273 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:32 smithi053 ceph-mon[99472]: Cluster is now healthy 2023-11-06T01:32:33.274 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:32 smithi053 ceph-mon[99472]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2023-11-06T01:32:33.274 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:32 smithi053 ceph-mon[99472]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-11-06T01:32:33.274 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:32 smithi053 ceph-mon[99472]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-11-06T01:32:33.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:32:33.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:32 smithi062 ceph-mon[103924]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:33.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:32 smithi062 ceph-mon[103924]: pgmap v255: 97 pgs: 97 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s wr, 12 op/s 2023-11-06T01:32:33.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:32 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] up:active 2023-11-06T01:32:33.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:32 smithi062 ceph-mon[103924]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-11-06T01:32:33.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:32 smithi062 ceph-mon[103924]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-06T01:32:33.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:32 smithi062 ceph-mon[103924]: Cluster is now healthy 2023-11-06T01:32:33.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:32 smithi062 ceph-mon[103924]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2023-11-06T01:32:33.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:32 smithi062 ceph-mon[103924]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-11-06T01:32:33.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:32 smithi062 ceph-mon[103924]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-11-06T01:32:33.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.400+0000 7f61a2962700 1 -- 172.21.15.53:0/4072348636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f619c0ffee0 msgr2=0x7f619c100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.400+0000 7f61a2962700 1 --2- 172.21.15.53:0/4072348636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f619c0ffee0 0x7f619c100360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6184009990 tx=0x7f6184009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.400+0000 7f61a2962700 1 -- 172.21.15.53:0/4072348636 shutdown_connections 2023-11-06T01:32:33.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.400+0000 7f61a2962700 1 --2- 172.21.15.53:0/4072348636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f619c101290 0x7f619c1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.400+0000 7f61a2962700 1 --2- 172.21.15.53:0/4072348636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f619c1008a0 0x7f619c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.400+0000 7f61a2962700 1 --2- 172.21.15.53:0/4072348636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f619c0ffee0 0x7f619c100360 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.400+0000 7f61a2962700 1 -- 172.21.15.53:0/4072348636 >> 172.21.15.53:0/4072348636 conn(0x7f619c0fa950 msgr2=0x7f619c0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:33.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.400+0000 7f61a2962700 1 -- 172.21.15.53:0/4072348636 shutdown_connections 2023-11-06T01:32:33.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.400+0000 7f61a2962700 1 -- 172.21.15.53:0/4072348636 wait complete. 2023-11-06T01:32:33.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.400+0000 7f61a2962700 1 Processor -- start 2023-11-06T01:32:33.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a2962700 1 -- start start 2023-11-06T01:32:33.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a2962700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f619c0ffee0 0x7f619c195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a2962700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f619c1008a0 0x7f619c195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a2962700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f619c101290 0x7f619c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a2962700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19d490 con 0x7f619c101290 2023-11-06T01:32:33.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a2962700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19d5d0 con 0x7f619c1008a0 2023-11-06T01:32:33.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a2962700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19d760 con 0x7f619c0ffee0 2023-11-06T01:32:33.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a0eff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f619c101290 0x7f619c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a0eff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f619c101290 0x7f619c19cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38138/0 (socket says 172.21.15.53:38138) 2023-11-06T01:32:33.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a0eff700 1 -- 172.21.15.53:0/1420887198 learned_addr learned my addr 172.21.15.53:0/1420887198 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:33.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f619b7fe700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f619c1008a0 0x7f619c195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f619bfff700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f619c0ffee0 0x7f619c195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a0eff700 1 -- 172.21.15.53:0/1420887198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f619c0ffee0 msgr2=0x7f619c195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a0eff700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f619c0ffee0 0x7f619c195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a0eff700 1 -- 172.21.15.53:0/1420887198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f619c1008a0 msgr2=0x7f619c195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a0eff700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f619c1008a0 0x7f619c195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a0eff700 1 -- 172.21.15.53:0/1420887198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61840095f0 con 0x7f619c101290 2023-11-06T01:32:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.401+0000 7f61a0eff700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f619c101290 0x7f619c19cec0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6194007ee0 tx=0x7f619400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.402+0000 7f61997fa700 1 -- 172.21.15.53:0/1420887198 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194017070 con 0x7f619c101290 2023-11-06T01:32:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.402+0000 7f61997fa700 1 -- 172.21.15.53:0/1420887198 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f619400f040 con 0x7f619c101290 2023-11-06T01:32:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.402+0000 7f61a2962700 1 -- 172.21.15.53:0/1420887198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c19da40 con 0x7f619c101290 2023-11-06T01:32:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.402+0000 7f61997fa700 1 -- 172.21.15.53:0/1420887198 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194005740 con 0x7f619c101290 2023-11-06T01:32:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.402+0000 7f61a2962700 1 -- 172.21.15.53:0/1420887198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c19df50 con 0x7f619c101290 2023-11-06T01:32:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.403+0000 7f61a2962700 1 -- 172.21.15.53:0/1420887198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c10b480 con 0x7f619c101290 2023-11-06T01:32:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.404+0000 7f61997fa700 1 -- 172.21.15.53:0/1420887198 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f619400dc60 con 0x7f619c101290 2023-11-06T01:32:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.404+0000 7f61997fa700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f61880a05f0 0x7f61880a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.405+0000 7f619bfff700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f61880a05f0 0x7f61880a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.405+0000 7f61997fa700 1 -- 172.21.15.53:0/1420887198 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f61940cb010 con 0x7f619c101290 2023-11-06T01:32:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.405+0000 7f619bfff700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f61880a05f0 0x7f61880a2ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f619c067cd0 tx=0x7f618c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.409+0000 7f61997fa700 1 -- 172.21.15.53:0/1420887198 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6194094d00 con 0x7f619c101290 2023-11-06T01:32:33.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:32 smithi163 ceph-mon[104416]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:33.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:32 smithi163 ceph-mon[104416]: pgmap v255: 97 pgs: 97 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s wr, 12 op/s 2023-11-06T01:32:33.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:32 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] up:active 2023-11-06T01:32:33.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:32 smithi163 ceph-mon[104416]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-11-06T01:32:33.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:32 smithi163 ceph-mon[104416]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-06T01:32:33.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:32 smithi163 ceph-mon[104416]: Cluster is now healthy 2023-11-06T01:32:33.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:32 smithi163 ceph-mon[104416]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2023-11-06T01:32:33.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:32 smithi163 ceph-mon[104416]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-11-06T01:32:33.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:32 smithi163 ceph-mon[104416]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-11-06T01:32:33.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.656+0000 7f61a2962700 1 -- 172.21.15.53:0/1420887198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f619c10b6a0 con 0x7f619c101290 2023-11-06T01:32:33.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.657+0000 7f61997fa700 1 -- 172.21.15.53:0/1420887198 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10815 (secure 0 0 0) 0x7f6194098320 con 0x7f619c101290 2023-11-06T01:32:33.658 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:33.658 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-06T01:32:26.093194+0000","modified":"2023-11-06T01:32:33.006369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24472,"mds_2":24464,"mds_3":24422,"mds_4":24395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24395":{"gid":24395,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T01:32:33.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 -- 172.21.15.53:0/1420887198 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f61880a05f0 msgr2=0x7f61880a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f61880a05f0 0x7f61880a2ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f619c067cd0 tx=0x7f618c009260 comp rx=0 tx=0).stop 2023-11-06T01:32:33.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 -- 172.21.15.53:0/1420887198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f619c101290 msgr2=0x7f619c19cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f619c101290 0x7f619c19cec0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6194007ee0 tx=0x7f619400d3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 -- 172.21.15.53:0/1420887198 shutdown_connections 2023-11-06T01:32:33.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f61880a05f0 0x7f61880a2ab0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f619c101290 0x7f619c19cec0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f619c0ffee0 0x7f619c195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 --2- 172.21.15.53:0/1420887198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f619c1008a0 0x7f619c195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 -- 172.21.15.53:0/1420887198 >> 172.21.15.53:0/1420887198 conn(0x7f619c0fa950 msgr2=0x7f619c0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:33.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 -- 172.21.15.53:0/1420887198 shutdown_connections 2023-11-06T01:32:33.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.659+0000 7f61a2962700 1 -- 172.21.15.53:0/1420887198 wait complete. 2023-11-06T01:32:33.663 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-11-06T01:32:33.673 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:32:33.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.798+0000 7f492cb0f700 1 -- 172.21.15.53:0/1451050174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4928101f80 msgr2=0x7f49281023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.798+0000 7f492cb0f700 1 --2- 172.21.15.53:0/1451050174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4928101f80 0x7f49281023e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f491c0099c0 tx=0x7f491c009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.798+0000 7f492cb0f700 1 -- 172.21.15.53:0/1451050174 shutdown_connections 2023-11-06T01:32:33.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.798+0000 7f492cb0f700 1 --2- 172.21.15.53:0/1451050174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4928100d90 0x7f49281011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.798+0000 7f492cb0f700 1 --2- 172.21.15.53:0/1451050174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4928102920 0x7f49281091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.798+0000 7f492cb0f700 1 --2- 172.21.15.53:0/1451050174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4928101f80 0x7f49281023e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.798+0000 7f492cb0f700 1 -- 172.21.15.53:0/1451050174 >> 172.21.15.53:0/1451050174 conn(0x7f49280fc2e0 msgr2=0x7f49280fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:33.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.798+0000 7f492cb0f700 1 -- 172.21.15.53:0/1451050174 shutdown_connections 2023-11-06T01:32:33.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.799+0000 7f492cb0f700 1 -- 172.21.15.53:0/1451050174 wait complete. 2023-11-06T01:32:33.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.799+0000 7f492cb0f700 1 Processor -- start 2023-11-06T01:32:33.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.799+0000 7f492cb0f700 1 -- start start 2023-11-06T01:32:33.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492cb0f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4928100d90 0x7f4928196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492cb0f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4928101f80 0x7f4928196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492cb0f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4928102920 0x7f492819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492cb0f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492819e490 con 0x7f4928100d90 2023-11-06T01:32:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492cb0f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492819e5d0 con 0x7f4928101f80 2023-11-06T01:32:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492cb0f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492819e760 con 0x7f4928102920 2023-11-06T01:32:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4928100d90 0x7f4928196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4928100d90 0x7f4928196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38148/0 (socket says 172.21.15.53:38148) 2023-11-06T01:32:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492659c700 1 -- 172.21.15.53:0/1434421414 learned_addr learned my addr 172.21.15.53:0/1434421414 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f4926d9d700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4928102920 0x7f492819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492659c700 1 -- 172.21.15.53:0/1434421414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4928102920 msgr2=0x7f492819dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492659c700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4928102920 0x7f492819dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f4925d9b700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4928101f80 0x7f4928196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492659c700 1 -- 172.21.15.53:0/1434421414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4928101f80 msgr2=0x7f4928196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:33.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492659c700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4928101f80 0x7f4928196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:33.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492659c700 1 -- 172.21.15.53:0/1434421414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c0095f0 con 0x7f4928100d90 2023-11-06T01:32:33.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f492659c700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4928100d90 0x7f4928196910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f491800c990 tx=0x7f491800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:33.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.800+0000 7f49177fe700 1 -- 172.21.15.53:0/1434421414 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918007b00 con 0x7f4928100d90 2023-11-06T01:32:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.801+0000 7f49177fe700 1 -- 172.21.15.53:0/1434421414 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4918007c60 con 0x7f4928100d90 2023-11-06T01:32:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.801+0000 7f492cb0f700 1 -- 172.21.15.53:0/1434421414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492819ea40 con 0x7f4928100d90 2023-11-06T01:32:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.801+0000 7f49177fe700 1 -- 172.21.15.53:0/1434421414 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918018660 con 0x7f4928100d90 2023-11-06T01:32:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.801+0000 7f492cb0f700 1 -- 172.21.15.53:0/1434421414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492819efa0 con 0x7f4928100d90 2023-11-06T01:32:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.801+0000 7f492cb0f700 1 -- 172.21.15.53:0/1434421414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492810c5a0 con 0x7f4928100d90 2023-11-06T01:32:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.803+0000 7f49177fe700 1 -- 172.21.15.53:0/1434421414 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4918004020 con 0x7f4928100d90 2023-11-06T01:32:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.803+0000 7f49177fe700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49100a05f0 0x7f49100a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.803+0000 7f49177fe700 1 -- 172.21.15.53:0/1434421414 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f49180cb900 con 0x7f4928100d90 2023-11-06T01:32:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.803+0000 7f4925d9b700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49100a05f0 0x7f49100a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.804+0000 7f4925d9b700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49100a05f0 0x7f49100a2ab0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f491c0098e0 tx=0x7f491c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:33.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:33.807+0000 7f49177fe700 1 -- 172.21.15.53:0/1434421414 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4918095700 con 0x7f4928100d90 2023-11-06T01:32:34.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.011+0000 7f492cb0f700 1 -- 172.21.15.53:0/1434421414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4928107c10 con 0x7f4928100d90 2023-11-06T01:32:34.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.013+0000 7f49177fe700 1 -- 172.21.15.53:0/1434421414 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f4918098d20 con 0x7f4928100d90 2023-11-06T01:32:34.014 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:34.015 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:32:27.094591+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:32:34.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 -- 172.21.15.53:0/1434421414 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49100a05f0 msgr2=0x7f49100a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49100a05f0 0x7f49100a2ab0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f491c0098e0 tx=0x7f491c00b540 comp rx=0 tx=0).stop 2023-11-06T01:32:34.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 -- 172.21.15.53:0/1434421414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4928100d90 msgr2=0x7f4928196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4928100d90 0x7f4928196910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f491800c990 tx=0x7f491800cd50 comp rx=0 tx=0).stop 2023-11-06T01:32:34.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 -- 172.21.15.53:0/1434421414 shutdown_connections 2023-11-06T01:32:34.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49100a05f0 0x7f49100a2ab0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4928102920 0x7f492819dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4928100d90 0x7f4928196910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 --2- 172.21.15.53:0/1434421414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4928101f80 0x7f4928196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 -- 172.21.15.53:0/1434421414 >> 172.21.15.53:0/1434421414 conn(0x7f49280fc2e0 msgr2=0x7f49280fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:34.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 -- 172.21.15.53:0/1434421414 shutdown_connections 2023-11-06T01:32:34.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.016+0000 7f492cb0f700 1 -- 172.21.15.53:0/1434421414 wait complete. 2023-11-06T01:32:34.035 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T01:32:34.036 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T01:32:34.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.163+0000 7f6944f03700 1 -- 172.21.15.53:0/2763355993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6940100d20 msgr2=0x7f6940101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.163+0000 7f6944f03700 1 --2- 172.21.15.53:0/2763355993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6940100d20 0x7f6940101140 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f69300099c0 tx=0x7f6930009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.163+0000 7f6944f03700 1 -- 172.21.15.53:0/2763355993 shutdown_connections 2023-11-06T01:32:34.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.163+0000 7f6944f03700 1 --2- 172.21.15.53:0/2763355993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69401028b0 0x7f6940109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.163+0000 7f6944f03700 1 --2- 172.21.15.53:0/2763355993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6940100d20 0x7f6940101140 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.163+0000 7f6944f03700 1 --2- 172.21.15.53:0/2763355993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6940101f10 0x7f6940102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.163+0000 7f6944f03700 1 -- 172.21.15.53:0/2763355993 >> 172.21.15.53:0/2763355993 conn(0x7f69400fc270 msgr2=0x7f69400fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:34.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.163+0000 7f6944f03700 1 -- 172.21.15.53:0/2763355993 shutdown_connections 2023-11-06T01:32:34.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.163+0000 7f6944f03700 1 -- 172.21.15.53:0/2763355993 wait complete. 2023-11-06T01:32:34.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f6944f03700 1 Processor -- start 2023-11-06T01:32:34.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f6944f03700 1 -- start start 2023-11-06T01:32:34.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f6944f03700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6940100d20 0x7f6940196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:34.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f6944f03700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6940101f10 0x7f6940196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:34.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f6944f03700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69401028b0 0x7f694019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:34.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f6944f03700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694019e5a0 con 0x7f6940101f10 2023-11-06T01:32:34.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f6944f03700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694019e6e0 con 0x7f6940100d20 2023-11-06T01:32:34.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f6944f03700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694019e870 con 0x7f69401028b0 2023-11-06T01:32:34.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f693dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6940101f10 0x7f6940196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:34.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f693dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6940101f10 0x7f6940196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38164/0 (socket says 172.21.15.53:38164) 2023-11-06T01:32:34.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f693dd9b700 1 -- 172.21.15.53:0/731552829 learned_addr learned my addr 172.21.15.53:0/731552829 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:34.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f693e59c700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6940100d20 0x7f6940196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:34.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.164+0000 7f693ed9d700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69401028b0 0x7f694019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:34.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.165+0000 7f693dd9b700 1 -- 172.21.15.53:0/731552829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69401028b0 msgr2=0x7f694019dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.165+0000 7f693dd9b700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69401028b0 0x7f694019dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.165+0000 7f693dd9b700 1 -- 172.21.15.53:0/731552829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6940100d20 msgr2=0x7f6940196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.165+0000 7f693dd9b700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6940100d20 0x7f6940196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.165+0000 7f693dd9b700 1 -- 172.21.15.53:0/731552829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69300095f0 con 0x7f6940101f10 2023-11-06T01:32:34.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.165+0000 7f693dd9b700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6940101f10 0x7f6940196f60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f693400e9a0 tx=0x7f693400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:34.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.165+0000 7f692f7fe700 1 -- 172.21.15.53:0/731552829 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934009790 con 0x7f6940101f10 2023-11-06T01:32:34.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.165+0000 7f692f7fe700 1 -- 172.21.15.53:0/731552829 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6934010430 con 0x7f6940101f10 2023-11-06T01:32:34.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.165+0000 7f6944f03700 1 -- 172.21.15.53:0/731552829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694019eb50 con 0x7f6940101f10 2023-11-06T01:32:34.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.165+0000 7f692f7fe700 1 -- 172.21.15.53:0/731552829 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934018680 con 0x7f6940101f10 2023-11-06T01:32:34.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.165+0000 7f6944f03700 1 -- 172.21.15.53:0/731552829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694019f0b0 con 0x7f6940101f10 2023-11-06T01:32:34.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.166+0000 7f6944f03700 1 -- 172.21.15.53:0/731552829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69401082b0 con 0x7f6940101f10 2023-11-06T01:32:34.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.167+0000 7f692f7fe700 1 -- 172.21.15.53:0/731552829 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f69340105a0 con 0x7f6940101f10 2023-11-06T01:32:34.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.168+0000 7f692f7fe700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69280a0670 0x7f69280a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:34.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.168+0000 7f693e59c700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69280a0670 0x7f69280a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:34.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.168+0000 7f692f7fe700 1 -- 172.21.15.53:0/731552829 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6934014070 con 0x7f6940101f10 2023-11-06T01:32:34.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.168+0000 7f693e59c700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69280a0670 0x7f69280a2b30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6930009990 tx=0x7f6930005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:34.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.171+0000 7f692f7fe700 1 -- 172.21.15.53:0/731552829 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f69340958f0 con 0x7f6940101f10 2023-11-06T01:32:34.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:34 smithi053 ceph-mon[99472]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:34.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:34 smithi053 ceph-mon[99472]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:34.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:34 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] up:active 2023-11-06T01:32:34.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:34 smithi053 ceph-mon[99472]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-11-06T01:32:34.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:34 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:32:34.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:34 smithi053 ceph-mon[99472]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:34.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:34 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1420887198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:34.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.369+0000 7f6944f03700 1 -- 172.21.15.53:0/731552829 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69400611d0 con 0x7f69280a0670 2023-11-06T01:32:34.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.381+0000 7f692f7fe700 1 -- 172.21.15.53:0/731552829 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335315 (secure 0 0 0) 0x7f69400611d0 con 0x7f69280a0670 2023-11-06T01:32:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:34.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 -- 172.21.15.53:0/731552829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69280a0670 msgr2=0x7f69280a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69280a0670 0x7f69280a2b30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6930009990 tx=0x7f6930005fb0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 -- 172.21.15.53:0/731552829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6940101f10 msgr2=0x7f6940196f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6940101f10 0x7f6940196f60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f693400e9a0 tx=0x7f693400ecb0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 -- 172.21.15.53:0/731552829 shutdown_connections 2023-11-06T01:32:34.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f69280a0670 0x7f69280a2b30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6940100d20 0x7f6940196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69401028b0 0x7f694019dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 --2- 172.21.15.53:0/731552829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6940101f10 0x7f6940196f60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 -- 172.21.15.53:0/731552829 >> 172.21.15.53:0/731552829 conn(0x7f69400fc270 msgr2=0x7f69401053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:34.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 -- 172.21.15.53:0/731552829 shutdown_connections 2023-11-06T01:32:34.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.385+0000 7f6944f03700 1 -- 172.21.15.53:0/731552829 wait complete. 2023-11-06T01:32:34.393 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-06T01:32:34.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:34 smithi062 ceph-mon[103924]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:34.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:34 smithi062 ceph-mon[103924]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:34.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:34 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] up:active 2023-11-06T01:32:34.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:34 smithi062 ceph-mon[103924]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-11-06T01:32:34.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:34 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:32:34.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:34 smithi062 ceph-mon[103924]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:34.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:34 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1420887198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:34.399 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2023-11-06T01:32:33.473247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593764,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":135,"num_read_kb":116,"num_write":296,"num_write_kb":2974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":283,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":333984,"kb_used_data":10560,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607840,"statfs":{"total":1151940427776,"available":1151598428160,"internally_reserved":0,"allocated":10813440,"data_stored":6799147,"data_compressed":108579,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3396,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":115,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.814836"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408521+0000","last_change":"2023-11-06T01:32:26.110793+0000","last_active":"2023-11-06T01:32:26.408521+0000","last_peered":"2023-11-06T01:32:26.408521+0000","last_clean":"2023-11-06T01:32:26.408521+0000","last_became_active":"2023-11-06T01:32:26.110489+0000","last_became_peered":"2023-11-06T01:32:26.110489+0000","last_unstale":"2023-11-06T01:32:26.408521+0000","last_undegraded":"2023-11-06T01:32:26.408521+0000","last_fullsized":"2023-11-06T01:32:26.408521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:01:08.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231988+0000","last_change":"2023-11-06T01:32:26.107273+0000","last_active":"2023-11-06T01:32:27.231988+0000","last_peered":"2023-11-06T01:32:27.231988+0000","last_clean":"2023-11-06T01:32:27.231988+0000","last_became_active":"2023-11-06T01:32:26.106961+0000","last_became_peered":"2023-11-06T01:32:26.106961+0000","last_unstale":"2023-11-06T01:32:27.231988+0000","last_undegraded":"2023-11-06T01:32:27.231988+0000","last_fullsized":"2023-11-06T01:32:27.231988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:40:39.073926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231979+0000","last_change":"2023-11-06T01:32:26.121316+0000","last_active":"2023-11-06T01:32:27.231979+0000","last_peered":"2023-11-06T01:32:27.231979+0000","last_clean":"2023-11-06T01:32:27.231979+0000","last_became_active":"2023-11-06T01:32:26.120756+0000","last_became_peered":"2023-11-06T01:32:26.120756+0000","last_unstale":"2023-11-06T01:32:27.231979+0000","last_undegraded":"2023-11-06T01:32:27.231979+0000","last_fullsized":"2023-11-06T01:32:27.231979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:02:45.814577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.100053+0000","last_change":"2023-11-06T01:32:26.121951+0000","last_active":"2023-11-06T01:32:27.100053+0000","last_peered":"2023-11-06T01:32:27.100053+0000","last_clean":"2023-11-06T01:32:27.100053+0000","last_became_active":"2023-11-06T01:32:26.121636+0000","last_became_peered":"2023-11-06T01:32:26.121636+0000","last_unstale":"2023-11-06T01:32:27.100053+0000","last_undegraded":"2023-11-06T01:32:27.100053+0000","last_fullsized":"2023-11-06T01:32:27.100053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:02.381355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.119944+0000","last_change":"2023-11-06T01:32:26.117788+0000","last_active":"2023-11-06T01:32:26.119944+0000","last_peered":"2023-11-06T01:32:26.119944+0000","last_clean":"2023-11-06T01:32:26.119944+0000","last_became_active":"2023-11-06T01:32:26.117161+0000","last_became_peered":"2023-11-06T01:32:26.117161+0000","last_unstale":"2023-11-06T01:32:26.119944+0000","last_undegraded":"2023-11-06T01:32:26.119944+0000","last_fullsized":"2023-11-06T01:32:26.119944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:31.665690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221591+0000","last_change":"2023-11-06T01:32:26.109532+0000","last_active":"2023-11-06T01:32:27.221591+0000","last_peered":"2023-11-06T01:32:27.221591+0000","last_clean":"2023-11-06T01:32:27.221591+0000","last_became_active":"2023-11-06T01:32:26.106144+0000","last_became_peered":"2023-11-06T01:32:26.106144+0000","last_unstale":"2023-11-06T01:32:27.221591+0000","last_undegraded":"2023-11-06T01:32:27.221591+0000","last_fullsized":"2023-11-06T01:32:27.221591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:54.220515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221306+0000","last_change":"2023-11-06T01:32:26.116280+0000","last_active":"2023-11-06T01:32:27.221306+0000","last_peered":"2023-11-06T01:32:27.221306+0000","last_clean":"2023-11-06T01:32:27.221306+0000","last_became_active":"2023-11-06T01:32:26.115957+0000","last_became_peered":"2023-11-06T01:32:26.115957+0000","last_unstale":"2023-11-06T01:32:27.221306+0000","last_undegraded":"2023-11-06T01:32:27.221306+0000","last_fullsized":"2023-11-06T01:32:27.221306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:25.956902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221244+0000","last_change":"2023-11-06T01:32:26.124818+0000","last_active":"2023-11-06T01:32:27.221244+0000","last_peered":"2023-11-06T01:32:27.221244+0000","last_clean":"2023-11-06T01:32:27.221244+0000","last_became_active":"2023-11-06T01:32:26.124511+0000","last_became_peered":"2023-11-06T01:32:26.124511+0000","last_unstale":"2023-11-06T01:32:27.221244+0000","last_undegraded":"2023-11-06T01:32:27.221244+0000","last_fullsized":"2023-11-06T01:32:27.221244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:35.853898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222751+0000","last_change":"2023-11-06T01:32:26.114243+0000","last_active":"2023-11-06T01:32:27.222751+0000","last_peered":"2023-11-06T01:32:27.222751+0000","last_clean":"2023-11-06T01:32:27.222751+0000","last_became_active":"2023-11-06T01:32:26.113923+0000","last_became_peered":"2023-11-06T01:32:26.113923+0000","last_unstale":"2023-11-06T01:32:27.222751+0000","last_undegraded":"2023-11-06T01:32:27.222751+0000","last_fullsized":"2023-11-06T01:32:27.222751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:48.197329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221817+0000","last_change":"2023-11-06T01:32:26.113819+0000","last_active":"2023-11-06T01:32:27.221817+0000","last_peered":"2023-11-06T01:32:27.221817+0000","last_clean":"2023-11-06T01:32:27.221817+0000","last_became_active":"2023-11-06T01:32:26.113506+0000","last_became_peered":"2023-11-06T01:32:26.113506+0000","last_unstale":"2023-11-06T01:32:27.221817+0000","last_undegraded":"2023-11-06T01:32:27.221817+0000","last_fullsized":"2023-11-06T01:32:27.221817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:33.122100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221622+0000","last_change":"2023-11-06T01:32:26.116504+0000","last_active":"2023-11-06T01:32:27.221622+0000","last_peered":"2023-11-06T01:32:27.221622+0000","last_clean":"2023-11-06T01:32:27.221622+0000","last_became_active":"2023-11-06T01:32:26.115220+0000","last_became_peered":"2023-11-06T01:32:26.115220+0000","last_unstale":"2023-11-06T01:32:27.221622+0000","last_undegraded":"2023-11-06T01:32:27.221622+0000","last_fullsized":"2023-11-06T01:32:27.221622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:34.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.100025+0000","last_change":"2023-11-06T01:32:26.111596+0000","last_active":"2023-11-06T01:32:27.100025+0000","last_peered":"2023-11-06T01:32:27.100025+0000","last_clean":"2023-11-06T01:32:27.100025+0000","last_became_active":"2023-11-06T01:32:26.111309+0000","last_became_peered":"2023-11-06T01:32:26.111309+0000","last_unstale":"2023-11-06T01:32:27.100025+0000","last_undegraded":"2023-11-06T01:32:27.100025+0000","last_fullsized":"2023-11-06T01:32:27.100025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:34.917068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.098303+0000","last_change":"2023-11-06T01:32:26.114536+0000","last_active":"2023-11-06T01:32:27.098303+0000","last_peered":"2023-11-06T01:32:27.098303+0000","last_clean":"2023-11-06T01:32:27.098303+0000","last_became_active":"2023-11-06T01:32:26.114124+0000","last_became_peered":"2023-11-06T01:32:26.114124+0000","last_unstale":"2023-11-06T01:32:27.098303+0000","last_undegraded":"2023-11-06T01:32:27.098303+0000","last_fullsized":"2023-11-06T01:32:27.098303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:05:10.855240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231958+0000","last_change":"2023-11-06T01:32:26.111139+0000","last_active":"2023-11-06T01:32:27.231958+0000","last_peered":"2023-11-06T01:32:27.231958+0000","last_clean":"2023-11-06T01:32:27.231958+0000","last_became_active":"2023-11-06T01:32:26.110764+0000","last_became_peered":"2023-11-06T01:32:26.110764+0000","last_unstale":"2023-11-06T01:32:27.231958+0000","last_undegraded":"2023-11-06T01:32:27.231958+0000","last_fullsized":"2023-11-06T01:32:27.231958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:46.422701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231938+0000","last_change":"2023-11-06T01:32:26.116096+0000","last_active":"2023-11-06T01:32:27.231938+0000","last_peered":"2023-11-06T01:32:27.231938+0000","last_clean":"2023-11-06T01:32:27.231938+0000","last_became_active":"2023-11-06T01:32:26.115754+0000","last_became_peered":"2023-11-06T01:32:26.115754+0000","last_unstale":"2023-11-06T01:32:27.231938+0000","last_undegraded":"2023-11-06T01:32:27.231938+0000","last_fullsized":"2023-11-06T01:32:27.231938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:15:58.109435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.220995+0000","last_change":"2023-11-06T01:32:26.118169+0000","last_active":"2023-11-06T01:32:27.220995+0000","last_peered":"2023-11-06T01:32:27.220995+0000","last_clean":"2023-11-06T01:32:27.220995+0000","last_became_active":"2023-11-06T01:32:26.117837+0000","last_became_peered":"2023-11-06T01:32:26.117837+0000","last_unstale":"2023-11-06T01:32:27.220995+0000","last_undegraded":"2023-11-06T01:32:27.220995+0000","last_fullsized":"2023-11-06T01:32:27.220995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:46.838927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408530+0000","last_change":"2023-11-06T01:32:26.110020+0000","last_active":"2023-11-06T01:32:26.408530+0000","last_peered":"2023-11-06T01:32:26.408530+0000","last_clean":"2023-11-06T01:32:26.408530+0000","last_became_active":"2023-11-06T01:32:26.109707+0000","last_became_peered":"2023-11-06T01:32:26.109707+0000","last_unstale":"2023-11-06T01:32:26.408530+0000","last_undegraded":"2023-11-06T01:32:26.408530+0000","last_fullsized":"2023-11-06T01:32:26.408530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:00.016258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222625+0000","last_change":"2023-11-06T01:32:26.119949+0000","last_active":"2023-11-06T01:32:27.222625+0000","last_peered":"2023-11-06T01:32:27.222625+0000","last_clean":"2023-11-06T01:32:27.222625+0000","last_became_active":"2023-11-06T01:32:26.117594+0000","last_became_peered":"2023-11-06T01:32:26.117594+0000","last_unstale":"2023-11-06T01:32:27.222625+0000","last_undegraded":"2023-11-06T01:32:27.222625+0000","last_fullsized":"2023-11-06T01:32:27.222625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:17:22.757846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221583+0000","last_change":"2023-11-06T01:32:26.123176+0000","last_active":"2023-11-06T01:32:27.221583+0000","last_peered":"2023-11-06T01:32:27.221583+0000","last_clean":"2023-11-06T01:32:27.221583+0000","last_became_active":"2023-11-06T01:32:26.116687+0000","last_became_peered":"2023-11-06T01:32:26.116687+0000","last_unstale":"2023-11-06T01:32:27.221583+0000","last_undegraded":"2023-11-06T01:32:27.221583+0000","last_fullsized":"2023-11-06T01:32:27.221583+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:30.422426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.012844+0000","last_change":"2023-11-06T01:32:25.682991+0000","last_active":"2023-11-06T01:32:32.012844+0000","last_peered":"2023-11-06T01:32:32.012844+0000","last_clean":"2023-11-06T01:32:32.012844+0000","last_became_active":"2023-11-06T01:32:25.682687+0000","last_became_peered":"2023-11-06T01:32:25.682687+0000","last_unstale":"2023-11-06T01:32:32.012844+0000","last_undegraded":"2023-11-06T01:32:32.012844+0000","last_fullsized":"2023-11-06T01:32:32.012844+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:12:54.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221983+0000","last_change":"2023-11-06T01:32:26.126350+0000","last_active":"2023-11-06T01:32:27.221983+0000","last_peered":"2023-11-06T01:32:27.221983+0000","last_clean":"2023-11-06T01:32:27.221983+0000","last_became_active":"2023-11-06T01:32:26.124703+0000","last_became_peered":"2023-11-06T01:32:26.124703+0000","last_unstale":"2023-11-06T01:32:27.221983+0000","last_undegraded":"2023-11-06T01:32:27.221983+0000","last_fullsized":"2023-11-06T01:32:27.221983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:08:25.866045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.124322+0000","last_change":"2023-11-06T01:32:25.680184+0000","last_active":"2023-11-06T01:32:26.124322+0000","last_peered":"2023-11-06T01:32:26.124322+0000","last_clean":"2023-11-06T01:32:26.124322+0000","last_became_active":"2023-11-06T01:32:25.679859+0000","last_became_peered":"2023-11-06T01:32:25.679859+0000","last_unstale":"2023-11-06T01:32:26.124322+0000","last_undegraded":"2023-11-06T01:32:26.124322+0000","last_fullsized":"2023-11-06T01:32:26.124322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:04:34.153562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.232017+0000","last_change":"2023-11-06T01:32:26.114511+0000","last_active":"2023-11-06T01:32:27.232017+0000","last_peered":"2023-11-06T01:32:27.232017+0000","last_clean":"2023-11-06T01:32:27.232017+0000","last_became_active":"2023-11-06T01:32:26.114182+0000","last_became_peered":"2023-11-06T01:32:26.114182+0000","last_unstale":"2023-11-06T01:32:27.232017+0000","last_undegraded":"2023-11-06T01:32:27.232017+0000","last_fullsized":"2023-11-06T01:32:27.232017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:49:10.694358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.455745+0000","last_change":"2023-11-06T01:32:25.681453+0000","last_active":"2023-11-06T01:32:30.455745+0000","last_peered":"2023-11-06T01:32:30.455745+0000","last_clean":"2023-11-06T01:32:30.455745+0000","last_became_active":"2023-11-06T01:32:25.681180+0000","last_became_peered":"2023-11-06T01:32:25.681180+0000","last_unstale":"2023-11-06T01:32:30.455745+0000","last_undegraded":"2023-11-06T01:32:30.455745+0000","last_fullsized":"2023-11-06T01:32:30.455745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:39:19.738479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222542+0000","last_change":"2023-11-06T01:32:26.118748+0000","last_active":"2023-11-06T01:32:27.222542+0000","last_peered":"2023-11-06T01:32:27.222542+0000","last_clean":"2023-11-06T01:32:27.222542+0000","last_became_active":"2023-11-06T01:32:26.118449+0000","last_became_peered":"2023-11-06T01:32:26.118449+0000","last_unstale":"2023-11-06T01:32:27.222542+0000","last_undegraded":"2023-11-06T01:32:27.222542+0000","last_fullsized":"2023-11-06T01:32:27.222542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:28.134553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.124105+0000","last_change":"2023-11-06T01:32:25.680967+0000","last_active":"2023-11-06T01:32:26.124105+0000","last_peered":"2023-11-06T01:32:26.124105+0000","last_clean":"2023-11-06T01:32:26.124105+0000","last_became_active":"2023-11-06T01:32:25.678829+0000","last_became_peered":"2023-11-06T01:32:25.678829+0000","last_unstale":"2023-11-06T01:32:26.124105+0000","last_undegraded":"2023-11-06T01:32:26.124105+0000","last_fullsized":"2023-11-06T01:32:26.124105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:40:07.691852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121396+0000","last_change":"2023-11-06T01:32:26.122692+0000","last_active":"2023-11-06T01:32:27.121396+0000","last_peered":"2023-11-06T01:32:27.121396+0000","last_clean":"2023-11-06T01:32:27.121396+0000","last_became_active":"2023-11-06T01:32:26.121960+0000","last_became_peered":"2023-11-06T01:32:26.121960+0000","last_unstale":"2023-11-06T01:32:27.121396+0000","last_undegraded":"2023-11-06T01:32:27.121396+0000","last_fullsized":"2023-11-06T01:32:27.121396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:48.010467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121241+0000","last_change":"2023-11-06T01:32:25.680404+0000","last_active":"2023-11-06T01:32:27.121241+0000","last_peered":"2023-11-06T01:32:27.121241+0000","last_clean":"2023-11-06T01:32:27.121241+0000","last_became_active":"2023-11-06T01:32:25.680060+0000","last_became_peered":"2023-11-06T01:32:25.680060+0000","last_unstale":"2023-11-06T01:32:27.121241+0000","last_undegraded":"2023-11-06T01:32:27.121241+0000","last_fullsized":"2023-11-06T01:32:27.121241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:55:00.476395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.119288+0000","last_change":"2023-11-06T01:32:26.116839+0000","last_active":"2023-11-06T01:32:26.119288+0000","last_peered":"2023-11-06T01:32:26.119288+0000","last_clean":"2023-11-06T01:32:26.119288+0000","last_became_active":"2023-11-06T01:32:26.116535+0000","last_became_peered":"2023-11-06T01:32:26.116535+0000","last_unstale":"2023-11-06T01:32:26.119288+0000","last_undegraded":"2023-11-06T01:32:26.119288+0000","last_fullsized":"2023-11-06T01:32:26.119288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:11.219626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.117519+0000","last_change":"2023-11-06T01:32:25.678426+0000","last_active":"2023-11-06T01:32:26.117519+0000","last_peered":"2023-11-06T01:32:26.117519+0000","last_clean":"2023-11-06T01:32:26.117519+0000","last_became_active":"2023-11-06T01:32:25.678072+0000","last_became_peered":"2023-11-06T01:32:25.678072+0000","last_unstale":"2023-11-06T01:32:26.117519+0000","last_undegraded":"2023-11-06T01:32:26.117519+0000","last_fullsized":"2023-11-06T01:32:26.117519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:09:41.630098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121277+0000","last_change":"2023-11-06T01:32:26.119184+0000","last_active":"2023-11-06T01:32:27.121277+0000","last_peered":"2023-11-06T01:32:27.121277+0000","last_clean":"2023-11-06T01:32:27.121277+0000","last_became_active":"2023-11-06T01:32:26.118896+0000","last_became_peered":"2023-11-06T01:32:26.118896+0000","last_unstale":"2023-11-06T01:32:27.121277+0000","last_undegraded":"2023-11-06T01:32:27.121277+0000","last_fullsized":"2023-11-06T01:32:27.121277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:09:17.351315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097703+0000","last_change":"2023-11-06T01:32:25.685621+0000","last_active":"2023-11-06T01:32:27.097703+0000","last_peered":"2023-11-06T01:32:27.097703+0000","last_clean":"2023-11-06T01:32:27.097703+0000","last_became_active":"2023-11-06T01:32:25.684965+0000","last_became_peered":"2023-11-06T01:32:25.684965+0000","last_unstale":"2023-11-06T01:32:27.097703+0000","last_undegraded":"2023-11-06T01:32:27.097703+0000","last_fullsized":"2023-11-06T01:32:27.097703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:28:00.922680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221174+0000","last_change":"2023-11-06T01:32:26.122750+0000","last_active":"2023-11-06T01:32:27.221174+0000","last_peered":"2023-11-06T01:32:27.221174+0000","last_clean":"2023-11-06T01:32:27.221174+0000","last_became_active":"2023-11-06T01:32:26.122241+0000","last_became_peered":"2023-11-06T01:32:26.122241+0000","last_unstale":"2023-11-06T01:32:27.221174+0000","last_undegraded":"2023-11-06T01:32:27.221174+0000","last_fullsized":"2023-11-06T01:32:27.221174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:04.402197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121070+0000","last_change":"2023-11-06T01:32:25.681594+0000","last_active":"2023-11-06T01:32:27.121070+0000","last_peered":"2023-11-06T01:32:27.121070+0000","last_clean":"2023-11-06T01:32:27.121070+0000","last_became_active":"2023-11-06T01:32:25.681303+0000","last_became_peered":"2023-11-06T01:32:25.681303+0000","last_unstale":"2023-11-06T01:32:27.121070+0000","last_undegraded":"2023-11-06T01:32:27.121070+0000","last_fullsized":"2023-11-06T01:32:27.121070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:33.141440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221062+0000","last_change":"2023-11-06T01:32:26.117234+0000","last_active":"2023-11-06T01:32:27.221062+0000","last_peered":"2023-11-06T01:32:27.221062+0000","last_clean":"2023-11-06T01:32:27.221062+0000","last_became_active":"2023-11-06T01:32:26.116915+0000","last_became_peered":"2023-11-06T01:32:26.116915+0000","last_unstale":"2023-11-06T01:32:27.221062+0000","last_undegraded":"2023-11-06T01:32:27.221062+0000","last_fullsized":"2023-11-06T01:32:27.221062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:05:27.521552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.448115+0000","last_change":"2023-11-06T01:32:25.682827+0000","last_active":"2023-11-06T01:32:29.448115+0000","last_peered":"2023-11-06T01:32:29.448115+0000","last_clean":"2023-11-06T01:32:29.448115+0000","last_became_active":"2023-11-06T01:32:25.682470+0000","last_became_peered":"2023-11-06T01:32:25.682470+0000","last_unstale":"2023-11-06T01:32:29.448115+0000","last_undegraded":"2023-11-06T01:32:29.448115+0000","last_fullsized":"2023-11-06T01:32:29.448115+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:43.632070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222801+0000","last_change":"2023-11-06T01:32:26.123918+0000","last_active":"2023-11-06T01:32:27.222801+0000","last_peered":"2023-11-06T01:32:27.222801+0000","last_clean":"2023-11-06T01:32:27.222801+0000","last_became_active":"2023-11-06T01:32:26.123450+0000","last_became_peered":"2023-11-06T01:32:26.123450+0000","last_unstale":"2023-11-06T01:32:27.222801+0000","last_undegraded":"2023-11-06T01:32:27.222801+0000","last_fullsized":"2023-11-06T01:32:27.222801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:38.227311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.448018+0000","last_change":"2023-11-06T01:32:25.680051+0000","last_active":"2023-11-06T01:32:29.448018+0000","last_peered":"2023-11-06T01:32:29.448018+0000","last_clean":"2023-11-06T01:32:29.448018+0000","last_became_active":"2023-11-06T01:32:25.679705+0000","last_became_peered":"2023-11-06T01:32:25.679705+0000","last_unstale":"2023-11-06T01:32:29.448018+0000","last_undegraded":"2023-11-06T01:32:29.448018+0000","last_fullsized":"2023-11-06T01:32:29.448018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:18:22.620036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.232007+0000","last_change":"2023-11-06T01:32:26.115101+0000","last_active":"2023-11-06T01:32:27.232007+0000","last_peered":"2023-11-06T01:32:27.232007+0000","last_clean":"2023-11-06T01:32:27.232007+0000","last_became_active":"2023-11-06T01:32:26.114480+0000","last_became_peered":"2023-11-06T01:32:26.114480+0000","last_unstale":"2023-11-06T01:32:27.232007+0000","last_undegraded":"2023-11-06T01:32:27.232007+0000","last_fullsized":"2023-11-06T01:32:27.232007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:17.770085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121423+0000","last_change":"2023-11-06T01:32:25.678689+0000","last_active":"2023-11-06T01:32:27.121423+0000","last_peered":"2023-11-06T01:32:27.121423+0000","last_clean":"2023-11-06T01:32:27.121423+0000","last_became_active":"2023-11-06T01:32:25.678355+0000","last_became_peered":"2023-11-06T01:32:25.678355+0000","last_unstale":"2023-11-06T01:32:27.121423+0000","last_undegraded":"2023-11-06T01:32:27.121423+0000","last_fullsized":"2023-11-06T01:32:27.121423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:23.007596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222655+0000","last_change":"2023-11-06T01:32:26.118120+0000","last_active":"2023-11-06T01:32:27.222655+0000","last_peered":"2023-11-06T01:32:27.222655+0000","last_clean":"2023-11-06T01:32:27.222655+0000","last_became_active":"2023-11-06T01:32:26.117762+0000","last_became_peered":"2023-11-06T01:32:26.117762+0000","last_unstale":"2023-11-06T01:32:27.222655+0000","last_undegraded":"2023-11-06T01:32:27.222655+0000","last_fullsized":"2023-11-06T01:32:27.222655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:28:42.417463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.445783+0000","last_change":"2023-11-06T01:32:25.676036+0000","last_active":"2023-11-06T01:32:29.445783+0000","last_peered":"2023-11-06T01:32:29.445783+0000","last_clean":"2023-11-06T01:32:29.445783+0000","last_became_active":"2023-11-06T01:32:25.675672+0000","last_became_peered":"2023-11-06T01:32:25.675672+0000","last_unstale":"2023-11-06T01:32:29.445783+0000","last_undegraded":"2023-11-06T01:32:29.445783+0000","last_fullsized":"2023-11-06T01:32:29.445783+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:30:54.060926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.120038+0000","last_change":"2023-11-06T01:32:26.118832+0000","last_active":"2023-11-06T01:32:26.120038+0000","last_peered":"2023-11-06T01:32:26.120038+0000","last_clean":"2023-11-06T01:32:26.120038+0000","last_became_active":"2023-11-06T01:32:26.118551+0000","last_became_peered":"2023-11-06T01:32:26.118551+0000","last_unstale":"2023-11-06T01:32:26.120038+0000","last_undegraded":"2023-11-06T01:32:26.120038+0000","last_fullsized":"2023-11-06T01:32:26.120038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:23.031203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.445689+0000","last_change":"2023-11-06T01:32:25.679433+0000","last_active":"2023-11-06T01:32:29.445689+0000","last_peered":"2023-11-06T01:32:29.445689+0000","last_clean":"2023-11-06T01:32:29.445689+0000","last_became_active":"2023-11-06T01:32:25.679090+0000","last_became_peered":"2023-11-06T01:32:25.679090+0000","last_unstale":"2023-11-06T01:32:29.445689+0000","last_undegraded":"2023-11-06T01:32:29.445689+0000","last_fullsized":"2023-11-06T01:32:29.445689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:14:57.795474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097850+0000","last_change":"2023-11-06T01:32:26.113421+0000","last_active":"2023-11-06T01:32:27.097850+0000","last_peered":"2023-11-06T01:32:27.097850+0000","last_clean":"2023-11-06T01:32:27.097850+0000","last_became_active":"2023-11-06T01:32:26.113085+0000","last_became_peered":"2023-11-06T01:32:26.113085+0000","last_unstale":"2023-11-06T01:32:27.097850+0000","last_undegraded":"2023-11-06T01:32:27.097850+0000","last_fullsized":"2023-11-06T01:32:27.097850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:15:04.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121199+0000","last_change":"2023-11-06T01:32:25.680293+0000","last_active":"2023-11-06T01:32:27.121199+0000","last_peered":"2023-11-06T01:32:27.121199+0000","last_clean":"2023-11-06T01:32:27.121199+0000","last_became_active":"2023-11-06T01:32:25.679803+0000","last_became_peered":"2023-11-06T01:32:25.679803+0000","last_unstale":"2023-11-06T01:32:27.121199+0000","last_undegraded":"2023-11-06T01:32:27.121199+0000","last_fullsized":"2023-11-06T01:32:27.121199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:28.562797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221634+0000","last_change":"2023-11-06T01:32:26.115996+0000","last_active":"2023-11-06T01:32:27.221634+0000","last_peered":"2023-11-06T01:32:27.221634+0000","last_clean":"2023-11-06T01:32:27.221634+0000","last_became_active":"2023-11-06T01:32:26.115407+0000","last_became_peered":"2023-11-06T01:32:26.115407+0000","last_unstale":"2023-11-06T01:32:27.221634+0000","last_undegraded":"2023-11-06T01:32:27.221634+0000","last_fullsized":"2023-11-06T01:32:27.221634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:08.069698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.004611+0000","last_change":"2023-11-06T01:32:25.680958+0000","last_active":"2023-11-06T01:32:31.004611+0000","last_peered":"2023-11-06T01:32:31.004611+0000","last_clean":"2023-11-06T01:32:31.004611+0000","last_became_active":"2023-11-06T01:32:25.680616+0000","last_became_peered":"2023-11-06T01:32:25.680616+0000","last_unstale":"2023-11-06T01:32:31.004611+0000","last_undegraded":"2023-11-06T01:32:31.004611+0000","last_fullsized":"2023-11-06T01:32:31.004611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:02.631755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221186+0000","last_change":"2023-11-06T01:32:26.115464+0000","last_active":"2023-11-06T01:32:27.221186+0000","last_peered":"2023-11-06T01:32:27.221186+0000","last_clean":"2023-11-06T01:32:27.221186+0000","last_became_active":"2023-11-06T01:32:26.115090+0000","last_became_peered":"2023-11-06T01:32:26.115090+0000","last_unstale":"2023-11-06T01:32:27.221186+0000","last_undegraded":"2023-11-06T01:32:27.221186+0000","last_fullsized":"2023-11-06T01:32:27.221186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:08.018044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.458054+0000","last_change":"2023-11-06T01:32:25.677787+0000","last_active":"2023-11-06T01:32:30.458054+0000","last_peered":"2023-11-06T01:32:30.458054+0000","last_clean":"2023-11-06T01:32:30.458054+0000","last_became_active":"2023-11-06T01:32:25.677527+0000","last_became_peered":"2023-11-06T01:32:25.677527+0000","last_unstale":"2023-11-06T01:32:30.458054+0000","last_undegraded":"2023-11-06T01:32:30.458054+0000","last_fullsized":"2023-11-06T01:32:30.458054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:34:09.452965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121118+0000","last_change":"2023-11-06T01:32:26.117792+0000","last_active":"2023-11-06T01:32:27.121118+0000","last_peered":"2023-11-06T01:32:27.121118+0000","last_clean":"2023-11-06T01:32:27.121118+0000","last_became_active":"2023-11-06T01:32:26.117469+0000","last_became_peered":"2023-11-06T01:32:26.117469+0000","last_unstale":"2023-11-06T01:32:27.121118+0000","last_undegraded":"2023-11-06T01:32:27.121118+0000","last_fullsized":"2023-11-06T01:32:27.121118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:44:54.409476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.121576+0000","last_change":"2023-11-06T01:32:25.677257+0000","last_active":"2023-11-06T01:32:26.121576+0000","last_peered":"2023-11-06T01:32:26.121576+0000","last_clean":"2023-11-06T01:32:26.121576+0000","last_became_active":"2023-11-06T01:32:25.676536+0000","last_became_peered":"2023-11-06T01:32:25.676536+0000","last_unstale":"2023-11-06T01:32:26.121576+0000","last_undegraded":"2023-11-06T01:32:26.121576+0000","last_fullsized":"2023-11-06T01:32:26.121576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:53:33.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408512+0000","last_change":"2023-11-06T01:32:26.111447+0000","last_active":"2023-11-06T01:32:26.408512+0000","last_peered":"2023-11-06T01:32:26.408512+0000","last_clean":"2023-11-06T01:32:26.408512+0000","last_became_active":"2023-11-06T01:32:26.111132+0000","last_became_peered":"2023-11-06T01:32:26.111132+0000","last_unstale":"2023-11-06T01:32:26.408512+0000","last_undegraded":"2023-11-06T01:32:26.408512+0000","last_fullsized":"2023-11-06T01:32:26.408512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:55.385003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.717374+0000","last_change":"2023-11-06T01:32:25.682562+0000","last_active":"2023-11-06T01:32:27.717374+0000","last_peered":"2023-11-06T01:32:27.717374+0000","last_clean":"2023-11-06T01:32:27.717374+0000","last_became_active":"2023-11-06T01:32:25.682227+0000","last_became_peered":"2023-11-06T01:32:25.682227+0000","last_unstale":"2023-11-06T01:32:27.717374+0000","last_undegraded":"2023-11-06T01:32:27.717374+0000","last_fullsized":"2023-11-06T01:32:27.717374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:40.113429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221900+0000","last_change":"2023-11-06T01:32:26.110589+0000","last_active":"2023-11-06T01:32:27.221900+0000","last_peered":"2023-11-06T01:32:27.221900+0000","last_clean":"2023-11-06T01:32:27.221900+0000","last_became_active":"2023-11-06T01:32:26.107413+0000","last_became_peered":"2023-11-06T01:32:26.107413+0000","last_unstale":"2023-11-06T01:32:27.221900+0000","last_undegraded":"2023-11-06T01:32:27.221900+0000","last_fullsized":"2023-11-06T01:32:27.221900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:39:08.613713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.102215+0000","last_change":"2023-11-06T01:32:25.688176+0000","last_active":"2023-11-06T01:32:26.102215+0000","last_peered":"2023-11-06T01:32:26.102215+0000","last_clean":"2023-11-06T01:32:26.102215+0000","last_became_active":"2023-11-06T01:32:25.687745+0000","last_became_peered":"2023-11-06T01:32:25.687745+0000","last_unstale":"2023-11-06T01:32:26.102215+0000","last_undegraded":"2023-11-06T01:32:26.102215+0000","last_fullsized":"2023-11-06T01:32:26.102215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:21:23.566389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221764+0000","last_change":"2023-11-06T01:32:26.111700+0000","last_active":"2023-11-06T01:32:27.221764+0000","last_peered":"2023-11-06T01:32:27.221764+0000","last_clean":"2023-11-06T01:32:27.221764+0000","last_became_active":"2023-11-06T01:32:26.111380+0000","last_became_peered":"2023-11-06T01:32:26.111380+0000","last_unstale":"2023-11-06T01:32:27.221764+0000","last_undegraded":"2023-11-06T01:32:27.221764+0000","last_fullsized":"2023-11-06T01:32:27.221764+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:56:47.874663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121155+0000","last_change":"2023-11-06T01:32:26.121906+0000","last_active":"2023-11-06T01:32:27.121155+0000","last_peered":"2023-11-06T01:32:27.121155+0000","last_clean":"2023-11-06T01:32:27.121155+0000","last_became_active":"2023-11-06T01:32:26.121292+0000","last_became_peered":"2023-11-06T01:32:26.121292+0000","last_unstale":"2023-11-06T01:32:27.121155+0000","last_undegraded":"2023-11-06T01:32:27.121155+0000","last_fullsized":"2023-11-06T01:32:27.121155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.532098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097619+0000","last_change":"2023-11-06T01:32:25.685814+0000","last_active":"2023-11-06T01:32:27.097619+0000","last_peered":"2023-11-06T01:32:27.097619+0000","last_clean":"2023-11-06T01:32:27.097619+0000","last_became_active":"2023-11-06T01:32:25.684890+0000","last_became_peered":"2023-11-06T01:32:25.684890+0000","last_unstale":"2023-11-06T01:32:27.097619+0000","last_undegraded":"2023-11-06T01:32:27.097619+0000","last_fullsized":"2023-11-06T01:32:27.097619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:21.950945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222017+0000","last_change":"2023-11-06T01:32:26.121752+0000","last_active":"2023-11-06T01:32:27.222017+0000","last_peered":"2023-11-06T01:32:27.222017+0000","last_clean":"2023-11-06T01:32:27.222017+0000","last_became_active":"2023-11-06T01:32:26.121447+0000","last_became_peered":"2023-11-06T01:32:26.121447+0000","last_unstale":"2023-11-06T01:32:27.222017+0000","last_undegraded":"2023-11-06T01:32:27.222017+0000","last_fullsized":"2023-11-06T01:32:27.222017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:51:19.727335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.004622+0000","last_change":"2023-11-06T01:32:25.682727+0000","last_active":"2023-11-06T01:32:31.004622+0000","last_peered":"2023-11-06T01:32:31.004622+0000","last_clean":"2023-11-06T01:32:31.004622+0000","last_became_active":"2023-11-06T01:32:25.682377+0000","last_became_peered":"2023-11-06T01:32:25.682377+0000","last_unstale":"2023-11-06T01:32:31.004622+0000","last_undegraded":"2023-11-06T01:32:31.004622+0000","last_fullsized":"2023-11-06T01:32:31.004622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:17:38.614350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":325,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.100201+0000","last_change":"2023-11-06T01:31:27.931076+0000","last_active":"2023-11-06T01:32:26.100201+0000","last_peered":"2023-11-06T01:32:26.100201+0000","last_clean":"2023-11-06T01:32:26.100201+0000","last_became_active":"2023-11-06T01:31:27.930669+0000","last_became_peered":"2023-11-06T01:31:27.930669+0000","last_unstale":"2023-11-06T01:32:26.100201+0000","last_undegraded":"2023-11-06T01:32:26.100201+0000","last_fullsized":"2023-11-06T01:32:26.100201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_clean_scrub_stamp":"2023-11-06T01:28:36.252549+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:47:48.729325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097629+0000","last_change":"2023-11-06T01:32:26.110055+0000","last_active":"2023-11-06T01:32:27.097629+0000","last_peered":"2023-11-06T01:32:27.097629+0000","last_clean":"2023-11-06T01:32:27.097629+0000","last_became_active":"2023-11-06T01:32:26.109640+0000","last_became_peered":"2023-11-06T01:32:26.109640+0000","last_unstale":"2023-11-06T01:32:27.097629+0000","last_undegraded":"2023-11-06T01:32:27.097629+0000","last_fullsized":"2023-11-06T01:32:27.097629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:45:43.076910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.220977+0000","last_change":"2023-11-06T01:32:25.679885+0000","last_active":"2023-11-06T01:32:27.220977+0000","last_peered":"2023-11-06T01:32:27.220977+0000","last_clean":"2023-11-06T01:32:27.220977+0000","last_became_active":"2023-11-06T01:32:25.679518+0000","last_became_peered":"2023-11-06T01:32:25.679518+0000","last_unstale":"2023-11-06T01:32:27.220977+0000","last_undegraded":"2023-11-06T01:32:27.220977+0000","last_fullsized":"2023-11-06T01:32:27.220977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:44.558534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221410+0000","last_change":"2023-11-06T01:32:25.682016+0000","last_active":"2023-11-06T01:32:27.221410+0000","last_peered":"2023-11-06T01:32:27.221410+0000","last_clean":"2023-11-06T01:32:27.221410+0000","last_became_active":"2023-11-06T01:32:25.681753+0000","last_became_peered":"2023-11-06T01:32:25.681753+0000","last_unstale":"2023-11-06T01:32:27.221410+0000","last_undegraded":"2023-11-06T01:32:27.221410+0000","last_fullsized":"2023-11-06T01:32:27.221410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:08.575728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097816+0000","last_change":"2023-11-06T01:32:26.117109+0000","last_active":"2023-11-06T01:32:27.097816+0000","last_peered":"2023-11-06T01:32:27.097816+0000","last_clean":"2023-11-06T01:32:27.097816+0000","last_became_active":"2023-11-06T01:32:26.116769+0000","last_became_peered":"2023-11-06T01:32:26.116769+0000","last_unstale":"2023-11-06T01:32:27.097816+0000","last_undegraded":"2023-11-06T01:32:27.097816+0000","last_fullsized":"2023-11-06T01:32:27.097816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:48:04.188806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.006423+0000","last_change":"2023-11-06T01:32:25.681013+0000","last_active":"2023-11-06T01:32:31.006423+0000","last_peered":"2023-11-06T01:32:31.006423+0000","last_clean":"2023-11-06T01:32:31.006423+0000","last_became_active":"2023-11-06T01:32:25.680699+0000","last_became_peered":"2023-11-06T01:32:25.680699+0000","last_unstale":"2023-11-06T01:32:31.006423+0000","last_undegraded":"2023-11-06T01:32:31.006423+0000","last_fullsized":"2023-11-06T01:32:31.006423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:31.755527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221905+0000","last_change":"2023-11-06T01:32:26.116351+0000","last_active":"2023-11-06T01:32:27.221905+0000","last_peered":"2023-11-06T01:32:27.221905+0000","last_clean":"2023-11-06T01:32:27.221905+0000","last_became_active":"2023-11-06T01:32:26.116044+0000","last_became_peered":"2023-11-06T01:32:26.116044+0000","last_unstale":"2023-11-06T01:32:27.221905+0000","last_undegraded":"2023-11-06T01:32:27.221905+0000","last_fullsized":"2023-11-06T01:32:27.221905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:20.816412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.016620+0000","last_change":"2023-11-06T01:32:25.676764+0000","last_active":"2023-11-06T01:32:32.016620+0000","last_peered":"2023-11-06T01:32:32.016620+0000","last_clean":"2023-11-06T01:32:32.016620+0000","last_became_active":"2023-11-06T01:32:25.676418+0000","last_became_peered":"2023-11-06T01:32:25.676418+0000","last_unstale":"2023-11-06T01:32:32.016620+0000","last_undegraded":"2023-11-06T01:32:32.016620+0000","last_fullsized":"2023-11-06T01:32:32.016620+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:49.320532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231997+0000","last_change":"2023-11-06T01:32:26.121107+0000","last_active":"2023-11-06T01:32:27.231997+0000","last_peered":"2023-11-06T01:32:27.231997+0000","last_clean":"2023-11-06T01:32:27.231997+0000","last_became_active":"2023-11-06T01:32:26.120790+0000","last_became_peered":"2023-11-06T01:32:26.120790+0000","last_unstale":"2023-11-06T01:32:27.231997+0000","last_undegraded":"2023-11-06T01:32:27.231997+0000","last_fullsized":"2023-11-06T01:32:27.231997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:34:46.660301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222419+0000","last_change":"2023-11-06T01:32:25.678959+0000","last_active":"2023-11-06T01:32:27.222419+0000","last_peered":"2023-11-06T01:32:27.222419+0000","last_clean":"2023-11-06T01:32:27.222419+0000","last_became_active":"2023-11-06T01:32:25.678564+0000","last_became_peered":"2023-11-06T01:32:25.678564+0000","last_unstale":"2023-11-06T01:32:27.222419+0000","last_undegraded":"2023-11-06T01:32:27.222419+0000","last_fullsized":"2023-11-06T01:32:27.222419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:30:06.304889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.114597+0000","last_change":"2023-11-06T01:32:26.113222+0000","last_active":"2023-11-06T01:32:26.114597+0000","last_peered":"2023-11-06T01:32:26.114597+0000","last_clean":"2023-11-06T01:32:26.114597+0000","last_became_active":"2023-11-06T01:32:26.112926+0000","last_became_peered":"2023-11-06T01:32:26.112926+0000","last_unstale":"2023-11-06T01:32:26.114597+0000","last_undegraded":"2023-11-06T01:32:26.114597+0000","last_fullsized":"2023-11-06T01:32:26.114597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:09:51.042469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.445582+0000","last_change":"2023-11-06T01:32:25.684985+0000","last_active":"2023-11-06T01:32:29.445582+0000","last_peered":"2023-11-06T01:32:29.445582+0000","last_clean":"2023-11-06T01:32:29.445582+0000","last_became_active":"2023-11-06T01:32:25.684656+0000","last_became_peered":"2023-11-06T01:32:25.684656+0000","last_unstale":"2023-11-06T01:32:29.445582+0000","last_undegraded":"2023-11-06T01:32:29.445582+0000","last_fullsized":"2023-11-06T01:32:29.445582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:42:29.406803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408460+0000","last_change":"2023-11-06T01:32:26.107484+0000","last_active":"2023-11-06T01:32:26.408460+0000","last_peered":"2023-11-06T01:32:26.408460+0000","last_clean":"2023-11-06T01:32:26.408460+0000","last_became_active":"2023-11-06T01:32:26.106092+0000","last_became_peered":"2023-11-06T01:32:26.106092+0000","last_unstale":"2023-11-06T01:32:26.408460+0000","last_undegraded":"2023-11-06T01:32:26.408460+0000","last_fullsized":"2023-11-06T01:32:26.408460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:43:39.481290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.101920+0000","last_change":"2023-11-06T01:32:25.679531+0000","last_active":"2023-11-06T01:32:26.101920+0000","last_peered":"2023-11-06T01:32:26.101920+0000","last_clean":"2023-11-06T01:32:26.101920+0000","last_became_active":"2023-11-06T01:32:25.679076+0000","last_became_peered":"2023-11-06T01:32:25.679076+0000","last_unstale":"2023-11-06T01:32:26.101920+0000","last_undegraded":"2023-11-06T01:32:26.101920+0000","last_fullsized":"2023-11-06T01:32:26.101920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:37.381547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222136+0000","last_change":"2023-11-06T01:32:26.118584+0000","last_active":"2023-11-06T01:32:27.222136+0000","last_peered":"2023-11-06T01:32:27.222136+0000","last_clean":"2023-11-06T01:32:27.222136+0000","last_became_active":"2023-11-06T01:32:26.118285+0000","last_became_peered":"2023-11-06T01:32:26.118285+0000","last_unstale":"2023-11-06T01:32:27.222136+0000","last_undegraded":"2023-11-06T01:32:27.222136+0000","last_fullsized":"2023-11-06T01:32:27.222136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:52:33.057377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.447677+0000","last_change":"2023-11-06T01:32:25.681214+0000","last_active":"2023-11-06T01:32:29.447677+0000","last_peered":"2023-11-06T01:32:29.447677+0000","last_clean":"2023-11-06T01:32:29.447677+0000","last_became_active":"2023-11-06T01:32:25.680907+0000","last_became_peered":"2023-11-06T01:32:25.680907+0000","last_unstale":"2023-11-06T01:32:29.447677+0000","last_undegraded":"2023-11-06T01:32:29.447677+0000","last_fullsized":"2023-11-06T01:32:29.447677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:04:58.248787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221859+0000","last_change":"2023-11-06T01:32:26.120862+0000","last_active":"2023-11-06T01:32:27.221859+0000","last_peered":"2023-11-06T01:32:27.221859+0000","last_clean":"2023-11-06T01:32:27.221859+0000","last_became_active":"2023-11-06T01:32:26.120307+0000","last_became_peered":"2023-11-06T01:32:26.120307+0000","last_unstale":"2023-11-06T01:32:27.221859+0000","last_undegraded":"2023-11-06T01:32:27.221859+0000","last_fullsized":"2023-11-06T01:32:27.221859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:45:35.611541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.102314+0000","last_change":"2023-11-06T01:32:25.675607+0000","last_active":"2023-11-06T01:32:26.102314+0000","last_peered":"2023-11-06T01:32:26.102314+0000","last_clean":"2023-11-06T01:32:26.102314+0000","last_became_active":"2023-11-06T01:32:25.675293+0000","last_became_peered":"2023-11-06T01:32:25.675293+0000","last_unstale":"2023-11-06T01:32:26.102314+0000","last_undegraded":"2023-11-06T01:32:26.102314+0000","last_fullsized":"2023-11-06T01:32:26.102314+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:05.190844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221522+0000","last_change":"2023-11-06T01:32:26.116987+0000","last_active":"2023-11-06T01:32:27.221522+0000","last_peered":"2023-11-06T01:32:27.221522+0000","last_clean":"2023-11-06T01:32:27.221522+0000","last_became_active":"2023-11-06T01:32:26.116756+0000","last_became_peered":"2023-11-06T01:32:26.116756+0000","last_unstale":"2023-11-06T01:32:27.221522+0000","last_undegraded":"2023-11-06T01:32:27.221522+0000","last_fullsized":"2023-11-06T01:32:27.221522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:44.039520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121352+0000","last_change":"2023-11-06T01:32:25.675483+0000","last_active":"2023-11-06T01:32:27.121352+0000","last_peered":"2023-11-06T01:32:27.121352+0000","last_clean":"2023-11-06T01:32:27.121352+0000","last_became_active":"2023-11-06T01:32:25.675137+0000","last_became_peered":"2023-11-06T01:32:25.675137+0000","last_unstale":"2023-11-06T01:32:27.121352+0000","last_undegraded":"2023-11-06T01:32:27.121352+0000","last_fullsized":"2023-11-06T01:32:27.121352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:42.415165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221104+0000","last_change":"2023-11-06T01:32:26.115575+0000","last_active":"2023-11-06T01:32:27.221104+0000","last_peered":"2023-11-06T01:32:27.221104+0000","last_clean":"2023-11-06T01:32:27.221104+0000","last_became_active":"2023-11-06T01:32:26.115142+0000","last_became_peered":"2023-11-06T01:32:26.115142+0000","last_unstale":"2023-11-06T01:32:27.221104+0000","last_undegraded":"2023-11-06T01:32:27.221104+0000","last_fullsized":"2023-11-06T01:32:27.221104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:52.859340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097857+0000","last_change":"2023-11-06T01:32:25.676972+0000","last_active":"2023-11-06T01:32:27.097857+0000","last_peered":"2023-11-06T01:32:27.097857+0000","last_clean":"2023-11-06T01:32:27.097857+0000","last_became_active":"2023-11-06T01:32:25.676685+0000","last_became_peered":"2023-11-06T01:32:25.676685+0000","last_unstale":"2023-11-06T01:32:27.097857+0000","last_undegraded":"2023-11-06T01:32:27.097857+0000","last_fullsized":"2023-11-06T01:32:27.097857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:30:21.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221207+0000","last_change":"2023-11-06T01:32:26.115164+0000","last_active":"2023-11-06T01:32:27.221207+0000","last_peered":"2023-11-06T01:32:27.221207+0000","last_clean":"2023-11-06T01:32:27.221207+0000","last_became_active":"2023-11-06T01:32:26.114868+0000","last_became_peered":"2023-11-06T01:32:26.114868+0000","last_unstale":"2023-11-06T01:32:27.221207+0000","last_undegraded":"2023-11-06T01:32:27.221207+0000","last_fullsized":"2023-11-06T01:32:27.221207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:01:34.149871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408478+0000","last_change":"2023-11-06T01:32:26.115600+0000","last_active":"2023-11-06T01:32:26.408478+0000","last_peered":"2023-11-06T01:32:26.408478+0000","last_clean":"2023-11-06T01:32:26.408478+0000","last_became_active":"2023-11-06T01:32:26.115290+0000","last_became_peered":"2023-11-06T01:32:26.115290+0000","last_unstale":"2023-11-06T01:32:26.408478+0000","last_undegraded":"2023-11-06T01:32:26.408478+0000","last_fullsized":"2023-11-06T01:32:26.408478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:04:06.393671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221461+0000","last_change":"2023-11-06T01:32:26.115336+0000","last_active":"2023-11-06T01:32:27.221461+0000","last_peered":"2023-11-06T01:32:27.221461+0000","last_clean":"2023-11-06T01:32:27.221461+0000","last_became_active":"2023-11-06T01:32:26.114946+0000","last_became_peered":"2023-11-06T01:32:26.114946+0000","last_unstale":"2023-11-06T01:32:27.221461+0000","last_undegraded":"2023-11-06T01:32:27.221461+0000","last_fullsized":"2023-11-06T01:32:27.221461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:19:13.651494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221633+0000","last_change":"2023-11-06T01:32:26.117794+0000","last_active":"2023-11-06T01:32:27.221633+0000","last_peered":"2023-11-06T01:32:27.221633+0000","last_clean":"2023-11-06T01:32:27.221633+0000","last_became_active":"2023-11-06T01:32:26.117425+0000","last_became_peered":"2023-11-06T01:32:26.117425+0000","last_unstale":"2023-11-06T01:32:27.221633+0000","last_undegraded":"2023-11-06T01:32:27.221633+0000","last_fullsized":"2023-11-06T01:32:27.221633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:00:24.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231970+0000","last_change":"2023-11-06T01:32:26.113662+0000","last_active":"2023-11-06T01:32:27.231970+0000","last_peered":"2023-11-06T01:32:27.231970+0000","last_clean":"2023-11-06T01:32:27.231970+0000","last_became_active":"2023-11-06T01:32:26.113124+0000","last_became_peered":"2023-11-06T01:32:26.113124+0000","last_unstale":"2023-11-06T01:32:27.231970+0000","last_undegraded":"2023-11-06T01:32:27.231970+0000","last_fullsized":"2023-11-06T01:32:27.231970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:50:24.349521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097929+0000","last_change":"2023-11-06T01:32:26.109910+0000","last_active":"2023-11-06T01:32:27.097929+0000","last_peered":"2023-11-06T01:32:27.097929+0000","last_clean":"2023-11-06T01:32:27.097929+0000","last_became_active":"2023-11-06T01:32:26.109568+0000","last_became_peered":"2023-11-06T01:32:26.109568+0000","last_unstale":"2023-11-06T01:32:27.097929+0000","last_undegraded":"2023-11-06T01:32:27.097929+0000","last_fullsized":"2023-11-06T01:32:27.097929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:11.049749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408501+0000","last_change":"2023-11-06T01:32:26.114824+0000","last_active":"2023-11-06T01:32:26.408501+0000","last_peered":"2023-11-06T01:32:26.408501+0000","last_clean":"2023-11-06T01:32:26.408501+0000","last_became_active":"2023-11-06T01:32:26.114539+0000","last_became_peered":"2023-11-06T01:32:26.114539+0000","last_unstale":"2023-11-06T01:32:26.408501+0000","last_undegraded":"2023-11-06T01:32:26.408501+0000","last_fullsized":"2023-11-06T01:32:26.408501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:03:02.018769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221808+0000","last_change":"2023-11-06T01:32:26.114743+0000","last_active":"2023-11-06T01:32:27.221808+0000","last_peered":"2023-11-06T01:32:27.221808+0000","last_clean":"2023-11-06T01:32:27.221808+0000","last_became_active":"2023-11-06T01:32:26.114432+0000","last_became_peered":"2023-11-06T01:32:26.114432+0000","last_unstale":"2023-11-06T01:32:27.221808+0000","last_undegraded":"2023-11-06T01:32:27.221808+0000","last_fullsized":"2023-11-06T01:32:27.221808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:27:18.209810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221801+0000","last_change":"2023-11-06T01:32:26.125724+0000","last_active":"2023-11-06T01:32:27.221801+0000","last_peered":"2023-11-06T01:32:27.221801+0000","last_clean":"2023-11-06T01:32:27.221801+0000","last_became_active":"2023-11-06T01:32:26.125455+0000","last_became_peered":"2023-11-06T01:32:26.125455+0000","last_unstale":"2023-11-06T01:32:27.221801+0000","last_undegraded":"2023-11-06T01:32:27.221801+0000","last_fullsized":"2023-11-06T01:32:27.221801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:49.099308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121127+0000","last_change":"2023-11-06T01:32:26.119328+0000","last_active":"2023-11-06T01:32:27.121127+0000","last_peered":"2023-11-06T01:32:27.121127+0000","last_clean":"2023-11-06T01:32:27.121127+0000","last_became_active":"2023-11-06T01:32:26.119042+0000","last_became_peered":"2023-11-06T01:32:26.119042+0000","last_unstale":"2023-11-06T01:32:27.121127+0000","last_undegraded":"2023-11-06T01:32:27.121127+0000","last_fullsized":"2023-11-06T01:32:27.121127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:36.143788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221676+0000","last_change":"2023-11-06T01:32:26.126538+0000","last_active":"2023-11-06T01:32:27.221676+0000","last_peered":"2023-11-06T01:32:27.221676+0000","last_clean":"2023-11-06T01:32:27.221676+0000","last_became_active":"2023-11-06T01:32:26.124898+0000","last_became_peered":"2023-11-06T01:32:26.124898+0000","last_unstale":"2023-11-06T01:32:27.221676+0000","last_undegraded":"2023-11-06T01:32:27.221676+0000","last_fullsized":"2023-11-06T01:32:27.221676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:33.019326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222542+0000","last_change":"2023-11-06T01:32:26.118751+0000","last_active":"2023-11-06T01:32:27.222542+0000","last_peered":"2023-11-06T01:32:27.222542+0000","last_clean":"2023-11-06T01:32:27.222542+0000","last_became_active":"2023-11-06T01:32:26.118359+0000","last_became_peered":"2023-11-06T01:32:26.118359+0000","last_unstale":"2023-11-06T01:32:27.222542+0000","last_undegraded":"2023-11-06T01:32:27.222542+0000","last_fullsized":"2023-11-06T01:32:27.222542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:55.960400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.098195+0000","last_change":"2023-11-06T01:32:26.110133+0000","last_active":"2023-11-06T01:32:27.098195+0000","last_peered":"2023-11-06T01:32:27.098195+0000","last_clean":"2023-11-06T01:32:27.098195+0000","last_became_active":"2023-11-06T01:32:26.109793+0000","last_became_peered":"2023-11-06T01:32:26.109793+0000","last_unstale":"2023-11-06T01:32:27.098195+0000","last_undegraded":"2023-11-06T01:32:27.098195+0000","last_fullsized":"2023-11-06T01:32:27.098195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:00:03.956947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408491+0000","last_change":"2023-11-06T01:32:26.109828+0000","last_active":"2023-11-06T01:32:26.408491+0000","last_peered":"2023-11-06T01:32:26.408491+0000","last_clean":"2023-11-06T01:32:26.408491+0000","last_became_active":"2023-11-06T01:32:26.109526+0000","last_became_peered":"2023-11-06T01:32:26.109526+0000","last_unstale":"2023-11-06T01:32:26.408491+0000","last_undegraded":"2023-11-06T01:32:26.408491+0000","last_fullsized":"2023-11-06T01:32:26.408491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:36.561908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250075,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3396,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":115,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":13574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481804,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":418689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":10,"up_from":70,"seq":300647710736,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":1010664,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.70699999999999996}]},{"osd":2,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.745}]},{"osd":3,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.66000000000000003}]},{"osd":4,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.68100000000000005}]},{"osd":5,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.89600000000000002}]},{"osd":6,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.84999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]}]},{"osd":9,"up_from":65,"seq":279172874261,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":419229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46899999999999997}]},{"osd":1,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.44500000000000001}]},{"osd":2,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.58099999999999996}]},{"osd":3,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.48899999999999999}]},{"osd":4,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.52000000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.68300000000000005}]},{"osd":8,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.73199999999999998}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.728}]}]},{"osd":8,"up_from":58,"seq":249108103192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":418514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.78000000000000003}]},{"osd":1,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57999999999999996}]},{"osd":2,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.47499999999999998}]},{"osd":3,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.77000000000000002}]},{"osd":4,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.68899999999999995}]},{"osd":5,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.60199999999999998}]},{"osd":6,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1,"5min":1,"15min":1},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.84999999999999998}]},{"osd":7,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.755},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.73699999999999999}]},{"osd":9,"last update":"Mon Nov 6 01:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.93700000000000006}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.91900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":7,"up_from":51,"seq":219043332123,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":1009057,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.873,"5min":1.157,"15min":1.157},"last":0.376}]},{"osd":1,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.32900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.45600000000000002,"15min":0.44800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.33700000000000002}]},{"osd":2,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57899999999999996,"15min":0.56799999999999995},"min":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.45400000000000001}]},{"osd":3,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.503},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85899999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.51300000000000001,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.379}]},{"osd":5,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.53200000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53900000000000003}]},{"osd":6,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97099999999999997,"5min":1.232,"15min":1.232},"last":0.58599999999999997}]},{"osd":8,"last update":"Mon Nov 6 01:31:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55200000000000005}]},{"osd":9,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61799999999999999}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.628}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495648,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":418107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.443},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42099999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81100000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.38},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.435},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66800000000000004}]},{"osd":3,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.46200000000000002}]},{"osd":4,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69699999999999995,"15min":0.69399999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.72499999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.71499999999999997,"15min":0.69999999999999996},"min":{"1min":0.55900000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52800000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.5}]},{"osd":8,"last update":"Mon Nov 6 01:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.55700000000000005}]},{"osd":9,"last update":"Mon Nov 6 01:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.71899999999999997}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.62},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.69199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":419671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.51300000000000001,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.433}]},{"osd":2,"last update":"Mon Nov 6 01:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85899999999999999,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.35499999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.65600000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.60499999999999998}]},{"osd":3,"last update":"Mon Nov 6 01:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.54700000000000004,"15min":0.52100000000000002},"min":{"1min":0.44900000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91800000000000004,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.47799999999999998,"15min":0.46100000000000002},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.53500000000000003,"15min":0.51000000000000001},"min":{"1min":0.36899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.47099999999999997}]},{"osd":5,"last update":"Mon Nov 6 01:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.754,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Nov 6 01:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.495,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.377}]},{"osd":7,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.60199999999999998,"15min":0.56799999999999995},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.61399999999999999,"15min":0.57199999999999995},"min":{"1min":0.50700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.50700000000000001}]},{"osd":8,"last update":"Mon Nov 6 01:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.67400000000000004}]},{"osd":9,"last update":"Mon Nov 6 01:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.54000000000000004}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.495},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.65700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":420125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50700000000000001},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.504},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.61299999999999999}]},{"osd":2,"last update":"Mon Nov 6 01:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57399999999999995,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.39600000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44,"15min":0.437},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.36799999999999999}]},{"osd":4,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.41499999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Nov 6 01:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.60399999999999998},"min":{"1min":0.48499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.502},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.52700000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.439}]},{"osd":6,"last update":"Mon Nov 6 01:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.58299999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.54300000000000004}]},{"osd":7,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.52800000000000002}]},{"osd":9,"last update":"Mon Nov 6 01:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.67000000000000004}]},{"osd":10,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.624},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.64900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":419705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.495,"15min":0.48599999999999999},"min":{"1min":0.36799999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.33500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.44700000000000001}]},{"osd":1,"last update":"Mon Nov 6 01:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.41999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.434}]},{"osd":3,"last update":"Mon Nov 6 01:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58699999999999997,"15min":0.57099999999999995},"min":{"1min":0.46999999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Mon Nov 6 01:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55500000000000005}]},{"osd":5,"last update":"Mon Nov 6 01:31:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.433},"min":{"1min":0.35899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.60299999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.375},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68600000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.503}]},{"osd":6,"last update":"Mon Nov 6 01:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56399999999999995}]},{"osd":7,"last update":"Mon Nov 6 01:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.54700000000000004}]},{"osd":8,"last update":"Mon Nov 6 01:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.53500000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.65000000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.61799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]},{"osd":3,"up_from":26,"seq":111669149741,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":418301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.371,"15min":0.36099999999999999},"min":{"1min":0.27300000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.377,"15min":0.36099999999999999},"min":{"1min":0.32200000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.47199999999999998}]},{"osd":1,"last update":"Mon Nov 6 01:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.86299999999999999,"5min":1.119,"15min":1.119},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.52000000000000002,"15min":0.48799999999999999},"min":{"1min":0.45200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57799999999999996}]},{"osd":2,"last update":"Mon Nov 6 01:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.47199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.46600000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.91700000000000004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Mon Nov 6 01:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.502,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.98799999999999999,"5min":1.159,"15min":1.159},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Nov 6 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56200000000000006}]},{"osd":6,"last update":"Mon Nov 6 01:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.34499999999999997,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.56499999999999995,"15min":0.54100000000000004},"min":{"1min":0.41199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.63}]},{"osd":7,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.66500000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.67600000000000005,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73699999999999999}]},{"osd":8,"last update":"Mon Nov 6 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.622},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.0430000000000001,"5min":6.0430000000000001,"15min":6.0430000000000001},"last":0.76500000000000001}]},{"osd":9,"last update":"Mon Nov 6 01:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.64400000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.749},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888105,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":418759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.45300000000000001},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.49399999999999999,"15min":0.47799999999999998},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.749,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42999999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.377,"15min":0.35799999999999998},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.36599999999999999},"min":{"1min":0.27000000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.503}]},{"osd":2,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.44900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.79700000000000004}]},{"osd":3,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.51400000000000001,"15min":0.49099999999999999},"min":{"1min":0.38900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.41199999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.53700000000000003}]},{"osd":5,"last update":"Mon Nov 6 01:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77100000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85099999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.88}]},{"osd":6,"last update":"Mon Nov 6 01:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78500000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.83599999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68999999999999995}]},{"osd":7,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.745},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.95499999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":1.0469999999999999}]},{"osd":9,"last update":"Mon Nov 6 01:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.78300000000000003}]},{"osd":10,"last update":"Mon Nov 6 01:32:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.86399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724581,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":1008326,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.41499999999999998}]},{"osd":1,"last update":"Mon Nov 6 01:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.34499999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54400000000000004,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Nov 6 01:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39100000000000001,"15min":0.38300000000000001},"min":{"1min":0.308,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.38,"15min":0.37},"min":{"1min":0.31,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.309}]},{"osd":3,"last update":"Mon Nov 6 01:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81499999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.45900000000000002}]},{"osd":4,"last update":"Mon Nov 6 01:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.496}]},{"osd":6,"last update":"Mon Nov 6 01:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.622},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48699999999999999}]},{"osd":7,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53400000000000003}]},{"osd":8,"last update":"Mon Nov 6 01:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54300000000000004}]},{"osd":9,"last update":"Mon Nov 6 01:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.56100000000000005}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T01:32:34.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T01:32:34.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:34 smithi163 ceph-mon[104416]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:34.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:34 smithi163 ceph-mon[104416]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:34.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:34 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] up:active 2023-11-06T01:32:34.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:34 smithi163 ceph-mon[104416]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-11-06T01:32:34.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:34 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:32:34.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:34 smithi163 ceph-mon[104416]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:32:34.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:34 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1420887198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:32:34.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.566+0000 7f3a81d33700 1 -- 172.21.15.53:0/1701991932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7c101800 msgr2=0x7f3a7c1080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.566+0000 7f3a81d33700 1 --2- 172.21.15.53:0/1701991932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7c101800 0x7f3a7c1080b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3a740099c0 tx=0x7f3a74009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.566+0000 7f3a81d33700 1 -- 172.21.15.53:0/1701991932 shutdown_connections 2023-11-06T01:32:34.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.566+0000 7f3a81d33700 1 --2- 172.21.15.53:0/1701991932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c0ffc70 0x7f3a7c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.566+0000 7f3a81d33700 1 --2- 172.21.15.53:0/1701991932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7c101800 0x7f3a7c1080b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.566+0000 7f3a81d33700 1 --2- 172.21.15.53:0/1701991932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c100e60 0x7f3a7c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.566+0000 7f3a81d33700 1 -- 172.21.15.53:0/1701991932 >> 172.21.15.53:0/1701991932 conn(0x7f3a7c0fb1c0 msgr2=0x7f3a7c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:34.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.566+0000 7f3a81d33700 1 -- 172.21.15.53:0/1701991932 shutdown_connections 2023-11-06T01:32:34.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.566+0000 7f3a81d33700 1 -- 172.21.15.53:0/1701991932 wait complete. 2023-11-06T01:32:34.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a81d33700 1 Processor -- start 2023-11-06T01:32:34.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a81d33700 1 -- start start 2023-11-06T01:32:34.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a81d33700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c0ffc70 0x7f3a7c068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:34.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a81d33700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c100e60 0x7f3a7c19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:34.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a81d33700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7c101800 0x7f3a7c19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:34.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a81d33700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c19e500 con 0x7f3a7c101800 2023-11-06T01:32:34.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a81d33700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c19e670 con 0x7f3a7c100e60 2023-11-06T01:32:34.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a81d33700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c19e830 con 0x7f3a7c0ffc70 2023-11-06T01:32:34.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7c101800 0x7f3a7c19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:34.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7c101800 0x7f3a7c19dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38180/0 (socket says 172.21.15.53:38180) 2023-11-06T01:32:34.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7bfff700 1 -- 172.21.15.53:0/755397100 learned_addr learned my addr 172.21.15.53:0/755397100 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:34.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7affd700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c100e60 0x7f3a7c19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:34.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7b7fe700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c0ffc70 0x7f3a7c068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:34.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7bfff700 1 -- 172.21.15.53:0/755397100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c0ffc70 msgr2=0x7f3a7c068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7bfff700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c0ffc70 0x7f3a7c068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7bfff700 1 -- 172.21.15.53:0/755397100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c100e60 msgr2=0x7f3a7c19b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7bfff700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c100e60 0x7f3a7c19b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7bfff700 1 -- 172.21.15.53:0/755397100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a740095f0 con 0x7f3a7c101800 2023-11-06T01:32:34.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7b7fe700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c0ffc70 0x7f3a7c068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:32:34.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.567+0000 7f3a7affd700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c100e60 0x7f3a7c19b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:32:34.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.568+0000 7f3a7bfff700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7c101800 0x7f3a7c19dd60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a740098e0 tx=0x7f3a7400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:34.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.568+0000 7f3a78ff9700 1 -- 172.21.15.53:0/755397100 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7401d070 con 0x7f3a7c101800 2023-11-06T01:32:34.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.568+0000 7f3a78ff9700 1 -- 172.21.15.53:0/755397100 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a740054f0 con 0x7f3a7c101800 2023-11-06T01:32:34.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.568+0000 7f3a81d33700 1 -- 172.21.15.53:0/755397100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c19eab0 con 0x7f3a7c101800 2023-11-06T01:32:34.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.568+0000 7f3a78ff9700 1 -- 172.21.15.53:0/755397100 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74004b40 con 0x7f3a7c101800 2023-11-06T01:32:34.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.568+0000 7f3a81d33700 1 -- 172.21.15.53:0/755397100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c19ef30 con 0x7f3a7c101800 2023-11-06T01:32:34.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.569+0000 7f3a81d33700 1 -- 172.21.15.53:0/755397100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c107190 con 0x7f3a7c101800 2023-11-06T01:32:34.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.570+0000 7f3a78ff9700 1 -- 172.21.15.53:0/755397100 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f3a74004d20 con 0x7f3a7c101800 2023-11-06T01:32:34.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.570+0000 7f3a78ff9700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3a680a0670 0x7f3a680a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:34.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.570+0000 7f3a78ff9700 1 -- 172.21.15.53:0/755397100 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3a740cc710 con 0x7f3a7c101800 2023-11-06T01:32:34.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.570+0000 7f3a7b7fe700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3a680a0670 0x7f3a680a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:34.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.571+0000 7f3a7b7fe700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3a680a0670 0x7f3a680a2b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c100cc0 tx=0x7f3a6c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:34.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.575+0000 7f3a78ff9700 1 -- 172.21.15.53:0/755397100 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3a74096390 con 0x7f3a7c101800 2023-11-06T01:32:34.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.773+0000 7f3a81d33700 1 -- 172.21.15.53:0/755397100 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a7c0611d0 con 0x7f3a680a0670 2023-11-06T01:32:34.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.784+0000 7f3a78ff9700 1 -- 172.21.15.53:0/755397100 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335315 (secure 0 0 0) 0x7f3a7c0611d0 con 0x7f3a680a0670 2023-11-06T01:32:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:34.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 -- 172.21.15.53:0/755397100 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3a680a0670 msgr2=0x7f3a680a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3a680a0670 0x7f3a680a2b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c100cc0 tx=0x7f3a6c00b040 comp rx=0 tx=0).stop 2023-11-06T01:32:34.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 -- 172.21.15.53:0/755397100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7c101800 msgr2=0x7f3a7c19dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:34.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7c101800 0x7f3a7c19dd60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a740098e0 tx=0x7f3a7400f740 comp rx=0 tx=0).stop 2023-11-06T01:32:34.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 -- 172.21.15.53:0/755397100 shutdown_connections 2023-11-06T01:32:34.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3a680a0670 0x7f3a680a2b30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7c101800 0x7f3a7c19dd60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c0ffc70 0x7f3a7c068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 --2- 172.21.15.53:0/755397100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c100e60 0x7f3a7c19b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:34.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 -- 172.21.15.53:0/755397100 >> 172.21.15.53:0/755397100 conn(0x7f3a7c0fb1c0 msgr2=0x7f3a7c1042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:34.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 -- 172.21.15.53:0/755397100 shutdown_connections 2023-11-06T01:32:34.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:34.788+0000 7f3a81d33700 1 -- 172.21.15.53:0/755397100 wait complete. 2023-11-06T01:32:34.794 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-06T01:32:34.801 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2023-11-06T01:32:33.473247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593764,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":135,"num_read_kb":116,"num_write":296,"num_write_kb":2974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":283,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":333984,"kb_used_data":10560,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607840,"statfs":{"total":1151940427776,"available":1151598428160,"internally_reserved":0,"allocated":10813440,"data_stored":6799147,"data_compressed":108579,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3396,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":115,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.814836"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408521+0000","last_change":"2023-11-06T01:32:26.110793+0000","last_active":"2023-11-06T01:32:26.408521+0000","last_peered":"2023-11-06T01:32:26.408521+0000","last_clean":"2023-11-06T01:32:26.408521+0000","last_became_active":"2023-11-06T01:32:26.110489+0000","last_became_peered":"2023-11-06T01:32:26.110489+0000","last_unstale":"2023-11-06T01:32:26.408521+0000","last_undegraded":"2023-11-06T01:32:26.408521+0000","last_fullsized":"2023-11-06T01:32:26.408521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:01:08.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231988+0000","last_change":"2023-11-06T01:32:26.107273+0000","last_active":"2023-11-06T01:32:27.231988+0000","last_peered":"2023-11-06T01:32:27.231988+0000","last_clean":"2023-11-06T01:32:27.231988+0000","last_became_active":"2023-11-06T01:32:26.106961+0000","last_became_peered":"2023-11-06T01:32:26.106961+0000","last_unstale":"2023-11-06T01:32:27.231988+0000","last_undegraded":"2023-11-06T01:32:27.231988+0000","last_fullsized":"2023-11-06T01:32:27.231988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:40:39.073926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231979+0000","last_change":"2023-11-06T01:32:26.121316+0000","last_active":"2023-11-06T01:32:27.231979+0000","last_peered":"2023-11-06T01:32:27.231979+0000","last_clean":"2023-11-06T01:32:27.231979+0000","last_became_active":"2023-11-06T01:32:26.120756+0000","last_became_peered":"2023-11-06T01:32:26.120756+0000","last_unstale":"2023-11-06T01:32:27.231979+0000","last_undegraded":"2023-11-06T01:32:27.231979+0000","last_fullsized":"2023-11-06T01:32:27.231979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:02:45.814577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.100053+0000","last_change":"2023-11-06T01:32:26.121951+0000","last_active":"2023-11-06T01:32:27.100053+0000","last_peered":"2023-11-06T01:32:27.100053+0000","last_clean":"2023-11-06T01:32:27.100053+0000","last_became_active":"2023-11-06T01:32:26.121636+0000","last_became_peered":"2023-11-06T01:32:26.121636+0000","last_unstale":"2023-11-06T01:32:27.100053+0000","last_undegraded":"2023-11-06T01:32:27.100053+0000","last_fullsized":"2023-11-06T01:32:27.100053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:02.381355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.119944+0000","last_change":"2023-11-06T01:32:26.117788+0000","last_active":"2023-11-06T01:32:26.119944+0000","last_peered":"2023-11-06T01:32:26.119944+0000","last_clean":"2023-11-06T01:32:26.119944+0000","last_became_active":"2023-11-06T01:32:26.117161+0000","last_became_peered":"2023-11-06T01:32:26.117161+0000","last_unstale":"2023-11-06T01:32:26.119944+0000","last_undegraded":"2023-11-06T01:32:26.119944+0000","last_fullsized":"2023-11-06T01:32:26.119944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:31.665690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221591+0000","last_change":"2023-11-06T01:32:26.109532+0000","last_active":"2023-11-06T01:32:27.221591+0000","last_peered":"2023-11-06T01:32:27.221591+0000","last_clean":"2023-11-06T01:32:27.221591+0000","last_became_active":"2023-11-06T01:32:26.106144+0000","last_became_peered":"2023-11-06T01:32:26.106144+0000","last_unstale":"2023-11-06T01:32:27.221591+0000","last_undegraded":"2023-11-06T01:32:27.221591+0000","last_fullsized":"2023-11-06T01:32:27.221591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:54.220515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221306+0000","last_change":"2023-11-06T01:32:26.116280+0000","last_active":"2023-11-06T01:32:27.221306+0000","last_peered":"2023-11-06T01:32:27.221306+0000","last_clean":"2023-11-06T01:32:27.221306+0000","last_became_active":"2023-11-06T01:32:26.115957+0000","last_became_peered":"2023-11-06T01:32:26.115957+0000","last_unstale":"2023-11-06T01:32:27.221306+0000","last_undegraded":"2023-11-06T01:32:27.221306+0000","last_fullsized":"2023-11-06T01:32:27.221306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:25.956902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221244+0000","last_change":"2023-11-06T01:32:26.124818+0000","last_active":"2023-11-06T01:32:27.221244+0000","last_peered":"2023-11-06T01:32:27.221244+0000","last_clean":"2023-11-06T01:32:27.221244+0000","last_became_active":"2023-11-06T01:32:26.124511+0000","last_became_peered":"2023-11-06T01:32:26.124511+0000","last_unstale":"2023-11-06T01:32:27.221244+0000","last_undegraded":"2023-11-06T01:32:27.221244+0000","last_fullsized":"2023-11-06T01:32:27.221244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:35.853898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222751+0000","last_change":"2023-11-06T01:32:26.114243+0000","last_active":"2023-11-06T01:32:27.222751+0000","last_peered":"2023-11-06T01:32:27.222751+0000","last_clean":"2023-11-06T01:32:27.222751+0000","last_became_active":"2023-11-06T01:32:26.113923+0000","last_became_peered":"2023-11-06T01:32:26.113923+0000","last_unstale":"2023-11-06T01:32:27.222751+0000","last_undegraded":"2023-11-06T01:32:27.222751+0000","last_fullsized":"2023-11-06T01:32:27.222751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:48.197329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221817+0000","last_change":"2023-11-06T01:32:26.113819+0000","last_active":"2023-11-06T01:32:27.221817+0000","last_peered":"2023-11-06T01:32:27.221817+0000","last_clean":"2023-11-06T01:32:27.221817+0000","last_became_active":"2023-11-06T01:32:26.113506+0000","last_became_peered":"2023-11-06T01:32:26.113506+0000","last_unstale":"2023-11-06T01:32:27.221817+0000","last_undegraded":"2023-11-06T01:32:27.221817+0000","last_fullsized":"2023-11-06T01:32:27.221817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:33.122100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221622+0000","last_change":"2023-11-06T01:32:26.116504+0000","last_active":"2023-11-06T01:32:27.221622+0000","last_peered":"2023-11-06T01:32:27.221622+0000","last_clean":"2023-11-06T01:32:27.221622+0000","last_became_active":"2023-11-06T01:32:26.115220+0000","last_became_peered":"2023-11-06T01:32:26.115220+0000","last_unstale":"2023-11-06T01:32:27.221622+0000","last_undegraded":"2023-11-06T01:32:27.221622+0000","last_fullsized":"2023-11-06T01:32:27.221622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:34.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.100025+0000","last_change":"2023-11-06T01:32:26.111596+0000","last_active":"2023-11-06T01:32:27.100025+0000","last_peered":"2023-11-06T01:32:27.100025+0000","last_clean":"2023-11-06T01:32:27.100025+0000","last_became_active":"2023-11-06T01:32:26.111309+0000","last_became_peered":"2023-11-06T01:32:26.111309+0000","last_unstale":"2023-11-06T01:32:27.100025+0000","last_undegraded":"2023-11-06T01:32:27.100025+0000","last_fullsized":"2023-11-06T01:32:27.100025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:34.917068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.098303+0000","last_change":"2023-11-06T01:32:26.114536+0000","last_active":"2023-11-06T01:32:27.098303+0000","last_peered":"2023-11-06T01:32:27.098303+0000","last_clean":"2023-11-06T01:32:27.098303+0000","last_became_active":"2023-11-06T01:32:26.114124+0000","last_became_peered":"2023-11-06T01:32:26.114124+0000","last_unstale":"2023-11-06T01:32:27.098303+0000","last_undegraded":"2023-11-06T01:32:27.098303+0000","last_fullsized":"2023-11-06T01:32:27.098303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:05:10.855240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231958+0000","last_change":"2023-11-06T01:32:26.111139+0000","last_active":"2023-11-06T01:32:27.231958+0000","last_peered":"2023-11-06T01:32:27.231958+0000","last_clean":"2023-11-06T01:32:27.231958+0000","last_became_active":"2023-11-06T01:32:26.110764+0000","last_became_peered":"2023-11-06T01:32:26.110764+0000","last_unstale":"2023-11-06T01:32:27.231958+0000","last_undegraded":"2023-11-06T01:32:27.231958+0000","last_fullsized":"2023-11-06T01:32:27.231958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:46.422701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231938+0000","last_change":"2023-11-06T01:32:26.116096+0000","last_active":"2023-11-06T01:32:27.231938+0000","last_peered":"2023-11-06T01:32:27.231938+0000","last_clean":"2023-11-06T01:32:27.231938+0000","last_became_active":"2023-11-06T01:32:26.115754+0000","last_became_peered":"2023-11-06T01:32:26.115754+0000","last_unstale":"2023-11-06T01:32:27.231938+0000","last_undegraded":"2023-11-06T01:32:27.231938+0000","last_fullsized":"2023-11-06T01:32:27.231938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:15:58.109435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.220995+0000","last_change":"2023-11-06T01:32:26.118169+0000","last_active":"2023-11-06T01:32:27.220995+0000","last_peered":"2023-11-06T01:32:27.220995+0000","last_clean":"2023-11-06T01:32:27.220995+0000","last_became_active":"2023-11-06T01:32:26.117837+0000","last_became_peered":"2023-11-06T01:32:26.117837+0000","last_unstale":"2023-11-06T01:32:27.220995+0000","last_undegraded":"2023-11-06T01:32:27.220995+0000","last_fullsized":"2023-11-06T01:32:27.220995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:46.838927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408530+0000","last_change":"2023-11-06T01:32:26.110020+0000","last_active":"2023-11-06T01:32:26.408530+0000","last_peered":"2023-11-06T01:32:26.408530+0000","last_clean":"2023-11-06T01:32:26.408530+0000","last_became_active":"2023-11-06T01:32:26.109707+0000","last_became_peered":"2023-11-06T01:32:26.109707+0000","last_unstale":"2023-11-06T01:32:26.408530+0000","last_undegraded":"2023-11-06T01:32:26.408530+0000","last_fullsized":"2023-11-06T01:32:26.408530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:00.016258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222625+0000","last_change":"2023-11-06T01:32:26.119949+0000","last_active":"2023-11-06T01:32:27.222625+0000","last_peered":"2023-11-06T01:32:27.222625+0000","last_clean":"2023-11-06T01:32:27.222625+0000","last_became_active":"2023-11-06T01:32:26.117594+0000","last_became_peered":"2023-11-06T01:32:26.117594+0000","last_unstale":"2023-11-06T01:32:27.222625+0000","last_undegraded":"2023-11-06T01:32:27.222625+0000","last_fullsized":"2023-11-06T01:32:27.222625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:17:22.757846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221583+0000","last_change":"2023-11-06T01:32:26.123176+0000","last_active":"2023-11-06T01:32:27.221583+0000","last_peered":"2023-11-06T01:32:27.221583+0000","last_clean":"2023-11-06T01:32:27.221583+0000","last_became_active":"2023-11-06T01:32:26.116687+0000","last_became_peered":"2023-11-06T01:32:26.116687+0000","last_unstale":"2023-11-06T01:32:27.221583+0000","last_undegraded":"2023-11-06T01:32:27.221583+0000","last_fullsized":"2023-11-06T01:32:27.221583+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:30.422426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.012844+0000","last_change":"2023-11-06T01:32:25.682991+0000","last_active":"2023-11-06T01:32:32.012844+0000","last_peered":"2023-11-06T01:32:32.012844+0000","last_clean":"2023-11-06T01:32:32.012844+0000","last_became_active":"2023-11-06T01:32:25.682687+0000","last_became_peered":"2023-11-06T01:32:25.682687+0000","last_unstale":"2023-11-06T01:32:32.012844+0000","last_undegraded":"2023-11-06T01:32:32.012844+0000","last_fullsized":"2023-11-06T01:32:32.012844+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:12:54.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221983+0000","last_change":"2023-11-06T01:32:26.126350+0000","last_active":"2023-11-06T01:32:27.221983+0000","last_peered":"2023-11-06T01:32:27.221983+0000","last_clean":"2023-11-06T01:32:27.221983+0000","last_became_active":"2023-11-06T01:32:26.124703+0000","last_became_peered":"2023-11-06T01:32:26.124703+0000","last_unstale":"2023-11-06T01:32:27.221983+0000","last_undegraded":"2023-11-06T01:32:27.221983+0000","last_fullsized":"2023-11-06T01:32:27.221983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:08:25.866045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.124322+0000","last_change":"2023-11-06T01:32:25.680184+0000","last_active":"2023-11-06T01:32:26.124322+0000","last_peered":"2023-11-06T01:32:26.124322+0000","last_clean":"2023-11-06T01:32:26.124322+0000","last_became_active":"2023-11-06T01:32:25.679859+0000","last_became_peered":"2023-11-06T01:32:25.679859+0000","last_unstale":"2023-11-06T01:32:26.124322+0000","last_undegraded":"2023-11-06T01:32:26.124322+0000","last_fullsized":"2023-11-06T01:32:26.124322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:04:34.153562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.232017+0000","last_change":"2023-11-06T01:32:26.114511+0000","last_active":"2023-11-06T01:32:27.232017+0000","last_peered":"2023-11-06T01:32:27.232017+0000","last_clean":"2023-11-06T01:32:27.232017+0000","last_became_active":"2023-11-06T01:32:26.114182+0000","last_became_peered":"2023-11-06T01:32:26.114182+0000","last_unstale":"2023-11-06T01:32:27.232017+0000","last_undegraded":"2023-11-06T01:32:27.232017+0000","last_fullsized":"2023-11-06T01:32:27.232017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:49:10.694358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.455745+0000","last_change":"2023-11-06T01:32:25.681453+0000","last_active":"2023-11-06T01:32:30.455745+0000","last_peered":"2023-11-06T01:32:30.455745+0000","last_clean":"2023-11-06T01:32:30.455745+0000","last_became_active":"2023-11-06T01:32:25.681180+0000","last_became_peered":"2023-11-06T01:32:25.681180+0000","last_unstale":"2023-11-06T01:32:30.455745+0000","last_undegraded":"2023-11-06T01:32:30.455745+0000","last_fullsized":"2023-11-06T01:32:30.455745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:39:19.738479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222542+0000","last_change":"2023-11-06T01:32:26.118748+0000","last_active":"2023-11-06T01:32:27.222542+0000","last_peered":"2023-11-06T01:32:27.222542+0000","last_clean":"2023-11-06T01:32:27.222542+0000","last_became_active":"2023-11-06T01:32:26.118449+0000","last_became_peered":"2023-11-06T01:32:26.118449+0000","last_unstale":"2023-11-06T01:32:27.222542+0000","last_undegraded":"2023-11-06T01:32:27.222542+0000","last_fullsized":"2023-11-06T01:32:27.222542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:28.134553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.124105+0000","last_change":"2023-11-06T01:32:25.680967+0000","last_active":"2023-11-06T01:32:26.124105+0000","last_peered":"2023-11-06T01:32:26.124105+0000","last_clean":"2023-11-06T01:32:26.124105+0000","last_became_active":"2023-11-06T01:32:25.678829+0000","last_became_peered":"2023-11-06T01:32:25.678829+0000","last_unstale":"2023-11-06T01:32:26.124105+0000","last_undegraded":"2023-11-06T01:32:26.124105+0000","last_fullsized":"2023-11-06T01:32:26.124105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:40:07.691852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121396+0000","last_change":"2023-11-06T01:32:26.122692+0000","last_active":"2023-11-06T01:32:27.121396+0000","last_peered":"2023-11-06T01:32:27.121396+0000","last_clean":"2023-11-06T01:32:27.121396+0000","last_became_active":"2023-11-06T01:32:26.121960+0000","last_became_peered":"2023-11-06T01:32:26.121960+0000","last_unstale":"2023-11-06T01:32:27.121396+0000","last_undegraded":"2023-11-06T01:32:27.121396+0000","last_fullsized":"2023-11-06T01:32:27.121396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:48.010467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121241+0000","last_change":"2023-11-06T01:32:25.680404+0000","last_active":"2023-11-06T01:32:27.121241+0000","last_peered":"2023-11-06T01:32:27.121241+0000","last_clean":"2023-11-06T01:32:27.121241+0000","last_became_active":"2023-11-06T01:32:25.680060+0000","last_became_peered":"2023-11-06T01:32:25.680060+0000","last_unstale":"2023-11-06T01:32:27.121241+0000","last_undegraded":"2023-11-06T01:32:27.121241+0000","last_fullsized":"2023-11-06T01:32:27.121241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:55:00.476395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.119288+0000","last_change":"2023-11-06T01:32:26.116839+0000","last_active":"2023-11-06T01:32:26.119288+0000","last_peered":"2023-11-06T01:32:26.119288+0000","last_clean":"2023-11-06T01:32:26.119288+0000","last_became_active":"2023-11-06T01:32:26.116535+0000","last_became_peered":"2023-11-06T01:32:26.116535+0000","last_unstale":"2023-11-06T01:32:26.119288+0000","last_undegraded":"2023-11-06T01:32:26.119288+0000","last_fullsized":"2023-11-06T01:32:26.119288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:11.219626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.117519+0000","last_change":"2023-11-06T01:32:25.678426+0000","last_active":"2023-11-06T01:32:26.117519+0000","last_peered":"2023-11-06T01:32:26.117519+0000","last_clean":"2023-11-06T01:32:26.117519+0000","last_became_active":"2023-11-06T01:32:25.678072+0000","last_became_peered":"2023-11-06T01:32:25.678072+0000","last_unstale":"2023-11-06T01:32:26.117519+0000","last_undegraded":"2023-11-06T01:32:26.117519+0000","last_fullsized":"2023-11-06T01:32:26.117519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:09:41.630098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121277+0000","last_change":"2023-11-06T01:32:26.119184+0000","last_active":"2023-11-06T01:32:27.121277+0000","last_peered":"2023-11-06T01:32:27.121277+0000","last_clean":"2023-11-06T01:32:27.121277+0000","last_became_active":"2023-11-06T01:32:26.118896+0000","last_became_peered":"2023-11-06T01:32:26.118896+0000","last_unstale":"2023-11-06T01:32:27.121277+0000","last_undegraded":"2023-11-06T01:32:27.121277+0000","last_fullsized":"2023-11-06T01:32:27.121277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:09:17.351315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097703+0000","last_change":"2023-11-06T01:32:25.685621+0000","last_active":"2023-11-06T01:32:27.097703+0000","last_peered":"2023-11-06T01:32:27.097703+0000","last_clean":"2023-11-06T01:32:27.097703+0000","last_became_active":"2023-11-06T01:32:25.684965+0000","last_became_peered":"2023-11-06T01:32:25.684965+0000","last_unstale":"2023-11-06T01:32:27.097703+0000","last_undegraded":"2023-11-06T01:32:27.097703+0000","last_fullsized":"2023-11-06T01:32:27.097703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:28:00.922680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221174+0000","last_change":"2023-11-06T01:32:26.122750+0000","last_active":"2023-11-06T01:32:27.221174+0000","last_peered":"2023-11-06T01:32:27.221174+0000","last_clean":"2023-11-06T01:32:27.221174+0000","last_became_active":"2023-11-06T01:32:26.122241+0000","last_became_peered":"2023-11-06T01:32:26.122241+0000","last_unstale":"2023-11-06T01:32:27.221174+0000","last_undegraded":"2023-11-06T01:32:27.221174+0000","last_fullsized":"2023-11-06T01:32:27.221174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:04.402197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121070+0000","last_change":"2023-11-06T01:32:25.681594+0000","last_active":"2023-11-06T01:32:27.121070+0000","last_peered":"2023-11-06T01:32:27.121070+0000","last_clean":"2023-11-06T01:32:27.121070+0000","last_became_active":"2023-11-06T01:32:25.681303+0000","last_became_peered":"2023-11-06T01:32:25.681303+0000","last_unstale":"2023-11-06T01:32:27.121070+0000","last_undegraded":"2023-11-06T01:32:27.121070+0000","last_fullsized":"2023-11-06T01:32:27.121070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:33.141440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221062+0000","last_change":"2023-11-06T01:32:26.117234+0000","last_active":"2023-11-06T01:32:27.221062+0000","last_peered":"2023-11-06T01:32:27.221062+0000","last_clean":"2023-11-06T01:32:27.221062+0000","last_became_active":"2023-11-06T01:32:26.116915+0000","last_became_peered":"2023-11-06T01:32:26.116915+0000","last_unstale":"2023-11-06T01:32:27.221062+0000","last_undegraded":"2023-11-06T01:32:27.221062+0000","last_fullsized":"2023-11-06T01:32:27.221062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:05:27.521552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.448115+0000","last_change":"2023-11-06T01:32:25.682827+0000","last_active":"2023-11-06T01:32:29.448115+0000","last_peered":"2023-11-06T01:32:29.448115+0000","last_clean":"2023-11-06T01:32:29.448115+0000","last_became_active":"2023-11-06T01:32:25.682470+0000","last_became_peered":"2023-11-06T01:32:25.682470+0000","last_unstale":"2023-11-06T01:32:29.448115+0000","last_undegraded":"2023-11-06T01:32:29.448115+0000","last_fullsized":"2023-11-06T01:32:29.448115+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:43.632070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222801+0000","last_change":"2023-11-06T01:32:26.123918+0000","last_active":"2023-11-06T01:32:27.222801+0000","last_peered":"2023-11-06T01:32:27.222801+0000","last_clean":"2023-11-06T01:32:27.222801+0000","last_became_active":"2023-11-06T01:32:26.123450+0000","last_became_peered":"2023-11-06T01:32:26.123450+0000","last_unstale":"2023-11-06T01:32:27.222801+0000","last_undegraded":"2023-11-06T01:32:27.222801+0000","last_fullsized":"2023-11-06T01:32:27.222801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:38.227311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.448018+0000","last_change":"2023-11-06T01:32:25.680051+0000","last_active":"2023-11-06T01:32:29.448018+0000","last_peered":"2023-11-06T01:32:29.448018+0000","last_clean":"2023-11-06T01:32:29.448018+0000","last_became_active":"2023-11-06T01:32:25.679705+0000","last_became_peered":"2023-11-06T01:32:25.679705+0000","last_unstale":"2023-11-06T01:32:29.448018+0000","last_undegraded":"2023-11-06T01:32:29.448018+0000","last_fullsized":"2023-11-06T01:32:29.448018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:18:22.620036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.232007+0000","last_change":"2023-11-06T01:32:26.115101+0000","last_active":"2023-11-06T01:32:27.232007+0000","last_peered":"2023-11-06T01:32:27.232007+0000","last_clean":"2023-11-06T01:32:27.232007+0000","last_became_active":"2023-11-06T01:32:26.114480+0000","last_became_peered":"2023-11-06T01:32:26.114480+0000","last_unstale":"2023-11-06T01:32:27.232007+0000","last_undegraded":"2023-11-06T01:32:27.232007+0000","last_fullsized":"2023-11-06T01:32:27.232007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:17.770085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121423+0000","last_change":"2023-11-06T01:32:25.678689+0000","last_active":"2023-11-06T01:32:27.121423+0000","last_peered":"2023-11-06T01:32:27.121423+0000","last_clean":"2023-11-06T01:32:27.121423+0000","last_became_active":"2023-11-06T01:32:25.678355+0000","last_became_peered":"2023-11-06T01:32:25.678355+0000","last_unstale":"2023-11-06T01:32:27.121423+0000","last_undegraded":"2023-11-06T01:32:27.121423+0000","last_fullsized":"2023-11-06T01:32:27.121423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:23.007596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222655+0000","last_change":"2023-11-06T01:32:26.118120+0000","last_active":"2023-11-06T01:32:27.222655+0000","last_peered":"2023-11-06T01:32:27.222655+0000","last_clean":"2023-11-06T01:32:27.222655+0000","last_became_active":"2023-11-06T01:32:26.117762+0000","last_became_peered":"2023-11-06T01:32:26.117762+0000","last_unstale":"2023-11-06T01:32:27.222655+0000","last_undegraded":"2023-11-06T01:32:27.222655+0000","last_fullsized":"2023-11-06T01:32:27.222655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:28:42.417463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.445783+0000","last_change":"2023-11-06T01:32:25.676036+0000","last_active":"2023-11-06T01:32:29.445783+0000","last_peered":"2023-11-06T01:32:29.445783+0000","last_clean":"2023-11-06T01:32:29.445783+0000","last_became_active":"2023-11-06T01:32:25.675672+0000","last_became_peered":"2023-11-06T01:32:25.675672+0000","last_unstale":"2023-11-06T01:32:29.445783+0000","last_undegraded":"2023-11-06T01:32:29.445783+0000","last_fullsized":"2023-11-06T01:32:29.445783+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:30:54.060926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.120038+0000","last_change":"2023-11-06T01:32:26.118832+0000","last_active":"2023-11-06T01:32:26.120038+0000","last_peered":"2023-11-06T01:32:26.120038+0000","last_clean":"2023-11-06T01:32:26.120038+0000","last_became_active":"2023-11-06T01:32:26.118551+0000","last_became_peered":"2023-11-06T01:32:26.118551+0000","last_unstale":"2023-11-06T01:32:26.120038+0000","last_undegraded":"2023-11-06T01:32:26.120038+0000","last_fullsized":"2023-11-06T01:32:26.120038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:23.031203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.445689+0000","last_change":"2023-11-06T01:32:25.679433+0000","last_active":"2023-11-06T01:32:29.445689+0000","last_peered":"2023-11-06T01:32:29.445689+0000","last_clean":"2023-11-06T01:32:29.445689+0000","last_became_active":"2023-11-06T01:32:25.679090+0000","last_became_peered":"2023-11-06T01:32:25.679090+0000","last_unstale":"2023-11-06T01:32:29.445689+0000","last_undegraded":"2023-11-06T01:32:29.445689+0000","last_fullsized":"2023-11-06T01:32:29.445689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:14:57.795474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097850+0000","last_change":"2023-11-06T01:32:26.113421+0000","last_active":"2023-11-06T01:32:27.097850+0000","last_peered":"2023-11-06T01:32:27.097850+0000","last_clean":"2023-11-06T01:32:27.097850+0000","last_became_active":"2023-11-06T01:32:26.113085+0000","last_became_peered":"2023-11-06T01:32:26.113085+0000","last_unstale":"2023-11-06T01:32:27.097850+0000","last_undegraded":"2023-11-06T01:32:27.097850+0000","last_fullsized":"2023-11-06T01:32:27.097850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:15:04.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121199+0000","last_change":"2023-11-06T01:32:25.680293+0000","last_active":"2023-11-06T01:32:27.121199+0000","last_peered":"2023-11-06T01:32:27.121199+0000","last_clean":"2023-11-06T01:32:27.121199+0000","last_became_active":"2023-11-06T01:32:25.679803+0000","last_became_peered":"2023-11-06T01:32:25.679803+0000","last_unstale":"2023-11-06T01:32:27.121199+0000","last_undegraded":"2023-11-06T01:32:27.121199+0000","last_fullsized":"2023-11-06T01:32:27.121199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:28.562797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221634+0000","last_change":"2023-11-06T01:32:26.115996+0000","last_active":"2023-11-06T01:32:27.221634+0000","last_peered":"2023-11-06T01:32:27.221634+0000","last_clean":"2023-11-06T01:32:27.221634+0000","last_became_active":"2023-11-06T01:32:26.115407+0000","last_became_peered":"2023-11-06T01:32:26.115407+0000","last_unstale":"2023-11-06T01:32:27.221634+0000","last_undegraded":"2023-11-06T01:32:27.221634+0000","last_fullsized":"2023-11-06T01:32:27.221634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:08.069698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.004611+0000","last_change":"2023-11-06T01:32:25.680958+0000","last_active":"2023-11-06T01:32:31.004611+0000","last_peered":"2023-11-06T01:32:31.004611+0000","last_clean":"2023-11-06T01:32:31.004611+0000","last_became_active":"2023-11-06T01:32:25.680616+0000","last_became_peered":"2023-11-06T01:32:25.680616+0000","last_unstale":"2023-11-06T01:32:31.004611+0000","last_undegraded":"2023-11-06T01:32:31.004611+0000","last_fullsized":"2023-11-06T01:32:31.004611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:02.631755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221186+0000","last_change":"2023-11-06T01:32:26.115464+0000","last_active":"2023-11-06T01:32:27.221186+0000","last_peered":"2023-11-06T01:32:27.221186+0000","last_clean":"2023-11-06T01:32:27.221186+0000","last_became_active":"2023-11-06T01:32:26.115090+0000","last_became_peered":"2023-11-06T01:32:26.115090+0000","last_unstale":"2023-11-06T01:32:27.221186+0000","last_undegraded":"2023-11-06T01:32:27.221186+0000","last_fullsized":"2023-11-06T01:32:27.221186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:08.018044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.458054+0000","last_change":"2023-11-06T01:32:25.677787+0000","last_active":"2023-11-06T01:32:30.458054+0000","last_peered":"2023-11-06T01:32:30.458054+0000","last_clean":"2023-11-06T01:32:30.458054+0000","last_became_active":"2023-11-06T01:32:25.677527+0000","last_became_peered":"2023-11-06T01:32:25.677527+0000","last_unstale":"2023-11-06T01:32:30.458054+0000","last_undegraded":"2023-11-06T01:32:30.458054+0000","last_fullsized":"2023-11-06T01:32:30.458054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:34:09.452965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121118+0000","last_change":"2023-11-06T01:32:26.117792+0000","last_active":"2023-11-06T01:32:27.121118+0000","last_peered":"2023-11-06T01:32:27.121118+0000","last_clean":"2023-11-06T01:32:27.121118+0000","last_became_active":"2023-11-06T01:32:26.117469+0000","last_became_peered":"2023-11-06T01:32:26.117469+0000","last_unstale":"2023-11-06T01:32:27.121118+0000","last_undegraded":"2023-11-06T01:32:27.121118+0000","last_fullsized":"2023-11-06T01:32:27.121118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:44:54.409476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.121576+0000","last_change":"2023-11-06T01:32:25.677257+0000","last_active":"2023-11-06T01:32:26.121576+0000","last_peered":"2023-11-06T01:32:26.121576+0000","last_clean":"2023-11-06T01:32:26.121576+0000","last_became_active":"2023-11-06T01:32:25.676536+0000","last_became_peered":"2023-11-06T01:32:25.676536+0000","last_unstale":"2023-11-06T01:32:26.121576+0000","last_undegraded":"2023-11-06T01:32:26.121576+0000","last_fullsized":"2023-11-06T01:32:26.121576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:53:33.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408512+0000","last_change":"2023-11-06T01:32:26.111447+0000","last_active":"2023-11-06T01:32:26.408512+0000","last_peered":"2023-11-06T01:32:26.408512+0000","last_clean":"2023-11-06T01:32:26.408512+0000","last_became_active":"2023-11-06T01:32:26.111132+0000","last_became_peered":"2023-11-06T01:32:26.111132+0000","last_unstale":"2023-11-06T01:32:26.408512+0000","last_undegraded":"2023-11-06T01:32:26.408512+0000","last_fullsized":"2023-11-06T01:32:26.408512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:55.385003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.717374+0000","last_change":"2023-11-06T01:32:25.682562+0000","last_active":"2023-11-06T01:32:27.717374+0000","last_peered":"2023-11-06T01:32:27.717374+0000","last_clean":"2023-11-06T01:32:27.717374+0000","last_became_active":"2023-11-06T01:32:25.682227+0000","last_became_peered":"2023-11-06T01:32:25.682227+0000","last_unstale":"2023-11-06T01:32:27.717374+0000","last_undegraded":"2023-11-06T01:32:27.717374+0000","last_fullsized":"2023-11-06T01:32:27.717374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:40.113429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221900+0000","last_change":"2023-11-06T01:32:26.110589+0000","last_active":"2023-11-06T01:32:27.221900+0000","last_peered":"2023-11-06T01:32:27.221900+0000","last_clean":"2023-11-06T01:32:27.221900+0000","last_became_active":"2023-11-06T01:32:26.107413+0000","last_became_peered":"2023-11-06T01:32:26.107413+0000","last_unstale":"2023-11-06T01:32:27.221900+0000","last_undegraded":"2023-11-06T01:32:27.221900+0000","last_fullsized":"2023-11-06T01:32:27.221900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:39:08.613713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.102215+0000","last_change":"2023-11-06T01:32:25.688176+0000","last_active":"2023-11-06T01:32:26.102215+0000","last_peered":"2023-11-06T01:32:26.102215+0000","last_clean":"2023-11-06T01:32:26.102215+0000","last_became_active":"2023-11-06T01:32:25.687745+0000","last_became_peered":"2023-11-06T01:32:25.687745+0000","last_unstale":"2023-11-06T01:32:26.102215+0000","last_undegraded":"2023-11-06T01:32:26.102215+0000","last_fullsized":"2023-11-06T01:32:26.102215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:21:23.566389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221764+0000","last_change":"2023-11-06T01:32:26.111700+0000","last_active":"2023-11-06T01:32:27.221764+0000","last_peered":"2023-11-06T01:32:27.221764+0000","last_clean":"2023-11-06T01:32:27.221764+0000","last_became_active":"2023-11-06T01:32:26.111380+0000","last_became_peered":"2023-11-06T01:32:26.111380+0000","last_unstale":"2023-11-06T01:32:27.221764+0000","last_undegraded":"2023-11-06T01:32:27.221764+0000","last_fullsized":"2023-11-06T01:32:27.221764+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:56:47.874663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121155+0000","last_change":"2023-11-06T01:32:26.121906+0000","last_active":"2023-11-06T01:32:27.121155+0000","last_peered":"2023-11-06T01:32:27.121155+0000","last_clean":"2023-11-06T01:32:27.121155+0000","last_became_active":"2023-11-06T01:32:26.121292+0000","last_became_peered":"2023-11-06T01:32:26.121292+0000","last_unstale":"2023-11-06T01:32:27.121155+0000","last_undegraded":"2023-11-06T01:32:27.121155+0000","last_fullsized":"2023-11-06T01:32:27.121155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.532098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097619+0000","last_change":"2023-11-06T01:32:25.685814+0000","last_active":"2023-11-06T01:32:27.097619+0000","last_peered":"2023-11-06T01:32:27.097619+0000","last_clean":"2023-11-06T01:32:27.097619+0000","last_became_active":"2023-11-06T01:32:25.684890+0000","last_became_peered":"2023-11-06T01:32:25.684890+0000","last_unstale":"2023-11-06T01:32:27.097619+0000","last_undegraded":"2023-11-06T01:32:27.097619+0000","last_fullsized":"2023-11-06T01:32:27.097619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:21.950945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222017+0000","last_change":"2023-11-06T01:32:26.121752+0000","last_active":"2023-11-06T01:32:27.222017+0000","last_peered":"2023-11-06T01:32:27.222017+0000","last_clean":"2023-11-06T01:32:27.222017+0000","last_became_active":"2023-11-06T01:32:26.121447+0000","last_became_peered":"2023-11-06T01:32:26.121447+0000","last_unstale":"2023-11-06T01:32:27.222017+0000","last_undegraded":"2023-11-06T01:32:27.222017+0000","last_fullsized":"2023-11-06T01:32:27.222017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:51:19.727335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.004622+0000","last_change":"2023-11-06T01:32:25.682727+0000","last_active":"2023-11-06T01:32:31.004622+0000","last_peered":"2023-11-06T01:32:31.004622+0000","last_clean":"2023-11-06T01:32:31.004622+0000","last_became_active":"2023-11-06T01:32:25.682377+0000","last_became_peered":"2023-11-06T01:32:25.682377+0000","last_unstale":"2023-11-06T01:32:31.004622+0000","last_undegraded":"2023-11-06T01:32:31.004622+0000","last_fullsized":"2023-11-06T01:32:31.004622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:17:38.614350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":325,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.100201+0000","last_change":"2023-11-06T01:31:27.931076+0000","last_active":"2023-11-06T01:32:26.100201+0000","last_peered":"2023-11-06T01:32:26.100201+0000","last_clean":"2023-11-06T01:32:26.100201+0000","last_became_active":"2023-11-06T01:31:27.930669+0000","last_became_peered":"2023-11-06T01:31:27.930669+0000","last_unstale":"2023-11-06T01:32:26.100201+0000","last_undegraded":"2023-11-06T01:32:26.100201+0000","last_fullsized":"2023-11-06T01:32:26.100201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_clean_scrub_stamp":"2023-11-06T01:28:36.252549+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:47:48.729325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097629+0000","last_change":"2023-11-06T01:32:26.110055+0000","last_active":"2023-11-06T01:32:27.097629+0000","last_peered":"2023-11-06T01:32:27.097629+0000","last_clean":"2023-11-06T01:32:27.097629+0000","last_became_active":"2023-11-06T01:32:26.109640+0000","last_became_peered":"2023-11-06T01:32:26.109640+0000","last_unstale":"2023-11-06T01:32:27.097629+0000","last_undegraded":"2023-11-06T01:32:27.097629+0000","last_fullsized":"2023-11-06T01:32:27.097629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:45:43.076910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.220977+0000","last_change":"2023-11-06T01:32:25.679885+0000","last_active":"2023-11-06T01:32:27.220977+0000","last_peered":"2023-11-06T01:32:27.220977+0000","last_clean":"2023-11-06T01:32:27.220977+0000","last_became_active":"2023-11-06T01:32:25.679518+0000","last_became_peered":"2023-11-06T01:32:25.679518+0000","last_unstale":"2023-11-06T01:32:27.220977+0000","last_undegraded":"2023-11-06T01:32:27.220977+0000","last_fullsized":"2023-11-06T01:32:27.220977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:44.558534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221410+0000","last_change":"2023-11-06T01:32:25.682016+0000","last_active":"2023-11-06T01:32:27.221410+0000","last_peered":"2023-11-06T01:32:27.221410+0000","last_clean":"2023-11-06T01:32:27.221410+0000","last_became_active":"2023-11-06T01:32:25.681753+0000","last_became_peered":"2023-11-06T01:32:25.681753+0000","last_unstale":"2023-11-06T01:32:27.221410+0000","last_undegraded":"2023-11-06T01:32:27.221410+0000","last_fullsized":"2023-11-06T01:32:27.221410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:08.575728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097816+0000","last_change":"2023-11-06T01:32:26.117109+0000","last_active":"2023-11-06T01:32:27.097816+0000","last_peered":"2023-11-06T01:32:27.097816+0000","last_clean":"2023-11-06T01:32:27.097816+0000","last_became_active":"2023-11-06T01:32:26.116769+0000","last_became_peered":"2023-11-06T01:32:26.116769+0000","last_unstale":"2023-11-06T01:32:27.097816+0000","last_undegraded":"2023-11-06T01:32:27.097816+0000","last_fullsized":"2023-11-06T01:32:27.097816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:48:04.188806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.006423+0000","last_change":"2023-11-06T01:32:25.681013+0000","last_active":"2023-11-06T01:32:31.006423+0000","last_peered":"2023-11-06T01:32:31.006423+0000","last_clean":"2023-11-06T01:32:31.006423+0000","last_became_active":"2023-11-06T01:32:25.680699+0000","last_became_peered":"2023-11-06T01:32:25.680699+0000","last_unstale":"2023-11-06T01:32:31.006423+0000","last_undegraded":"2023-11-06T01:32:31.006423+0000","last_fullsized":"2023-11-06T01:32:31.006423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:31.755527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221905+0000","last_change":"2023-11-06T01:32:26.116351+0000","last_active":"2023-11-06T01:32:27.221905+0000","last_peered":"2023-11-06T01:32:27.221905+0000","last_clean":"2023-11-06T01:32:27.221905+0000","last_became_active":"2023-11-06T01:32:26.116044+0000","last_became_peered":"2023-11-06T01:32:26.116044+0000","last_unstale":"2023-11-06T01:32:27.221905+0000","last_undegraded":"2023-11-06T01:32:27.221905+0000","last_fullsized":"2023-11-06T01:32:27.221905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:20.816412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.016620+0000","last_change":"2023-11-06T01:32:25.676764+0000","last_active":"2023-11-06T01:32:32.016620+0000","last_peered":"2023-11-06T01:32:32.016620+0000","last_clean":"2023-11-06T01:32:32.016620+0000","last_became_active":"2023-11-06T01:32:25.676418+0000","last_became_peered":"2023-11-06T01:32:25.676418+0000","last_unstale":"2023-11-06T01:32:32.016620+0000","last_undegraded":"2023-11-06T01:32:32.016620+0000","last_fullsized":"2023-11-06T01:32:32.016620+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:49.320532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231997+0000","last_change":"2023-11-06T01:32:26.121107+0000","last_active":"2023-11-06T01:32:27.231997+0000","last_peered":"2023-11-06T01:32:27.231997+0000","last_clean":"2023-11-06T01:32:27.231997+0000","last_became_active":"2023-11-06T01:32:26.120790+0000","last_became_peered":"2023-11-06T01:32:26.120790+0000","last_unstale":"2023-11-06T01:32:27.231997+0000","last_undegraded":"2023-11-06T01:32:27.231997+0000","last_fullsized":"2023-11-06T01:32:27.231997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:34:46.660301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222419+0000","last_change":"2023-11-06T01:32:25.678959+0000","last_active":"2023-11-06T01:32:27.222419+0000","last_peered":"2023-11-06T01:32:27.222419+0000","last_clean":"2023-11-06T01:32:27.222419+0000","last_became_active":"2023-11-06T01:32:25.678564+0000","last_became_peered":"2023-11-06T01:32:25.678564+0000","last_unstale":"2023-11-06T01:32:27.222419+0000","last_undegraded":"2023-11-06T01:32:27.222419+0000","last_fullsized":"2023-11-06T01:32:27.222419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:30:06.304889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.114597+0000","last_change":"2023-11-06T01:32:26.113222+0000","last_active":"2023-11-06T01:32:26.114597+0000","last_peered":"2023-11-06T01:32:26.114597+0000","last_clean":"2023-11-06T01:32:26.114597+0000","last_became_active":"2023-11-06T01:32:26.112926+0000","last_became_peered":"2023-11-06T01:32:26.112926+0000","last_unstale":"2023-11-06T01:32:26.114597+0000","last_undegraded":"2023-11-06T01:32:26.114597+0000","last_fullsized":"2023-11-06T01:32:26.114597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:09:51.042469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.445582+0000","last_change":"2023-11-06T01:32:25.684985+0000","last_active":"2023-11-06T01:32:29.445582+0000","last_peered":"2023-11-06T01:32:29.445582+0000","last_clean":"2023-11-06T01:32:29.445582+0000","last_became_active":"2023-11-06T01:32:25.684656+0000","last_became_peered":"2023-11-06T01:32:25.684656+0000","last_unstale":"2023-11-06T01:32:29.445582+0000","last_undegraded":"2023-11-06T01:32:29.445582+0000","last_fullsized":"2023-11-06T01:32:29.445582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:42:29.406803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408460+0000","last_change":"2023-11-06T01:32:26.107484+0000","last_active":"2023-11-06T01:32:26.408460+0000","last_peered":"2023-11-06T01:32:26.408460+0000","last_clean":"2023-11-06T01:32:26.408460+0000","last_became_active":"2023-11-06T01:32:26.106092+0000","last_became_peered":"2023-11-06T01:32:26.106092+0000","last_unstale":"2023-11-06T01:32:26.408460+0000","last_undegraded":"2023-11-06T01:32:26.408460+0000","last_fullsized":"2023-11-06T01:32:26.408460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:43:39.481290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.101920+0000","last_change":"2023-11-06T01:32:25.679531+0000","last_active":"2023-11-06T01:32:26.101920+0000","last_peered":"2023-11-06T01:32:26.101920+0000","last_clean":"2023-11-06T01:32:26.101920+0000","last_became_active":"2023-11-06T01:32:25.679076+0000","last_became_peered":"2023-11-06T01:32:25.679076+0000","last_unstale":"2023-11-06T01:32:26.101920+0000","last_undegraded":"2023-11-06T01:32:26.101920+0000","last_fullsized":"2023-11-06T01:32:26.101920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:37.381547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222136+0000","last_change":"2023-11-06T01:32:26.118584+0000","last_active":"2023-11-06T01:32:27.222136+0000","last_peered":"2023-11-06T01:32:27.222136+0000","last_clean":"2023-11-06T01:32:27.222136+0000","last_became_active":"2023-11-06T01:32:26.118285+0000","last_became_peered":"2023-11-06T01:32:26.118285+0000","last_unstale":"2023-11-06T01:32:27.222136+0000","last_undegraded":"2023-11-06T01:32:27.222136+0000","last_fullsized":"2023-11-06T01:32:27.222136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:52:33.057377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.447677+0000","last_change":"2023-11-06T01:32:25.681214+0000","last_active":"2023-11-06T01:32:29.447677+0000","last_peered":"2023-11-06T01:32:29.447677+0000","last_clean":"2023-11-06T01:32:29.447677+0000","last_became_active":"2023-11-06T01:32:25.680907+0000","last_became_peered":"2023-11-06T01:32:25.680907+0000","last_unstale":"2023-11-06T01:32:29.447677+0000","last_undegraded":"2023-11-06T01:32:29.447677+0000","last_fullsized":"2023-11-06T01:32:29.447677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:04:58.248787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221859+0000","last_change":"2023-11-06T01:32:26.120862+0000","last_active":"2023-11-06T01:32:27.221859+0000","last_peered":"2023-11-06T01:32:27.221859+0000","last_clean":"2023-11-06T01:32:27.221859+0000","last_became_active":"2023-11-06T01:32:26.120307+0000","last_became_peered":"2023-11-06T01:32:26.120307+0000","last_unstale":"2023-11-06T01:32:27.221859+0000","last_undegraded":"2023-11-06T01:32:27.221859+0000","last_fullsized":"2023-11-06T01:32:27.221859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:45:35.611541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.102314+0000","last_change":"2023-11-06T01:32:25.675607+0000","last_active":"2023-11-06T01:32:26.102314+0000","last_peered":"2023-11-06T01:32:26.102314+0000","last_clean":"2023-11-06T01:32:26.102314+0000","last_became_active":"2023-11-06T01:32:25.675293+0000","last_became_peered":"2023-11-06T01:32:25.675293+0000","last_unstale":"2023-11-06T01:32:26.102314+0000","last_undegraded":"2023-11-06T01:32:26.102314+0000","last_fullsized":"2023-11-06T01:32:26.102314+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:05.190844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221522+0000","last_change":"2023-11-06T01:32:26.116987+0000","last_active":"2023-11-06T01:32:27.221522+0000","last_peered":"2023-11-06T01:32:27.221522+0000","last_clean":"2023-11-06T01:32:27.221522+0000","last_became_active":"2023-11-06T01:32:26.116756+0000","last_became_peered":"2023-11-06T01:32:26.116756+0000","last_unstale":"2023-11-06T01:32:27.221522+0000","last_undegraded":"2023-11-06T01:32:27.221522+0000","last_fullsized":"2023-11-06T01:32:27.221522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:44.039520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121352+0000","last_change":"2023-11-06T01:32:25.675483+0000","last_active":"2023-11-06T01:32:27.121352+0000","last_peered":"2023-11-06T01:32:27.121352+0000","last_clean":"2023-11-06T01:32:27.121352+0000","last_became_active":"2023-11-06T01:32:25.675137+0000","last_became_peered":"2023-11-06T01:32:25.675137+0000","last_unstale":"2023-11-06T01:32:27.121352+0000","last_undegraded":"2023-11-06T01:32:27.121352+0000","last_fullsized":"2023-11-06T01:32:27.121352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:42.415165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221104+0000","last_change":"2023-11-06T01:32:26.115575+0000","last_active":"2023-11-06T01:32:27.221104+0000","last_peered":"2023-11-06T01:32:27.221104+0000","last_clean":"2023-11-06T01:32:27.221104+0000","last_became_active":"2023-11-06T01:32:26.115142+0000","last_became_peered":"2023-11-06T01:32:26.115142+0000","last_unstale":"2023-11-06T01:32:27.221104+0000","last_undegraded":"2023-11-06T01:32:27.221104+0000","last_fullsized":"2023-11-06T01:32:27.221104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:52.859340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097857+0000","last_change":"2023-11-06T01:32:25.676972+0000","last_active":"2023-11-06T01:32:27.097857+0000","last_peered":"2023-11-06T01:32:27.097857+0000","last_clean":"2023-11-06T01:32:27.097857+0000","last_became_active":"2023-11-06T01:32:25.676685+0000","last_became_peered":"2023-11-06T01:32:25.676685+0000","last_unstale":"2023-11-06T01:32:27.097857+0000","last_undegraded":"2023-11-06T01:32:27.097857+0000","last_fullsized":"2023-11-06T01:32:27.097857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:30:21.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221207+0000","last_change":"2023-11-06T01:32:26.115164+0000","last_active":"2023-11-06T01:32:27.221207+0000","last_peered":"2023-11-06T01:32:27.221207+0000","last_clean":"2023-11-06T01:32:27.221207+0000","last_became_active":"2023-11-06T01:32:26.114868+0000","last_became_peered":"2023-11-06T01:32:26.114868+0000","last_unstale":"2023-11-06T01:32:27.221207+0000","last_undegraded":"2023-11-06T01:32:27.221207+0000","last_fullsized":"2023-11-06T01:32:27.221207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:01:34.149871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408478+0000","last_change":"2023-11-06T01:32:26.115600+0000","last_active":"2023-11-06T01:32:26.408478+0000","last_peered":"2023-11-06T01:32:26.408478+0000","last_clean":"2023-11-06T01:32:26.408478+0000","last_became_active":"2023-11-06T01:32:26.115290+0000","last_became_peered":"2023-11-06T01:32:26.115290+0000","last_unstale":"2023-11-06T01:32:26.408478+0000","last_undegraded":"2023-11-06T01:32:26.408478+0000","last_fullsized":"2023-11-06T01:32:26.408478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:04:06.393671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221461+0000","last_change":"2023-11-06T01:32:26.115336+0000","last_active":"2023-11-06T01:32:27.221461+0000","last_peered":"2023-11-06T01:32:27.221461+0000","last_clean":"2023-11-06T01:32:27.221461+0000","last_became_active":"2023-11-06T01:32:26.114946+0000","last_became_peered":"2023-11-06T01:32:26.114946+0000","last_unstale":"2023-11-06T01:32:27.221461+0000","last_undegraded":"2023-11-06T01:32:27.221461+0000","last_fullsized":"2023-11-06T01:32:27.221461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:19:13.651494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221633+0000","last_change":"2023-11-06T01:32:26.117794+0000","last_active":"2023-11-06T01:32:27.221633+0000","last_peered":"2023-11-06T01:32:27.221633+0000","last_clean":"2023-11-06T01:32:27.221633+0000","last_became_active":"2023-11-06T01:32:26.117425+0000","last_became_peered":"2023-11-06T01:32:26.117425+0000","last_unstale":"2023-11-06T01:32:27.221633+0000","last_undegraded":"2023-11-06T01:32:27.221633+0000","last_fullsized":"2023-11-06T01:32:27.221633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:00:24.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231970+0000","last_change":"2023-11-06T01:32:26.113662+0000","last_active":"2023-11-06T01:32:27.231970+0000","last_peered":"2023-11-06T01:32:27.231970+0000","last_clean":"2023-11-06T01:32:27.231970+0000","last_became_active":"2023-11-06T01:32:26.113124+0000","last_became_peered":"2023-11-06T01:32:26.113124+0000","last_unstale":"2023-11-06T01:32:27.231970+0000","last_undegraded":"2023-11-06T01:32:27.231970+0000","last_fullsized":"2023-11-06T01:32:27.231970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:50:24.349521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097929+0000","last_change":"2023-11-06T01:32:26.109910+0000","last_active":"2023-11-06T01:32:27.097929+0000","last_peered":"2023-11-06T01:32:27.097929+0000","last_clean":"2023-11-06T01:32:27.097929+0000","last_became_active":"2023-11-06T01:32:26.109568+0000","last_became_peered":"2023-11-06T01:32:26.109568+0000","last_unstale":"2023-11-06T01:32:27.097929+0000","last_undegraded":"2023-11-06T01:32:27.097929+0000","last_fullsized":"2023-11-06T01:32:27.097929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:11.049749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408501+0000","last_change":"2023-11-06T01:32:26.114824+0000","last_active":"2023-11-06T01:32:26.408501+0000","last_peered":"2023-11-06T01:32:26.408501+0000","last_clean":"2023-11-06T01:32:26.408501+0000","last_became_active":"2023-11-06T01:32:26.114539+0000","last_became_peered":"2023-11-06T01:32:26.114539+0000","last_unstale":"2023-11-06T01:32:26.408501+0000","last_undegraded":"2023-11-06T01:32:26.408501+0000","last_fullsized":"2023-11-06T01:32:26.408501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:03:02.018769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221808+0000","last_change":"2023-11-06T01:32:26.114743+0000","last_active":"2023-11-06T01:32:27.221808+0000","last_peered":"2023-11-06T01:32:27.221808+0000","last_clean":"2023-11-06T01:32:27.221808+0000","last_became_active":"2023-11-06T01:32:26.114432+0000","last_became_peered":"2023-11-06T01:32:26.114432+0000","last_unstale":"2023-11-06T01:32:27.221808+0000","last_undegraded":"2023-11-06T01:32:27.221808+0000","last_fullsized":"2023-11-06T01:32:27.221808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:27:18.209810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221801+0000","last_change":"2023-11-06T01:32:26.125724+0000","last_active":"2023-11-06T01:32:27.221801+0000","last_peered":"2023-11-06T01:32:27.221801+0000","last_clean":"2023-11-06T01:32:27.221801+0000","last_became_active":"2023-11-06T01:32:26.125455+0000","last_became_peered":"2023-11-06T01:32:26.125455+0000","last_unstale":"2023-11-06T01:32:27.221801+0000","last_undegraded":"2023-11-06T01:32:27.221801+0000","last_fullsized":"2023-11-06T01:32:27.221801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:49.099308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121127+0000","last_change":"2023-11-06T01:32:26.119328+0000","last_active":"2023-11-06T01:32:27.121127+0000","last_peered":"2023-11-06T01:32:27.121127+0000","last_clean":"2023-11-06T01:32:27.121127+0000","last_became_active":"2023-11-06T01:32:26.119042+0000","last_became_peered":"2023-11-06T01:32:26.119042+0000","last_unstale":"2023-11-06T01:32:27.121127+0000","last_undegraded":"2023-11-06T01:32:27.121127+0000","last_fullsized":"2023-11-06T01:32:27.121127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:36.143788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221676+0000","last_change":"2023-11-06T01:32:26.126538+0000","last_active":"2023-11-06T01:32:27.221676+0000","last_peered":"2023-11-06T01:32:27.221676+0000","last_clean":"2023-11-06T01:32:27.221676+0000","last_became_active":"2023-11-06T01:32:26.124898+0000","last_became_peered":"2023-11-06T01:32:26.124898+0000","last_unstale":"2023-11-06T01:32:27.221676+0000","last_undegraded":"2023-11-06T01:32:27.221676+0000","last_fullsized":"2023-11-06T01:32:27.221676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:33.019326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222542+0000","last_change":"2023-11-06T01:32:26.118751+0000","last_active":"2023-11-06T01:32:27.222542+0000","last_peered":"2023-11-06T01:32:27.222542+0000","last_clean":"2023-11-06T01:32:27.222542+0000","last_became_active":"2023-11-06T01:32:26.118359+0000","last_became_peered":"2023-11-06T01:32:26.118359+0000","last_unstale":"2023-11-06T01:32:27.222542+0000","last_undegraded":"2023-11-06T01:32:27.222542+0000","last_fullsized":"2023-11-06T01:32:27.222542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:55.960400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.098195+0000","last_change":"2023-11-06T01:32:26.110133+0000","last_active":"2023-11-06T01:32:27.098195+0000","last_peered":"2023-11-06T01:32:27.098195+0000","last_clean":"2023-11-06T01:32:27.098195+0000","last_became_active":"2023-11-06T01:32:26.109793+0000","last_became_peered":"2023-11-06T01:32:26.109793+0000","last_unstale":"2023-11-06T01:32:27.098195+0000","last_undegraded":"2023-11-06T01:32:27.098195+0000","last_fullsized":"2023-11-06T01:32:27.098195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:00:03.956947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408491+0000","last_change":"2023-11-06T01:32:26.109828+0000","last_active":"2023-11-06T01:32:26.408491+0000","last_peered":"2023-11-06T01:32:26.408491+0000","last_clean":"2023-11-06T01:32:26.408491+0000","last_became_active":"2023-11-06T01:32:26.109526+0000","last_became_peered":"2023-11-06T01:32:26.109526+0000","last_unstale":"2023-11-06T01:32:26.408491+0000","last_undegraded":"2023-11-06T01:32:26.408491+0000","last_fullsized":"2023-11-06T01:32:26.408491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:36.561908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250075,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3396,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":115,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":13574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481804,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":418689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":10,"up_from":70,"seq":300647710736,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":1010664,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.70699999999999996}]},{"osd":2,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.745}]},{"osd":3,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.66000000000000003}]},{"osd":4,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.68100000000000005}]},{"osd":5,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.89600000000000002}]},{"osd":6,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.84999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]}]},{"osd":9,"up_from":65,"seq":279172874261,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":419229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46899999999999997}]},{"osd":1,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.44500000000000001}]},{"osd":2,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.58099999999999996}]},{"osd":3,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.48899999999999999}]},{"osd":4,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.52000000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.68300000000000005}]},{"osd":8,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.73199999999999998}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.728}]}]},{"osd":8,"up_from":58,"seq":249108103192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":418514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.78000000000000003}]},{"osd":1,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57999999999999996}]},{"osd":2,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.47499999999999998}]},{"osd":3,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.77000000000000002}]},{"osd":4,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.68899999999999995}]},{"osd":5,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.60199999999999998}]},{"osd":6,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1,"5min":1,"15min":1},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.84999999999999998}]},{"osd":7,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.755},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.73699999999999999}]},{"osd":9,"last update":"Mon Nov 6 01:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.93700000000000006}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.91900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":7,"up_from":51,"seq":219043332123,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":1009057,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.873,"5min":1.157,"15min":1.157},"last":0.376}]},{"osd":1,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.32900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.45600000000000002,"15min":0.44800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.33700000000000002}]},{"osd":2,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57899999999999996,"15min":0.56799999999999995},"min":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.45400000000000001}]},{"osd":3,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.503},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85899999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.51300000000000001,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.379}]},{"osd":5,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.53200000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53900000000000003}]},{"osd":6,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97099999999999997,"5min":1.232,"15min":1.232},"last":0.58599999999999997}]},{"osd":8,"last update":"Mon Nov 6 01:31:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55200000000000005}]},{"osd":9,"last update":"Mon Nov 6 01:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61799999999999999}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.628}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495648,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":418107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.443},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42099999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81100000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.38},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.435},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66800000000000004}]},{"osd":3,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.46200000000000002}]},{"osd":4,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69699999999999995,"15min":0.69399999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.72499999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.71499999999999997,"15min":0.69999999999999996},"min":{"1min":0.55900000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52800000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.5}]},{"osd":8,"last update":"Mon Nov 6 01:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.55700000000000005}]},{"osd":9,"last update":"Mon Nov 6 01:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.71899999999999997}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.62},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.69199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":419671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.51300000000000001,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.433}]},{"osd":2,"last update":"Mon Nov 6 01:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85899999999999999,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.35499999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.65600000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.60499999999999998}]},{"osd":3,"last update":"Mon Nov 6 01:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.54700000000000004,"15min":0.52100000000000002},"min":{"1min":0.44900000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91800000000000004,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.47799999999999998,"15min":0.46100000000000002},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.53500000000000003,"15min":0.51000000000000001},"min":{"1min":0.36899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.47099999999999997}]},{"osd":5,"last update":"Mon Nov 6 01:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.754,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Nov 6 01:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.495,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.377}]},{"osd":7,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.60199999999999998,"15min":0.56799999999999995},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.61399999999999999,"15min":0.57199999999999995},"min":{"1min":0.50700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.50700000000000001}]},{"osd":8,"last update":"Mon Nov 6 01:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.67400000000000004}]},{"osd":9,"last update":"Mon Nov 6 01:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.54000000000000004}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.495},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.65700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":420125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50700000000000001},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.504},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.61299999999999999}]},{"osd":2,"last update":"Mon Nov 6 01:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57399999999999995,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.39600000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44,"15min":0.437},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.36799999999999999}]},{"osd":4,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.41499999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Nov 6 01:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.60399999999999998},"min":{"1min":0.48499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.502},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.52700000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.439}]},{"osd":6,"last update":"Mon Nov 6 01:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.58299999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.54300000000000004}]},{"osd":7,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.52800000000000002}]},{"osd":9,"last update":"Mon Nov 6 01:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.67000000000000004}]},{"osd":10,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.624},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.64900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":419705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.495,"15min":0.48599999999999999},"min":{"1min":0.36799999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.33500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.44700000000000001}]},{"osd":1,"last update":"Mon Nov 6 01:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.41999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.434}]},{"osd":3,"last update":"Mon Nov 6 01:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58699999999999997,"15min":0.57099999999999995},"min":{"1min":0.46999999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Mon Nov 6 01:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55500000000000005}]},{"osd":5,"last update":"Mon Nov 6 01:31:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.433},"min":{"1min":0.35899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.60299999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.375},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68600000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.503}]},{"osd":6,"last update":"Mon Nov 6 01:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56399999999999995}]},{"osd":7,"last update":"Mon Nov 6 01:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.54700000000000004}]},{"osd":8,"last update":"Mon Nov 6 01:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.53500000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.65000000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.61799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]},{"osd":3,"up_from":26,"seq":111669149741,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":418301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.371,"15min":0.36099999999999999},"min":{"1min":0.27300000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.377,"15min":0.36099999999999999},"min":{"1min":0.32200000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.47199999999999998}]},{"osd":1,"last update":"Mon Nov 6 01:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.86299999999999999,"5min":1.119,"15min":1.119},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.52000000000000002,"15min":0.48799999999999999},"min":{"1min":0.45200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57799999999999996}]},{"osd":2,"last update":"Mon Nov 6 01:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.47199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.46600000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.91700000000000004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Mon Nov 6 01:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.502,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.98799999999999999,"5min":1.159,"15min":1.159},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Nov 6 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56200000000000006}]},{"osd":6,"last update":"Mon Nov 6 01:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.34499999999999997,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.56499999999999995,"15min":0.54100000000000004},"min":{"1min":0.41199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.63}]},{"osd":7,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.66500000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.67600000000000005,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73699999999999999}]},{"osd":8,"last update":"Mon Nov 6 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.622},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.0430000000000001,"5min":6.0430000000000001,"15min":6.0430000000000001},"last":0.76500000000000001}]},{"osd":9,"last update":"Mon Nov 6 01:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.64400000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.749},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888105,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":418759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.45300000000000001},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.49399999999999999,"15min":0.47799999999999998},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.749,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42999999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.377,"15min":0.35799999999999998},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.36599999999999999},"min":{"1min":0.27000000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.503}]},{"osd":2,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.44900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.79700000000000004}]},{"osd":3,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.51400000000000001,"15min":0.49099999999999999},"min":{"1min":0.38900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.41199999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.53700000000000003}]},{"osd":5,"last update":"Mon Nov 6 01:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77100000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85099999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.88}]},{"osd":6,"last update":"Mon Nov 6 01:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78500000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.83599999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68999999999999995}]},{"osd":7,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.745},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.95499999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":1.0469999999999999}]},{"osd":9,"last update":"Mon Nov 6 01:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.78300000000000003}]},{"osd":10,"last update":"Mon Nov 6 01:32:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.86399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724581,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":1008326,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.41499999999999998}]},{"osd":1,"last update":"Mon Nov 6 01:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.34499999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54400000000000004,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Nov 6 01:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39100000000000001,"15min":0.38300000000000001},"min":{"1min":0.308,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.38,"15min":0.37},"min":{"1min":0.31,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.309}]},{"osd":3,"last update":"Mon Nov 6 01:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81499999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.45900000000000002}]},{"osd":4,"last update":"Mon Nov 6 01:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.496}]},{"osd":6,"last update":"Mon Nov 6 01:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.622},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48699999999999999}]},{"osd":7,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53400000000000003}]},{"osd":8,"last update":"Mon Nov 6 01:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54300000000000004}]},{"osd":9,"last update":"Mon Nov 6 01:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.56100000000000005}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T01:32:34.810 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T01:32:34.811 INFO:tasks.cephadm:Setting up client nodes... 2023-11-06T01:32:34.811 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-11-06T01:32:35.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:35 smithi053 ceph-mon[99472]: pgmap v256: 97 pgs: 97 active+clean; 580 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 917 B/s rd, 4.6 KiB/s wr, 15 op/s 2023-11-06T01:32:35.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:35 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1434421414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:35.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:35 smithi053 ceph-mon[99472]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-11-06T01:32:35.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:35 smithi062 ceph-mon[103924]: pgmap v256: 97 pgs: 97 active+clean; 580 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 917 B/s rd, 4.6 KiB/s wr, 15 op/s 2023-11-06T01:32:35.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:35 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1434421414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:35.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:35 smithi062 ceph-mon[103924]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-11-06T01:32:35.396 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:35.489 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:32:35 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-i[140736]: 2023-11-06T01:32:35.024+0000 7fcf223c8700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-11-06T01:32:35.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:35 smithi163 ceph-mon[104416]: pgmap v256: 97 pgs: 97 active+clean; 580 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 917 B/s rd, 4.6 KiB/s wr, 15 op/s 2023-11-06T01:32:35.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:35 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1434421414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:35.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:35 smithi163 ceph-mon[104416]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-11-06T01:32:36.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:36 smithi053 ceph-mon[99472]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:32:36.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:36 smithi053 ceph-mon[99472]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:32:36.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:36 smithi062 ceph-mon[103924]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:32:36.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:36 smithi062 ceph-mon[103924]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:32:36.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:36 smithi163 ceph-mon[104416]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:32:36.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:36 smithi163 ceph-mon[104416]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:32:36.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.646+0000 7f8c54f93700 1 -- 172.21.15.53:0/124195364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c500ffd50 msgr2=0x7f8c50100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:36.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.646+0000 7f8c54f93700 1 --2- 172.21.15.53:0/124195364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c500ffd50 0x7f8c50100170 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8c400099c0 tx=0x7f8c40009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.646+0000 7f8c54f93700 1 -- 172.21.15.53:0/124195364 shutdown_connections 2023-11-06T01:32:36.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.646+0000 7f8c54f93700 1 --2- 172.21.15.53:0/124195364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c501018f0 0x7f8c501081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.646+0000 7f8c54f93700 1 --2- 172.21.15.53:0/124195364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c500ffd50 0x7f8c50100170 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.646+0000 7f8c54f93700 1 --2- 172.21.15.53:0/124195364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c50100f50 0x7f8c501013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.646+0000 7f8c54f93700 1 -- 172.21.15.53:0/124195364 >> 172.21.15.53:0/124195364 conn(0x7f8c500fb2d0 msgr2=0x7f8c500fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:36.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.646+0000 7f8c54f93700 1 -- 172.21.15.53:0/124195364 shutdown_connections 2023-11-06T01:32:36.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.646+0000 7f8c54f93700 1 -- 172.21.15.53:0/124195364 wait complete. 2023-11-06T01:32:36.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c54f93700 1 Processor -- start 2023-11-06T01:32:36.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c54f93700 1 -- start start 2023-11-06T01:32:36.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c54f93700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c500ffd50 0x7f8c50069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:36.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c54f93700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c50100f50 0x7f8c5019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:36.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c54f93700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c501018f0 0x7f8c5019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:36.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c54f93700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5019e630 con 0x7f8c501018f0 2023-11-06T01:32:36.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c54f93700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5019e7a0 con 0x7f8c50100f50 2023-11-06T01:32:36.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c54f93700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5019e960 con 0x7f8c500ffd50 2023-11-06T01:32:36.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c4ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c501018f0 0x7f8c5019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:36.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c4ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c501018f0 0x7f8c5019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38192/0 (socket says 172.21.15.53:38192) 2023-11-06T01:32:36.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c4ed9d700 1 -- 172.21.15.53:0/4294484816 learned_addr learned my addr 172.21.15.53:0/4294484816 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:36.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c4e59c700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c500ffd50 0x7f8c50069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:36.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c4dd9b700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c50100f50 0x7f8c5019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:36.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c4ed9d700 1 -- 172.21.15.53:0/4294484816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c500ffd50 msgr2=0x7f8c50069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:36.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c4ed9d700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c500ffd50 0x7f8c50069410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c4ed9d700 1 -- 172.21.15.53:0/4294484816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c50100f50 msgr2=0x7f8c5019b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:36.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c4ed9d700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c50100f50 0x7f8c5019b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c4ed9d700 1 -- 172.21.15.53:0/4294484816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c400095f0 con 0x7f8c501018f0 2023-11-06T01:32:36.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.647+0000 7f8c4dd9b700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c50100f50 0x7f8c5019b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:36.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.648+0000 7f8c4ed9d700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c501018f0 0x7f8c5019de90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800ef30 tx=0x7f8c4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:36.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.648+0000 7f8c3f7fe700 1 -- 172.21.15.53:0/4294484816 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48018070 con 0x7f8c501018f0 2023-11-06T01:32:36.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.648+0000 7f8c3f7fe700 1 -- 172.21.15.53:0/4294484816 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c4800f040 con 0x7f8c501018f0 2023-11-06T01:32:36.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.648+0000 7f8c54f93700 1 -- 172.21.15.53:0/4294484816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5019ebe0 con 0x7f8c501018f0 2023-11-06T01:32:36.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.648+0000 7f8c3f7fe700 1 -- 172.21.15.53:0/4294484816 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4800cdb0 con 0x7f8c501018f0 2023-11-06T01:32:36.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.648+0000 7f8c54f93700 1 -- 172.21.15.53:0/4294484816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5019f090 con 0x7f8c501018f0 2023-11-06T01:32:36.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.649+0000 7f8c54f93700 1 -- 172.21.15.53:0/4294484816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c50106ce0 con 0x7f8c501018f0 2023-11-06T01:32:36.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.650+0000 7f8c3f7fe700 1 -- 172.21.15.53:0/4294484816 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f8c48003a00 con 0x7f8c501018f0 2023-11-06T01:32:36.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.650+0000 7f8c3f7fe700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8c380a05b0 0x7f8c380a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:36.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.651+0000 7f8c4e59c700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8c380a05b0 0x7f8c380a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:36.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.651+0000 7f8c3f7fe700 1 -- 172.21.15.53:0/4294484816 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8c480cb260 con 0x7f8c501018f0 2023-11-06T01:32:36.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.651+0000 7f8c4e59c700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8c380a05b0 0x7f8c380a2a70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009990 tx=0x7f8c40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:36.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.655+0000 7f8c3f7fe700 1 -- 172.21.15.53:0/4294484816 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8c48094de0 con 0x7f8c501018f0 2023-11-06T01:32:36.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.934+0000 7f8c54f93700 1 -- 172.21.15.53:0/4294484816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8c5002cd10 con 0x7f8c501018f0 2023-11-06T01:32:36.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.938+0000 7f8c3f7fe700 1 -- 172.21.15.53:0/4294484816 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8c48098400 con 0x7f8c501018f0 2023-11-06T01:32:36.939 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-11-06T01:32:36.940 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA0QkhlB8u+NxAAdZKNOBWVlbWuBHKOTy026g== 2023-11-06T01:32:36.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 -- 172.21.15.53:0/4294484816 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8c380a05b0 msgr2=0x7f8c380a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:36.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8c380a05b0 0x7f8c380a2a70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009990 tx=0x7f8c40005fb0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 -- 172.21.15.53:0/4294484816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c501018f0 msgr2=0x7f8c5019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:36.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c501018f0 0x7f8c5019de90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800ef30 tx=0x7f8c4800c3a0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 -- 172.21.15.53:0/4294484816 shutdown_connections 2023-11-06T01:32:36.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8c380a05b0 0x7f8c380a2a70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c501018f0 0x7f8c5019de90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c500ffd50 0x7f8c50069410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 --2- 172.21.15.53:0/4294484816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c50100f50 0x7f8c5019b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:36.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 -- 172.21.15.53:0/4294484816 >> 172.21.15.53:0/4294484816 conn(0x7f8c500fb2d0 msgr2=0x7f8c500fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:36.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 -- 172.21.15.53:0/4294484816 shutdown_connections 2023-11-06T01:32:36.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:36.941+0000 7f8c54f93700 1 -- 172.21.15.53:0/4294484816 wait complete. 2023-11-06T01:32:37.317 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:37 smithi053 ceph-mon[99472]: pgmap v257: 97 pgs: 97 active+clean; 622 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 26 op/s 2023-11-06T01:32:37.317 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:37 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/4294484816' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-06T01:32:37.317 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:37 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/4294484816' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-06T01:32:37.344 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:32:37.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-11-06T01:32:37.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-06T01:32:37.387 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-06T01:32:37.388 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-11-06T01:32:37.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph mgr dump --format=json 2023-11-06T01:32:37.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:37 smithi062 ceph-mon[103924]: pgmap v257: 97 pgs: 97 active+clean; 622 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 26 op/s 2023-11-06T01:32:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:37 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/4294484816' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-06T01:32:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:37 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/4294484816' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-06T01:32:37.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:37 smithi163 ceph-mon[104416]: pgmap v257: 97 pgs: 97 active+clean; 622 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 26 op/s 2023-11-06T01:32:37.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:37 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/4294484816' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-06T01:32:37.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:37 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/4294484816' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-06T01:32:37.984 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:39.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.324+0000 7f0e50529700 1 -- 172.21.15.53:0/275565870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e481019c0 msgr2=0x7f0e48105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:39.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.324+0000 7f0e50529700 1 --2- 172.21.15.53:0/275565870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e481019c0 0x7f0e48105a10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009990 tx=0x7f0e38009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:39.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.324+0000 7f0e50529700 1 -- 172.21.15.53:0/275565870 shutdown_connections 2023-11-06T01:32:39.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.324+0000 7f0e50529700 1 --2- 172.21.15.53:0/275565870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e481060f0 0x7f0e4810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:39.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.324+0000 7f0e50529700 1 --2- 172.21.15.53:0/275565870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e48101010 0x7f0e481013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:39.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.324+0000 7f0e50529700 1 --2- 172.21.15.53:0/275565870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e481019c0 0x7f0e48105a10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:39.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.324+0000 7f0e50529700 1 -- 172.21.15.53:0/275565870 >> 172.21.15.53:0/275565870 conn(0x7f0e480fc880 msgr2=0x7f0e480feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:39.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.324+0000 7f0e50529700 1 -- 172.21.15.53:0/275565870 shutdown_connections 2023-11-06T01:32:39.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.324+0000 7f0e50529700 1 -- 172.21.15.53:0/275565870 wait complete. 2023-11-06T01:32:39.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e50529700 1 Processor -- start 2023-11-06T01:32:39.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e50529700 1 -- start start 2023-11-06T01:32:39.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e50529700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e48101010 0x7f0e48073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:39.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e50529700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e481019c0 0x7f0e480735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:39.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e50529700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e481060f0 0x7f0e48073b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:39.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e50529700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e481a5290 con 0x7f0e481060f0 2023-11-06T01:32:39.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e50529700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e481a5400 con 0x7f0e48101010 2023-11-06T01:32:39.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e50529700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e481a55c0 con 0x7f0e481019c0 2023-11-06T01:32:39.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e4eac6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e481060f0 0x7f0e48073b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:39.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e4eac6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e481060f0 0x7f0e48073b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38204/0 (socket says 172.21.15.53:38204) 2023-11-06T01:32:39.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e4eac6700 1 -- 172.21.15.53:0/877956020 learned_addr learned my addr 172.21.15.53:0/877956020 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:39.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e4dac4700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e481019c0 0x7f0e480735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:39.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e4e2c5700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e48101010 0x7f0e48073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:39.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e4eac6700 1 -- 172.21.15.53:0/877956020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e481019c0 msgr2=0x7f0e480735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:39.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e4eac6700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e481019c0 0x7f0e480735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:39.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e4eac6700 1 -- 172.21.15.53:0/877956020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e48101010 msgr2=0x7f0e48073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:39.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e4eac6700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e48101010 0x7f0e48073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:39.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e4eac6700 1 -- 172.21.15.53:0/877956020 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e380095f0 con 0x7f0e481060f0 2023-11-06T01:32:39.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.325+0000 7f0e4e2c5700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e48101010 0x7f0e48073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:32:39.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.326+0000 7f0e4eac6700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e481060f0 0x7f0e48073b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000ada0 tx=0x7f0e4000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:39.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.326+0000 7f0e3f7fe700 1 -- 172.21.15.53:0/877956020 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4000f070 con 0x7f0e481060f0 2023-11-06T01:32:39.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.326+0000 7f0e50529700 1 -- 172.21.15.53:0/877956020 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e481a58a0 con 0x7f0e481060f0 2023-11-06T01:32:39.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.326+0000 7f0e3f7fe700 1 -- 172.21.15.53:0/877956020 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e40007b30 con 0x7f0e481060f0 2023-11-06T01:32:39.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.326+0000 7f0e3f7fe700 1 -- 172.21.15.53:0/877956020 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40005440 con 0x7f0e481060f0 2023-11-06T01:32:39.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.326+0000 7f0e50529700 1 -- 172.21.15.53:0/877956020 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e481a5da0 con 0x7f0e481060f0 2023-11-06T01:32:39.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.327+0000 7f0e50529700 1 -- 172.21.15.53:0/877956020 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4804ea90 con 0x7f0e481060f0 2023-11-06T01:32:39.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.328+0000 7f0e3f7fe700 1 -- 172.21.15.53:0/877956020 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0e40004ad0 con 0x7f0e481060f0 2023-11-06T01:32:39.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.328+0000 7f0e3f7fe700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e340a05b0 0x7f0e340a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:39.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.328+0000 7f0e4e2c5700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e340a05b0 0x7f0e340a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:39.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.328+0000 7f0e3f7fe700 1 -- 172.21.15.53:0/877956020 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0e400cafe0 con 0x7f0e481060f0 2023-11-06T01:32:39.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.329+0000 7f0e4e2c5700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e340a05b0 0x7f0e340a2a70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e44005fd0 tx=0x7f0e44005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:39.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.333+0000 7f0e3f7fe700 1 -- 172.21.15.53:0/877956020 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0e40094c60 con 0x7f0e481060f0 2023-11-06T01:32:39.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:39 smithi053 ceph-mon[99472]: pgmap v258: 97 pgs: 97 active+clean; 623 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 KiB/s wr, 27 op/s 2023-11-06T01:32:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:39 smithi062 ceph-mon[103924]: pgmap v258: 97 pgs: 97 active+clean; 623 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 KiB/s wr, 27 op/s 2023-11-06T01:32:39.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:39 smithi163 ceph-mon[104416]: pgmap v258: 97 pgs: 97 active+clean; 623 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 KiB/s wr, 27 op/s 2023-11-06T01:32:39.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.587+0000 7f0e50529700 1 -- 172.21.15.53:0/877956020 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0e481a5fc0 con 0x7f0e481060f0 2023-11-06T01:32:39.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.591+0000 7f0e3f7fe700 1 -- 172.21.15.53:0/877956020 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+258829 (secure 0 0 0) 0x7f0e40083090 con 0x7f0e481060f0 2023-11-06T01:32:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:39.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.594+0000 7f0e50529700 1 -- 172.21.15.53:0/877956020 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e340a05b0 msgr2=0x7f0e340a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:39.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.594+0000 7f0e50529700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e340a05b0 0x7f0e340a2a70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e44005fd0 tx=0x7f0e44005e20 comp rx=0 tx=0).stop 2023-11-06T01:32:39.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.594+0000 7f0e50529700 1 -- 172.21.15.53:0/877956020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e481060f0 msgr2=0x7f0e48073b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:39.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.594+0000 7f0e50529700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e481060f0 0x7f0e48073b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000ada0 tx=0x7f0e4000c330 comp rx=0 tx=0).stop 2023-11-06T01:32:39.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.594+0000 7f0e50529700 1 -- 172.21.15.53:0/877956020 shutdown_connections 2023-11-06T01:32:39.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.594+0000 7f0e50529700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0e340a05b0 0x7f0e340a2a70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:39.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.594+0000 7f0e50529700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e48101010 0x7f0e48073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:39.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.594+0000 7f0e50529700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e481060f0 0x7f0e48073b50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:39.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.594+0000 7f0e50529700 1 --2- 172.21.15.53:0/877956020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e481019c0 0x7f0e480735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:39.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.594+0000 7f0e50529700 1 -- 172.21.15.53:0/877956020 >> 172.21.15.53:0/877956020 conn(0x7f0e480fc880 msgr2=0x7f0e480fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:39.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.595+0000 7f0e50529700 1 -- 172.21.15.53:0/877956020 shutdown_connections 2023-11-06T01:32:39.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:39.595+0000 7f0e50529700 1 -- 172.21.15.53:0/877956020 wait complete. 2023-11-06T01:32:40.036 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1491433716},{"type":"v1","addr":"172.21.15.53:6801","nonce":1491433716}]},"active_addr":"172.21.15.53:6801/1491433716","active_change":"2023-11-06T01:26:25.398279+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.53:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":4069330909}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":1030194757}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":439783877}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":2297095197}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":2744828136}]}]} 2023-11-06T01:32:40.042 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-11-06T01:32:40.042 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-11-06T01:32:40.043 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd dump --format=json 2023-11-06T01:32:40.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:40 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/877956020' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-06T01:32:40.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:40 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/877956020' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-06T01:32:40.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:40 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/877956020' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-06T01:32:40.604 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:41.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:41 smithi053 ceph-mon[99472]: pgmap v259: 97 pgs: 97 active+clean; 623 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2023-11-06T01:32:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:41 smithi062 ceph-mon[103924]: pgmap v259: 97 pgs: 97 active+clean; 623 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2023-11-06T01:32:41.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:41 smithi163 ceph-mon[104416]: pgmap v259: 97 pgs: 97 active+clean; 623 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2023-11-06T01:32:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.843+0000 7fb929af8700 1 -- 172.21.15.53:0/239184629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb924105d80 msgr2=0x7fb924106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:41.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.843+0000 7fb929af8700 1 --2- 172.21.15.53:0/239184629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb924105d80 0x7fb924106160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb914009990 tx=0x7fb914009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.843+0000 7fb929af8700 1 -- 172.21.15.53:0/239184629 shutdown_connections 2023-11-06T01:32:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.843+0000 7fb929af8700 1 --2- 172.21.15.53:0/239184629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb924100690 0x7fb92410cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.843+0000 7fb929af8700 1 --2- 172.21.15.53:0/239184629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb924105d80 0x7fb924106160 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:41.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.843+0000 7fb929af8700 1 --2- 172.21.15.53:0/239184629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9240ffcd0 0x7fb924100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:41.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.843+0000 7fb929af8700 1 -- 172.21.15.53:0/239184629 >> 172.21.15.53:0/239184629 conn(0x7fb924076d90 msgr2=0x7fb9240771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:41.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.843+0000 7fb929af8700 1 -- 172.21.15.53:0/239184629 shutdown_connections 2023-11-06T01:32:41.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.843+0000 7fb929af8700 1 -- 172.21.15.53:0/239184629 wait complete. 2023-11-06T01:32:41.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.844+0000 7fb929af8700 1 Processor -- start 2023-11-06T01:32:41.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.844+0000 7fb929af8700 1 -- start start 2023-11-06T01:32:41.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb929af8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9240ffcd0 0x7fb92419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:41.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb929af8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb924100690 0x7fb92419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:41.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb929af8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb924105d80 0x7fb924195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:41.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb929af8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9241961c0 con 0x7fb924100690 2023-11-06T01:32:41.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb929af8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb924196330 con 0x7fb924105d80 2023-11-06T01:32:41.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb929af8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9241964f0 con 0x7fb9240ffcd0 2023-11-06T01:32:41.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb922ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb924100690 0x7fb92419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:41.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb922ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb924100690 0x7fb92419c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38214/0 (socket says 172.21.15.53:38214) 2023-11-06T01:32:41.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb922ffd700 1 -- 172.21.15.53:0/1941409339 learned_addr learned my addr 172.21.15.53:0/1941409339 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:41.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb923fff700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb924105d80 0x7fb924195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:41.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb9237fe700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9240ffcd0 0x7fb92419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:41.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb922ffd700 1 -- 172.21.15.53:0/1941409339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9240ffcd0 msgr2=0x7fb92419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:41.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb922ffd700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9240ffcd0 0x7fb92419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:41.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb922ffd700 1 -- 172.21.15.53:0/1941409339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb924105d80 msgr2=0x7fb924195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:41.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb922ffd700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb924105d80 0x7fb924195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:41.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb922ffd700 1 -- 172.21.15.53:0/1941409339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9140095f0 con 0x7fb924100690 2023-11-06T01:32:41.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb923fff700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb924105d80 0x7fb924195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:41.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb922ffd700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb924100690 0x7fb92419c010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00b530 tx=0x7fb90c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:41.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb920ff9700 1 -- 172.21.15.53:0/1941409339 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c011830 con 0x7fb924100690 2023-11-06T01:32:41.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb920ff9700 1 -- 172.21.15.53:0/1941409339 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb90c011e50 con 0x7fb924100690 2023-11-06T01:32:41.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb920ff9700 1 -- 172.21.15.53:0/1941409339 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c00f2e0 con 0x7fb924100690 2023-11-06T01:32:41.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.845+0000 7fb929af8700 1 -- 172.21.15.53:0/1941409339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9241967d0 con 0x7fb924100690 2023-11-06T01:32:41.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.846+0000 7fb929af8700 1 -- 172.21.15.53:0/1941409339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9241a2ac0 con 0x7fb924100690 2023-11-06T01:32:41.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.847+0000 7fb929af8700 1 -- 172.21.15.53:0/1941409339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb92404ea90 con 0x7fb924100690 2023-11-06T01:32:41.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.848+0000 7fb920ff9700 1 -- 172.21.15.53:0/1941409339 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fb90c010580 con 0x7fb924100690 2023-11-06T01:32:41.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.848+0000 7fb920ff9700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9100a0630 0x7fb9100a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:41.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.848+0000 7fb920ff9700 1 -- 172.21.15.53:0/1941409339 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb90c0cb870 con 0x7fb924100690 2023-11-06T01:32:41.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.848+0000 7fb9237fe700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9100a0630 0x7fb9100a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:41.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.849+0000 7fb9237fe700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9100a0630 0x7fb9100a2af0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb914006010 tx=0x7fb91400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:41.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:41.852+0000 7fb920ff9700 1 -- 172.21.15.53:0/1941409339 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb90c095570 con 0x7fb924100690 2023-11-06T01:32:42.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.053+0000 7fb929af8700 1 -- 172.21.15.53:0/1941409339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb924196ff0 con 0x7fb924100690 2023-11-06T01:32:42.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.055+0000 7fb920ff9700 1 -- 172.21.15.53:0/1941409339 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22071 (secure 0 0 0) 0x7fb90c015090 con 0x7fb924100690 2023-11-06T01:32:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:42.057 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:32:27.094591+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:32:42.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 -- 172.21.15.53:0/1941409339 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9100a0630 msgr2=0x7fb9100a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:42.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9100a0630 0x7fb9100a2af0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb914006010 tx=0x7fb91400b540 comp rx=0 tx=0).stop 2023-11-06T01:32:42.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 -- 172.21.15.53:0/1941409339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb924100690 msgr2=0x7fb92419c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:42.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb924100690 0x7fb92419c010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00b530 tx=0x7fb90c00b8f0 comp rx=0 tx=0).stop 2023-11-06T01:32:42.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 -- 172.21.15.53:0/1941409339 shutdown_connections 2023-11-06T01:32:42.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb9100a0630 0x7fb9100a2af0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:42.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb924105d80 0x7fb924195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:42.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9240ffcd0 0x7fb92419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:42.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 --2- 172.21.15.53:0/1941409339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb924100690 0x7fb92419c010 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:42.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 -- 172.21.15.53:0/1941409339 >> 172.21.15.53:0/1941409339 conn(0x7fb924076d90 msgr2=0x7fb9240fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:42.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 -- 172.21.15.53:0/1941409339 shutdown_connections 2023-11-06T01:32:42.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:42.057+0000 7fb929af8700 1 -- 172.21.15.53:0/1941409339 wait complete. 2023-11-06T01:32:42.454 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-11-06T01:32:42.455 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd dump --format=json 2023-11-06T01:32:43.016 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:43.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:43 smithi053 ceph-mon[99472]: pgmap v260: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2023-11-06T01:32:43.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:43 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1941409339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:43 smithi062 ceph-mon[103924]: pgmap v260: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2023-11-06T01:32:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:43 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1941409339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:43.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:43 smithi163 ceph-mon[104416]: pgmap v260: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2023-11-06T01:32:43.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:43 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1941409339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:44.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.593+0000 7f99d4286700 1 -- 172.21.15.53:0/46424538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99cc1022b0 msgr2=0x7f99cc10cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:44.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.593+0000 7f99d4286700 1 --2- 172.21.15.53:0/46424538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99cc1022b0 0x7f99cc10cc90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f99c400b0d0 tx=0x7f99c400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.593+0000 7f99d4286700 1 -- 172.21.15.53:0/46424538 shutdown_connections 2023-11-06T01:32:44.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.593+0000 7f99d4286700 1 --2- 172.21.15.53:0/46424538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99cc100010 0x7f99cc1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.593+0000 7f99d4286700 1 --2- 172.21.15.53:0/46424538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99cc1022b0 0x7f99cc10cc90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.593+0000 7f99d4286700 1 --2- 172.21.15.53:0/46424538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99cc1018f0 0x7f99cc101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.593+0000 7f99d4286700 1 -- 172.21.15.53:0/46424538 >> 172.21.15.53:0/46424538 conn(0x7f99cc076d80 msgr2=0x7f99cc077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.593+0000 7f99d4286700 1 -- 172.21.15.53:0/46424538 shutdown_connections 2023-11-06T01:32:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.593+0000 7f99d4286700 1 -- 172.21.15.53:0/46424538 wait complete. 2023-11-06T01:32:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d4286700 1 Processor -- start 2023-11-06T01:32:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d4286700 1 -- start start 2023-11-06T01:32:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d4286700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99cc100010 0x7f99cc197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d4286700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99cc1018f0 0x7f99cc197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d4286700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99cc1022b0 0x7f99cc19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d4286700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc19c010 con 0x7f99cc1018f0 2023-11-06T01:32:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d4286700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc19c180 con 0x7f99cc1022b0 2023-11-06T01:32:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d4286700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc19c340 con 0x7f99cc100010 2023-11-06T01:32:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d1821700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99cc1018f0 0x7f99cc197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d1821700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99cc1018f0 0x7f99cc197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38226/0 (socket says 172.21.15.53:38226) 2023-11-06T01:32:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d1821700 1 -- 172.21.15.53:0/1139720897 learned_addr learned my addr 172.21.15.53:0/1139720897 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d2022700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99cc100010 0x7f99cc197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d2823700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99cc1022b0 0x7f99cc19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d1821700 1 -- 172.21.15.53:0/1139720897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99cc100010 msgr2=0x7f99cc197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d1821700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99cc100010 0x7f99cc197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d1821700 1 -- 172.21.15.53:0/1139720897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99cc1022b0 msgr2=0x7f99cc19ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d1821700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99cc1022b0 0x7f99cc19ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d1821700 1 -- 172.21.15.53:0/1139720897 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4009d00 con 0x7f99cc1018f0 2023-11-06T01:32:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d2022700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99cc100010 0x7f99cc197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.594+0000 7f99d2823700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99cc1022b0 0x7f99cc19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:32:44.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.595+0000 7f99d1821700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99cc1018f0 0x7f99cc197590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c920 tx=0x7f99c800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:44.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.595+0000 7f99c37fe700 1 -- 172.21.15.53:0/1139720897 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8007b00 con 0x7f99cc1018f0 2023-11-06T01:32:44.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.595+0000 7f99c37fe700 1 -- 172.21.15.53:0/1139720897 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99c8004cf0 con 0x7f99cc1018f0 2023-11-06T01:32:44.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.595+0000 7f99d4286700 1 -- 172.21.15.53:0/1139720897 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc19c620 con 0x7f99cc1018f0 2023-11-06T01:32:44.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.595+0000 7f99c37fe700 1 -- 172.21.15.53:0/1139720897 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8005610 con 0x7f99cc1018f0 2023-11-06T01:32:44.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.595+0000 7f99d4286700 1 -- 172.21.15.53:0/1139720897 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc1a3e80 con 0x7f99cc1018f0 2023-11-06T01:32:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.596+0000 7f99d4286700 1 -- 172.21.15.53:0/1139720897 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc04ea90 con 0x7f99cc1018f0 2023-11-06T01:32:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.597+0000 7f99c37fe700 1 -- 172.21.15.53:0/1139720897 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f99c8004750 con 0x7f99cc1018f0 2023-11-06T01:32:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.598+0000 7f99c37fe700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f99b80a05b0 0x7f99b80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.598+0000 7f99d2022700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f99b80a05b0 0x7f99b80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.598+0000 7f99c37fe700 1 -- 172.21.15.53:0/1139720897 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f99c80cb960 con 0x7f99cc1018f0 2023-11-06T01:32:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.598+0000 7f99d2022700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f99b80a05b0 0x7f99b80a2a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005950 tx=0x7f99bc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.602+0000 7f99c37fe700 1 -- 172.21.15.53:0/1139720897 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f99c8095550 con 0x7f99cc1018f0 2023-11-06T01:32:44.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.802+0000 7f99d4286700 1 -- 172.21.15.53:0/1139720897 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99cc1983b0 con 0x7f99cc1018f0 2023-11-06T01:32:44.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.803+0000 7f99c37fe700 1 -- 172.21.15.53:0/1139720897 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f99c801d090 con 0x7f99cc1018f0 2023-11-06T01:32:44.805 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:32:44.806 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:32:27.094591+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:32:44.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.805+0000 7f99d4286700 1 -- 172.21.15.53:0/1139720897 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f99b80a05b0 msgr2=0x7f99b80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:44.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.805+0000 7f99d4286700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f99b80a05b0 0x7f99b80a2a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005950 tx=0x7f99bc0058e0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.805+0000 7f99d4286700 1 -- 172.21.15.53:0/1139720897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99cc1018f0 msgr2=0x7f99cc197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.805+0000 7f99d4286700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99cc1018f0 0x7f99cc197590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c920 tx=0x7f99c800cce0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.805+0000 7f99d4286700 1 -- 172.21.15.53:0/1139720897 shutdown_connections 2023-11-06T01:32:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.805+0000 7f99d4286700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f99b80a05b0 0x7f99b80a2a70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.805+0000 7f99d4286700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99cc1022b0 0x7f99cc19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.805+0000 7f99d4286700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99cc100010 0x7f99cc197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.805+0000 7f99d4286700 1 --2- 172.21.15.53:0/1139720897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99cc1018f0 0x7f99cc197590 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:44.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.805+0000 7f99d4286700 1 -- 172.21.15.53:0/1139720897 >> 172.21.15.53:0/1139720897 conn(0x7f99cc076d80 msgr2=0x7f99cc0690c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:44.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.806+0000 7f99d4286700 1 -- 172.21.15.53:0/1139720897 shutdown_connections 2023-11-06T01:32:44.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:44.806+0000 7f99d4286700 1 -- 172.21.15.53:0/1139720897 wait complete. 2023-11-06T01:32:45.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:45 smithi053 ceph-mon[99472]: pgmap v261: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.8 KiB/s wr, 22 op/s 2023-11-06T01:32:45.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:45 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1139720897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:45.279 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-11-06T01:32:45.279 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-11-06T01:32:45.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-11-06T01:32:45.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-11-06T01:32:45.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-11-06T01:32:45.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-11-06T01:32:45.282 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-11-06T01:32:45.283 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-11-06T01:32:45.283 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-11-06T01:32:45.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-11-06T01:32:45.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-11-06T01:32:45.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-11-06T01:32:45.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:45 smithi062 ceph-mon[103924]: pgmap v261: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.8 KiB/s wr, 22 op/s 2023-11-06T01:32:45.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:45 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1139720897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:45.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:45 smithi163 ceph-mon[104416]: pgmap v261: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.8 KiB/s wr, 22 op/s 2023-11-06T01:32:45.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:45 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1139720897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:32:46.117 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:46.223 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:46.301 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:46.302 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:46.328 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:46.328 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:46.511 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:46.704 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:47.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:47 smithi053 ceph-mon[99472]: pgmap v262: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 22 op/s 2023-11-06T01:32:47.143 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:47.145 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:47.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:47 smithi062 ceph-mon[103924]: pgmap v262: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 22 op/s 2023-11-06T01:32:47.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:47 smithi163 ceph-mon[104416]: pgmap v262: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 22 op/s 2023-11-06T01:32:48.146 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:49.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:49 smithi053 ceph-mon[99472]: pgmap v263: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-11-06T01:32:49.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:49 smithi062 ceph-mon[103924]: pgmap v263: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-11-06T01:32:49.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:49 smithi163 ceph-mon[104416]: pgmap v263: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-11-06T01:32:51.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:51 smithi053 ceph-mon[99472]: pgmap v264: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-11-06T01:32:51.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:51 smithi062 ceph-mon[103924]: pgmap v264: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-11-06T01:32:51.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:51 smithi163 ceph-mon[104416]: pgmap v264: 97 pgs: 97 active+clean; 634 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-11-06T01:32:52.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.565+0000 7f68738d3700 1 -- 172.21.15.53:0/4007453701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f686c0688b0 msgr2=0x7f686c068d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.565+0000 7f68738d3700 1 --2- 172.21.15.53:0/4007453701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f686c0688b0 0x7f686c068d30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f685c009990 tx=0x7f685c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.565+0000 7f68738d3700 1 -- 172.21.15.53:0/4007453701 shutdown_connections 2023-11-06T01:32:52.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.565+0000 7f68738d3700 1 --2- 172.21.15.53:0/4007453701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c069270 0x7f686c10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.565+0000 7f68738d3700 1 --2- 172.21.15.53:0/4007453701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f686c106f20 0x7f686c107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.565+0000 7f68738d3700 1 --2- 172.21.15.53:0/4007453701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f686c0688b0 0x7f686c068d30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.565+0000 7f68738d3700 1 -- 172.21.15.53:0/4007453701 >> 172.21.15.53:0/4007453701 conn(0x7f686c076d90 msgr2=0x7f686c0771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:52.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.565+0000 7f68738d3700 1 -- 172.21.15.53:0/4007453701 shutdown_connections 2023-11-06T01:32:52.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.565+0000 7f68738d3700 1 -- 172.21.15.53:0/4007453701 wait complete. 2023-11-06T01:32:52.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f68738d3700 1 Processor -- start 2023-11-06T01:32:52.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f68738d3700 1 -- start start 2023-11-06T01:32:52.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f68738d3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f686c0688b0 0x7f686c196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:52.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f68738d3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f686c069270 0x7f686c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:52.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f68738d3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c106f20 0x7f686c19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:52.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f68738d3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c19bdd0 con 0x7f686c0688b0 2023-11-06T01:32:52.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f68738d3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c19bf40 con 0x7f686c106f20 2023-11-06T01:32:52.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f68738d3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c19c100 con 0x7f686c069270 2023-11-06T01:32:52.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f687166f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f686c0688b0 0x7f686c196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:52.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f687166f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f686c0688b0 0x7f686c196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38234/0 (socket says 172.21.15.53:38234) 2023-11-06T01:32:52.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f687166f700 1 -- 172.21.15.53:0/1415121690 learned_addr learned my addr 172.21.15.53:0/1415121690 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:52.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f6870e6e700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f686c069270 0x7f686c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:52.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.566+0000 7f6871e70700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c106f20 0x7f686c19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:52.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.567+0000 7f687166f700 1 -- 172.21.15.53:0/1415121690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f686c069270 msgr2=0x7f686c197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.567+0000 7f687166f700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f686c069270 0x7f686c197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.567+0000 7f687166f700 1 -- 172.21.15.53:0/1415121690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c106f20 msgr2=0x7f686c19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.567+0000 7f687166f700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c106f20 0x7f686c19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.567+0000 7f687166f700 1 -- 172.21.15.53:0/1415121690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0095f0 con 0x7f686c0688b0 2023-11-06T01:32:52.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.567+0000 7f687166f700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f686c0688b0 0x7f686c196e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f686800d950 tx=0x7f686800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:52.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.567+0000 7f68627fc700 1 -- 172.21.15.53:0/1415121690 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800f890 con 0x7f686c0688b0 2023-11-06T01:32:52.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.567+0000 7f68627fc700 1 -- 172.21.15.53:0/1415121690 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6868010460 con 0x7f686c0688b0 2023-11-06T01:32:52.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.567+0000 7f68738d3700 1 -- 172.21.15.53:0/1415121690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c19c3e0 con 0x7f686c0688b0 2023-11-06T01:32:52.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.567+0000 7f68627fc700 1 -- 172.21.15.53:0/1415121690 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800e610 con 0x7f686c0688b0 2023-11-06T01:32:52.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.567+0000 7f68738d3700 1 -- 172.21.15.53:0/1415121690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c1a3c40 con 0x7f686c0688b0 2023-11-06T01:32:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.568+0000 7f68738d3700 1 -- 172.21.15.53:0/1415121690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f686c110ff0 con 0x7f686c0688b0 2023-11-06T01:32:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.569+0000 7f68627fc700 1 -- 172.21.15.53:0/1415121690 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f68680108d0 con 0x7f686c0688b0 2023-11-06T01:32:52.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.570+0000 7f68627fc700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68580a0630 0x7f68580a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:52.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.570+0000 7f6870e6e700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68580a0630 0x7f68580a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:52.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.570+0000 7f68627fc700 1 -- 172.21.15.53:0/1415121690 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f68680cbca0 con 0x7f686c0688b0 2023-11-06T01:32:52.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.570+0000 7f68627fc700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861] conn(0x7f68580a9e10 0x7f68580ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:52.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.570+0000 7f68627fc700 1 -- 172.21.15.53:0/1415121690 --> [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68580ac8e0 con 0x7f68580a9e10 2023-11-06T01:32:52.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.570+0000 7f68627fc700 1 -- 172.21.15.53:0/1415121690 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68680cc010 con 0x7f686c0688b0 2023-11-06T01:32:52.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.570+0000 7f6870e6e700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68580a0630 0x7f68580a2af0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f686c198430 tx=0x7f685c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:52.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.570+0000 7f6871e70700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861] conn(0x7f68580a9e10 0x7f68580ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:52.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.571+0000 7f6871e70700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861] conn(0x7f68580a9e10 0x7f68580ac230 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:52.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.571+0000 7f68627fc700 1 -- 172.21.15.53:0/1415121690 <== osd.10 v2:172.21.15.62:6824/1978188861 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f68580ac8e0 con 0x7f68580a9e10 2023-11-06T01:32:52.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.587+0000 7f68738d3700 1 -- 172.21.15.53:0/1415121690 --> [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f686c04ea90 con 0x7f68580a9e10 2023-11-06T01:32:52.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f68627fc700 1 -- 172.21.15.53:0/1415121690 <== osd.10 v2:172.21.15.62:6824/1978188861 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f686c04ea90 con 0x7f68580a9e10 2023-11-06T01:32:52.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f68738d3700 1 -- 172.21.15.53:0/1415121690 >> [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861] conn(0x7f68580a9e10 msgr2=0x7f68580ac230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f68738d3700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861] conn(0x7f68580a9e10 0x7f68580ac230 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f68738d3700 1 -- 172.21.15.53:0/1415121690 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68580a0630 msgr2=0x7f68580a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f68738d3700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68580a0630 0x7f68580a2af0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f686c198430 tx=0x7f685c005fd0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f68738d3700 1 -- 172.21.15.53:0/1415121690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f686c0688b0 msgr2=0x7f686c196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f68738d3700 1 --2- 172.21.15.53:0/1415121690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f686c0688b0 0x7f686c196e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f686800d950 tx=0x7f686800dc60 comp rx=0 tx=0).stop 2023-11-06T01:32:52.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f6871e70700 1 -- 172.21.15.53:0/1415121690 reap_dead start 2023-11-06T01:32:52.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f68738d3700 1 -- 172.21.15.53:0/1415121690 shutdown_connections 2023-11-06T01:32:52.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f68738d3700 1 -- 172.21.15.53:0/1415121690 >> 172.21.15.53:0/1415121690 conn(0x7f686c076d90 msgr2=0x7f686c0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:52.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f68738d3700 1 -- 172.21.15.53:0/1415121690 shutdown_connections 2023-11-06T01:32:52.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.588+0000 7f68738d3700 1 -- 172.21.15.53:0/1415121690 wait complete. 2023-11-06T01:32:52.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.808+0000 7f079de0c700 1 -- 172.21.15.53:0/3189548106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07981007e0 msgr2=0x7f0798100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.808+0000 7f079de0c700 1 --2- 172.21.15.53:0/3189548106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07981007e0 0x7f0798100bc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0788009990 tx=0x7f0788009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.808+0000 7f079de0c700 1 -- 172.21.15.53:0/3189548106 shutdown_connections 2023-11-06T01:32:52.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.808+0000 7f079de0c700 1 --2- 172.21.15.53:0/3189548106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f079810d820 0x7f079810fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.808+0000 7f079de0c700 1 --2- 172.21.15.53:0/3189548106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07981007e0 0x7f0798100bc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.808+0000 7f079de0c700 1 --2- 172.21.15.53:0/3189548106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0798101100 0x7f079810d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.808+0000 7f079de0c700 1 -- 172.21.15.53:0/3189548106 >> 172.21.15.53:0/3189548106 conn(0x7f0798076d80 msgr2=0x7f0798077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:52.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.808+0000 7f079de0c700 1 -- 172.21.15.53:0/3189548106 shutdown_connections 2023-11-06T01:32:52.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.808+0000 7f079de0c700 1 -- 172.21.15.53:0/3189548106 wait complete. 2023-11-06T01:32:52.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f079de0c700 1 Processor -- start 2023-11-06T01:32:52.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f079de0c700 1 -- start start 2023-11-06T01:32:52.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f079de0c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07981007e0 0x7f07981992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:52.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f079de0c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0798101100 0x7f07981997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:52.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f079de0c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f079810d820 0x7f079819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:52.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f079de0c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079819e270 con 0x7f0798101100 2023-11-06T01:32:52.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f079de0c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079819e3e0 con 0x7f07981007e0 2023-11-06T01:32:52.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f079de0c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079819e5a0 con 0x7f079810d820 2023-11-06T01:32:52.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f0796ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0798101100 0x7f07981997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:52.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f0796ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0798101100 0x7f07981997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38252/0 (socket says 172.21.15.53:38252) 2023-11-06T01:32:52.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f0796ffd700 1 -- 172.21.15.53:0/1219975499 learned_addr learned my addr 172.21.15.53:0/1219975499 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:52.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.809+0000 7f07977fe700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07981007e0 0x7f07981992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:52.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f0797fff700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f079810d820 0x7f079819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:52.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f0796ffd700 1 -- 172.21.15.53:0/1219975499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f079810d820 msgr2=0x7f079819dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f0796ffd700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f079810d820 0x7f079819dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f0796ffd700 1 -- 172.21.15.53:0/1219975499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07981007e0 msgr2=0x7f07981992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f0796ffd700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07981007e0 0x7f07981992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f0796ffd700 1 -- 172.21.15.53:0/1219975499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880095f0 con 0x7f0798101100 2023-11-06T01:32:52.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f07977fe700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07981007e0 0x7f07981992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:52.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f0796ffd700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0798101100 0x7f07981997f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f078c00d950 tx=0x7f078c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:52.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f0794ff9700 1 -- 172.21.15.53:0/1219975499 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c009b40 con 0x7f0798101100 2023-11-06T01:32:52.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f079de0c700 1 -- 172.21.15.53:0/1219975499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079819e880 con 0x7f0798101100 2023-11-06T01:32:52.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f0794ff9700 1 -- 172.21.15.53:0/1219975499 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f078c009ca0 con 0x7f0798101100 2023-11-06T01:32:52.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f0794ff9700 1 -- 172.21.15.53:0/1219975499 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00f620 con 0x7f0798101100 2023-11-06T01:32:52.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.810+0000 7f079de0c700 1 -- 172.21.15.53:0/1219975499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07981a60e0 con 0x7f0798101100 2023-11-06T01:32:52.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.811+0000 7f079de0c700 1 -- 172.21.15.53:0/1219975499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0798113370 con 0x7f0798101100 2023-11-06T01:32:52.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.812+0000 7f0794ff9700 1 -- 172.21.15.53:0/1219975499 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f078c010460 con 0x7f0798101100 2023-11-06T01:32:52.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.813+0000 7f0794ff9700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f07800a05b0 0x7f07800a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.813+0000 7f07977fe700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f07800a05b0 0x7f07800a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.813+0000 7f0794ff9700 1 -- 172.21.15.53:0/1219975499 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f078c0cbf80 con 0x7f0798101100 2023-11-06T01:32:52.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.813+0000 7f0794ff9700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718] conn(0x7f07800a9d90 0x7f07800ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:52.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.813+0000 7f0794ff9700 1 -- 172.21.15.53:0/1219975499 --> [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07800ac860 con 0x7f07800a9d90 2023-11-06T01:32:52.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.813+0000 7f0794ff9700 1 -- 172.21.15.53:0/1219975499 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078c0cc420 con 0x7f0798101100 2023-11-06T01:32:52.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.813+0000 7f07977fe700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f07800a05b0 0x7f07800a2a70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f078800b5c0 tx=0x7f0788005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:52.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.814+0000 7f0797fff700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718] conn(0x7f07800a9d90 0x7f07800ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:52.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.814+0000 7f0797fff700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718] conn(0x7f07800a9d90 0x7f07800ac1b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:52.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.815+0000 7f0794ff9700 1 -- 172.21.15.53:0/1219975499 <== osd.4 v2:172.21.15.62:6808/1956539718 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f07800ac860 con 0x7f07800a9d90 2023-11-06T01:32:52.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.831+0000 7f079de0c700 1 -- 172.21.15.53:0/1219975499 --> [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f079804ea90 con 0x7f07800a9d90 2023-11-06T01:32:52.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.831+0000 7f0794ff9700 1 -- 172.21.15.53:0/1219975499 <== osd.4 v2:172.21.15.62:6808/1956539718 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f079804ea90 con 0x7f07800a9d90 2023-11-06T01:32:52.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.831+0000 7f079de0c700 1 -- 172.21.15.53:0/1219975499 >> [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718] conn(0x7f07800a9d90 msgr2=0x7f07800ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.831+0000 7f079de0c700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718] conn(0x7f07800a9d90 0x7f07800ac1b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.831+0000 7f079de0c700 1 -- 172.21.15.53:0/1219975499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f07800a05b0 msgr2=0x7f07800a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.831+0000 7f079de0c700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f07800a05b0 0x7f07800a2a70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f078800b5c0 tx=0x7f0788005fb0 comp rx=0 tx=0).stop 2023-11-06T01:32:52.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.831+0000 7f079de0c700 1 -- 172.21.15.53:0/1219975499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0798101100 msgr2=0x7f07981997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:52.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.831+0000 7f079de0c700 1 --2- 172.21.15.53:0/1219975499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0798101100 0x7f07981997f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f078c00d950 tx=0x7f078c00dd10 comp rx=0 tx=0).stop 2023-11-06T01:32:52.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.831+0000 7f0797fff700 1 -- 172.21.15.53:0/1219975499 reap_dead start 2023-11-06T01:32:52.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.832+0000 7f079de0c700 1 -- 172.21.15.53:0/1219975499 shutdown_connections 2023-11-06T01:32:52.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.832+0000 7f079de0c700 1 -- 172.21.15.53:0/1219975499 >> 172.21.15.53:0/1219975499 conn(0x7f0798076d80 msgr2=0x7f079810eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:52.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.832+0000 7f079de0c700 1 -- 172.21.15.53:0/1219975499 shutdown_connections 2023-11-06T01:32:52.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:52.832+0000 7f079de0c700 1 -- 172.21.15.53:0/1219975499 wait complete. 2023-11-06T01:32:53.043 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:53 smithi053 ceph-mon[99472]: pgmap v265: 97 pgs: 97 active+clean; 636 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-11-06T01:32:53.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.362+0000 7f1d73b5d700 1 -- 172.21.15.53:0/2964181394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d6c105d80 msgr2=0x7f1d6c106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.362+0000 7f1d73b5d700 1 --2- 172.21.15.53:0/2964181394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d6c105d80 0x7f1d6c106160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009990 tx=0x7f1d68009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.362+0000 7f1d73b5d700 1 -- 172.21.15.53:0/2964181394 shutdown_connections 2023-11-06T01:32:53.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.362+0000 7f1d73b5d700 1 --2- 172.21.15.53:0/2964181394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6c100690 0x7f1d6c10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.362+0000 7f1d73b5d700 1 --2- 172.21.15.53:0/2964181394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d6c105d80 0x7f1d6c106160 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.362+0000 7f1d73b5d700 1 --2- 172.21.15.53:0/2964181394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d6c0ffcd0 0x7f1d6c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.362+0000 7f1d73b5d700 1 -- 172.21.15.53:0/2964181394 >> 172.21.15.53:0/2964181394 conn(0x7f1d6c076d90 msgr2=0x7f1d6c0771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:53.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.362+0000 7f1d73b5d700 1 -- 172.21.15.53:0/2964181394 shutdown_connections 2023-11-06T01:32:53.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.362+0000 7f1d73b5d700 1 -- 172.21.15.53:0/2964181394 wait complete. 2023-11-06T01:32:53.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.363+0000 7f1d73b5d700 1 Processor -- start 2023-11-06T01:32:53.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.363+0000 7f1d73b5d700 1 -- start start 2023-11-06T01:32:53.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d73b5d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6c0ffcd0 0x7f1d6c196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d73b5d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d6c100690 0x7f1d6c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d73b5d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d6c105d80 0x7f1d6c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d73b5d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c19be20 con 0x7f1d6c105d80 2023-11-06T01:32:53.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d73b5d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c19bf90 con 0x7f1d6c0ffcd0 2023-11-06T01:32:53.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d73b5d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c19c150 con 0x7f1d6c100690 2023-11-06T01:32:53.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d720fa700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d6c105d80 0x7f1d6c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d720fa700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d6c105d80 0x7f1d6c19b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38270/0 (socket says 172.21.15.53:38270) 2023-11-06T01:32:53.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d720fa700 1 -- 172.21.15.53:0/1459039520 learned_addr learned my addr 172.21.15.53:0/1459039520 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:53.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d710f8700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d6c100690 0x7f1d6c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d718f9700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6c0ffcd0 0x7f1d6c196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d720fa700 1 -- 172.21.15.53:0/1459039520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d6c100690 msgr2=0x7f1d6c1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d720fa700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d6c100690 0x7f1d6c1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d720fa700 1 -- 172.21.15.53:0/1459039520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6c0ffcd0 msgr2=0x7f1d6c196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d720fa700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6c0ffcd0 0x7f1d6c196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d720fa700 1 -- 172.21.15.53:0/1459039520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d680095f0 con 0x7f1d6c105d80 2023-11-06T01:32:53.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.364+0000 7f1d720fa700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d6c105d80 0x7f1d6c19b810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400b6d0 tx=0x7f1d6400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:53.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.365+0000 7f1d62ffd700 1 -- 172.21.15.53:0/1459039520 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400e070 con 0x7f1d6c105d80 2023-11-06T01:32:53.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.365+0000 7f1d62ffd700 1 -- 172.21.15.53:0/1459039520 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d64009820 con 0x7f1d6c105d80 2023-11-06T01:32:53.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.365+0000 7f1d73b5d700 1 -- 172.21.15.53:0/1459039520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6c19c430 con 0x7f1d6c105d80 2023-11-06T01:32:53.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.365+0000 7f1d62ffd700 1 -- 172.21.15.53:0/1459039520 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64012b30 con 0x7f1d6c105d80 2023-11-06T01:32:53.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.365+0000 7f1d73b5d700 1 -- 172.21.15.53:0/1459039520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c10d400 con 0x7f1d6c105d80 2023-11-06T01:32:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.365+0000 7f1d73b5d700 1 -- 172.21.15.53:0/1459039520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d6c04ea90 con 0x7f1d6c105d80 2023-11-06T01:32:53.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.367+0000 7f1d62ffd700 1 -- 172.21.15.53:0/1459039520 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f1d64011d60 con 0x7f1d6c105d80 2023-11-06T01:32:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.367+0000 7f1d62ffd700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1d580a0630 0x7f1d580a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.367+0000 7f1d62ffd700 1 -- 172.21.15.53:0/1459039520 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1d640cbf40 con 0x7f1d6c105d80 2023-11-06T01:32:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.367+0000 7f1d718f9700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1d580a0630 0x7f1d580a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.368+0000 7f1d62ffd700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439] conn(0x7f1d580a9e10 0x7f1d580ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.368+0000 7f1d62ffd700 1 -- 172.21.15.53:0/1459039520 --> [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d580ac8e0 con 0x7f1d580a9e10 2023-11-06T01:32:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.368+0000 7f1d62ffd700 1 -- 172.21.15.53:0/1459039520 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d640cc3e0 con 0x7f1d6c105d80 2023-11-06T01:32:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.368+0000 7f1d718f9700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1d580a0630 0x7f1d580a2af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1d68006010 tx=0x7f1d68005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.368+0000 7f1d710f8700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439] conn(0x7f1d580a9e10 0x7f1d580ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.368+0000 7f1d710f8700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439] conn(0x7f1d580a9e10 0x7f1d580ac230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.369+0000 7f1d62ffd700 1 -- 172.21.15.53:0/1459039520 <== osd.5 v2:172.21.15.163:6808/464090439 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1d580ac8e0 con 0x7f1d580a9e10 2023-11-06T01:32:53.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.385+0000 7f1d73b5d700 1 -- 172.21.15.53:0/1459039520 --> [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1d6c111090 con 0x7f1d580a9e10 2023-11-06T01:32:53.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.386+0000 7f1d62ffd700 1 -- 172.21.15.53:0/1459039520 <== osd.5 v2:172.21.15.163:6808/464090439 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1d5c007cf0 con 0x7f1d580a9e10 2023-11-06T01:32:53.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.386+0000 7f1d73b5d700 1 -- 172.21.15.53:0/1459039520 >> [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439] conn(0x7f1d580a9e10 msgr2=0x7f1d580ac230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.386+0000 7f1d73b5d700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439] conn(0x7f1d580a9e10 0x7f1d580ac230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.386+0000 7f1d73b5d700 1 -- 172.21.15.53:0/1459039520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1d580a0630 msgr2=0x7f1d580a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.386+0000 7f1d73b5d700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1d580a0630 0x7f1d580a2af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1d68006010 tx=0x7f1d68005ab0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.386+0000 7f1d73b5d700 1 -- 172.21.15.53:0/1459039520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d6c105d80 msgr2=0x7f1d6c19b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.386+0000 7f1d73b5d700 1 --2- 172.21.15.53:0/1459039520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d6c105d80 0x7f1d6c19b810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400b6d0 tx=0x7f1d6400b9e0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.386+0000 7f1d720fa700 1 -- 172.21.15.53:0/1459039520 reap_dead start 2023-11-06T01:32:53.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.386+0000 7f1d73b5d700 1 -- 172.21.15.53:0/1459039520 shutdown_connections 2023-11-06T01:32:53.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.386+0000 7f1d73b5d700 1 -- 172.21.15.53:0/1459039520 >> 172.21.15.53:0/1459039520 conn(0x7f1d6c076d90 msgr2=0x7f1d6c103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:53.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.387+0000 7f1d73b5d700 1 -- 172.21.15.53:0/1459039520 shutdown_connections 2023-11-06T01:32:53.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.387+0000 7f1d73b5d700 1 -- 172.21.15.53:0/1459039520 wait complete. 2023-11-06T01:32:53.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:53 smithi062 ceph-mon[103924]: pgmap v265: 97 pgs: 97 active+clean; 636 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-11-06T01:32:53.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:53 smithi163 ceph-mon[104416]: pgmap v265: 97 pgs: 97 active+clean; 636 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-11-06T01:32:53.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.442+0000 7fda47f08700 1 -- 172.21.15.53:0/122999024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda40105d80 msgr2=0x7fda40106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.442+0000 7fda47f08700 1 --2- 172.21.15.53:0/122999024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda40105d80 0x7fda40106160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009990 tx=0x7fda3c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.442+0000 7fda47f08700 1 -- 172.21.15.53:0/122999024 shutdown_connections 2023-11-06T01:32:53.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.442+0000 7fda47f08700 1 --2- 172.21.15.53:0/122999024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda40100690 0x7fda4010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.442+0000 7fda47f08700 1 --2- 172.21.15.53:0/122999024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda40105d80 0x7fda40106160 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.442+0000 7fda47f08700 1 --2- 172.21.15.53:0/122999024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda400ffcd0 0x7fda40100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.442+0000 7fda47f08700 1 -- 172.21.15.53:0/122999024 >> 172.21.15.53:0/122999024 conn(0x7fda40076d90 msgr2=0x7fda400771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:53.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.442+0000 7fda47f08700 1 -- 172.21.15.53:0/122999024 shutdown_connections 2023-11-06T01:32:53.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.442+0000 7fda47f08700 1 -- 172.21.15.53:0/122999024 wait complete. 2023-11-06T01:32:53.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda47f08700 1 Processor -- start 2023-11-06T01:32:53.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda47f08700 1 -- start start 2023-11-06T01:32:53.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda47f08700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda400ffcd0 0x7fda40198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda47f08700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda40100690 0x7fda40199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda47f08700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda40105d80 0x7fda4019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda47f08700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4019dfb0 con 0x7fda40100690 2023-11-06T01:32:53.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda47f08700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4019e120 con 0x7fda40105d80 2023-11-06T01:32:53.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda47f08700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4019e2e0 con 0x7fda400ffcd0 2023-11-06T01:32:53.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda454a3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda40100690 0x7fda40199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda454a3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda40100690 0x7fda40199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38284/0 (socket says 172.21.15.53:38284) 2023-11-06T01:32:53.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda454a3700 1 -- 172.21.15.53:0/3394570610 learned_addr learned my addr 172.21.15.53:0/3394570610 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:53.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda45ca4700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda400ffcd0 0x7fda40198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda464a5700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda40105d80 0x7fda4019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda454a3700 1 -- 172.21.15.53:0/3394570610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda400ffcd0 msgr2=0x7fda40198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.443+0000 7fda454a3700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda400ffcd0 0x7fda40198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.444+0000 7fda454a3700 1 -- 172.21.15.53:0/3394570610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda40105d80 msgr2=0x7fda4019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.444+0000 7fda454a3700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda40105d80 0x7fda4019d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.444+0000 7fda454a3700 1 -- 172.21.15.53:0/3394570610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c0095f0 con 0x7fda40100690 2023-11-06T01:32:53.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.444+0000 7fda454a3700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda40100690 0x7fda40199530 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fda3000d980 tx=0x7fda3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:53.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.444+0000 7fda36ffd700 1 -- 172.21.15.53:0/3394570610 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30004220 con 0x7fda40100690 2023-11-06T01:32:53.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.444+0000 7fda47f08700 1 -- 172.21.15.53:0/3394570610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4019e5c0 con 0x7fda40100690 2023-11-06T01:32:53.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.444+0000 7fda36ffd700 1 -- 172.21.15.53:0/3394570610 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda30007640 con 0x7fda40100690 2023-11-06T01:32:53.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.444+0000 7fda36ffd700 1 -- 172.21.15.53:0/3394570610 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30005690 con 0x7fda40100690 2023-11-06T01:32:53.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.444+0000 7fda47f08700 1 -- 172.21.15.53:0/3394570610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda401a5e20 con 0x7fda40100690 2023-11-06T01:32:53.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.445+0000 7fda47f08700 1 -- 172.21.15.53:0/3394570610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda4010fea0 con 0x7fda40100690 2023-11-06T01:32:53.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.446+0000 7fda36ffd700 1 -- 172.21.15.53:0/3394570610 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fda30004750 con 0x7fda40100690 2023-11-06T01:32:53.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.446+0000 7fda36ffd700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda2c0a05b0 0x7fda2c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.447+0000 7fda45ca4700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda2c0a05b0 0x7fda2c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.447+0000 7fda36ffd700 1 -- 172.21.15.53:0/3394570610 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fda300cbe30 con 0x7fda40100690 2023-11-06T01:32:53.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.447+0000 7fda36ffd700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345] conn(0x7fda2c0a9d90 0x7fda2c0ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.447+0000 7fda36ffd700 1 -- 172.21.15.53:0/3394570610 --> [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda2c0ac860 con 0x7fda2c0a9d90 2023-11-06T01:32:53.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.447+0000 7fda36ffd700 1 -- 172.21.15.53:0/3394570610 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda300cc2d0 con 0x7fda40100690 2023-11-06T01:32:53.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.447+0000 7fda464a5700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345] conn(0x7fda2c0a9d90 0x7fda2c0ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.447+0000 7fda45ca4700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda2c0a05b0 0x7fda2c0a2a70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00b5c0 tx=0x7fda3c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:53.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.447+0000 7fda464a5700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345] conn(0x7fda2c0a9d90 0x7fda2c0ac1b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:53.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.448+0000 7fda36ffd700 1 -- 172.21.15.53:0/3394570610 <== osd.3 v2:172.21.15.53:6810/3352913345 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fda2c0ac860 con 0x7fda2c0a9d90 2023-11-06T01:32:53.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda47f08700 1 -- 172.21.15.53:0/3394570610 --> [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda4004ea90 con 0x7fda2c0a9d90 2023-11-06T01:32:53.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda36ffd700 1 -- 172.21.15.53:0/3394570610 <== osd.3 v2:172.21.15.53:6810/3352913345 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fda4004ea90 con 0x7fda2c0a9d90 2023-11-06T01:32:53.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda47f08700 1 -- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345] conn(0x7fda2c0a9d90 msgr2=0x7fda2c0ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda47f08700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345] conn(0x7fda2c0a9d90 0x7fda2c0ac1b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda47f08700 1 -- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda2c0a05b0 msgr2=0x7fda2c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda47f08700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda2c0a05b0 0x7fda2c0a2a70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00b5c0 tx=0x7fda3c005960 comp rx=0 tx=0).stop 2023-11-06T01:32:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda47f08700 1 -- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda40100690 msgr2=0x7fda40199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda47f08700 1 --2- 172.21.15.53:0/3394570610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda40100690 0x7fda40199530 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fda3000d980 tx=0x7fda3000dc90 comp rx=0 tx=0).stop 2023-11-06T01:32:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda464a5700 1 -- 172.21.15.53:0/3394570610 reap_dead start 2023-11-06T01:32:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda47f08700 1 -- 172.21.15.53:0/3394570610 shutdown_connections 2023-11-06T01:32:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda47f08700 1 -- 172.21.15.53:0/3394570610 >> 172.21.15.53:0/3394570610 conn(0x7fda40076d90 msgr2=0x7fda400fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.464+0000 7fda47f08700 1 -- 172.21.15.53:0/3394570610 shutdown_connections 2023-11-06T01:32:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.465+0000 7fda47f08700 1 -- 172.21.15.53:0/3394570610 wait complete. 2023-11-06T01:32:53.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.711+0000 7f3823387700 1 -- 172.21.15.53:0/1023558544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c100690 msgr2=0x7f381c10cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.711+0000 7f3823387700 1 --2- 172.21.15.53:0/1023558544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c100690 0x7f381c10cb30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f381400b0a0 tx=0x7f381400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.711+0000 7f3823387700 1 -- 172.21.15.53:0/1023558544 shutdown_connections 2023-11-06T01:32:53.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.711+0000 7f3823387700 1 --2- 172.21.15.53:0/1023558544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f381c105d80 0x7f381c106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.711+0000 7f3823387700 1 --2- 172.21.15.53:0/1023558544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c100690 0x7f381c10cb30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.711+0000 7f3823387700 1 --2- 172.21.15.53:0/1023558544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f381c0ffcd0 0x7f381c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.711+0000 7f3823387700 1 -- 172.21.15.53:0/1023558544 >> 172.21.15.53:0/1023558544 conn(0x7f381c076d90 msgr2=0x7f381c0771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:53.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.711+0000 7f3823387700 1 -- 172.21.15.53:0/1023558544 shutdown_connections 2023-11-06T01:32:53.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.711+0000 7f3823387700 1 -- 172.21.15.53:0/1023558544 wait complete. 2023-11-06T01:32:53.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.712+0000 7f3823387700 1 Processor -- start 2023-11-06T01:32:53.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.712+0000 7f3823387700 1 -- start start 2023-11-06T01:32:53.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.712+0000 7f3823387700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f381c0ffcd0 0x7f381c195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.712+0000 7f3823387700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f381c100690 0x7f381c196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.712+0000 7f3823387700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c105d80 0x7f381c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.712+0000 7f3823387700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c19ad90 con 0x7f381c105d80 2023-11-06T01:32:53.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.712+0000 7f3823387700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c19af00 con 0x7f381c100690 2023-11-06T01:32:53.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.712+0000 7f3823387700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c19b0c0 con 0x7f381c0ffcd0 2023-11-06T01:32:53.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3821924700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c105d80 0x7f381c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3821924700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c105d80 0x7f381c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38302/0 (socket says 172.21.15.53:38302) 2023-11-06T01:32:53.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3821924700 1 -- 172.21.15.53:0/4060821070 learned_addr learned my addr 172.21.15.53:0/4060821070 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:53.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3821123700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f381c0ffcd0 0x7f381c195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3820922700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f381c100690 0x7f381c196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3821924700 1 -- 172.21.15.53:0/4060821070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f381c0ffcd0 msgr2=0x7f381c195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3821924700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f381c0ffcd0 0x7f381c195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3821924700 1 -- 172.21.15.53:0/4060821070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f381c100690 msgr2=0x7f381c196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3821924700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f381c100690 0x7f381c196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3821924700 1 -- 172.21.15.53:0/4060821070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3814009d00 con 0x7f381c105d80 2023-11-06T01:32:53.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3821123700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f381c0ffcd0 0x7f381c195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:53.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3821924700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c105d80 0x7f381c19a780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f381400b780 tx=0x7f3814007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:53.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f38127fc700 1 -- 172.21.15.53:0/4060821070 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381400e040 con 0x7f381c105d80 2023-11-06T01:32:53.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3823387700 1 -- 172.21.15.53:0/4060821070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c19b340 con 0x7f381c105d80 2023-11-06T01:32:53.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f38127fc700 1 -- 172.21.15.53:0/4060821070 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3814004c50 con 0x7f381c105d80 2023-11-06T01:32:53.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f38127fc700 1 -- 172.21.15.53:0/4060821070 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38140052a0 con 0x7f381c105d80 2023-11-06T01:32:53.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.713+0000 7f3823387700 1 -- 172.21.15.53:0/4060821070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c1a2c00 con 0x7f381c105d80 2023-11-06T01:32:53.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.715+0000 7f3823387700 1 -- 172.21.15.53:0/4060821070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f381c04ea90 con 0x7f381c105d80 2023-11-06T01:32:53.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.715+0000 7f38127fc700 1 -- 172.21.15.53:0/4060821070 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f381401d470 con 0x7f381c105d80 2023-11-06T01:32:53.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.716+0000 7f38127fc700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f38080a05b0 0x7f38080a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.716+0000 7f3821123700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f38080a05b0 0x7f38080a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.716+0000 7f38127fc700 1 -- 172.21.15.53:0/4060821070 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f38140ccfd0 con 0x7f381c105d80 2023-11-06T01:32:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.716+0000 7f38127fc700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705] conn(0x7f38080a9d90 0x7f38080ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.716+0000 7f38127fc700 1 -- 172.21.15.53:0/4060821070 --> [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f38080ac860 con 0x7f38080a9d90 2023-11-06T01:32:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.716+0000 7f38127fc700 1 -- 172.21.15.53:0/4060821070 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38140cf6e0 con 0x7f381c105d80 2023-11-06T01:32:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.717+0000 7f3821123700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f38080a05b0 0x7f38080a2a70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3818009cb0 tx=0x7f3818009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.717+0000 7f3820922700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705] conn(0x7f38080a9d90 0x7f38080ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.717+0000 7f3820922700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705] conn(0x7f38080a9d90 0x7f38080ac1b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:53.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.718+0000 7f38127fc700 1 -- 172.21.15.53:0/4060821070 <== osd.11 v2:172.21.15.163:6824/1021137705 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f38080ac860 con 0x7f38080a9d90 2023-11-06T01:32:53.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.733+0000 7f3823387700 1 -- 172.21.15.53:0/4060821070 --> [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f381c10fee0 con 0x7f38080a9d90 2023-11-06T01:32:53.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.734+0000 7f38127fc700 1 -- 172.21.15.53:0/4060821070 <== osd.11 v2:172.21.15.163:6824/1021137705 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f380c00b870 con 0x7f38080a9d90 2023-11-06T01:32:53.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.734+0000 7f3823387700 1 -- 172.21.15.53:0/4060821070 >> [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705] conn(0x7f38080a9d90 msgr2=0x7f38080ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.734+0000 7f3823387700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705] conn(0x7f38080a9d90 0x7f38080ac1b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.734+0000 7f3823387700 1 -- 172.21.15.53:0/4060821070 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f38080a05b0 msgr2=0x7f38080a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.734+0000 7f3823387700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f38080a05b0 0x7f38080a2a70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3818009cb0 tx=0x7f3818009260 comp rx=0 tx=0).stop 2023-11-06T01:32:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.734+0000 7f3823387700 1 -- 172.21.15.53:0/4060821070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c105d80 msgr2=0x7f381c19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.734+0000 7f3823387700 1 --2- 172.21.15.53:0/4060821070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f381c105d80 0x7f381c19a780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f381400b780 tx=0x7f3814007a40 comp rx=0 tx=0).stop 2023-11-06T01:32:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.735+0000 7f3821924700 1 -- 172.21.15.53:0/4060821070 reap_dead start 2023-11-06T01:32:53.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.735+0000 7f3823387700 1 -- 172.21.15.53:0/4060821070 shutdown_connections 2023-11-06T01:32:53.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.735+0000 7f3823387700 1 -- 172.21.15.53:0/4060821070 >> 172.21.15.53:0/4060821070 conn(0x7f381c076d90 msgr2=0x7f381c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:53.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.735+0000 7f3823387700 1 -- 172.21.15.53:0/4060821070 shutdown_connections 2023-11-06T01:32:53.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.735+0000 7f3823387700 1 -- 172.21.15.53:0/4060821070 wait complete. 2023-11-06T01:32:53.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.788+0000 7f72b6255700 1 -- 172.21.15.53:0/2041775455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b0105d80 msgr2=0x7f72b0106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.788+0000 7f72b6255700 1 --2- 172.21.15.53:0/2041775455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b0105d80 0x7f72b0106160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72a800b0a0 tx=0x7f72a800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.788+0000 7f72b6255700 1 -- 172.21.15.53:0/2041775455 shutdown_connections 2023-11-06T01:32:53.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.788+0000 7f72b6255700 1 --2- 172.21.15.53:0/2041775455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b0100690 0x7f72b010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.788+0000 7f72b6255700 1 --2- 172.21.15.53:0/2041775455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b0105d80 0x7f72b0106160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.788+0000 7f72b6255700 1 --2- 172.21.15.53:0/2041775455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72b00ffcd0 0x7f72b0100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.788+0000 7f72b6255700 1 -- 172.21.15.53:0/2041775455 >> 172.21.15.53:0/2041775455 conn(0x7f72b0076d90 msgr2=0x7f72b00771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:53.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.788+0000 7f72b6255700 1 -- 172.21.15.53:0/2041775455 shutdown_connections 2023-11-06T01:32:53.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.788+0000 7f72b6255700 1 -- 172.21.15.53:0/2041775455 wait complete. 2023-11-06T01:32:53.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72b6255700 1 Processor -- start 2023-11-06T01:32:53.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72b6255700 1 -- start start 2023-11-06T01:32:53.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72b6255700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b00ffcd0 0x7f72b0197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72b6255700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72b0100690 0x7f72b01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72b6255700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b0105d80 0x7f72b01a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:53.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72b6255700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b0198bc0 con 0x7f72b00ffcd0 2023-11-06T01:32:53.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72b6255700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b01a3330 con 0x7f72b0105d80 2023-11-06T01:32:53.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72b6255700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b01a34a0 con 0x7f72b0100690 2023-11-06T01:32:53.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72af7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b00ffcd0 0x7f72b0197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72af7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b00ffcd0 0x7f72b0197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38314/0 (socket says 172.21.15.53:38314) 2023-11-06T01:32:53.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72af7fe700 1 -- 172.21.15.53:0/458545798 learned_addr learned my addr 172.21.15.53:0/458545798 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:53.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.789+0000 7f72aeffd700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72b0100690 0x7f72b01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72affff700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b0105d80 0x7f72b01a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:53.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72af7fe700 1 -- 172.21.15.53:0/458545798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72b0100690 msgr2=0x7f72b01982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:53.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72af7fe700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72b0100690 0x7f72b01982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:53.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72af7fe700 1 -- 172.21.15.53:0/458545798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b0105d80 msgr2=0x7f72b01a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:54.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72af7fe700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b0105d80 0x7f72b01a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:54.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72af7fe700 1 -- 172.21.15.53:0/458545798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a8009d00 con 0x7f72b00ffcd0 2023-11-06T01:32:54.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72affff700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b0105d80 0x7f72b01a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:54.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72af7fe700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b00ffcd0 0x7f72b0197d70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f72a80062a0 tx=0x7f72a800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:54.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72acff9700 1 -- 172.21.15.53:0/458545798 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a800e040 con 0x7f72b00ffcd0 2023-11-06T01:32:54.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72acff9700 1 -- 172.21.15.53:0/458545798 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72a8003e10 con 0x7f72b00ffcd0 2023-11-06T01:32:54.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72b6255700 1 -- 172.21.15.53:0/458545798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b01a3720 con 0x7f72b00ffcd0 2023-11-06T01:32:54.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72acff9700 1 -- 172.21.15.53:0/458545798 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a801b6c0 con 0x7f72b00ffcd0 2023-11-06T01:32:54.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.790+0000 7f72b6255700 1 -- 172.21.15.53:0/458545798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b01a3c10 con 0x7f72b00ffcd0 2023-11-06T01:32:54.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.791+0000 7f72b6255700 1 -- 172.21.15.53:0/458545798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f72b004ea90 con 0x7f72b00ffcd0 2023-11-06T01:32:54.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.792+0000 7f72acff9700 1 -- 172.21.15.53:0/458545798 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f72a801d470 con 0x7f72b00ffcd0 2023-11-06T01:32:54.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.793+0000 7f72acff9700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f729c0a0630 0x7f729c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:54.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.793+0000 7f72aeffd700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f729c0a0630 0x7f729c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:54.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.793+0000 7f72acff9700 1 -- 172.21.15.53:0/458545798 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72a80cd170 con 0x7f72b00ffcd0 2023-11-06T01:32:54.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.793+0000 7f72acff9700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317] conn(0x7f729c0a9e10 0x7f729c0ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:54.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.793+0000 7f72acff9700 1 -- 172.21.15.53:0/458545798 --> [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f729c0ac8e0 con 0x7f729c0a9e10 2023-11-06T01:32:54.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.793+0000 7f72acff9700 1 -- 172.21.15.53:0/458545798 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72a80cd610 con 0x7f72b00ffcd0 2023-11-06T01:32:54.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.793+0000 7f72aeffd700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f729c0a0630 0x7f729c0a2af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f72b0069100 tx=0x7f7298009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:54.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.793+0000 7f72affff700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317] conn(0x7f729c0a9e10 0x7f729c0ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:54.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.794+0000 7f72affff700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317] conn(0x7f729c0a9e10 0x7f729c0ac230 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:54.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.795+0000 7f72acff9700 1 -- 172.21.15.53:0/458545798 <== osd.8 v2:172.21.15.163:6816/2361123317 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f729c0ac8e0 con 0x7f729c0a9e10 2023-11-06T01:32:54.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.812+0000 7f72b6255700 1 -- 172.21.15.53:0/458545798 --> [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f72b010fee0 con 0x7f729c0a9e10 2023-11-06T01:32:54.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.813+0000 7f72acff9700 1 -- 172.21.15.53:0/458545798 <== osd.8 v2:172.21.15.163:6816/2361123317 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f72a000a8e0 con 0x7f729c0a9e10 2023-11-06T01:32:54.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.814+0000 7f72b6255700 1 -- 172.21.15.53:0/458545798 >> [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317] conn(0x7f729c0a9e10 msgr2=0x7f729c0ac230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:54.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.814+0000 7f72b6255700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317] conn(0x7f729c0a9e10 0x7f729c0ac230 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:54.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.814+0000 7f72b6255700 1 -- 172.21.15.53:0/458545798 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f729c0a0630 msgr2=0x7f729c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:54.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.814+0000 7f72b6255700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f729c0a0630 0x7f729c0a2af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f72b0069100 tx=0x7f7298009260 comp rx=0 tx=0).stop 2023-11-06T01:32:54.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.814+0000 7f72b6255700 1 -- 172.21.15.53:0/458545798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b00ffcd0 msgr2=0x7f72b0197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:54.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.814+0000 7f72b6255700 1 --2- 172.21.15.53:0/458545798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b00ffcd0 0x7f72b0197d70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f72a80062a0 tx=0x7f72a800bd60 comp rx=0 tx=0).stop 2023-11-06T01:32:54.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.814+0000 7f72affff700 1 -- 172.21.15.53:0/458545798 reap_dead start 2023-11-06T01:32:54.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.814+0000 7f72b6255700 1 -- 172.21.15.53:0/458545798 shutdown_connections 2023-11-06T01:32:54.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.814+0000 7f72b6255700 1 -- 172.21.15.53:0/458545798 >> 172.21.15.53:0/458545798 conn(0x7f72b0076d90 msgr2=0x7f72b00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:54.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.814+0000 7f72b6255700 1 -- 172.21.15.53:0/458545798 shutdown_connections 2023-11-06T01:32:54.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:53.814+0000 7f72b6255700 1 -- 172.21.15.53:0/458545798 wait complete. 2023-11-06T01:32:54.441 INFO:teuthology.orchestra.run.smithi053.stdout:300647710741 2023-11-06T01:32:54.441 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-11-06T01:32:54.674 INFO:teuthology.orchestra.run.smithi053.stdout:167503724586 2023-11-06T01:32:54.674 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-11-06T01:32:54.878 INFO:teuthology.orchestra.run.smithi053.stdout:146028888110 2023-11-06T01:32:54.879 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-11-06T01:32:55.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.929+0000 7f8277292700 1 -- 172.21.15.53:0/3161465084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82701060f0 msgr2=0x7f827010dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:55.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.929+0000 7f8277292700 1 --2- 172.21.15.53:0/3161465084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82701060f0 0x7f827010dcc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f826800b0a0 tx=0x7f826800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:55.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.929+0000 7f8277292700 1 -- 172.21.15.53:0/3161465084 shutdown_connections 2023-11-06T01:32:55.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.929+0000 7f8277292700 1 --2- 172.21.15.53:0/3161465084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8270101010 0x7f82701013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:55.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.929+0000 7f8277292700 1 --2- 172.21.15.53:0/3161465084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82701060f0 0x7f827010dcc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:55.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.929+0000 7f8277292700 1 --2- 172.21.15.53:0/3161465084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82701019c0 0x7f8270105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:55.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.929+0000 7f8277292700 1 -- 172.21.15.53:0/3161465084 >> 172.21.15.53:0/3161465084 conn(0x7f82700fc880 msgr2=0x7f82700feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.929+0000 7f8277292700 1 -- 172.21.15.53:0/3161465084 shutdown_connections 2023-11-06T01:32:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.929+0000 7f8277292700 1 -- 172.21.15.53:0/3161465084 wait complete. 2023-11-06T01:32:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.929+0000 7f8277292700 1 Processor -- start 2023-11-06T01:32:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.929+0000 7f8277292700 1 -- start start 2023-11-06T01:32:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f8277292700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8270101010 0x7f8270196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f8277292700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82701019c0 0x7f82701974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f8277292700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82701060f0 0x7f827019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f8277292700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827019bf50 con 0x7f82701060f0 2023-11-06T01:32:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f8277292700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827019c0c0 con 0x7f82701019c0 2023-11-06T01:32:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f8277292700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827019c280 con 0x7f8270101010 2023-11-06T01:32:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827582f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82701060f0 0x7f827019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827582f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82701060f0 0x7f827019b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38332/0 (socket says 172.21.15.53:38332) 2023-11-06T01:32:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827582f700 1 -- 172.21.15.53:0/3968795820 learned_addr learned my addr 172.21.15.53:0/3968795820 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827502e700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8270101010 0x7f8270196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827482d700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82701019c0 0x7f82701974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827582f700 1 -- 172.21.15.53:0/3968795820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8270101010 msgr2=0x7f8270196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827582f700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8270101010 0x7f8270196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827582f700 1 -- 172.21.15.53:0/3968795820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82701019c0 msgr2=0x7f82701974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827582f700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82701019c0 0x7f82701974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:55.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827582f700 1 -- 172.21.15.53:0/3968795820 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268009d00 con 0x7f82701060f0 2023-11-06T01:32:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827502e700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8270101010 0x7f8270196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:32:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f827582f700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82701060f0 0x7f827019b940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f826800b780 tx=0x7f8268007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f82667fc700 1 -- 172.21.15.53:0/3968795820 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826800e040 con 0x7f82701060f0 2023-11-06T01:32:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f8277292700 1 -- 172.21.15.53:0/3968795820 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827019c500 con 0x7f82701060f0 2023-11-06T01:32:55.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f82667fc700 1 -- 172.21.15.53:0/3968795820 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8268004c50 con 0x7f82701060f0 2023-11-06T01:32:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f82667fc700 1 -- 172.21.15.53:0/3968795820 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82680052a0 con 0x7f82701060f0 2023-11-06T01:32:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.930+0000 7f8277292700 1 -- 172.21.15.53:0/3968795820 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82701a3dc0 con 0x7f82701060f0 2023-11-06T01:32:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.931+0000 7f8277292700 1 -- 172.21.15.53:0/3968795820 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f827004ea90 con 0x7f82701060f0 2023-11-06T01:32:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.933+0000 7f82667fc700 1 -- 172.21.15.53:0/3968795820 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f826801d470 con 0x7f82701060f0 2023-11-06T01:32:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.933+0000 7f82667fc700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f825c0a05b0 0x7f825c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.933+0000 7f827502e700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f825c0a05b0 0x7f825c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.933+0000 7f82667fc700 1 -- 172.21.15.53:0/3968795820 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f82680cd280 con 0x7f82701060f0 2023-11-06T01:32:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.933+0000 7f82667fc700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393] conn(0x7f825c0a9d90 0x7f825c0ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.933+0000 7f82667fc700 1 -- 172.21.15.53:0/3968795820 --> [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f825c0ac860 con 0x7f825c0a9d90 2023-11-06T01:32:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.934+0000 7f82667fc700 1 -- 172.21.15.53:0/3968795820 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82680cd720 con 0x7f82701060f0 2023-11-06T01:32:55.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.934+0000 7f827502e700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f825c0a05b0 0x7f825c0a2a70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f826c009cb0 tx=0x7f826c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:55.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.934+0000 7f827482d700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393] conn(0x7f825c0a9d90 0x7f825c0ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:55.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.934+0000 7f827482d700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393] conn(0x7f825c0a9d90 0x7f825c0ac1b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.935+0000 7f82667fc700 1 -- 172.21.15.53:0/3968795820 <== osd.1 v2:172.21.15.62:6800/395697393 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f825c0ac860 con 0x7f825c0a9d90 2023-11-06T01:32:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.952+0000 7f8277292700 1 -- 172.21.15.53:0/3968795820 --> [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8270111030 con 0x7f825c0a9d90 2023-11-06T01:32:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.952+0000 7f82667fc700 1 -- 172.21.15.53:0/3968795820 <== osd.1 v2:172.21.15.62:6800/395697393 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8260009900 con 0x7f825c0a9d90 2023-11-06T01:32:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.953+0000 7f8277292700 1 -- 172.21.15.53:0/3968795820 >> [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393] conn(0x7f825c0a9d90 msgr2=0x7f825c0ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.953+0000 7f8277292700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393] conn(0x7f825c0a9d90 0x7f825c0ac1b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.953+0000 7f8277292700 1 -- 172.21.15.53:0/3968795820 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f825c0a05b0 msgr2=0x7f825c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.953+0000 7f8277292700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f825c0a05b0 0x7f825c0a2a70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f826c009cb0 tx=0x7f826c009260 comp rx=0 tx=0).stop 2023-11-06T01:32:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.953+0000 7f8277292700 1 -- 172.21.15.53:0/3968795820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82701060f0 msgr2=0x7f827019b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.953+0000 7f8277292700 1 --2- 172.21.15.53:0/3968795820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82701060f0 0x7f827019b940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f826800b780 tx=0x7f8268007a40 comp rx=0 tx=0).stop 2023-11-06T01:32:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.953+0000 7f827582f700 1 -- 172.21.15.53:0/3968795820 reap_dead start 2023-11-06T01:32:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.953+0000 7f8277292700 1 -- 172.21.15.53:0/3968795820 shutdown_connections 2023-11-06T01:32:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.953+0000 7f8277292700 1 -- 172.21.15.53:0/3968795820 >> 172.21.15.53:0/3968795820 conn(0x7f82700fc880 msgr2=0x7f82700fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.953+0000 7f8277292700 1 -- 172.21.15.53:0/3968795820 shutdown_connections 2023-11-06T01:32:55.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:54.953+0000 7f8277292700 1 -- 172.21.15.53:0/3968795820 wait complete. 2023-11-06T01:32:55.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:55 smithi053 ceph-mon[99472]: pgmap v266: 97 pgs: 97 active+clean; 636 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 10 op/s 2023-11-06T01:32:55.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:55 smithi062 ceph-mon[103924]: pgmap v266: 97 pgs: 97 active+clean; 636 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 10 op/s 2023-11-06T01:32:55.447 INFO:teuthology.orchestra.run.smithi053.stdout:330712481809 2023-11-06T01:32:55.448 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-11-06T01:32:55.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:55 smithi163 ceph-mon[104416]: pgmap v266: 97 pgs: 97 active+clean; 636 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 10 op/s 2023-11-06T01:32:56.526 INFO:teuthology.orchestra.run.smithi053.stdout:55834574908 2023-11-06T01:32:56.526 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-11-06T01:32:56.604 INFO:teuthology.orchestra.run.smithi053.stdout:111669149746 2023-11-06T01:32:56.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-11-06T01:32:56.722 INFO:teuthology.orchestra.run.smithi053.stdout:249108103197 2023-11-06T01:32:56.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-11-06T01:32:56.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.728+0000 7f9a3106c700 1 -- 172.21.15.53:0/4233163309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a2c105e80 msgr2=0x7f9a2c1007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:56.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.728+0000 7f9a3106c700 1 --2- 172.21.15.53:0/4233163309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a2c105e80 0x7f9a2c1007e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009990 tx=0x7f9a1c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.728+0000 7f9a3106c700 1 -- 172.21.15.53:0/4233163309 shutdown_connections 2023-11-06T01:32:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.728+0000 7f9a3106c700 1 --2- 172.21.15.53:0/4233163309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a2c103b00 0x7f9a2c103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.728+0000 7f9a3106c700 1 --2- 172.21.15.53:0/4233163309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a2c105e80 0x7f9a2c1007e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.728+0000 7f9a3106c700 1 --2- 172.21.15.53:0/4233163309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a2c100db0 0x7f9a2c101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.728+0000 7f9a3106c700 1 -- 172.21.15.53:0/4233163309 >> 172.21.15.53:0/4233163309 conn(0x7f9a2c076d80 msgr2=0x7f9a2c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.728+0000 7f9a3106c700 1 -- 172.21.15.53:0/4233163309 shutdown_connections 2023-11-06T01:32:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.728+0000 7f9a3106c700 1 -- 172.21.15.53:0/4233163309 wait complete. 2023-11-06T01:32:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.729+0000 7f9a3106c700 1 Processor -- start 2023-11-06T01:32:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.730+0000 7f9a3106c700 1 -- start start 2023-11-06T01:32:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.730+0000 7f9a3106c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a2c100db0 0x7f9a2c1a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:56.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.730+0000 7f9a3106c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a2c103b00 0x7f9a2c1a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:56.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.730+0000 7f9a3106c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a2c105e80 0x7f9a2c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:56.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.730+0000 7f9a3106c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c19dbe0 con 0x7f9a2c100db0 2023-11-06T01:32:56.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.730+0000 7f9a3106c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c19dd50 con 0x7f9a2c103b00 2023-11-06T01:32:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.730+0000 7f9a3106c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c19df10 con 0x7f9a2c105e80 2023-11-06T01:32:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.730+0000 7f9a2b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a2c105e80 0x7f9a2c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.730+0000 7f9a2b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a2c105e80 0x7f9a2c19d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:52584/0 (socket says 172.21.15.53:52584) 2023-11-06T01:32:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.731+0000 7f9a2b59e700 1 -- 172.21.15.53:0/1432496509 learned_addr learned my addr 172.21.15.53:0/1432496509 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.731+0000 7f9a2b59e700 1 -- 172.21.15.53:0/1432496509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a2c103b00 msgr2=0x7f9a2c1a3a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:32:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.731+0000 7f9a2b59e700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a2c103b00 0x7f9a2c1a3a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.731+0000 7f9a2b59e700 1 -- 172.21.15.53:0/1432496509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a2c100db0 msgr2=0x7f9a2c1a34f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:32:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.731+0000 7f9a2b59e700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a2c100db0 0x7f9a2c1a34f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.731+0000 7f9a2b59e700 1 -- 172.21.15.53:0/1432496509 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c0095f0 con 0x7f9a2c105e80 2023-11-06T01:32:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.731+0000 7f9a2b59e700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a2c105e80 0x7f9a2c19d570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a24007ee0 tx=0x7f9a2400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.732+0000 7f9a13fff700 1 -- 172.21.15.53:0/1432496509 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a24017070 con 0x7f9a2c105e80 2023-11-06T01:32:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.732+0000 7f9a13fff700 1 -- 172.21.15.53:0/1432496509 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a2400f040 con 0x7f9a2c105e80 2023-11-06T01:32:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.732+0000 7f9a13fff700 1 -- 172.21.15.53:0/1432496509 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a24005740 con 0x7f9a2c105e80 2023-11-06T01:32:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.733+0000 7f9a3106c700 1 -- 172.21.15.53:0/1432496509 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2c19e1f0 con 0x7f9a2c105e80 2023-11-06T01:32:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.733+0000 7f9a3106c700 1 -- 172.21.15.53:0/1432496509 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c106460 con 0x7f9a2c105e80 2023-11-06T01:32:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.734+0000 7f9a3106c700 1 -- 172.21.15.53:0/1432496509 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a2c117790 con 0x7f9a2c105e80 2023-11-06T01:32:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.737+0000 7f9a13fff700 1 -- 172.21.15.53:0/1432496509 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f9a240058a0 con 0x7f9a2c105e80 2023-11-06T01:32:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.737+0000 7f9a13fff700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9a140a0630 0x7f9a140a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:56.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.737+0000 7f9a13fff700 1 -- 172.21.15.53:0/1432496509 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9a240cb3a0 con 0x7f9a2c105e80 2023-11-06T01:32:56.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.737+0000 7f9a13fff700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016] conn(0x7f9a140a9e10 0x7f9a140ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:56.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.738+0000 7f9a13fff700 1 -- 172.21.15.53:0/1432496509 --> [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a140ac8e0 con 0x7f9a140a9e10 2023-11-06T01:32:56.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.738+0000 7f9a13fff700 1 -- 172.21.15.53:0/1432496509 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a240cb7c0 con 0x7f9a2c105e80 2023-11-06T01:32:56.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.738+0000 7f9a2a59c700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016] conn(0x7f9a140a9e10 0x7f9a140ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:56.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.738+0000 7f9a2a59c700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016] conn(0x7f9a140a9e10 0x7f9a140ac230 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:56.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.739+0000 7f9a13fff700 1 -- 172.21.15.53:0/1432496509 <== osd.2 v2:172.21.15.163:6800/1520683016 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9a140ac8e0 con 0x7f9a140a9e10 2023-11-06T01:32:56.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.742+0000 7f9a2ad9d700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9a140a0630 0x7f9a140a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.745+0000 7f9a2ad9d700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9a140a0630 0x7f9a140a2af0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00b5c0 tx=0x7f9a1c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.764+0000 7f9a3106c700 1 -- 172.21.15.53:0/1432496509 --> [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9a2c04ea90 con 0x7f9a140a9e10 2023-11-06T01:32:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.765+0000 7f9a13fff700 1 -- 172.21.15.53:0/1432496509 <== osd.2 v2:172.21.15.163:6800/1520683016 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9a2c04ea90 con 0x7f9a140a9e10 2023-11-06T01:32:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.768+0000 7f9a3106c700 1 -- 172.21.15.53:0/1432496509 >> [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016] conn(0x7f9a140a9e10 msgr2=0x7f9a140ac230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.768+0000 7f9a3106c700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016] conn(0x7f9a140a9e10 0x7f9a140ac230 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.768+0000 7f9a3106c700 1 -- 172.21.15.53:0/1432496509 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9a140a0630 msgr2=0x7f9a140a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.768+0000 7f9a3106c700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9a140a0630 0x7f9a140a2af0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00b5c0 tx=0x7f9a1c000bc0 comp rx=0 tx=0).stop 2023-11-06T01:32:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.768+0000 7f9a3106c700 1 -- 172.21.15.53:0/1432496509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a2c105e80 msgr2=0x7f9a2c19d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.768+0000 7f9a3106c700 1 --2- 172.21.15.53:0/1432496509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a2c105e80 0x7f9a2c19d570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a24007ee0 tx=0x7f9a2400d3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.768+0000 7f9a2b59e700 1 -- 172.21.15.53:0/1432496509 reap_dead start 2023-11-06T01:32:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.768+0000 7f9a3106c700 1 -- 172.21.15.53:0/1432496509 shutdown_connections 2023-11-06T01:32:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.768+0000 7f9a3106c700 1 -- 172.21.15.53:0/1432496509 >> 172.21.15.53:0/1432496509 conn(0x7f9a2c076d80 msgr2=0x7f9a2c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.768+0000 7f9a3106c700 1 -- 172.21.15.53:0/1432496509 shutdown_connections 2023-11-06T01:32:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:56.768+0000 7f9a3106c700 1 -- 172.21.15.53:0/1432496509 wait complete. 2023-11-06T01:32:57.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.113+0000 7f3293861700 1 -- 172.21.15.53:0/2480027843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f328c100010 msgr2=0x7f328c1003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.113+0000 7f3293861700 1 --2- 172.21.15.53:0/2480027843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f328c100010 0x7f328c1003f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3280009990 tx=0x7f3280009ca0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.113+0000 7f3293861700 1 -- 172.21.15.53:0/2480027843 shutdown_connections 2023-11-06T01:32:57.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.113+0000 7f3293861700 1 --2- 172.21.15.53:0/2480027843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f328c100010 0x7f328c1003f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.113+0000 7f3293861700 1 --2- 172.21.15.53:0/2480027843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c1022b0 0x7f328c10cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.113+0000 7f3293861700 1 --2- 172.21.15.53:0/2480027843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c1018f0 0x7f328c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.113+0000 7f3293861700 1 -- 172.21.15.53:0/2480027843 >> 172.21.15.53:0/2480027843 conn(0x7f328c076d80 msgr2=0x7f328c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:57.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.113+0000 7f3293861700 1 -- 172.21.15.53:0/2480027843 shutdown_connections 2023-11-06T01:32:57.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.114+0000 7f3293861700 1 -- 172.21.15.53:0/2480027843 wait complete. 2023-11-06T01:32:57.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3293861700 1 Processor -- start 2023-11-06T01:32:57.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3293861700 1 -- start start 2023-11-06T01:32:57.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3293861700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100010 0x7f328c194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3293861700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c1018f0 0x7f328c1952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3293861700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f328c1022b0 0x7f328c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3293861700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c199d40 con 0x7f328c1018f0 2023-11-06T01:32:57.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3293861700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c199eb0 con 0x7f328c1022b0 2023-11-06T01:32:57.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3293861700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c19a070 con 0x7f328c100010 2023-11-06T01:32:57.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3290dfc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c1018f0 0x7f328c1952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3290dfc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c1018f0 0x7f328c1952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38372/0 (socket says 172.21.15.53:38372) 2023-11-06T01:32:57.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3290dfc700 1 -- 172.21.15.53:0/3879676300 learned_addr learned my addr 172.21.15.53:0/3879676300 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:57.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f3291dfe700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f328c1022b0 0x7f328c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.115+0000 7f32915fd700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100010 0x7f328c194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.116+0000 7f3290dfc700 1 -- 172.21.15.53:0/3879676300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100010 msgr2=0x7f328c194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.116+0000 7f3290dfc700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100010 0x7f328c194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.116+0000 7f3290dfc700 1 -- 172.21.15.53:0/3879676300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f328c1022b0 msgr2=0x7f328c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.116+0000 7f3290dfc700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f328c1022b0 0x7f328c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.116+0000 7f3290dfc700 1 -- 172.21.15.53:0/3879676300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32800095f0 con 0x7f328c1018f0 2023-11-06T01:32:57.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.116+0000 7f3290dfc700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c1018f0 0x7f328c1952c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f328400ef10 tx=0x7f328400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.117+0000 7f327e7fc700 1 -- 172.21.15.53:0/3879676300 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328400c9c0 con 0x7f328c1018f0 2023-11-06T01:32:57.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.117+0000 7f327e7fc700 1 -- 172.21.15.53:0/3879676300 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f328400f040 con 0x7f328c1018f0 2023-11-06T01:32:57.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.117+0000 7f327e7fc700 1 -- 172.21.15.53:0/3879676300 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284014480 con 0x7f328c1018f0 2023-11-06T01:32:57.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.117+0000 7f3293861700 1 -- 172.21.15.53:0/3879676300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c19a2f0 con 0x7f328c1018f0 2023-11-06T01:32:57.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.117+0000 7f3293861700 1 -- 172.21.15.53:0/3879676300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c1a5fc0 con 0x7f328c1018f0 2023-11-06T01:32:57.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.119+0000 7f3293861700 1 -- 172.21.15.53:0/3879676300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3270000ff0 con 0x7f328c1018f0 2023-11-06T01:32:57.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.119+0000 7f327e7fc700 1 -- 172.21.15.53:0/3879676300 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f328400cb20 con 0x7f328c1018f0 2023-11-06T01:32:57.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.120+0000 7f327e7fc700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f32780a06e0 0x7f32780a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.120+0000 7f32915fd700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f32780a06e0 0x7f32780a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.120+0000 7f327e7fc700 1 -- 172.21.15.53:0/3879676300 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f32840cbaa0 con 0x7f328c1018f0 2023-11-06T01:32:57.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.120+0000 7f327e7fc700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850] conn(0x7f32780a9ec0 0x7f32780ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.120+0000 7f327e7fc700 1 -- 172.21.15.53:0/3879676300 --> [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32780ac990 con 0x7f32780a9ec0 2023-11-06T01:32:57.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.120+0000 7f327e7fc700 1 -- 172.21.15.53:0/3879676300 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32840d02c0 con 0x7f328c1018f0 2023-11-06T01:32:57.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.120+0000 7f3291dfe700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850] conn(0x7f32780a9ec0 0x7f32780ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.120+0000 7f32915fd700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f32780a06e0 0x7f32780a2ba0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f328000b2e0 tx=0x7f3280005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.121+0000 7f3291dfe700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850] conn(0x7f32780a9ec0 0x7f32780ac2e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.122+0000 7f327e7fc700 1 -- 172.21.15.53:0/3879676300 <== osd.9 v2:172.21.15.53:6826/825228850 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f32780ac990 con 0x7f32780a9ec0 2023-11-06T01:32:57.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.143+0000 7f49e2017700 1 -- 172.21.15.53:0/3577000278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49dc1018f0 msgr2=0x7f49dc101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.143+0000 7f49e2017700 1 --2- 172.21.15.53:0/3577000278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49dc1018f0 0x7f49dc101d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009920 tx=0x7f49d0009c30 comp rx=0 tx=0).stop 2023-11-06T01:32:57.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.143+0000 7f49e2017700 1 -- 172.21.15.53:0/3577000278 shutdown_connections 2023-11-06T01:32:57.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.143+0000 7f49e2017700 1 --2- 172.21.15.53:0/3577000278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49dc100010 0x7f49dc1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.143+0000 7f49e2017700 1 --2- 172.21.15.53:0/3577000278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc1022b0 0x7f49dc10cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.143+0000 7f49e2017700 1 --2- 172.21.15.53:0/3577000278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49dc1018f0 0x7f49dc101d70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.143+0000 7f49e2017700 1 -- 172.21.15.53:0/3577000278 >> 172.21.15.53:0/3577000278 conn(0x7f49dc076d80 msgr2=0x7f49dc077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:57.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.143+0000 7f49e2017700 1 -- 172.21.15.53:0/3577000278 shutdown_connections 2023-11-06T01:32:57.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.143+0000 7f49e2017700 1 -- 172.21.15.53:0/3577000278 wait complete. 2023-11-06T01:32:57.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.144+0000 7f49e2017700 1 Processor -- start 2023-11-06T01:32:57.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49e2017700 1 -- start start 2023-11-06T01:32:57.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49e2017700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49dc100010 0x7f49dc197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49e2017700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc1018f0 0x7f49dc197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49e2017700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49dc1022b0 0x7f49dc19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49e2017700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc19c0c0 con 0x7f49dc1018f0 2023-11-06T01:32:57.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49e2017700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc19c230 con 0x7f49dc1022b0 2023-11-06T01:32:57.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49e2017700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc19c3f0 con 0x7f49dc100010 2023-11-06T01:32:57.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49daffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc1018f0 0x7f49dc197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49db7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49dc100010 0x7f49dc197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49daffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc1018f0 0x7f49dc197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38388/0 (socket says 172.21.15.53:38388) 2023-11-06T01:32:57.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49daffd700 1 -- 172.21.15.53:0/463499113 learned_addr learned my addr 172.21.15.53:0/463499113 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:57.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.145+0000 7f49dbfff700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49dc1022b0 0x7f49dc19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f49daffd700 1 -- 172.21.15.53:0/463499113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49dc100010 msgr2=0x7f49dc197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f49daffd700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49dc100010 0x7f49dc197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f49daffd700 1 -- 172.21.15.53:0/463499113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49dc1022b0 msgr2=0x7f49dc19bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f49daffd700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49dc1022b0 0x7f49dc19bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f49daffd700 1 -- 172.21.15.53:0/463499113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4009d00 con 0x7f49dc1018f0 2023-11-06T01:32:57.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f3293861700 1 -- 172.21.15.53:0/3879676300 --> [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3270002d70 con 0x7f32780a9ec0 2023-11-06T01:32:57.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f49daffd700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc1018f0 0x7f49dc197640 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009230 tx=0x7f49d000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f327e7fc700 1 -- 172.21.15.53:0/3879676300 <== osd.9 v2:172.21.15.53:6826/825228850 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3270002d70 con 0x7f32780a9ec0 2023-11-06T01:32:57.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f49d8ff9700 1 -- 172.21.15.53:0/463499113 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d001c070 con 0x7f49dc1018f0 2023-11-06T01:32:57.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f49e2017700 1 -- 172.21.15.53:0/463499113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d0009520 con 0x7f49dc1018f0 2023-11-06T01:32:57.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f49e2017700 1 -- 172.21.15.53:0/463499113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc10d590 con 0x7f49dc1018f0 2023-11-06T01:32:57.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f49d8ff9700 1 -- 172.21.15.53:0/463499113 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49d000bb20 con 0x7f49dc1018f0 2023-11-06T01:32:57.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.146+0000 7f49d8ff9700 1 -- 172.21.15.53:0/463499113 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d00038a0 con 0x7f49dc1018f0 2023-11-06T01:32:57.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.147+0000 7f3293861700 1 -- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850] conn(0x7f32780a9ec0 msgr2=0x7f32780ac2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.147+0000 7f3293861700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850] conn(0x7f32780a9ec0 0x7f32780ac2e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.147+0000 7f49e2017700 1 -- 172.21.15.53:0/463499113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49c8000ff0 con 0x7f49dc1018f0 2023-11-06T01:32:57.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.147+0000 7f3293861700 1 -- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f32780a06e0 msgr2=0x7f32780a2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.147+0000 7f3293861700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f32780a06e0 0x7f32780a2ba0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f328000b2e0 tx=0x7f3280005fb0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.147+0000 7f3293861700 1 -- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c1018f0 msgr2=0x7f328c1952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.147+0000 7f3293861700 1 --2- 172.21.15.53:0/3879676300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c1018f0 0x7f328c1952c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f328400ef10 tx=0x7f328400c3a0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.148+0000 7f3291dfe700 1 -- 172.21.15.53:0/3879676300 reap_dead start 2023-11-06T01:32:57.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.148+0000 7f3293861700 1 -- 172.21.15.53:0/3879676300 shutdown_connections 2023-11-06T01:32:57.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.148+0000 7f3293861700 1 -- 172.21.15.53:0/3879676300 >> 172.21.15.53:0/3879676300 conn(0x7f328c076d80 msgr2=0x7f328c103890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:57.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.089+0000 7f97b8076700 1 -- 172.21.15.53:0/1845721703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b010b3d0 msgr2=0x7f97b010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.089+0000 7f97b8076700 1 --2- 172.21.15.53:0/1845721703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b010b3d0 0x7f97b010d840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f97a800b0a0 tx=0x7f97a800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.090+0000 7f97b8076700 1 -- 172.21.15.53:0/1845721703 shutdown_connections 2023-11-06T01:32:57.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.090+0000 7f97b8076700 1 --2- 172.21.15.53:0/1845721703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b010b3d0 0x7f97b010d840 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.090+0000 7f97b8076700 1 --2- 172.21.15.53:0/1845721703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b0101fa0 0x7f97b0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.090+0000 7f97b8076700 1 --2- 172.21.15.53:0/1845721703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97b01028c0 0x7f97b010adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.090+0000 7f97b8076700 1 -- 172.21.15.53:0/1845721703 >> 172.21.15.53:0/1845721703 conn(0x7f97b00fb750 msgr2=0x7f97b00fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:57.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.090+0000 7f97b8076700 1 -- 172.21.15.53:0/1845721703 shutdown_connections 2023-11-06T01:32:57.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.090+0000 7f97b8076700 1 -- 172.21.15.53:0/1845721703 wait complete. 2023-11-06T01:32:57.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b8076700 1 Processor -- start 2023-11-06T01:32:57.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b8076700 1 -- start start 2023-11-06T01:32:57.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b8076700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97b0101fa0 0x7f97b00ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b8076700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b01028c0 0x7f97b0100140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b8076700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b010b3d0 0x7f97b01006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b8076700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01a53f0 con 0x7f97b01028c0 2023-11-06T01:32:57.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b8076700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01a5560 con 0x7f97b0101fa0 2023-11-06T01:32:57.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b8076700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01a5720 con 0x7f97b010b3d0 2023-11-06T01:32:57.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b5611700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b01028c0 0x7f97b0100140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b5611700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b01028c0 0x7f97b0100140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38358/0 (socket says 172.21.15.53:38358) 2023-11-06T01:32:57.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b5611700 1 -- 172.21.15.53:0/1216484600 learned_addr learned my addr 172.21.15.53:0/1216484600 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:57.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b6613700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b010b3d0 0x7f97b01006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.091+0000 7f97b5e12700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97b0101fa0 0x7f97b00ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.092+0000 7f97b5611700 1 -- 172.21.15.53:0/1216484600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b010b3d0 msgr2=0x7f97b01006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.092+0000 7f97b5611700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b010b3d0 0x7f97b01006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.092+0000 7f97b5611700 1 -- 172.21.15.53:0/1216484600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97b0101fa0 msgr2=0x7f97b00ffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.092+0000 7f97b5611700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97b0101fa0 0x7f97b00ffc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.092+0000 7f97b5611700 1 -- 172.21.15.53:0/1216484600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8009d00 con 0x7f97b01028c0 2023-11-06T01:32:57.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.092+0000 7f97b5611700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b01028c0 0x7f97b0100140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f97a000d950 tx=0x7f97a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.092+0000 7f97a6ffd700 1 -- 172.21.15.53:0/1216484600 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0004220 con 0x7f97b01028c0 2023-11-06T01:32:57.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.092+0000 7f97a6ffd700 1 -- 172.21.15.53:0/1216484600 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97a0007620 con 0x7f97b01028c0 2023-11-06T01:32:57.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.092+0000 7f97b8076700 1 -- 172.21.15.53:0/1216484600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b01a5a00 con 0x7f97b01028c0 2023-11-06T01:32:57.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.092+0000 7f97a6ffd700 1 -- 172.21.15.53:0/1216484600 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a00056c0 con 0x7f97b01028c0 2023-11-06T01:32:57.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.092+0000 7f97b8076700 1 -- 172.21.15.53:0/1216484600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b01a5f00 con 0x7f97b01028c0 2023-11-06T01:32:57.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.093+0000 7f97b8076700 1 -- 172.21.15.53:0/1216484600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97b0110fd0 con 0x7f97b01028c0 2023-11-06T01:32:57.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.095+0000 7f97a6ffd700 1 -- 172.21.15.53:0/1216484600 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f97a0004380 con 0x7f97b01028c0 2023-11-06T01:32:57.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.096+0000 7f97a6ffd700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f979c0a05b0 0x7f979c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.096+0000 7f97b5e12700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f979c0a05b0 0x7f979c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.096+0000 7f97a6ffd700 1 -- 172.21.15.53:0/1216484600 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f97a00cbcc0 con 0x7f97b01028c0 2023-11-06T01:32:57.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.096+0000 7f97a6ffd700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819] conn(0x7f979c0a9d90 0x7f979c0ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.096+0000 7f97a6ffd700 1 -- 172.21.15.53:0/1216484600 --> [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f979c0ac860 con 0x7f979c0a9d90 2023-11-06T01:32:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.096+0000 7f97a6ffd700 1 -- 172.21.15.53:0/1216484600 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97a00cc160 con 0x7f97b01028c0 2023-11-06T01:32:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.097+0000 7f97b6613700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819] conn(0x7f979c0a9d90 0x7f979c0ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.097+0000 7f97b5e12700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f979c0a05b0 0x7f979c0a2a70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009c60 tx=0x7f97ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.097+0000 7f97b6613700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819] conn(0x7f979c0a9d90 0x7f979c0ac1b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.098+0000 7f97a6ffd700 1 -- 172.21.15.53:0/1216484600 <== osd.6 v2:172.21.15.53:6818/4140594819 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f979c0ac860 con 0x7f979c0a9d90 2023-11-06T01:32:57.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.131+0000 7f97b8076700 1 -- 172.21.15.53:0/1216484600 --> [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f97b0068a20 con 0x7f979c0a9d90 2023-11-06T01:32:57.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.132+0000 7f97a6ffd700 1 -- 172.21.15.53:0/1216484600 <== osd.6 v2:172.21.15.53:6818/4140594819 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f97b0068a20 con 0x7f979c0a9d90 2023-11-06T01:32:57.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.132+0000 7f97b8076700 1 -- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819] conn(0x7f979c0a9d90 msgr2=0x7f979c0ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.132+0000 7f97b8076700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819] conn(0x7f979c0a9d90 0x7f979c0ac1b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.132+0000 7f97b8076700 1 -- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f979c0a05b0 msgr2=0x7f979c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.132+0000 7f97b8076700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f979c0a05b0 0x7f979c0a2a70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009c60 tx=0x7f97ac009260 comp rx=0 tx=0).stop 2023-11-06T01:32:57.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.132+0000 7f97b8076700 1 -- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b01028c0 msgr2=0x7f97b0100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.132+0000 7f97b8076700 1 --2- 172.21.15.53:0/1216484600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b01028c0 0x7f97b0100140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f97a000d950 tx=0x7f97a000dc60 comp rx=0 tx=0).stop 2023-11-06T01:32:57.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.132+0000 7f97b6613700 1 -- 172.21.15.53:0/1216484600 reap_dead start 2023-11-06T01:32:57.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.132+0000 7f97b8076700 1 -- 172.21.15.53:0/1216484600 shutdown_connections 2023-11-06T01:32:57.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.133+0000 7f97b8076700 1 -- 172.21.15.53:0/1216484600 >> 172.21.15.53:0/1216484600 conn(0x7f97b00fb750 msgr2=0x7f97b0102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:57.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.133+0000 7f97b8076700 1 -- 172.21.15.53:0/1216484600 shutdown_connections 2023-11-06T01:32:57.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.133+0000 7f97b8076700 1 -- 172.21.15.53:0/1216484600 wait complete. 2023-11-06T01:32:57.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.149+0000 7f49d8ff9700 1 -- 172.21.15.53:0/463499113 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f49d0020030 con 0x7f49dc1018f0 2023-11-06T01:32:57.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.150+0000 7f49d8ff9700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49c40a0630 0x7f49c40a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.150+0000 7f49d8ff9700 1 -- 172.21.15.53:0/463499113 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f49d00ccbf0 con 0x7f49dc1018f0 2023-11-06T01:32:57.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.151+0000 7f49d8ff9700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587] conn(0x7f49c40a9e10 0x7f49c40ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.151+0000 7f49d8ff9700 1 -- 172.21.15.53:0/463499113 --> [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49c40ac8e0 con 0x7f49c40a9e10 2023-11-06T01:32:57.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.151+0000 7f49d8ff9700 1 -- 172.21.15.53:0/463499113 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d00cd090 con 0x7f49dc1018f0 2023-11-06T01:32:57.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.151+0000 7f49db7fe700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49c40a0630 0x7f49c40a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.151+0000 7f49dbfff700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587] conn(0x7f49c40a9e10 0x7f49c40ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.151+0000 7f49db7fe700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49c40a0630 0x7f49c40a2af0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009c60 tx=0x7f49cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.148+0000 7f3293861700 1 -- 172.21.15.53:0/3879676300 shutdown_connections 2023-11-06T01:32:57.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.152+0000 7f49dbfff700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587] conn(0x7f49c40a9e10 0x7f49c40ac230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.152+0000 7f3293861700 1 -- 172.21.15.53:0/3879676300 wait complete. 2023-11-06T01:32:57.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.152+0000 7f49d8ff9700 1 -- 172.21.15.53:0/463499113 <== osd.7 v2:172.21.15.62:6816/1541716587 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f49c40ac8e0 con 0x7f49c40a9e10 2023-11-06T01:32:57.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.171+0000 7f49e2017700 1 -- 172.21.15.53:0/463499113 --> [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f49c8002d70 con 0x7f49c40a9e10 2023-11-06T01:32:57.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.171+0000 7f49d8ff9700 1 -- 172.21.15.53:0/463499113 <== osd.7 v2:172.21.15.62:6816/1541716587 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f49c8002d70 con 0x7f49c40a9e10 2023-11-06T01:32:57.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.172+0000 7f49e2017700 1 -- 172.21.15.53:0/463499113 >> [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587] conn(0x7f49c40a9e10 msgr2=0x7f49c40ac230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.172+0000 7f49e2017700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587] conn(0x7f49c40a9e10 0x7f49c40ac230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.172+0000 7f49e2017700 1 -- 172.21.15.53:0/463499113 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49c40a0630 msgr2=0x7f49c40a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.172+0000 7f49e2017700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49c40a0630 0x7f49c40a2af0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009c60 tx=0x7f49cc009260 comp rx=0 tx=0).stop 2023-11-06T01:32:57.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.172+0000 7f49e2017700 1 -- 172.21.15.53:0/463499113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc1018f0 msgr2=0x7f49dc197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.172+0000 7f49e2017700 1 --2- 172.21.15.53:0/463499113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc1018f0 0x7f49dc197640 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009230 tx=0x7f49d000fa00 comp rx=0 tx=0).stop 2023-11-06T01:32:57.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.172+0000 7f49dbfff700 1 -- 172.21.15.53:0/463499113 reap_dead start 2023-11-06T01:32:57.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.172+0000 7f49e2017700 1 -- 172.21.15.53:0/463499113 shutdown_connections 2023-11-06T01:32:57.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.172+0000 7f49e2017700 1 -- 172.21.15.53:0/463499113 >> 172.21.15.53:0/463499113 conn(0x7f49dc076d80 msgr2=0x7f49dc103890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:57.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.172+0000 7f49e2017700 1 -- 172.21.15.53:0/463499113 shutdown_connections 2023-11-06T01:32:57.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.172+0000 7f49e2017700 1 -- 172.21.15.53:0/463499113 wait complete. 2023-11-06T01:32:57.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.189+0000 7fdfc07ee700 1 -- 172.21.15.53:0/3689303580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb8100010 msgr2=0x7fdfb81003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.189+0000 7fdfc07ee700 1 --2- 172.21.15.53:0/3689303580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb8100010 0x7fdfb81003f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40099c0 tx=0x7fdfb4009cd0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.189+0000 7fdfc07ee700 1 -- 172.21.15.53:0/3689303580 shutdown_connections 2023-11-06T01:32:57.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.189+0000 7fdfc07ee700 1 --2- 172.21.15.53:0/3689303580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfb81022b0 0x7fdfb810cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.189+0000 7fdfc07ee700 1 --2- 172.21.15.53:0/3689303580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb8100010 0x7fdfb81003f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.189+0000 7fdfc07ee700 1 --2- 172.21.15.53:0/3689303580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfb81018f0 0x7fdfb8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.189+0000 7fdfc07ee700 1 -- 172.21.15.53:0/3689303580 >> 172.21.15.53:0/3689303580 conn(0x7fdfb8076d80 msgr2=0x7fdfb8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:57.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.189+0000 7fdfc07ee700 1 -- 172.21.15.53:0/3689303580 shutdown_connections 2023-11-06T01:32:57.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.189+0000 7fdfc07ee700 1 -- 172.21.15.53:0/3689303580 wait complete. 2023-11-06T01:32:57.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.190+0000 7fdfc07ee700 1 Processor -- start 2023-11-06T01:32:57.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.190+0000 7fdfc07ee700 1 -- start start 2023-11-06T01:32:57.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.190+0000 7fdfc07ee700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfb8100010 0x7fdfb8195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.190+0000 7fdfc07ee700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb81018f0 0x7fdfb8196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.190+0000 7fdfc07ee700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfb81022b0 0x7fdfb819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.190+0000 7fdfc07ee700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb819af00 con 0x7fdfb81018f0 2023-11-06T01:32:57.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.190+0000 7fdfc07ee700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb819b070 con 0x7fdfb81022b0 2023-11-06T01:32:57.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.190+0000 7fdfc07ee700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb819b230 con 0x7fdfb8100010 2023-11-06T01:32:57.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfbdd89700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb81018f0 0x7fdfb8196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfbdd89700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb81018f0 0x7fdfb8196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38402/0 (socket says 172.21.15.53:38402) 2023-11-06T01:32:57.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfbdd89700 1 -- 172.21.15.53:0/469801258 learned_addr learned my addr 172.21.15.53:0/469801258 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:32:57.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfbed8b700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfb81022b0 0x7fdfb819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfbdd89700 1 -- 172.21.15.53:0/469801258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfb8100010 msgr2=0x7fdfb8195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfbdd89700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfb8100010 0x7fdfb8195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfbdd89700 1 -- 172.21.15.53:0/469801258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfb81022b0 msgr2=0x7fdfb819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfbdd89700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfb81022b0 0x7fdfb819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfbdd89700 1 -- 172.21.15.53:0/469801258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb40095f0 con 0x7fdfb81018f0 2023-11-06T01:32:57.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfbed8b700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfb81022b0 0x7fdfb819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:32:57.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfbdd89700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb81018f0 0x7fdfb8196480 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8007b50 tx=0x7fdfa8007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfaf7fe700 1 -- 172.21.15.53:0/469801258 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8011070 con 0x7fdfb81018f0 2023-11-06T01:32:57.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfc07ee700 1 -- 172.21.15.53:0/469801258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb819b510 con 0x7fdfb81018f0 2023-11-06T01:32:57.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfaf7fe700 1 -- 172.21.15.53:0/469801258 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfa800c2e0 con 0x7fdfb81018f0 2023-11-06T01:32:57.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfaf7fe700 1 -- 172.21.15.53:0/469801258 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa800e5c0 con 0x7fdfb81018f0 2023-11-06T01:32:57.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.191+0000 7fdfc07ee700 1 -- 172.21.15.53:0/469801258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb80692b0 con 0x7fdfb81018f0 2023-11-06T01:32:57.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.192+0000 7fdfc07ee700 1 -- 172.21.15.53:0/469801258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdfb8110030 con 0x7fdfb81018f0 2023-11-06T01:32:57.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.193+0000 7fdfaf7fe700 1 -- 172.21.15.53:0/469801258 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fdfa8004750 con 0x7fdfb81018f0 2023-11-06T01:32:57.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.194+0000 7fdfaf7fe700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdfa40a05b0 0x7fdfa40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.194+0000 7fdfbe58a700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdfa40a05b0 0x7fdfa40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.194+0000 7fdfaf7fe700 1 -- 172.21.15.53:0/469801258 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdfa80cd7c0 con 0x7fdfb81018f0 2023-11-06T01:32:57.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.194+0000 7fdfaf7fe700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073] conn(0x7fdfa40a9d90 0x7fdfa40ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:32:57.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.194+0000 7fdfaf7fe700 1 -- 172.21.15.53:0/469801258 --> [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfa40ac860 con 0x7fdfa40a9d90 2023-11-06T01:32:57.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.194+0000 7fdfaf7fe700 1 -- 172.21.15.53:0/469801258 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfa80cdb30 con 0x7fdfb81018f0 2023-11-06T01:32:57.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.194+0000 7fdfbed8b700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073] conn(0x7fdfa40a9d90 0x7fdfa40ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:32:57.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.195+0000 7fdfbe58a700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdfa40a05b0 0x7fdfa40a2a70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009990 tx=0x7fdfb4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.195+0000 7fdfbed8b700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073] conn(0x7fdfa40a9d90 0x7fdfa40ac1b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:32:57.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.196+0000 7fdfaf7fe700 1 -- 172.21.15.53:0/469801258 <== osd.0 v2:172.21.15.53:6802/2480021073 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdfa40ac860 con 0x7fdfa40a9d90 2023-11-06T01:32:57.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.213+0000 7fdfc07ee700 1 -- 172.21.15.53:0/469801258 --> [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdfb804ea90 con 0x7fdfa40a9d90 2023-11-06T01:32:57.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfaf7fe700 1 -- 172.21.15.53:0/469801258 <== osd.0 v2:172.21.15.53:6802/2480021073 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdfb804ea90 con 0x7fdfa40a9d90 2023-11-06T01:32:57.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfc07ee700 1 -- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073] conn(0x7fdfa40a9d90 msgr2=0x7fdfa40ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfc07ee700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073] conn(0x7fdfa40a9d90 0x7fdfa40ac1b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfc07ee700 1 -- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdfa40a05b0 msgr2=0x7fdfa40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfc07ee700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdfa40a05b0 0x7fdfa40a2a70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009990 tx=0x7fdfb4000bc0 comp rx=0 tx=0).stop 2023-11-06T01:32:57.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfc07ee700 1 -- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb81018f0 msgr2=0x7fdfb8196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:32:57.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfc07ee700 1 --2- 172.21.15.53:0/469801258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb81018f0 0x7fdfb8196480 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8007b50 tx=0x7fdfa8007f10 comp rx=0 tx=0).stop 2023-11-06T01:32:57.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfbed8b700 1 -- 172.21.15.53:0/469801258 reap_dead start 2023-11-06T01:32:57.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfc07ee700 1 -- 172.21.15.53:0/469801258 shutdown_connections 2023-11-06T01:32:57.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfc07ee700 1 -- 172.21.15.53:0/469801258 >> 172.21.15.53:0/469801258 conn(0x7fdfb8076d80 msgr2=0x7fdfb8103890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:32:57.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfc07ee700 1 -- 172.21.15.53:0/469801258 shutdown_connections 2023-11-06T01:32:57.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:32:57.214+0000 7fdfc07ee700 1 -- 172.21.15.53:0/469801258 wait complete. 2023-11-06T01:32:57.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:57 smithi053 ceph-mon[99472]: pgmap v267: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 11 op/s 2023-11-06T01:32:57.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:57 smithi062 ceph-mon[103924]: pgmap v267: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 11 op/s 2023-11-06T01:32:57.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:57 smithi163 ceph-mon[104416]: pgmap v267: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 11 op/s 2023-11-06T01:32:57.749 INFO:teuthology.orchestra.run.smithi053.stdout:219043332129 2023-11-06T01:32:57.750 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-11-06T01:32:58.085 INFO:teuthology.orchestra.run.smithi053.stdout:34359738432 2023-11-06T01:32:58.086 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-11-06T01:32:58.458 INFO:teuthology.orchestra.run.smithi053.stdout:197568495654 2023-11-06T01:32:58.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-11-06T01:32:58.502 INFO:teuthology.orchestra.run.smithi053.stdout:279172874266 2023-11-06T01:32:58.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-11-06T01:32:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:58.807 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:59.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:32:59 smithi053 ceph-mon[99472]: pgmap v268: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2023-11-06T01:32:59.153 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:77309411384 2023-11-06T01:32:59.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-11-06T01:32:59.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:32:59 smithi062 ceph-mon[103924]: pgmap v268: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2023-11-06T01:32:59.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:32:59 smithi163 ceph-mon[104416]: pgmap v268: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2023-11-06T01:32:59.847 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:32:59.920 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:00.794 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:00.981 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:01.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:01 smithi053 ceph-mon[99472]: pgmap v269: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2023-11-06T01:33:01.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.357+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1513879899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3cc069270 msgr2=0x7fb3cc10dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:01.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.357+0000 7fb3d0ed5700 1 --2- 172.21.15.53:0/1513879899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3cc069270 0x7fb3cc10dbd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400b0a0 tx=0x7fb3c400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:01.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.357+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1513879899 shutdown_connections 2023-11-06T01:33:01.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.357+0000 7fb3d0ed5700 1 --2- 172.21.15.53:0/1513879899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3cc106f20 0x7fb3cc107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:01.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.357+0000 7fb3d0ed5700 1 --2- 172.21.15.53:0/1513879899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3cc069270 0x7fb3cc10dbd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:01.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.357+0000 7fb3d0ed5700 1 --2- 172.21.15.53:0/1513879899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc0688b0 0x7fb3cc068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:01.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.357+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1513879899 >> 172.21.15.53:0/1513879899 conn(0x7fb3cc076d90 msgr2=0x7fb3cc0771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:01.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.357+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1513879899 shutdown_connections 2023-11-06T01:33:01.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.357+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1513879899 wait complete. 2023-11-06T01:33:01.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.358+0000 7fb3d0ed5700 1 Processor -- start 2023-11-06T01:33:01.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.358+0000 7fb3d0ed5700 1 -- start start 2023-11-06T01:33:01.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3d0ed5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3cc0688b0 0x7fb3cc196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:01.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3d0ed5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3cc069270 0x7fb3cc197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:01.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3d0ed5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc106f20 0x7fb3cc19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:01.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3d0ed5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc19bef0 con 0x7fb3cc069270 2023-11-06T01:33:01.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3d0ed5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc19c060 con 0x7fb3cc0688b0 2023-11-06T01:33:01.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3d0ed5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc19c220 con 0x7fb3cc106f20 2023-11-06T01:33:01.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3ca59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3cc0688b0 0x7fb3cc196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:01.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3cad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc106f20 0x7fb3cc19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:01.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3c9d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3cc069270 0x7fb3cc197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:01.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3cad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc106f20 0x7fb3cc19b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:52660/0 (socket says 172.21.15.53:52660) 2023-11-06T01:33:01.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3ca59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3cc0688b0 0x7fb3cc196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:49854/0 (socket says 172.21.15.53:49854) 2023-11-06T01:33:01.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3cad9d700 1 -- 172.21.15.53:0/1305792343 learned_addr learned my addr 172.21.15.53:0/1305792343 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:01.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3c9d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3cc069270 0x7fb3cc197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38418/0 (socket says 172.21.15.53:38418) 2023-11-06T01:33:01.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3c9d9b700 1 -- 172.21.15.53:0/1305792343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc106f20 msgr2=0x7fb3cc19b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:01.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3c9d9b700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc106f20 0x7fb3cc19b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:01.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3c9d9b700 1 -- 172.21.15.53:0/1305792343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3cc0688b0 msgr2=0x7fb3cc196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:01.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3c9d9b700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3cc0688b0 0x7fb3cc196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:01.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3c9d9b700 1 -- 172.21.15.53:0/1305792343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c4009d00 con 0x7fb3cc069270 2023-11-06T01:33:01.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.359+0000 7fb3c9d9b700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3cc069270 0x7fb3cc197470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000d750 tx=0x7fb3c000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:01.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.360+0000 7fb3bb7fe700 1 -- 172.21.15.53:0/1305792343 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0004a30 con 0x7fb3cc069270 2023-11-06T01:33:01.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.360+0000 7fb3bb7fe700 1 -- 172.21.15.53:0/1305792343 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3c00054e0 con 0x7fb3cc069270 2023-11-06T01:33:01.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.360+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1305792343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc19c500 con 0x7fb3cc069270 2023-11-06T01:33:01.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.360+0000 7fb3bb7fe700 1 -- 172.21.15.53:0/1305792343 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c000fb50 con 0x7fb3cc069270 2023-11-06T01:33:01.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.360+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1305792343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc1a3d60 con 0x7fb3cc069270 2023-11-06T01:33:01.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.361+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1305792343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc04ea90 con 0x7fb3cc069270 2023-11-06T01:33:01.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.363+0000 7fb3bb7fe700 1 -- 172.21.15.53:0/1305792343 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fb3c0004b90 con 0x7fb3cc069270 2023-11-06T01:33:01.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.363+0000 7fb3bb7fe700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3b40a05b0 0x7fb3b40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:01.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.363+0000 7fb3ca59c700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3b40a05b0 0x7fb3b40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:01.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.363+0000 7fb3bb7fe700 1 -- 172.21.15.53:0/1305792343 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb3c00cb3f0 con 0x7fb3cc069270 2023-11-06T01:33:01.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.364+0000 7fb3ca59c700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3b40a05b0 0x7fb3b40a2a70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005950 tx=0x7fb3bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:01.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.369+0000 7fb3bb7fe700 1 -- 172.21.15.53:0/1305792343 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb3c0094ee0 con 0x7fb3cc069270 2023-11-06T01:33:01.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:01 smithi062 ceph-mon[103924]: pgmap v269: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2023-11-06T01:33:01.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:01 smithi163 ceph-mon[104416]: pgmap v269: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2023-11-06T01:33:01.533 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:01.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.605+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1305792343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fb3cc1a3ef0 con 0x7fb3cc069270 2023-11-06T01:33:01.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.605+0000 7fb3bb7fe700 1 -- 172.21.15.53:0/1305792343 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fb3c0098500 con 0x7fb3cc069270 2023-11-06T01:33:01.607 INFO:teuthology.orchestra.run.smithi053.stdout:300647710742 2023-11-06T01:33:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.607+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1305792343 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3b40a05b0 msgr2=0x7fb3b40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.607+0000 7fb3d0ed5700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3b40a05b0 0x7fb3b40a2a70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005950 tx=0x7fb3bc00b410 comp rx=0 tx=0).stop 2023-11-06T01:33:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.607+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1305792343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3cc069270 msgr2=0x7fb3cc197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.607+0000 7fb3d0ed5700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3cc069270 0x7fb3cc197470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000d750 tx=0x7fb3c000da60 comp rx=0 tx=0).stop 2023-11-06T01:33:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.608+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1305792343 shutdown_connections 2023-11-06T01:33:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.608+0000 7fb3d0ed5700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3b40a05b0 0x7fb3b40a2a70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.608+0000 7fb3d0ed5700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3cc0688b0 0x7fb3cc196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.608+0000 7fb3d0ed5700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc106f20 0x7fb3cc19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:01.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.608+0000 7fb3d0ed5700 1 --2- 172.21.15.53:0/1305792343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3cc069270 0x7fb3cc197470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:01.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.608+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1305792343 >> 172.21.15.53:0/1305792343 conn(0x7fb3cc076d90 msgr2=0x7fb3cc0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:01.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.608+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1305792343 shutdown_connections 2023-11-06T01:33:01.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:01.608+0000 7fb3d0ed5700 1 -- 172.21.15.53:0/1305792343 wait complete. 2023-11-06T01:33:02.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:02 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1305792343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-06T01:33:02.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:02 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1305792343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-06T01:33:02.461 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710741 got 300647710742 for osd.10 2023-11-06T01:33:02.462 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:02.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:02 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1305792343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-06T01:33:02.794 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:03.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.214+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1304527331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c1019e0 msgr2=0x7f4b7c10dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.214+0000 7f4b83c7a700 1 --2- 172.21.15.53:0/1304527331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c1019e0 0x7f4b7c10dc80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400b0d0 tx=0x7f4b7400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.214+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1304527331 shutdown_connections 2023-11-06T01:33:03.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.214+0000 7f4b83c7a700 1 --2- 172.21.15.53:0/1304527331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b7c106fa0 0x7f4b7c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.214+0000 7f4b83c7a700 1 --2- 172.21.15.53:0/1304527331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c1019e0 0x7f4b7c10dc80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.214+0000 7f4b83c7a700 1 --2- 172.21.15.53:0/1304527331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b7c101020 0x7f4b7c1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.214+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1304527331 >> 172.21.15.53:0/1304527331 conn(0x7f4b7c078db0 msgr2=0x7f4b7c0791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.214+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1304527331 shutdown_connections 2023-11-06T01:33:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.214+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1304527331 wait complete. 2023-11-06T01:33:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.214+0000 7f4b83c7a700 1 Processor -- start 2023-11-06T01:33:03.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.214+0000 7f4b83c7a700 1 -- start start 2023-11-06T01:33:03.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b83c7a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c101020 0x7f4b7c196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b83c7a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b7c1019e0 0x7f4b7c197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b83c7a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b7c106fa0 0x7f4b7c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b83c7a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c19bf00 con 0x7f4b7c101020 2023-11-06T01:33:03.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b83c7a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c19c070 con 0x7f4b7c1019e0 2023-11-06T01:33:03.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b83c7a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c19c230 con 0x7f4b7c106fa0 2023-11-06T01:33:03.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b81a16700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c101020 0x7f4b7c196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:03.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b81a16700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c101020 0x7f4b7c196f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38444/0 (socket says 172.21.15.53:38444) 2023-11-06T01:33:03.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b81a16700 1 -- 172.21.15.53:0/1447419541 learned_addr learned my addr 172.21.15.53:0/1447419541 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:03.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b81215700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b7c1019e0 0x7f4b7c197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b82217700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b7c106fa0 0x7f4b7c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b81a16700 1 -- 172.21.15.53:0/1447419541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b7c106fa0 msgr2=0x7f4b7c19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b81a16700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b7c106fa0 0x7f4b7c19b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b81a16700 1 -- 172.21.15.53:0/1447419541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b7c1019e0 msgr2=0x7f4b7c197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b81a16700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b7c1019e0 0x7f4b7c197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b81a16700 1 -- 172.21.15.53:0/1447419541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b74009d00 con 0x7f4b7c101020 2023-11-06T01:33:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b81a16700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c101020 0x7f4b7c196f40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800d950 tx=0x7f4b7800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b72ffd700 1 -- 172.21.15.53:0/1447419541 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78009b40 con 0x7f4b7c101020 2023-11-06T01:33:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1447419541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c19c510 con 0x7f4b7c101020 2023-11-06T01:33:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b72ffd700 1 -- 172.21.15.53:0/1447419541 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b78009ca0 con 0x7f4b7c101020 2023-11-06T01:33:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b72ffd700 1 -- 172.21.15.53:0/1447419541 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800f620 con 0x7f4b7c101020 2023-11-06T01:33:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.215+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1447419541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7c1a3d70 con 0x7f4b7c101020 2023-11-06T01:33:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.216+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1447419541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7c04ea90 con 0x7f4b7c101020 2023-11-06T01:33:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.217+0000 7f4b72ffd700 1 -- 172.21.15.53:0/1447419541 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f4b78010460 con 0x7f4b7c101020 2023-11-06T01:33:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.218+0000 7f4b72ffd700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b680a05b0 0x7f4b680a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.218+0000 7f4b81215700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b680a05b0 0x7f4b680a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.218+0000 7f4b72ffd700 1 -- 172.21.15.53:0/1447419541 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4b780cbcb0 con 0x7f4b7c101020 2023-11-06T01:33:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.218+0000 7f4b81215700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b680a05b0 0x7f4b680a2a70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c005fd0 tx=0x7f4b6c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.222+0000 7f4b72ffd700 1 -- 172.21.15.53:0/1447419541 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4b780958a0 con 0x7f4b7c101020 2023-11-06T01:33:03.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:03 smithi053 ceph-mon[99472]: pgmap v270: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-11-06T01:33:03.324 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:03 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1553638265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-06T01:33:03.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:03 smithi062 ceph-mon[103924]: pgmap v270: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-11-06T01:33:03.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:03 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1553638265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-06T01:33:03.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.429+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1447419541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4b7c1982a0 con 0x7f4b7c101020 2023-11-06T01:33:03.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.429+0000 7f4b72ffd700 1 -- 172.21.15.53:0/1447419541 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b78020020 con 0x7f4b7c101020 2023-11-06T01:33:03.431 INFO:teuthology.orchestra.run.smithi053.stdout:55834574909 2023-11-06T01:33:03.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.432+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1447419541 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b680a05b0 msgr2=0x7f4b680a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.432+0000 7f4b83c7a700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b680a05b0 0x7f4b680a2a70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c005fd0 tx=0x7f4b6c005f60 comp rx=0 tx=0).stop 2023-11-06T01:33:03.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.432+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1447419541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c101020 msgr2=0x7f4b7c196f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.432+0000 7f4b83c7a700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c101020 0x7f4b7c196f40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800d950 tx=0x7f4b7800dd10 comp rx=0 tx=0).stop 2023-11-06T01:33:03.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.433+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1447419541 shutdown_connections 2023-11-06T01:33:03.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.433+0000 7f4b83c7a700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b680a05b0 0x7f4b680a2a70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.433+0000 7f4b83c7a700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b7c106fa0 0x7f4b7c19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.433+0000 7f4b83c7a700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c101020 0x7f4b7c196f40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.433+0000 7f4b83c7a700 1 --2- 172.21.15.53:0/1447419541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b7c1019e0 0x7f4b7c197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.433+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1447419541 >> 172.21.15.53:0/1447419541 conn(0x7f4b7c078db0 msgr2=0x7f4b7c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:03.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.433+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1447419541 shutdown_connections 2023-11-06T01:33:03.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.433+0000 7f4b83c7a700 1 -- 172.21.15.53:0/1447419541 wait complete. 2023-11-06T01:33:03.443 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:03.475 INFO:teuthology.orchestra.run.smithi053.stdout:167503724588 2023-11-06T01:33:03.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.724+0000 7f123979d700 1 -- 172.21.15.53:0/4151905627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12340ffcd0 msgr2=0x7f1234100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.724+0000 7f123979d700 1 --2- 172.21.15.53:0/4151905627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12340ffcd0 0x7f1234100150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f121c009990 tx=0x7f121c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.724+0000 7f123979d700 1 -- 172.21.15.53:0/4151905627 shutdown_connections 2023-11-06T01:33:03.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.724+0000 7f123979d700 1 --2- 172.21.15.53:0/4151905627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1234105d80 0x7f1234106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.724+0000 7f123979d700 1 --2- 172.21.15.53:0/4151905627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1234100690 0x7f123410cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.724+0000 7f123979d700 1 --2- 172.21.15.53:0/4151905627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12340ffcd0 0x7f1234100150 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.724+0000 7f123979d700 1 -- 172.21.15.53:0/4151905627 >> 172.21.15.53:0/4151905627 conn(0x7f1234076d90 msgr2=0x7f12340771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:03.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.724+0000 7f123979d700 1 -- 172.21.15.53:0/4151905627 shutdown_connections 2023-11-06T01:33:03.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.724+0000 7f123979d700 1 -- 172.21.15.53:0/4151905627 wait complete. 2023-11-06T01:33:03.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.724+0000 7f123979d700 1 Processor -- start 2023-11-06T01:33:03.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f123979d700 1 -- start start 2023-11-06T01:33:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f123979d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12340ffcd0 0x7f123419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f123979d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1234100690 0x7f123419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f123979d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1234105d80 0x7f1234195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f123979d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1234196140 con 0x7f1234105d80 2023-11-06T01:33:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f123979d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12341962b0 con 0x7f12340ffcd0 2023-11-06T01:33:03.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f123979d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1234196470 con 0x7f1234100690 2023-11-06T01:33:03.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f12337fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1234105d80 0x7f1234195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:03.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f12337fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1234105d80 0x7f1234195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38434/0 (socket says 172.21.15.53:38434) 2023-11-06T01:33:03.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f12337fe700 1 -- 172.21.15.53:0/1553638265 learned_addr learned my addr 172.21.15.53:0/1553638265 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:03.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f12327fc700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1234100690 0x7f123419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:03.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f1232ffd700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12340ffcd0 0x7f123419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:03.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f12337fe700 1 -- 172.21.15.53:0/1553638265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1234100690 msgr2=0x7f123419bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f12337fe700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1234100690 0x7f123419bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f12337fe700 1 -- 172.21.15.53:0/1553638265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12340ffcd0 msgr2=0x7f123419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f12337fe700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12340ffcd0 0x7f123419ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f12337fe700 1 -- 172.21.15.53:0/1553638265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c0095f0 con 0x7f1234105d80 2023-11-06T01:33:03.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.725+0000 7f12337fe700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1234105d80 0x7f1234195ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f122c007f10 tx=0x7f122c00f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:03.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.726+0000 7f122bfff700 1 -- 172.21.15.53:0/1553638265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c00fd80 con 0x7f1234105d80 2023-11-06T01:33:03.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.726+0000 7f123979d700 1 -- 172.21.15.53:0/1553638265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1234196750 con 0x7f1234105d80 2023-11-06T01:33:03.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.726+0000 7f122bfff700 1 -- 172.21.15.53:0/1553638265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f122c009360 con 0x7f1234105d80 2023-11-06T01:33:03.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.726+0000 7f122bfff700 1 -- 172.21.15.53:0/1553638265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c005020 con 0x7f1234105d80 2023-11-06T01:33:03.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.726+0000 7f123979d700 1 -- 172.21.15.53:0/1553638265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1234068a20 con 0x7f1234105d80 2023-11-06T01:33:03.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.727+0000 7f123979d700 1 -- 172.21.15.53:0/1553638265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123404ea90 con 0x7f1234105d80 2023-11-06T01:33:03.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.728+0000 7f122bfff700 1 -- 172.21.15.53:0/1553638265 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f122c012460 con 0x7f1234105d80 2023-11-06T01:33:03.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.729+0000 7f122bfff700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a05b0 0x7f12200a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.729+0000 7f122bfff700 1 -- 172.21.15.53:0/1553638265 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f122c0ce020 con 0x7f1234105d80 2023-11-06T01:33:03.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.729+0000 7f1232ffd700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a05b0 0x7f12200a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:03.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.729+0000 7f1232ffd700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a05b0 0x7f12200a2a70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f122400ba60 tx=0x7f1224005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:03.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.733+0000 7f122bfff700 1 -- 172.21.15.53:0/1553638265 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f122c097c10 con 0x7f1234105d80 2023-11-06T01:33:03.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.951+0000 7f123979d700 1 -- 172.21.15.53:0/1553638265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1234196fe0 con 0x7f1234105d80 2023-11-06T01:33:03.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.951+0000 7f122bfff700 1 -- 172.21.15.53:0/1553638265 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f122c020090 con 0x7f1234105d80 2023-11-06T01:33:03.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.953+0000 7f123979d700 1 -- 172.21.15.53:0/1553638265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a05b0 msgr2=0x7f12200a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.953+0000 7f123979d700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a05b0 0x7f12200a2a70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f122400ba60 tx=0x7f1224005d50 comp rx=0 tx=0).stop 2023-11-06T01:33:03.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.954+0000 7f123979d700 1 -- 172.21.15.53:0/1553638265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1234105d80 msgr2=0x7f1234195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.954+0000 7f123979d700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1234105d80 0x7f1234195ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f122c007f10 tx=0x7f122c00f5c0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.954+0000 7f123979d700 1 -- 172.21.15.53:0/1553638265 shutdown_connections 2023-11-06T01:33:03.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.954+0000 7f123979d700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f12200a05b0 0x7f12200a2a70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.954+0000 7f123979d700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12340ffcd0 0x7f123419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.954+0000 7f123979d700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1234105d80 0x7f1234195ad0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.954+0000 7f123979d700 1 --2- 172.21.15.53:0/1553638265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1234100690 0x7f123419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.954+0000 7f123979d700 1 -- 172.21.15.53:0/1553638265 >> 172.21.15.53:0/1553638265 conn(0x7f1234076d90 msgr2=0x7f12340fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:03.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.954+0000 7f123979d700 1 -- 172.21.15.53:0/1553638265 shutdown_connections 2023-11-06T01:33:03.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:02.954+0000 7f123979d700 1 -- 172.21.15.53:0/1553638265 wait complete. 2023-11-06T01:33:03.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.471+0000 7faf41de0700 1 -- 172.21.15.53:0/4286421154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c0688b0 msgr2=0x7faf3c068d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.471+0000 7faf41de0700 1 --2- 172.21.15.53:0/4286421154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c0688b0 0x7faf3c068d30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faf30009990 tx=0x7faf30009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.472+0000 7faf41de0700 1 -- 172.21.15.53:0/4286421154 shutdown_connections 2023-11-06T01:33:03.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.472+0000 7faf41de0700 1 --2- 172.21.15.53:0/4286421154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf3c106f20 0x7faf3c107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.472+0000 7faf41de0700 1 --2- 172.21.15.53:0/4286421154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3c069270 0x7faf3c10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.472+0000 7faf41de0700 1 --2- 172.21.15.53:0/4286421154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c0688b0 0x7faf3c068d30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.472+0000 7faf41de0700 1 -- 172.21.15.53:0/4286421154 >> 172.21.15.53:0/4286421154 conn(0x7faf3c076d90 msgr2=0x7faf3c0771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:03.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.472+0000 7faf41de0700 1 -- 172.21.15.53:0/4286421154 shutdown_connections 2023-11-06T01:33:03.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.472+0000 7faf41de0700 1 -- 172.21.15.53:0/4286421154 wait complete. 2023-11-06T01:33:03.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.472+0000 7faf41de0700 1 Processor -- start 2023-11-06T01:33:03.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.472+0000 7faf41de0700 1 -- start start 2023-11-06T01:33:03.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf41de0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3c0688b0 0x7faf3c194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf41de0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf3c069270 0x7faf3c195210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf41de0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c106f20 0x7faf3c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf41de0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c198bd0 con 0x7faf3c106f20 2023-11-06T01:33:03.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf41de0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c198d40 con 0x7faf3c069270 2023-11-06T01:33:03.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf41de0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c198f00 con 0x7faf3c0688b0 2023-11-06T01:33:03.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf3bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c106f20 0x7faf3c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:03.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf3bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c106f20 0x7faf3c1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38462/0 (socket says 172.21.15.53:38462) 2023-11-06T01:33:03.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf3bfff700 1 -- 172.21.15.53:0/2164769041 learned_addr learned my addr 172.21.15.53:0/2164769041 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:03.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf3b7fe700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3c0688b0 0x7faf3c194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:03.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf3bfff700 1 -- 172.21.15.53:0/2164769041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3c0688b0 msgr2=0x7faf3c194cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf3bfff700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3c0688b0 0x7faf3c194cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf3bfff700 1 -- 172.21.15.53:0/2164769041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf3c069270 msgr2=0x7faf3c195210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf3bfff700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf3c069270 0x7faf3c195210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.473+0000 7faf3bfff700 1 -- 172.21.15.53:0/2164769041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf300095f0 con 0x7faf3c106f20 2023-11-06T01:33:03.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.474+0000 7faf3bfff700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c106f20 0x7faf3c1985d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faf34007e10 tx=0x7faf3400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.474+0000 7faf38ff9700 1 -- 172.21.15.53:0/2164769041 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34009de0 con 0x7faf3c106f20 2023-11-06T01:33:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.474+0000 7faf41de0700 1 -- 172.21.15.53:0/2164769041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c1991e0 con 0x7faf3c106f20 2023-11-06T01:33:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.474+0000 7faf38ff9700 1 -- 172.21.15.53:0/2164769041 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf34013070 con 0x7faf3c106f20 2023-11-06T01:33:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.474+0000 7faf38ff9700 1 -- 172.21.15.53:0/2164769041 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf340043a0 con 0x7faf3c106f20 2023-11-06T01:33:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.474+0000 7faf41de0700 1 -- 172.21.15.53:0/2164769041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c1a1b30 con 0x7faf3c106f20 2023-11-06T01:33:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.476+0000 7faf41de0700 1 -- 172.21.15.53:0/2164769041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c04ea90 con 0x7faf3c106f20 2023-11-06T01:33:03.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.476+0000 7faf38ff9700 1 -- 172.21.15.53:0/2164769041 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7faf34003a90 con 0x7faf3c106f20 2023-11-06T01:33:03.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.477+0000 7faf38ff9700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faf240a05b0 0x7faf240a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:03.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.477+0000 7faf3b7fe700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faf240a05b0 0x7faf240a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:03.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.477+0000 7faf38ff9700 1 -- 172.21.15.53:0/2164769041 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faf34015070 con 0x7faf3c106f20 2023-11-06T01:33:03.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.478+0000 7faf3b7fe700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faf240a05b0 0x7faf240a2a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009a10 tx=0x7faf2c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:03.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.482+0000 7faf38ff9700 1 -- 172.21.15.53:0/2164769041 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faf34094920 con 0x7faf3c106f20 2023-11-06T01:33:03.497 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:03 smithi163 ceph-mon[104416]: pgmap v270: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-11-06T01:33:03.498 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:03 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1553638265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-06T01:33:03.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.726+0000 7faf41de0700 1 -- 172.21.15.53:0/2164769041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7faf3c1a1e50 con 0x7faf3c106f20 2023-11-06T01:33:03.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.727+0000 7faf38ff9700 1 -- 172.21.15.53:0/2164769041 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf3401e090 con 0x7faf3c106f20 2023-11-06T01:33:03.729 INFO:teuthology.orchestra.run.smithi053.stdout:146028888111 2023-11-06T01:33:03.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 -- 172.21.15.53:0/2164769041 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faf240a05b0 msgr2=0x7faf240a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faf240a05b0 0x7faf240a2a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009a10 tx=0x7faf2c009190 comp rx=0 tx=0).stop 2023-11-06T01:33:03.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 -- 172.21.15.53:0/2164769041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c106f20 msgr2=0x7faf3c1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:03.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c106f20 0x7faf3c1985d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faf34007e10 tx=0x7faf3400d3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 -- 172.21.15.53:0/2164769041 shutdown_connections 2023-11-06T01:33:03.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faf240a05b0 0x7faf240a2a70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c106f20 0x7faf3c1985d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3c0688b0 0x7faf3c194cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 --2- 172.21.15.53:0/2164769041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf3c069270 0x7faf3c195210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:03.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 -- 172.21.15.53:0/2164769041 >> 172.21.15.53:0/2164769041 conn(0x7faf3c076d90 msgr2=0x7faf3c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:03.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 -- 172.21.15.53:0/2164769041 shutdown_connections 2023-11-06T01:33:03.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:03.729+0000 7faf41de0700 1 -- 172.21.15.53:0/2164769041 wait complete. 2023-11-06T01:33:04.298 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:04 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1447419541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-06T01:33:04.299 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:04 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2164769041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-06T01:33:04.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:04 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1447419541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-06T01:33:04.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:04 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2164769041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-06T01:33:04.403 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888110 got 146028888111 for osd.4 2023-11-06T01:33:04.403 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:04.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:04 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1447419541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-06T01:33:04.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:04 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2164769041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-06T01:33:05.057 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:05 smithi053 ceph-mon[99472]: pgmap v271: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2023-11-06T01:33:05.120 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724586 got 167503724588 for osd.5 2023-11-06T01:33:05.120 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:05.250 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574908 got 55834574909 for osd.1 2023-11-06T01:33:05.251 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:05.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:05 smithi062 ceph-mon[103924]: pgmap v271: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2023-11-06T01:33:05.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:05 smithi163 ceph-mon[104416]: pgmap v271: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2023-11-06T01:33:06.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.097+0000 7fac324d9700 1 -- 172.21.15.53:0/564529883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac2c101010 msgr2=0x7fac2c1013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:06.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.097+0000 7fac324d9700 1 --2- 172.21.15.53:0/564529883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac2c101010 0x7fac2c1013f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009990 tx=0x7fac1c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.097+0000 7fac324d9700 1 -- 172.21.15.53:0/564529883 shutdown_connections 2023-11-06T01:33:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.097+0000 7fac324d9700 1 --2- 172.21.15.53:0/564529883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac2c1060f0 0x7fac2c10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.097+0000 7fac324d9700 1 --2- 172.21.15.53:0/564529883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac2c101010 0x7fac2c1013f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.097+0000 7fac324d9700 1 --2- 172.21.15.53:0/564529883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac2c1019c0 0x7fac2c105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.097+0000 7fac324d9700 1 -- 172.21.15.53:0/564529883 >> 172.21.15.53:0/564529883 conn(0x7fac2c0fc880 msgr2=0x7fac2c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.097+0000 7fac324d9700 1 -- 172.21.15.53:0/564529883 shutdown_connections 2023-11-06T01:33:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.097+0000 7fac324d9700 1 -- 172.21.15.53:0/564529883 wait complete. 2023-11-06T01:33:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.098+0000 7fac324d9700 1 Processor -- start 2023-11-06T01:33:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.098+0000 7fac324d9700 1 -- start start 2023-11-06T01:33:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac324d9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac2c101010 0x7fac2c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:06.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac324d9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac2c1019c0 0x7fac2c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:06.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac324d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac2c1060f0 0x7fac2c073b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:06.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac324d9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c1a5300 con 0x7fac2c1019c0 2023-11-06T01:33:06.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac324d9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c1a5470 con 0x7fac2c1060f0 2023-11-06T01:33:06.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac324d9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c1a5630 con 0x7fac2c101010 2023-11-06T01:33:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac2b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac2c1019c0 0x7fac2c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac2b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac2c1019c0 0x7fac2c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38470/0 (socket says 172.21.15.53:38470) 2023-11-06T01:33:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac2b7fe700 1 -- 172.21.15.53:0/665870734 learned_addr learned my addr 172.21.15.53:0/665870734 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac2bfff700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac2c101010 0x7fac2c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac2b7fe700 1 -- 172.21.15.53:0/665870734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac2c101010 msgr2=0x7fac2c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac2b7fe700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac2c101010 0x7fac2c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac2b7fe700 1 -- 172.21.15.53:0/665870734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac2c1060f0 msgr2=0x7fac2c073b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:06.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac2b7fe700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac2c1060f0 0x7fac2c073b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac2b7fe700 1 -- 172.21.15.53:0/665870734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c0095f0 con 0x7fac2c1019c0 2023-11-06T01:33:06.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac2bfff700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac2c101010 0x7fac2c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:06.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac2b7fe700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac2c1019c0 0x7fac2c0735d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fac2000d980 tx=0x7fac2000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:06.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac297fa700 1 -- 172.21.15.53:0/665870734 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20009b40 con 0x7fac2c1019c0 2023-11-06T01:33:06.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac297fa700 1 -- 172.21.15.53:0/665870734 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac2000de60 con 0x7fac2c1019c0 2023-11-06T01:33:06.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac324d9700 1 -- 172.21.15.53:0/665870734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2c1a5910 con 0x7fac2c1019c0 2023-11-06T01:33:06.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac297fa700 1 -- 172.21.15.53:0/665870734 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000f620 con 0x7fac2c1019c0 2023-11-06T01:33:06.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.099+0000 7fac324d9700 1 -- 172.21.15.53:0/665870734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c1a5e10 con 0x7fac2c1019c0 2023-11-06T01:33:06.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.100+0000 7fac324d9700 1 -- 172.21.15.53:0/665870734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2c04ea90 con 0x7fac2c1019c0 2023-11-06T01:33:06.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.101+0000 7fac297fa700 1 -- 172.21.15.53:0/665870734 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fac20010460 con 0x7fac2c1019c0 2023-11-06T01:33:06.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.102+0000 7fac297fa700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fac140a05b0 0x7fac140a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:06.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.102+0000 7fac2bfff700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fac140a05b0 0x7fac140a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.102+0000 7fac297fa700 1 -- 172.21.15.53:0/665870734 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fac200cb650 con 0x7fac2c1019c0 2023-11-06T01:33:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.102+0000 7fac2bfff700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fac140a05b0 0x7fac140a2a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006010 tx=0x7fac1c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.106+0000 7fac297fa700 1 -- 172.21.15.53:0/665870734 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fac20095140 con 0x7fac2c1019c0 2023-11-06T01:33:06.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.328+0000 7fac324d9700 1 -- 172.21.15.53:0/665870734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fac2c1a6130 con 0x7fac2c1019c0 2023-11-06T01:33:06.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.328+0000 7fac297fa700 1 -- 172.21.15.53:0/665870734 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fac20098760 con 0x7fac2c1019c0 2023-11-06T01:33:06.331 INFO:teuthology.orchestra.run.smithi053.stdout:249108103199 2023-11-06T01:33:06.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 -- 172.21.15.53:0/665870734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fac140a05b0 msgr2=0x7fac140a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:06.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fac140a05b0 0x7fac140a2a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006010 tx=0x7fac1c00b540 comp rx=0 tx=0).stop 2023-11-06T01:33:06.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 -- 172.21.15.53:0/665870734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac2c1019c0 msgr2=0x7fac2c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:06.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac2c1019c0 0x7fac2c0735d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fac2000d980 tx=0x7fac2000dd40 comp rx=0 tx=0).stop 2023-11-06T01:33:06.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 -- 172.21.15.53:0/665870734 shutdown_connections 2023-11-06T01:33:06.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fac140a05b0 0x7fac140a2a70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac2c1060f0 0x7fac2c073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac2c101010 0x7fac2c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 --2- 172.21.15.53:0/665870734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac2c1019c0 0x7fac2c0735d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 -- 172.21.15.53:0/665870734 >> 172.21.15.53:0/665870734 conn(0x7fac2c0fc880 msgr2=0x7fac2c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:06.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 -- 172.21.15.53:0/665870734 shutdown_connections 2023-11-06T01:33:06.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.331+0000 7fac324d9700 1 -- 172.21.15.53:0/665870734 wait complete. 2023-11-06T01:33:06.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.501+0000 7f56d04fe700 1 -- 172.21.15.53:0/4026570832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c81008c0 msgr2=0x7f56c810cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:06.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.501+0000 7f56d04fe700 1 --2- 172.21.15.53:0/4026570832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c81008c0 0x7f56c810cb10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56c000b0a0 tx=0x7f56c000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.501+0000 7f56d04fe700 1 -- 172.21.15.53:0/4026570832 shutdown_connections 2023-11-06T01:33:06.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.501+0000 7f56d04fe700 1 --2- 172.21.15.53:0/4026570832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c8105e60 0x7f56c8106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.501+0000 7f56d04fe700 1 --2- 172.21.15.53:0/4026570832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c81008c0 0x7f56c810cb10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.501+0000 7f56d04fe700 1 --2- 172.21.15.53:0/4026570832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c80fff00 0x7f56c8100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.501+0000 7f56d04fe700 1 -- 172.21.15.53:0/4026570832 >> 172.21.15.53:0/4026570832 conn(0x7f56c8076bc0 msgr2=0x7f56c8076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:06.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.502+0000 7f56d04fe700 1 -- 172.21.15.53:0/4026570832 shutdown_connections 2023-11-06T01:33:06.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.502+0000 7f56d04fe700 1 -- 172.21.15.53:0/4026570832 wait complete. 2023-11-06T01:33:06.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.502+0000 7f56d04fe700 1 Processor -- start 2023-11-06T01:33:06.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.502+0000 7f56d04fe700 1 -- start start 2023-11-06T01:33:06.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56d04fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c80fff00 0x7f56c8197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56d04fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c81008c0 0x7f56c81975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56d04fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c8105e60 0x7f56c819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56d04fe700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c819c040 con 0x7f56c81008c0 2023-11-06T01:33:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56d04fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c819c1b0 con 0x7f56c80fff00 2023-11-06T01:33:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56d04fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c819c370 con 0x7f56c8105e60 2023-11-06T01:33:06.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56cda99700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c81008c0 0x7f56c81975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:06.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56cda99700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c81008c0 0x7f56c81975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38484/0 (socket says 172.21.15.53:38484) 2023-11-06T01:33:06.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56cda99700 1 -- 172.21.15.53:0/3313190310 learned_addr learned my addr 172.21.15.53:0/3313190310 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:06.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56cea9b700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c8105e60 0x7f56c819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:06.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56ce29a700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c80fff00 0x7f56c8197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:06.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56cda99700 1 -- 172.21.15.53:0/3313190310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c8105e60 msgr2=0x7f56c819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:06.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56cda99700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c8105e60 0x7f56c819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56cda99700 1 -- 172.21.15.53:0/3313190310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c80fff00 msgr2=0x7f56c8197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:06.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56cda99700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c80fff00 0x7f56c8197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56cda99700 1 -- 172.21.15.53:0/3313190310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c0009d00 con 0x7f56c81008c0 2023-11-06T01:33:06.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56ce29a700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c80fff00 0x7f56c8197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:06.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56cda99700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c81008c0 0x7f56c81975c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d950 tx=0x7f56b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:06.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.503+0000 7f56bf7fe700 1 -- 172.21.15.53:0/3313190310 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8009940 con 0x7f56c81008c0 2023-11-06T01:33:06.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.504+0000 7f56bf7fe700 1 -- 172.21.15.53:0/3313190310 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b8010460 con 0x7f56c81008c0 2023-11-06T01:33:06.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.504+0000 7f56d04fe700 1 -- 172.21.15.53:0/3313190310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c819c650 con 0x7f56c81008c0 2023-11-06T01:33:06.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.504+0000 7f56bf7fe700 1 -- 172.21.15.53:0/3313190310 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b800f620 con 0x7f56c81008c0 2023-11-06T01:33:06.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.504+0000 7f56d04fe700 1 -- 172.21.15.53:0/3313190310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c81a3eb0 con 0x7f56c81008c0 2023-11-06T01:33:06.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.505+0000 7f56d04fe700 1 -- 172.21.15.53:0/3313190310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c804ea90 con 0x7f56c81008c0 2023-11-06T01:33:06.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.506+0000 7f56bf7fe700 1 -- 172.21.15.53:0/3313190310 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f56b80108d0 con 0x7f56c81008c0 2023-11-06T01:33:06.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.507+0000 7f56bf7fe700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56b40a06b0 0x7f56b40a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:06.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.507+0000 7f56bf7fe700 1 -- 172.21.15.53:0/3313190310 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f56b80cb870 con 0x7f56c81008c0 2023-11-06T01:33:06.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.509+0000 7f56ce29a700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56b40a06b0 0x7f56b40a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:06.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.509+0000 7f56ce29a700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56b40a06b0 0x7f56b40a2b70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005e10 tx=0x7f56c4005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:06.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.513+0000 7f56bf7fe700 1 -- 172.21.15.53:0/3313190310 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f56b8095360 con 0x7f56c81008c0 2023-11-06T01:33:06.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.713+0000 7f56d04fe700 1 -- 172.21.15.53:0/3313190310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f56c81a40e0 con 0x7f56c81008c0 2023-11-06T01:33:06.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.713+0000 7f56bf7fe700 1 -- 172.21.15.53:0/3313190310 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f56b8016090 con 0x7f56c81008c0 2023-11-06T01:33:06.715 INFO:teuthology.orchestra.run.smithi053.stdout:111669149749 2023-11-06T01:33:06.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 -- 172.21.15.53:0/3313190310 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56b40a06b0 msgr2=0x7f56b40a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:06.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56b40a06b0 0x7f56b40a2b70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005e10 tx=0x7f56c4005d60 comp rx=0 tx=0).stop 2023-11-06T01:33:06.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 -- 172.21.15.53:0/3313190310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c81008c0 msgr2=0x7f56c81975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:06.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c81008c0 0x7f56c81975c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d950 tx=0x7f56b800dc60 comp rx=0 tx=0).stop 2023-11-06T01:33:06.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 -- 172.21.15.53:0/3313190310 shutdown_connections 2023-11-06T01:33:06.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56b40a06b0 0x7f56b40a2b70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c80fff00 0x7f56c8197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c8105e60 0x7f56c819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 --2- 172.21.15.53:0/3313190310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c81008c0 0x7f56c81975c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:06.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 -- 172.21.15.53:0/3313190310 >> 172.21.15.53:0/3313190310 conn(0x7f56c8076bc0 msgr2=0x7f56c80fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:06.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 -- 172.21.15.53:0/3313190310 shutdown_connections 2023-11-06T01:33:06.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:06.715+0000 7f56d04fe700 1 -- 172.21.15.53:0/3313190310 wait complete. 2023-11-06T01:33:06.974 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103197 got 249108103199 for osd.8 2023-11-06T01:33:06.975 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:07.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:07 smithi053 ceph-mon[99472]: pgmap v272: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2023-11-06T01:33:07.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:07 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/665870734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-06T01:33:07.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:07 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3313190310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-06T01:33:07.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:07 smithi062 ceph-mon[103924]: pgmap v272: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2023-11-06T01:33:07.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:07 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/665870734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-06T01:33:07.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:07 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3313190310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-06T01:33:07.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:07 smithi163 ceph-mon[104416]: pgmap v272: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2023-11-06T01:33:07.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:07 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/665870734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-06T01:33:07.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:07 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3313190310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-06T01:33:08.011 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:08.040 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149746 got 111669149749 for osd.3 2023-11-06T01:33:08.040 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:08.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.267+0000 7f0c30ba2700 1 -- 172.21.15.53:0/3885834057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c105f50 msgr2=0x7f0c2c10db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:08.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.267+0000 7f0c30ba2700 1 --2- 172.21.15.53:0/3885834057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c105f50 0x7f0c2c10db20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400b0a0 tx=0x7f0c2400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:08.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.267+0000 7f0c30ba2700 1 -- 172.21.15.53:0/3885834057 shutdown_connections 2023-11-06T01:33:08.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.267+0000 7f0c30ba2700 1 --2- 172.21.15.53:0/3885834057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c105f50 0x7f0c2c10db20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:08.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.267+0000 7f0c30ba2700 1 --2- 172.21.15.53:0/3885834057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c2c100e70 0x7f0c2c101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:08.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.267+0000 7f0c30ba2700 1 --2- 172.21.15.53:0/3885834057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c2c101820 0x7f0c2c105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:08.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.267+0000 7f0c30ba2700 1 -- 172.21.15.53:0/3885834057 >> 172.21.15.53:0/3885834057 conn(0x7f0c2c0fc760 msgr2=0x7f0c2c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:08.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.267+0000 7f0c30ba2700 1 -- 172.21.15.53:0/3885834057 shutdown_connections 2023-11-06T01:33:08.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.268+0000 7f0c30ba2700 1 -- 172.21.15.53:0/3885834057 wait complete. 2023-11-06T01:33:08.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.268+0000 7f0c30ba2700 1 Processor -- start 2023-11-06T01:33:08.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c30ba2700 1 -- start start 2023-11-06T01:33:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c30ba2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c2c100e70 0x7f0c2c196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c30ba2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c2c101820 0x7f0c2c197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c30ba2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c105f50 0x7f0c2c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c30ba2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c19abc0 con 0x7f0c2c105f50 2023-11-06T01:33:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c30ba2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c19ad30 con 0x7f0c2c101820 2023-11-06T01:33:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c30ba2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c19aef0 con 0x7f0c2c100e70 2023-11-06T01:33:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c2bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c105f50 0x7f0c2c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c2bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c105f50 0x7f0c2c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38498/0 (socket says 172.21.15.53:38498) 2023-11-06T01:33:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c2bfff700 1 -- 172.21.15.53:0/744491469 learned_addr learned my addr 172.21.15.53:0/744491469 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c2b7fe700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c2c100e70 0x7f0c2c196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:08.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c23fff700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c2c101820 0x7f0c2c197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:08.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c2bfff700 1 -- 172.21.15.53:0/744491469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c2c100e70 msgr2=0x7f0c2c196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:08.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c2bfff700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c2c100e70 0x7f0c2c196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:08.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c2bfff700 1 -- 172.21.15.53:0/744491469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c2c101820 msgr2=0x7f0c2c197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c2bfff700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c2c101820 0x7f0c2c197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c2bfff700 1 -- 172.21.15.53:0/744491469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c24009d00 con 0x7f0c2c105f50 2023-11-06T01:33:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.269+0000 7f0c23fff700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c2c101820 0x7f0c2c197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.270+0000 7f0c2bfff700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c105f50 0x7f0c2c19a5c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0c240062a0 tx=0x7f0c2400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.270+0000 7f0c297fa700 1 -- 172.21.15.53:0/744491469 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400e040 con 0x7f0c2c105f50 2023-11-06T01:33:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.270+0000 7f0c30ba2700 1 -- 172.21.15.53:0/744491469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c19b170 con 0x7f0c2c105f50 2023-11-06T01:33:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.270+0000 7f0c297fa700 1 -- 172.21.15.53:0/744491469 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c24004020 con 0x7f0c2c105f50 2023-11-06T01:33:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.270+0000 7f0c297fa700 1 -- 172.21.15.53:0/744491469 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2401b690 con 0x7f0c2c105f50 2023-11-06T01:33:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.270+0000 7f0c30ba2700 1 -- 172.21.15.53:0/744491469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2c1a3b20 con 0x7f0c2c105f50 2023-11-06T01:33:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.271+0000 7f0c30ba2700 1 -- 172.21.15.53:0/744491469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2c04f2e0 con 0x7f0c2c105f50 2023-11-06T01:33:08.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.272+0000 7f0c297fa700 1 -- 172.21.15.53:0/744491469 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0c24004400 con 0x7f0c2c105f50 2023-11-06T01:33:08.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.272+0000 7f0c297fa700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c180a05b0 0x7f0c180a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:08.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.272+0000 7f0c2b7fe700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c180a05b0 0x7f0c180a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:08.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.272+0000 7f0c297fa700 1 -- 172.21.15.53:0/744491469 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0c240cda90 con 0x7f0c2c105f50 2023-11-06T01:33:08.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.273+0000 7f0c2b7fe700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c180a05b0 0x7f0c180a2a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00ac50 tx=0x7f0c1c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:08.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.276+0000 7f0c297fa700 1 -- 172.21.15.53:0/744491469 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0c24097710 con 0x7f0c2c105f50 2023-11-06T01:33:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.484+0000 7f0c30ba2700 1 -- 172.21.15.53:0/744491469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f0c2c19b360 con 0x7f0c2c105f50 2023-11-06T01:33:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.485+0000 7f0c297fa700 1 -- 172.21.15.53:0/744491469 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f0c2409ad30 con 0x7f0c2c105f50 2023-11-06T01:33:08.487 INFO:teuthology.orchestra.run.smithi053.stdout:330712481812 2023-11-06T01:33:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 -- 172.21.15.53:0/744491469 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c180a05b0 msgr2=0x7f0c180a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c180a05b0 0x7f0c180a2a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00ac50 tx=0x7f0c1c00a380 comp rx=0 tx=0).stop 2023-11-06T01:33:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 -- 172.21.15.53:0/744491469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c105f50 msgr2=0x7f0c2c19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c105f50 0x7f0c2c19a5c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0c240062a0 tx=0x7f0c2400cf70 comp rx=0 tx=0).stop 2023-11-06T01:33:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 -- 172.21.15.53:0/744491469 shutdown_connections 2023-11-06T01:33:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c180a05b0 0x7f0c180a2a70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2c105f50 0x7f0c2c19a5c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c2c100e70 0x7f0c2c196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 --2- 172.21.15.53:0/744491469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c2c101820 0x7f0c2c197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 -- 172.21.15.53:0/744491469 >> 172.21.15.53:0/744491469 conn(0x7f0c2c0fc760 msgr2=0x7f0c2c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 -- 172.21.15.53:0/744491469 shutdown_connections 2023-11-06T01:33:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.487+0000 7f0c30ba2700 1 -- 172.21.15.53:0/744491469 wait complete. 2023-11-06T01:33:08.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.892+0000 7ff704e80700 1 -- 172.21.15.53:0/648024410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff70010d650 msgr2=0x7ff70010fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.892+0000 7ff704e80700 1 --2- 172.21.15.53:0/648024410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff70010d650 0x7ff70010fa60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800b0a0 tx=0x7ff6f800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.892+0000 7ff704e80700 1 -- 172.21.15.53:0/648024410 shutdown_connections 2023-11-06T01:33:09.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.892+0000 7ff704e80700 1 --2- 172.21.15.53:0/648024410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff70010d650 0x7ff70010fa60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.892+0000 7ff704e80700 1 --2- 172.21.15.53:0/648024410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff700073160 0x7ff700073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.892+0000 7ff704e80700 1 --2- 172.21.15.53:0/648024410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700073a80 0x7ff70010d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.892+0000 7ff704e80700 1 -- 172.21.15.53:0/648024410 >> 172.21.15.53:0/648024410 conn(0x7ff7000fc880 msgr2=0x7ff7000feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:09.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.892+0000 7ff704e80700 1 -- 172.21.15.53:0/648024410 shutdown_connections 2023-11-06T01:33:09.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.892+0000 7ff704e80700 1 -- 172.21.15.53:0/648024410 wait complete. 2023-11-06T01:33:09.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff704e80700 1 Processor -- start 2023-11-06T01:33:09.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff704e80700 1 -- start start 2023-11-06T01:33:09.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff704e80700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700073160 0x7ff7001991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff704e80700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff700073a80 0x7ff7001996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff704e80700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff70010d650 0x7ff70019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff704e80700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70019e170 con 0x7ff700073a80 2023-11-06T01:33:09.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff704e80700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70019e2e0 con 0x7ff700073160 2023-11-06T01:33:09.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff704e80700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70019e4a0 con 0x7ff70010d650 2023-11-06T01:33:09.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff6fdd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff700073a80 0x7ff7001996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:09.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff6fdd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff700073a80 0x7ff7001996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38508/0 (socket says 172.21.15.53:38508) 2023-11-06T01:33:09.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff6fdd9b700 1 -- 172.21.15.53:0/1137783660 learned_addr learned my addr 172.21.15.53:0/1137783660 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:09.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.893+0000 7ff6fe59c700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700073160 0x7ff7001991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:09.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff6fed9d700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff70010d650 0x7ff70019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:09.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff6fdd9b700 1 -- 172.21.15.53:0/1137783660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff70010d650 msgr2=0x7ff70019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff6fdd9b700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff70010d650 0x7ff70019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff6fdd9b700 1 -- 172.21.15.53:0/1137783660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700073160 msgr2=0x7ff7001991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff6fdd9b700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700073160 0x7ff7001991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff6fdd9b700 1 -- 172.21.15.53:0/1137783660 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f8009d00 con 0x7ff700073a80 2023-11-06T01:33:09.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff6fdd9b700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff700073a80 0x7ff7001996f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400d950 tx=0x7ff6f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:09.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff6ef7fe700 1 -- 172.21.15.53:0/1137783660 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4009940 con 0x7ff700073a80 2023-11-06T01:33:09.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff704e80700 1 -- 172.21.15.53:0/1137783660 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70019e780 con 0x7ff700073a80 2023-11-06T01:33:09.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff6ef7fe700 1 -- 172.21.15.53:0/1137783660 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6f4010460 con 0x7ff700073a80 2023-11-06T01:33:09.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff6ef7fe700 1 -- 172.21.15.53:0/1137783660 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400f620 con 0x7ff700073a80 2023-11-06T01:33:09.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.894+0000 7ff704e80700 1 -- 172.21.15.53:0/1137783660 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7001a5fe0 con 0x7ff700073a80 2023-11-06T01:33:09.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.895+0000 7ff704e80700 1 -- 172.21.15.53:0/1137783660 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70004ea90 con 0x7ff700073a80 2023-11-06T01:33:09.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.896+0000 7ff6ef7fe700 1 -- 172.21.15.53:0/1137783660 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7ff6f40108d0 con 0x7ff700073a80 2023-11-06T01:33:09.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.897+0000 7ff6ef7fe700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6e80a05b0 0x7ff6e80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.897+0000 7ff6fe59c700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6e80a05b0 0x7ff6e80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.897+0000 7ff6ef7fe700 1 -- 172.21.15.53:0/1137783660 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff6f40cb770 con 0x7ff700073a80 2023-11-06T01:33:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.897+0000 7ff6fe59c700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6e80a05b0 0x7ff6e80a2a70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0005950 tx=0x7ff6f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.901+0000 7ff6ef7fe700 1 -- 172.21.15.53:0/1137783660 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff6f4095360 con 0x7ff700073a80 2023-11-06T01:33:09.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.130+0000 7ff704e80700 1 -- 172.21.15.53:0/1137783660 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7ff70019a510 con 0x7ff700073a80 2023-11-06T01:33:09.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.130+0000 7ff6ef7fe700 1 -- 172.21.15.53:0/1137783660 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff6f4098980 con 0x7ff700073a80 2023-11-06T01:33:09.133 INFO:teuthology.orchestra.run.smithi053.stdout:279172874268 2023-11-06T01:33:09.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.132+0000 7ff704e80700 1 -- 172.21.15.53:0/1137783660 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6e80a05b0 msgr2=0x7ff6e80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.132+0000 7ff704e80700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6e80a05b0 0x7ff6e80a2a70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0005950 tx=0x7ff6f000b410 comp rx=0 tx=0).stop 2023-11-06T01:33:09.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.132+0000 7ff704e80700 1 -- 172.21.15.53:0/1137783660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff700073a80 msgr2=0x7ff7001996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.132+0000 7ff704e80700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff700073a80 0x7ff7001996f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400d950 tx=0x7ff6f400dc60 comp rx=0 tx=0).stop 2023-11-06T01:33:09.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.132+0000 7ff704e80700 1 -- 172.21.15.53:0/1137783660 shutdown_connections 2023-11-06T01:33:09.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.132+0000 7ff704e80700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6e80a05b0 0x7ff6e80a2a70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.132+0000 7ff704e80700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700073160 0x7ff7001991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.132+0000 7ff704e80700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff70010d650 0x7ff70019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.132+0000 7ff704e80700 1 --2- 172.21.15.53:0/1137783660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff700073a80 0x7ff7001996f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.132+0000 7ff704e80700 1 -- 172.21.15.53:0/1137783660 >> 172.21.15.53:0/1137783660 conn(0x7ff7000fc880 msgr2=0x7ff70010f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:09.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.133+0000 7ff704e80700 1 -- 172.21.15.53:0/1137783660 shutdown_connections 2023-11-06T01:33:09.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.133+0000 7ff704e80700 1 -- 172.21.15.53:0/1137783660 wait complete. 2023-11-06T01:33:09.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:09 smithi053 ceph-mon[99472]: pgmap v273: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:09.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:09 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/744491469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-06T01:33:09.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:09 smithi062 ceph-mon[103924]: pgmap v273: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:09.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:09 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/744491469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-06T01:33:09.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:09 smithi163 ceph-mon[104416]: pgmap v273: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:09.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:09 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/744491469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-06T01:33:09.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.222+0000 7f0b8e251700 1 -- 172.21.15.53:0/3407677017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880ff420 msgr2=0x7f0b880ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.222+0000 7f0b8e251700 1 --2- 172.21.15.53:0/3407677017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880ff420 0x7f0b880ff8a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c210 tx=0x7f0b8000c520 comp rx=0 tx=0).stop 2023-11-06T01:33:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.222+0000 7f0b8e251700 1 -- 172.21.15.53:0/3407677017 shutdown_connections 2023-11-06T01:33:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.222+0000 7f0b8e251700 1 --2- 172.21.15.53:0/3407677017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b881054b0 0x7f0b88105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.222+0000 7f0b8e251700 1 --2- 172.21.15.53:0/3407677017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b880ffde0 0x7f0b8810c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.222+0000 7f0b8e251700 1 --2- 172.21.15.53:0/3407677017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880ff420 0x7f0b880ff8a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.222+0000 7f0b8e251700 1 -- 172.21.15.53:0/3407677017 >> 172.21.15.53:0/3407677017 conn(0x7f0b88076200 msgr2=0x7f0b880fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.222+0000 7f0b8e251700 1 -- 172.21.15.53:0/3407677017 shutdown_connections 2023-11-06T01:33:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.222+0000 7f0b8e251700 1 -- 172.21.15.53:0/3407677017 wait complete. 2023-11-06T01:33:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.223+0000 7f0b8e251700 1 Processor -- start 2023-11-06T01:33:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.223+0000 7f0b8e251700 1 -- start start 2023-11-06T01:33:09.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b8e251700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b880ff420 0x7f0b88194ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b8e251700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880ffde0 0x7f0b88195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b8e251700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b881054b0 0x7f0b881998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b8e251700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b88199eb0 con 0x7f0b880ff420 2023-11-06T01:33:09.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b8e251700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8819a020 con 0x7f0b881054b0 2023-11-06T01:33:09.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b8e251700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8819a1e0 con 0x7f0b880ffde0 2023-11-06T01:33:09.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b877fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b880ff420 0x7f0b88194ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:09.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b877fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b880ff420 0x7f0b88194ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38532/0 (socket says 172.21.15.53:38532) 2023-11-06T01:33:09.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b877fe700 1 -- 172.21.15.53:0/70469447 learned_addr learned my addr 172.21.15.53:0/70469447 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:09.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b86ffd700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880ffde0 0x7f0b88195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:09.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b86ffd700 1 -- 172.21.15.53:0/70469447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b881054b0 msgr2=0x7f0b881998a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:33:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b86ffd700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b881054b0 0x7f0b881998a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b86ffd700 1 -- 172.21.15.53:0/70469447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b880ff420 msgr2=0x7f0b88194ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b86ffd700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b880ff420 0x7f0b88194ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.224+0000 7f0b86ffd700 1 -- 172.21.15.53:0/70469447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c009520 con 0x7f0b880ffde0 2023-11-06T01:33:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.225+0000 7f0b86ffd700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880ffde0 0x7f0b88195430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c210 tx=0x7f0b800082f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:09.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.225+0000 7f0b84ff9700 1 -- 172.21.15.53:0/70469447 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000f040 con 0x7f0b880ffde0 2023-11-06T01:33:09.612 INFO:teuthology.orchestra.run.smithi053.stdout:34359738435 2023-11-06T01:33:09.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.225+0000 7f0b84ff9700 1 -- 172.21.15.53:0/70469447 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b80004e40 con 0x7f0b880ffde0 2023-11-06T01:33:09.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.225+0000 7f0b8e251700 1 -- 172.21.15.53:0/70469447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b78009520 con 0x7f0b880ffde0 2023-11-06T01:33:09.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.225+0000 7f0b84ff9700 1 -- 172.21.15.53:0/70469447 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b80007e70 con 0x7f0b880ffde0 2023-11-06T01:33:09.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.225+0000 7f0b8e251700 1 -- 172.21.15.53:0/70469447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8000ae10 con 0x7f0b880ffde0 2023-11-06T01:33:09.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.226+0000 7f0b8e251700 1 -- 172.21.15.53:0/70469447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b88107a40 con 0x7f0b880ffde0 2023-11-06T01:33:09.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.228+0000 7f0b84ff9700 1 -- 172.21.15.53:0/70469447 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0b80007530 con 0x7f0b880ffde0 2023-11-06T01:33:09.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.228+0000 7f0b84ff9700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b700a0630 0x7f0b700a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.228+0000 7f0b84ff9700 1 -- 172.21.15.53:0/70469447 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0b800cce60 con 0x7f0b880ffde0 2023-11-06T01:33:09.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.228+0000 7f0b877fe700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b700a0630 0x7f0b700a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:09.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.229+0000 7f0b877fe700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b700a0630 0x7f0b700a2af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009e70 tx=0x7f0b78010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:09.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.233+0000 7f0b84ff9700 1 -- 172.21.15.53:0/70469447 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0b80096ae0 con 0x7f0b880ffde0 2023-11-06T01:33:09.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.442+0000 7f0b8e251700 1 -- 172.21.15.53:0/70469447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0b8804ea90 con 0x7f0b880ffde0 2023-11-06T01:33:09.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.443+0000 7f0b84ff9700 1 -- 172.21.15.53:0/70469447 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0b8009a100 con 0x7f0b880ffde0 2023-11-06T01:33:09.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.446+0000 7f0b8e251700 1 -- 172.21.15.53:0/70469447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b700a0630 msgr2=0x7f0b700a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.446+0000 7f0b8e251700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b700a0630 0x7f0b700a2af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009e70 tx=0x7f0b78010040 comp rx=0 tx=0).stop 2023-11-06T01:33:09.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.446+0000 7f0b8e251700 1 -- 172.21.15.53:0/70469447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880ffde0 msgr2=0x7f0b88195430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.446+0000 7f0b8e251700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880ffde0 0x7f0b88195430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c210 tx=0x7f0b800082f0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.446+0000 7f0b8e251700 1 -- 172.21.15.53:0/70469447 shutdown_connections 2023-11-06T01:33:09.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.446+0000 7f0b8e251700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b700a0630 0x7f0b700a2af0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.446+0000 7f0b8e251700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b881054b0 0x7f0b881998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.446+0000 7f0b8e251700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b880ff420 0x7f0b88194ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.446+0000 7f0b8e251700 1 --2- 172.21.15.53:0/70469447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880ffde0 0x7f0b88195430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.446+0000 7f0b8e251700 1 -- 172.21.15.53:0/70469447 >> 172.21.15.53:0/70469447 conn(0x7f0b88076200 msgr2=0x7f0b88102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.447+0000 7f0b8e251700 1 -- 172.21.15.53:0/70469447 shutdown_connections 2023-11-06T01:33:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.447+0000 7f0b8e251700 1 -- 172.21.15.53:0/70469447 wait complete. 2023-11-06T01:33:09.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.972+0000 7f0f59d3e700 1 -- 172.21.15.53:0/4156894464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54105d80 msgr2=0x7f0f54106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.972+0000 7f0f59d3e700 1 --2- 172.21.15.53:0/4156894464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54105d80 0x7f0f54106160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009990 tx=0x7f0f44009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.972+0000 7f0f59d3e700 1 -- 172.21.15.53:0/4156894464 shutdown_connections 2023-11-06T01:33:09.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.972+0000 7f0f59d3e700 1 --2- 172.21.15.53:0/4156894464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f54100690 0x7f0f5410cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.972+0000 7f0f59d3e700 1 --2- 172.21.15.53:0/4156894464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54105d80 0x7f0f54106160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.972+0000 7f0f59d3e700 1 --2- 172.21.15.53:0/4156894464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f540ffcd0 0x7f0f54100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.972+0000 7f0f59d3e700 1 -- 172.21.15.53:0/4156894464 >> 172.21.15.53:0/4156894464 conn(0x7f0f54076d90 msgr2=0x7f0f540771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:09.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.972+0000 7f0f59d3e700 1 -- 172.21.15.53:0/4156894464 shutdown_connections 2023-11-06T01:33:09.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.972+0000 7f0f59d3e700 1 -- 172.21.15.53:0/4156894464 wait complete. 2023-11-06T01:33:09.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f59d3e700 1 Processor -- start 2023-11-06T01:33:09.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f59d3e700 1 -- start start 2023-11-06T01:33:09.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f59d3e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f540ffcd0 0x7f0f54197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f59d3e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54100690 0x7f0f54198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f59d3e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f54105d80 0x7f0f541a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f59d3e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f54198c40 con 0x7f0f54100690 2023-11-06T01:33:09.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f59d3e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f541a3380 con 0x7f0f540ffcd0 2023-11-06T01:33:09.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f59d3e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f541a3540 con 0x7f0f54105d80 2023-11-06T01:33:09.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f52ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54100690 0x7f0f54198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:09.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f52ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54100690 0x7f0f54198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38520/0 (socket says 172.21.15.53:38520) 2023-11-06T01:33:09.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f52ffd700 1 -- 172.21.15.53:0/3300858506 learned_addr learned my addr 172.21.15.53:0/3300858506 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:09.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.973+0000 7f0f537fe700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f540ffcd0 0x7f0f54197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:09.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f52ffd700 1 -- 172.21.15.53:0/3300858506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f54105d80 msgr2=0x7f0f541a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f52ffd700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f54105d80 0x7f0f541a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f52ffd700 1 -- 172.21.15.53:0/3300858506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f540ffcd0 msgr2=0x7f0f54197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f52ffd700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f540ffcd0 0x7f0f54197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f52ffd700 1 -- 172.21.15.53:0/3300858506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f440095f0 con 0x7f0f54100690 2023-11-06T01:33:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f537fe700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f540ffcd0 0x7f0f54197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f52ffd700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54100690 0x7f0f54198300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800d950 tx=0x7f0f4800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f50ff9700 1 -- 172.21.15.53:0/3300858506 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48009b40 con 0x7f0f54100690 2023-11-06T01:33:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f59d3e700 1 -- 172.21.15.53:0/3300858506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f541a3820 con 0x7f0f54100690 2023-11-06T01:33:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f50ff9700 1 -- 172.21.15.53:0/3300858506 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f48009ca0 con 0x7f0f54100690 2023-11-06T01:33:09.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f50ff9700 1 -- 172.21.15.53:0/3300858506 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800f620 con 0x7f0f54100690 2023-11-06T01:33:09.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.974+0000 7f0f59d3e700 1 -- 172.21.15.53:0/3300858506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f541a3d70 con 0x7f0f54100690 2023-11-06T01:33:09.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.975+0000 7f0f59d3e700 1 -- 172.21.15.53:0/3300858506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5404ea90 con 0x7f0f54100690 2023-11-06T01:33:09.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.976+0000 7f0f50ff9700 1 -- 172.21.15.53:0/3300858506 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0f48010460 con 0x7f0f54100690 2023-11-06T01:33:09.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.977+0000 7f0f50ff9700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0f3c0a05b0 0x7f0f3c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:09.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.977+0000 7f0f537fe700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0f3c0a05b0 0x7f0f3c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:09.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.977+0000 7f0f50ff9700 1 -- 172.21.15.53:0/3300858506 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0f480cbd20 con 0x7f0f54100690 2023-11-06T01:33:09.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.977+0000 7f0f537fe700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0f3c0a05b0 0x7f0f3c0a2a70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400b5c0 tx=0x7f0f44005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:09.881 INFO:teuthology.orchestra.run.smithi053.stdout:219043332131 2023-11-06T01:33:09.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:08.980+0000 7f0f50ff9700 1 -- 172.21.15.53:0/3300858506 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0f48095910 con 0x7f0f54100690 2023-11-06T01:33:09.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.191+0000 7f0f59d3e700 1 -- 172.21.15.53:0/3300858506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0f541a4030 con 0x7f0f54100690 2023-11-06T01:33:09.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.191+0000 7f0f50ff9700 1 -- 172.21.15.53:0/3300858506 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f48020020 con 0x7f0f54100690 2023-11-06T01:33:09.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 -- 172.21.15.53:0/3300858506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0f3c0a05b0 msgr2=0x7f0f3c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0f3c0a05b0 0x7f0f3c0a2a70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400b5c0 tx=0x7f0f44005fb0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 -- 172.21.15.53:0/3300858506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54100690 msgr2=0x7f0f54198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:09.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54100690 0x7f0f54198300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800d950 tx=0x7f0f4800dd10 comp rx=0 tx=0).stop 2023-11-06T01:33:09.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 -- 172.21.15.53:0/3300858506 shutdown_connections 2023-11-06T01:33:09.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0f3c0a05b0 0x7f0f3c0a2a70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f540ffcd0 0x7f0f54197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f54105d80 0x7f0f541a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 --2- 172.21.15.53:0/3300858506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54100690 0x7f0f54198300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:09.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 -- 172.21.15.53:0/3300858506 >> 172.21.15.53:0/3300858506 conn(0x7f0f54076d90 msgr2=0x7f0f540fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:09.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 -- 172.21.15.53:0/3300858506 shutdown_connections 2023-11-06T01:33:09.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:09.193+0000 7f0f59d3e700 1 -- 172.21.15.53:0/3300858506 wait complete. 2023-11-06T01:33:09.996 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874266 got 279172874268 for osd.9 2023-11-06T01:33:09.996 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:10.022 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481809 got 330712481812 for osd.11 2023-11-06T01:33:10.023 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:10.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:10 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1137783660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-06T01:33:10.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:10 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3300858506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-06T01:33:10.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:10 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/70469447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-06T01:33:10.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:10 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1137783660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-06T01:33:10.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:10 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3300858506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-06T01:33:10.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:10 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/70469447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-06T01:33:10.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.400+0000 7f4421731700 1 -- 172.21.15.53:0/2562758139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441c1019b0 msgr2=0x7f441c105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:10.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.400+0000 7f4421731700 1 --2- 172.21.15.53:0/2562758139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441c1019b0 0x7f441c105a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4410009990 tx=0x7f4410009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:10.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.400+0000 7f4421731700 1 -- 172.21.15.53:0/2562758139 shutdown_connections 2023-11-06T01:33:10.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.400+0000 7f4421731700 1 --2- 172.21.15.53:0/2562758139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f441c1060e0 0x7f441c10dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:10.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.400+0000 7f4421731700 1 --2- 172.21.15.53:0/2562758139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c101000 0x7f441c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:10.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.400+0000 7f4421731700 1 --2- 172.21.15.53:0/2562758139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441c1019b0 0x7f441c105a00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:10.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.400+0000 7f4421731700 1 -- 172.21.15.53:0/2562758139 >> 172.21.15.53:0/2562758139 conn(0x7f441c0fc890 msgr2=0x7f441c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:10.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.400+0000 7f4421731700 1 -- 172.21.15.53:0/2562758139 shutdown_connections 2023-11-06T01:33:10.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.400+0000 7f4421731700 1 -- 172.21.15.53:0/2562758139 wait complete. 2023-11-06T01:33:10.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.400+0000 7f4421731700 1 Processor -- start 2023-11-06T01:33:10.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f4421731700 1 -- start start 2023-11-06T01:33:10.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f4421731700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c101000 0x7f441c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:10.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f4421731700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441c1019b0 0x7f441c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:10.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f4421731700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f441c1060e0 0x7f441c073b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:10.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f4421731700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1a5360 con 0x7f441c1019b0 2023-11-06T01:33:10.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f4421731700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1a54d0 con 0x7f441c1060e0 2023-11-06T01:33:10.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f4421731700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1a5690 con 0x7f441c101000 2023-11-06T01:33:10.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f441a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441c1019b0 0x7f441c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:10.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f441a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441c1019b0 0x7f441c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38546/0 (socket says 172.21.15.53:38546) 2023-11-06T01:33:10.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f441a7fc700 1 -- 172.21.15.53:0/3803162467 learned_addr learned my addr 172.21.15.53:0/3803162467 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:10.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f441affd700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c101000 0x7f441c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:10.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f441a7fc700 1 -- 172.21.15.53:0/3803162467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c101000 msgr2=0x7f441c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:10.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f441a7fc700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c101000 0x7f441c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:10.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f441a7fc700 1 -- 172.21.15.53:0/3803162467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f441c1060e0 msgr2=0x7f441c073b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:10.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f441b7fe700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f441c1060e0 0x7f441c073b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:10.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f441a7fc700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f441c1060e0 0x7f441c073b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:10.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f441a7fc700 1 -- 172.21.15.53:0/3803162467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44100095f0 con 0x7f441c1019b0 2023-11-06T01:33:10.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.401+0000 7f441a7fc700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441c1019b0 0x7f441c0735d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4410003730 tx=0x7f441000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:10.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.402+0000 7f4403fff700 1 -- 172.21.15.53:0/3803162467 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441001d070 con 0x7f441c1019b0 2023-11-06T01:33:10.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.402+0000 7f4403fff700 1 -- 172.21.15.53:0/3803162467 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f441000fb80 con 0x7f441c1019b0 2023-11-06T01:33:10.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.402+0000 7f4403fff700 1 -- 172.21.15.53:0/3803162467 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44100173f0 con 0x7f441c1019b0 2023-11-06T01:33:10.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.402+0000 7f4421731700 1 -- 172.21.15.53:0/3803162467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c1a5910 con 0x7f441c1019b0 2023-11-06T01:33:10.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.402+0000 7f4421731700 1 -- 172.21.15.53:0/3803162467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c1a5cd0 con 0x7f441c1019b0 2023-11-06T01:33:10.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.403+0000 7f4421731700 1 -- 172.21.15.53:0/3803162467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c04ea90 con 0x7f441c1019b0 2023-11-06T01:33:10.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.404+0000 7f4403fff700 1 -- 172.21.15.53:0/3803162467 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f4410022470 con 0x7f441c1019b0 2023-11-06T01:33:10.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.404+0000 7f4403fff700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44040a05b0 0x7f44040a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:10.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.405+0000 7f441affd700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44040a05b0 0x7f44040a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:10.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.405+0000 7f4403fff700 1 -- 172.21.15.53:0/3803162467 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f44100cd490 con 0x7f441c1019b0 2023-11-06T01:33:10.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.405+0000 7f441affd700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44040a05b0 0x7f44040a2a70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f440c009a10 tx=0x7f440c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:10.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.408+0000 7f4403fff700 1 -- 172.21.15.53:0/3803162467 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4410097110 con 0x7f441c1019b0 2023-11-06T01:33:10.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:10 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1137783660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-06T01:33:10.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:10 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3300858506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-06T01:33:10.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:10 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/70469447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-06T01:33:10.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.618+0000 7f4421731700 1 -- 172.21.15.53:0/3803162467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f441c0745c0 con 0x7f441c1019b0 2023-11-06T01:33:10.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.619+0000 7f4403fff700 1 -- 172.21.15.53:0/3803162467 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f441009a730 con 0x7f441c1019b0 2023-11-06T01:33:10.621 INFO:teuthology.orchestra.run.smithi053.stdout:197568495657 2023-11-06T01:33:10.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 -- 172.21.15.53:0/3803162467 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44040a05b0 msgr2=0x7f44040a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44040a05b0 0x7f44040a2a70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f440c009a10 tx=0x7f440c009190 comp rx=0 tx=0).stop 2023-11-06T01:33:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 -- 172.21.15.53:0/3803162467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441c1019b0 msgr2=0x7f441c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441c1019b0 0x7f441c0735d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4410003730 tx=0x7f441000f690 comp rx=0 tx=0).stop 2023-11-06T01:33:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 -- 172.21.15.53:0/3803162467 shutdown_connections 2023-11-06T01:33:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44040a05b0 0x7f44040a2a70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f441c1060e0 0x7f441c073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f441c101000 0x7f441c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 --2- 172.21.15.53:0/3803162467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441c1019b0 0x7f441c0735d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 -- 172.21.15.53:0/3803162467 >> 172.21.15.53:0/3803162467 conn(0x7f441c0fc890 msgr2=0x7f441c104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 -- 172.21.15.53:0/3803162467 shutdown_connections 2023-11-06T01:33:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:10.621+0000 7f4421731700 1 -- 172.21.15.53:0/3803162467 wait complete. 2023-11-06T01:33:10.993 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738432 got 34359738435 for osd.0 2023-11-06T01:33:10.994 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:11.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:11 smithi053 ceph-mon[99472]: pgmap v274: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:11.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:11 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3803162467' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-06T01:33:11.263 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332129 got 219043332131 for osd.7 2023-11-06T01:33:11.263 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:11.362 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495654 got 197568495657 for osd.6 2023-11-06T01:33:11.362 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:11.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:11 smithi062 ceph-mon[103924]: pgmap v274: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:11.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:11 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3803162467' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-06T01:33:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:11 smithi163 ceph-mon[104416]: pgmap v274: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:11 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3803162467' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-06T01:33:11.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.667+0000 7f9149c3c700 1 -- 172.21.15.53:0/2482163309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91441060f0 msgr2=0x7f914410dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:11.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.667+0000 7f9149c3c700 1 --2- 172.21.15.53:0/2482163309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91441060f0 0x7f914410dcc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f913c00b0a0 tx=0x7f913c00b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.667+0000 7f9149c3c700 1 -- 172.21.15.53:0/2482163309 shutdown_connections 2023-11-06T01:33:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.667+0000 7f9149c3c700 1 --2- 172.21.15.53:0/2482163309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91441060f0 0x7f914410dcc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.667+0000 7f9149c3c700 1 --2- 172.21.15.53:0/2482163309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9144101010 0x7f91441013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.667+0000 7f9149c3c700 1 --2- 172.21.15.53:0/2482163309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91441019c0 0x7f9144105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.667+0000 7f9149c3c700 1 -- 172.21.15.53:0/2482163309 >> 172.21.15.53:0/2482163309 conn(0x7f91440fc880 msgr2=0x7f91440feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.667+0000 7f9149c3c700 1 -- 172.21.15.53:0/2482163309 shutdown_connections 2023-11-06T01:33:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.667+0000 7f9149c3c700 1 -- 172.21.15.53:0/2482163309 wait complete. 2023-11-06T01:33:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f9149c3c700 1 Processor -- start 2023-11-06T01:33:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f9149c3c700 1 -- start start 2023-11-06T01:33:11.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f9149c3c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9144101010 0x7f9144073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:11.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f9149c3c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91441019c0 0x7f91440735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:11.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f9149c3c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91441060f0 0x7f9144073b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:11.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f9149c3c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91441a54f0 con 0x7f91441019c0 2023-11-06T01:33:11.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f9149c3c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91441a5660 con 0x7f9144101010 2023-11-06T01:33:11.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f9149c3c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91441a5820 con 0x7f91441060f0 2023-11-06T01:33:11.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f9142ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91441019c0 0x7f91440735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:11.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f9142ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91441019c0 0x7f91440735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38560/0 (socket says 172.21.15.53:38560) 2023-11-06T01:33:11.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f9142ffd700 1 -- 172.21.15.53:0/700990582 learned_addr learned my addr 172.21.15.53:0/700990582 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:11.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.668+0000 7f91437fe700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9144101010 0x7f9144073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:11.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9143fff700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91441060f0 0x7f9144073b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:11.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9142ffd700 1 -- 172.21.15.53:0/700990582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91441060f0 msgr2=0x7f9144073b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:11.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9142ffd700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91441060f0 0x7f9144073b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:11.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9142ffd700 1 -- 172.21.15.53:0/700990582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9144101010 msgr2=0x7f9144073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:11.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9142ffd700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9144101010 0x7f9144073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:11.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9142ffd700 1 -- 172.21.15.53:0/700990582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c009d00 con 0x7f91441019c0 2023-11-06T01:33:11.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9143fff700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91441060f0 0x7f9144073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:11.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9142ffd700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91441019c0 0x7f91440735d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f913800c8f0 tx=0x7f913800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:11.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9140ff9700 1 -- 172.21.15.53:0/700990582 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138007940 con 0x7f91441019c0 2023-11-06T01:33:11.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9140ff9700 1 -- 172.21.15.53:0/700990582 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f913800f450 con 0x7f91441019c0 2023-11-06T01:33:11.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9149c3c700 1 -- 172.21.15.53:0/700990582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91441a5b00 con 0x7f91441019c0 2023-11-06T01:33:11.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9140ff9700 1 -- 172.21.15.53:0/700990582 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138018660 con 0x7f91441019c0 2023-11-06T01:33:11.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.669+0000 7f9149c3c700 1 -- 172.21.15.53:0/700990582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91441a6000 con 0x7f91441019c0 2023-11-06T01:33:11.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.670+0000 7f9149c3c700 1 -- 172.21.15.53:0/700990582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914404ea90 con 0x7f91441019c0 2023-11-06T01:33:11.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.671+0000 7f9140ff9700 1 -- 172.21.15.53:0/700990582 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f913800f5c0 con 0x7f91441019c0 2023-11-06T01:33:11.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.672+0000 7f9140ff9700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f912c0a05b0 0x7f912c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:11.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.672+0000 7f9140ff9700 1 -- 172.21.15.53:0/700990582 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f91380cb8c0 con 0x7f91441019c0 2023-11-06T01:33:11.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.672+0000 7f91437fe700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f912c0a05b0 0x7f912c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:11.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.672+0000 7f91437fe700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f912c0a05b0 0x7f912c0a2a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9134005950 tx=0x7f913400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:11.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.676+0000 7f9140ff9700 1 -- 172.21.15.53:0/700990582 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f91380953b0 con 0x7f91441019c0 2023-11-06T01:33:11.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.874+0000 7f9149c3c700 1 -- 172.21.15.53:0/700990582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f91441a62f0 con 0x7f91441019c0 2023-11-06T01:33:11.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.875+0000 7f9140ff9700 1 -- 172.21.15.53:0/700990582 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f91380989d0 con 0x7f91441019c0 2023-11-06T01:33:11.877 INFO:teuthology.orchestra.run.smithi053.stdout:77309411386 2023-11-06T01:33:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 -- 172.21.15.53:0/700990582 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f912c0a05b0 msgr2=0x7f912c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f912c0a05b0 0x7f912c0a2a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9134005950 tx=0x7f913400b410 comp rx=0 tx=0).stop 2023-11-06T01:33:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 -- 172.21.15.53:0/700990582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91441019c0 msgr2=0x7f91440735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91441019c0 0x7f91440735d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f913800c8f0 tx=0x7f913800cc00 comp rx=0 tx=0).stop 2023-11-06T01:33:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 -- 172.21.15.53:0/700990582 shutdown_connections 2023-11-06T01:33:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f912c0a05b0 0x7f912c0a2a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9144101010 0x7f9144073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91441060f0 0x7f9144073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 --2- 172.21.15.53:0/700990582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91441019c0 0x7f91440735d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 -- 172.21.15.53:0/700990582 >> 172.21.15.53:0/700990582 conn(0x7f91440fc880 msgr2=0x7f91440fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 -- 172.21.15.53:0/700990582 shutdown_connections 2023-11-06T01:33:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:11.877+0000 7f9149c3c700 1 -- 172.21.15.53:0/700990582 wait complete. 2023-11-06T01:33:12.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:12 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/700990582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-06T01:33:12.331 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411384 got 77309411386 for osd.2 2023-11-06T01:33:12.332 DEBUG:teuthology.parallel:result is None 2023-11-06T01:33:12.332 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-11-06T01:33:12.333 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph pg dump --format=json 2023-11-06T01:33:12.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:12 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/700990582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-06T01:33:12.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:12 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/700990582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-06T01:33:12.893 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:13.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:13 smithi053 ceph-mon[99472]: pgmap v275: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-11-06T01:33:13.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:13 smithi062 ceph-mon[103924]: pgmap v275: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-11-06T01:33:13.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:13 smithi163 ceph-mon[104416]: pgmap v275: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-11-06T01:33:14.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.142+0000 7f2f59a3f700 1 -- 172.21.15.53:0/679555178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f540688b0 msgr2=0x7f2f54068d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:14.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.142+0000 7f2f59a3f700 1 --2- 172.21.15.53:0/679555178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f540688b0 0x7f2f54068d30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009990 tx=0x7f2f3c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:14.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.142+0000 7f2f59a3f700 1 -- 172.21.15.53:0/679555178 shutdown_connections 2023-11-06T01:33:14.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.142+0000 7f2f59a3f700 1 --2- 172.21.15.53:0/679555178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f54106f20 0x7f2f54107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:14.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.142+0000 7f2f59a3f700 1 --2- 172.21.15.53:0/679555178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f54069270 0x7f2f5410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:14.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.142+0000 7f2f59a3f700 1 --2- 172.21.15.53:0/679555178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f540688b0 0x7f2f54068d30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:14.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.142+0000 7f2f59a3f700 1 -- 172.21.15.53:0/679555178 >> 172.21.15.53:0/679555178 conn(0x7f2f54076d90 msgr2=0x7f2f540771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:14.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.142+0000 7f2f59a3f700 1 -- 172.21.15.53:0/679555178 shutdown_connections 2023-11-06T01:33:14.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.142+0000 7f2f59a3f700 1 -- 172.21.15.53:0/679555178 wait complete. 2023-11-06T01:33:14.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f59a3f700 1 Processor -- start 2023-11-06T01:33:14.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f59a3f700 1 -- start start 2023-11-06T01:33:14.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f59a3f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f540688b0 0x7f2f54196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:14.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f59a3f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f54069270 0x7f2f54197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:14.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f59a3f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f54106f20 0x7f2f5419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:14.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f59a3f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5419bda0 con 0x7f2f540688b0 2023-11-06T01:33:14.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f59a3f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5419bf10 con 0x7f2f54069270 2023-11-06T01:33:14.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f59a3f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5419c0d0 con 0x7f2f54106f20 2023-11-06T01:33:14.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f52ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f540688b0 0x7f2f54196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:14.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f52ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f540688b0 0x7f2f54196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38572/0 (socket says 172.21.15.53:38572) 2023-11-06T01:33:14.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f52ffd700 1 -- 172.21.15.53:0/1954928706 learned_addr learned my addr 172.21.15.53:0/1954928706 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:14.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f537fe700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f54106f20 0x7f2f5419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:14.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f527fc700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f54069270 0x7f2f54197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:14.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f52ffd700 1 -- 172.21.15.53:0/1954928706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f54106f20 msgr2=0x7f2f5419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:14.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f52ffd700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f54106f20 0x7f2f5419b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:14.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f52ffd700 1 -- 172.21.15.53:0/1954928706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f54069270 msgr2=0x7f2f54197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:14.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f52ffd700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f54069270 0x7f2f54197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:14.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f52ffd700 1 -- 172.21.15.53:0/1954928706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0095f0 con 0x7f2f540688b0 2023-11-06T01:33:14.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f527fc700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f54069270 0x7f2f54197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:14.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.143+0000 7f2f537fe700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f54106f20 0x7f2f5419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:14.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.144+0000 7f2f52ffd700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f540688b0 0x7f2f54196de0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c9b0 tx=0x7f2f4400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:14.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.144+0000 7f2f58a3d700 1 -- 172.21.15.53:0/1954928706 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44007b00 con 0x7f2f540688b0 2023-11-06T01:33:14.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.144+0000 7f2f58a3d700 1 -- 172.21.15.53:0/1954928706 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f44007c60 con 0x7f2f540688b0 2023-11-06T01:33:14.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.144+0000 7f2f59a3f700 1 -- 172.21.15.53:0/1954928706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5419c3b0 con 0x7f2f540688b0 2023-11-06T01:33:14.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.144+0000 7f2f58a3d700 1 -- 172.21.15.53:0/1954928706 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44018660 con 0x7f2f540688b0 2023-11-06T01:33:14.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.144+0000 7f2f59a3f700 1 -- 172.21.15.53:0/1954928706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f541a3c10 con 0x7f2f540688b0 2023-11-06T01:33:14.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.145+0000 7f2f59a3f700 1 -- 172.21.15.53:0/1954928706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5404ea90 con 0x7f2f540688b0 2023-11-06T01:33:14.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.146+0000 7f2f58a3d700 1 -- 172.21.15.53:0/1954928706 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f2f44018880 con 0x7f2f540688b0 2023-11-06T01:33:14.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.146+0000 7f2f58a3d700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2f400a05b0 0x7f2f400a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:14.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.147+0000 7f2f527fc700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2f400a05b0 0x7f2f400a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:14.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.147+0000 7f2f58a3d700 1 -- 172.21.15.53:0/1954928706 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2f440cba20 con 0x7f2f540688b0 2023-11-06T01:33:14.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.147+0000 7f2f527fc700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2f400a05b0 0x7f2f400a2a70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00b5c0 tx=0x7f2f3c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:14.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.151+0000 7f2f58a3d700 1 -- 172.21.15.53:0/1954928706 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2f44095610 con 0x7f2f540688b0 2023-11-06T01:33:14.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.345+0000 7f2f59a3f700 1 -- 172.21.15.53:0/1954928706 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f541051c0 con 0x7f2f400a05b0 2023-11-06T01:33:14.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.357+0000 7f2f58a3d700 1 -- 172.21.15.53:0/1954928706 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339676 (secure 0 0 0) 0x7f2f541051c0 con 0x7f2f400a05b0 2023-11-06T01:33:14.361 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:14.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.360+0000 7f2f59a3f700 1 -- 172.21.15.53:0/1954928706 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2f400a05b0 msgr2=0x7f2f400a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:14.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.360+0000 7f2f59a3f700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2f400a05b0 0x7f2f400a2a70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00b5c0 tx=0x7f2f3c005fb0 comp rx=0 tx=0).stop 2023-11-06T01:33:14.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.360+0000 7f2f59a3f700 1 -- 172.21.15.53:0/1954928706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f540688b0 msgr2=0x7f2f54196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:14.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.360+0000 7f2f59a3f700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f540688b0 0x7f2f54196de0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c9b0 tx=0x7f2f4400cd70 comp rx=0 tx=0).stop 2023-11-06T01:33:14.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.360+0000 7f2f59a3f700 1 -- 172.21.15.53:0/1954928706 shutdown_connections 2023-11-06T01:33:14.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.360+0000 7f2f59a3f700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2f400a05b0 0x7f2f400a2a70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:14.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.360+0000 7f2f59a3f700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f54106f20 0x7f2f5419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:14.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.360+0000 7f2f59a3f700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f540688b0 0x7f2f54196de0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:14.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.360+0000 7f2f59a3f700 1 --2- 172.21.15.53:0/1954928706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f54069270 0x7f2f54197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:14.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.360+0000 7f2f59a3f700 1 -- 172.21.15.53:0/1954928706 >> 172.21.15.53:0/1954928706 conn(0x7f2f54076d90 msgr2=0x7f2f54104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:14.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.361+0000 7f2f59a3f700 1 -- 172.21.15.53:0/1954928706 shutdown_connections 2023-11-06T01:33:14.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:14.361+0000 7f2f59a3f700 1 -- 172.21.15.53:0/1954928706 wait complete. 2023-11-06T01:33:14.370 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-06T01:33:14.745 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2023-11-06T01:33:13.487532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651276,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":550,"num_read_kb":388,"num_write":410,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338,"ondisk_log_size":338,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334344,"kb_used_data":10920,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607480,"statfs":{"total":1151940427776,"available":1151598059520,"internally_reserved":0,"allocated":11182080,"data_stored":6970119,"data_compressed":108579,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003597"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408521+0000","last_change":"2023-11-06T01:32:26.110793+0000","last_active":"2023-11-06T01:32:26.408521+0000","last_peered":"2023-11-06T01:32:26.408521+0000","last_clean":"2023-11-06T01:32:26.408521+0000","last_became_active":"2023-11-06T01:32:26.110489+0000","last_became_peered":"2023-11-06T01:32:26.110489+0000","last_unstale":"2023-11-06T01:32:26.408521+0000","last_undegraded":"2023-11-06T01:32:26.408521+0000","last_fullsized":"2023-11-06T01:32:26.408521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:01:08.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231988+0000","last_change":"2023-11-06T01:32:26.107273+0000","last_active":"2023-11-06T01:32:27.231988+0000","last_peered":"2023-11-06T01:32:27.231988+0000","last_clean":"2023-11-06T01:32:27.231988+0000","last_became_active":"2023-11-06T01:32:26.106961+0000","last_became_peered":"2023-11-06T01:32:26.106961+0000","last_unstale":"2023-11-06T01:32:27.231988+0000","last_undegraded":"2023-11-06T01:32:27.231988+0000","last_fullsized":"2023-11-06T01:32:27.231988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:40:39.073926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231979+0000","last_change":"2023-11-06T01:32:26.121316+0000","last_active":"2023-11-06T01:32:27.231979+0000","last_peered":"2023-11-06T01:32:27.231979+0000","last_clean":"2023-11-06T01:32:27.231979+0000","last_became_active":"2023-11-06T01:32:26.120756+0000","last_became_peered":"2023-11-06T01:32:26.120756+0000","last_unstale":"2023-11-06T01:32:27.231979+0000","last_undegraded":"2023-11-06T01:32:27.231979+0000","last_fullsized":"2023-11-06T01:32:27.231979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:02:45.814577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.100053+0000","last_change":"2023-11-06T01:32:26.121951+0000","last_active":"2023-11-06T01:32:27.100053+0000","last_peered":"2023-11-06T01:32:27.100053+0000","last_clean":"2023-11-06T01:32:27.100053+0000","last_became_active":"2023-11-06T01:32:26.121636+0000","last_became_peered":"2023-11-06T01:32:26.121636+0000","last_unstale":"2023-11-06T01:32:27.100053+0000","last_undegraded":"2023-11-06T01:32:27.100053+0000","last_fullsized":"2023-11-06T01:32:27.100053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:02.381355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.119944+0000","last_change":"2023-11-06T01:32:26.117788+0000","last_active":"2023-11-06T01:32:26.119944+0000","last_peered":"2023-11-06T01:32:26.119944+0000","last_clean":"2023-11-06T01:32:26.119944+0000","last_became_active":"2023-11-06T01:32:26.117161+0000","last_became_peered":"2023-11-06T01:32:26.117161+0000","last_unstale":"2023-11-06T01:32:26.119944+0000","last_undegraded":"2023-11-06T01:32:26.119944+0000","last_fullsized":"2023-11-06T01:32:26.119944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:31.665690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221591+0000","last_change":"2023-11-06T01:32:26.109532+0000","last_active":"2023-11-06T01:32:27.221591+0000","last_peered":"2023-11-06T01:32:27.221591+0000","last_clean":"2023-11-06T01:32:27.221591+0000","last_became_active":"2023-11-06T01:32:26.106144+0000","last_became_peered":"2023-11-06T01:32:26.106144+0000","last_unstale":"2023-11-06T01:32:27.221591+0000","last_undegraded":"2023-11-06T01:32:27.221591+0000","last_fullsized":"2023-11-06T01:32:27.221591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:54.220515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221306+0000","last_change":"2023-11-06T01:32:26.116280+0000","last_active":"2023-11-06T01:32:27.221306+0000","last_peered":"2023-11-06T01:32:27.221306+0000","last_clean":"2023-11-06T01:32:27.221306+0000","last_became_active":"2023-11-06T01:32:26.115957+0000","last_became_peered":"2023-11-06T01:32:26.115957+0000","last_unstale":"2023-11-06T01:32:27.221306+0000","last_undegraded":"2023-11-06T01:32:27.221306+0000","last_fullsized":"2023-11-06T01:32:27.221306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:25.956902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221244+0000","last_change":"2023-11-06T01:32:26.124818+0000","last_active":"2023-11-06T01:32:27.221244+0000","last_peered":"2023-11-06T01:32:27.221244+0000","last_clean":"2023-11-06T01:32:27.221244+0000","last_became_active":"2023-11-06T01:32:26.124511+0000","last_became_peered":"2023-11-06T01:32:26.124511+0000","last_unstale":"2023-11-06T01:32:27.221244+0000","last_undegraded":"2023-11-06T01:32:27.221244+0000","last_fullsized":"2023-11-06T01:32:27.221244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:35.853898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222751+0000","last_change":"2023-11-06T01:32:26.114243+0000","last_active":"2023-11-06T01:32:27.222751+0000","last_peered":"2023-11-06T01:32:27.222751+0000","last_clean":"2023-11-06T01:32:27.222751+0000","last_became_active":"2023-11-06T01:32:26.113923+0000","last_became_peered":"2023-11-06T01:32:26.113923+0000","last_unstale":"2023-11-06T01:32:27.222751+0000","last_undegraded":"2023-11-06T01:32:27.222751+0000","last_fullsized":"2023-11-06T01:32:27.222751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:48.197329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221817+0000","last_change":"2023-11-06T01:32:26.113819+0000","last_active":"2023-11-06T01:32:27.221817+0000","last_peered":"2023-11-06T01:32:27.221817+0000","last_clean":"2023-11-06T01:32:27.221817+0000","last_became_active":"2023-11-06T01:32:26.113506+0000","last_became_peered":"2023-11-06T01:32:26.113506+0000","last_unstale":"2023-11-06T01:32:27.221817+0000","last_undegraded":"2023-11-06T01:32:27.221817+0000","last_fullsized":"2023-11-06T01:32:27.221817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:33.122100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221622+0000","last_change":"2023-11-06T01:32:26.116504+0000","last_active":"2023-11-06T01:32:27.221622+0000","last_peered":"2023-11-06T01:32:27.221622+0000","last_clean":"2023-11-06T01:32:27.221622+0000","last_became_active":"2023-11-06T01:32:26.115220+0000","last_became_peered":"2023-11-06T01:32:26.115220+0000","last_unstale":"2023-11-06T01:32:27.221622+0000","last_undegraded":"2023-11-06T01:32:27.221622+0000","last_fullsized":"2023-11-06T01:32:27.221622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:34.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.100025+0000","last_change":"2023-11-06T01:32:26.111596+0000","last_active":"2023-11-06T01:32:27.100025+0000","last_peered":"2023-11-06T01:32:27.100025+0000","last_clean":"2023-11-06T01:32:27.100025+0000","last_became_active":"2023-11-06T01:32:26.111309+0000","last_became_peered":"2023-11-06T01:32:26.111309+0000","last_unstale":"2023-11-06T01:32:27.100025+0000","last_undegraded":"2023-11-06T01:32:27.100025+0000","last_fullsized":"2023-11-06T01:32:27.100025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:34.917068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.098303+0000","last_change":"2023-11-06T01:32:26.114536+0000","last_active":"2023-11-06T01:32:27.098303+0000","last_peered":"2023-11-06T01:32:27.098303+0000","last_clean":"2023-11-06T01:32:27.098303+0000","last_became_active":"2023-11-06T01:32:26.114124+0000","last_became_peered":"2023-11-06T01:32:26.114124+0000","last_unstale":"2023-11-06T01:32:27.098303+0000","last_undegraded":"2023-11-06T01:32:27.098303+0000","last_fullsized":"2023-11-06T01:32:27.098303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:05:10.855240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231958+0000","last_change":"2023-11-06T01:32:26.111139+0000","last_active":"2023-11-06T01:32:27.231958+0000","last_peered":"2023-11-06T01:32:27.231958+0000","last_clean":"2023-11-06T01:32:27.231958+0000","last_became_active":"2023-11-06T01:32:26.110764+0000","last_became_peered":"2023-11-06T01:32:26.110764+0000","last_unstale":"2023-11-06T01:32:27.231958+0000","last_undegraded":"2023-11-06T01:32:27.231958+0000","last_fullsized":"2023-11-06T01:32:27.231958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:46.422701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231938+0000","last_change":"2023-11-06T01:32:26.116096+0000","last_active":"2023-11-06T01:32:27.231938+0000","last_peered":"2023-11-06T01:32:27.231938+0000","last_clean":"2023-11-06T01:32:27.231938+0000","last_became_active":"2023-11-06T01:32:26.115754+0000","last_became_peered":"2023-11-06T01:32:26.115754+0000","last_unstale":"2023-11-06T01:32:27.231938+0000","last_undegraded":"2023-11-06T01:32:27.231938+0000","last_fullsized":"2023-11-06T01:32:27.231938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:15:58.109435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.220995+0000","last_change":"2023-11-06T01:32:26.118169+0000","last_active":"2023-11-06T01:32:27.220995+0000","last_peered":"2023-11-06T01:32:27.220995+0000","last_clean":"2023-11-06T01:32:27.220995+0000","last_became_active":"2023-11-06T01:32:26.117837+0000","last_became_peered":"2023-11-06T01:32:26.117837+0000","last_unstale":"2023-11-06T01:32:27.220995+0000","last_undegraded":"2023-11-06T01:32:27.220995+0000","last_fullsized":"2023-11-06T01:32:27.220995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:46.838927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408530+0000","last_change":"2023-11-06T01:32:26.110020+0000","last_active":"2023-11-06T01:32:26.408530+0000","last_peered":"2023-11-06T01:32:26.408530+0000","last_clean":"2023-11-06T01:32:26.408530+0000","last_became_active":"2023-11-06T01:32:26.109707+0000","last_became_peered":"2023-11-06T01:32:26.109707+0000","last_unstale":"2023-11-06T01:32:26.408530+0000","last_undegraded":"2023-11-06T01:32:26.408530+0000","last_fullsized":"2023-11-06T01:32:26.408530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:00.016258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222625+0000","last_change":"2023-11-06T01:32:26.119949+0000","last_active":"2023-11-06T01:32:27.222625+0000","last_peered":"2023-11-06T01:32:27.222625+0000","last_clean":"2023-11-06T01:32:27.222625+0000","last_became_active":"2023-11-06T01:32:26.117594+0000","last_became_peered":"2023-11-06T01:32:26.117594+0000","last_unstale":"2023-11-06T01:32:27.222625+0000","last_undegraded":"2023-11-06T01:32:27.222625+0000","last_fullsized":"2023-11-06T01:32:27.222625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:17:22.757846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221583+0000","last_change":"2023-11-06T01:32:26.123176+0000","last_active":"2023-11-06T01:32:27.221583+0000","last_peered":"2023-11-06T01:32:27.221583+0000","last_clean":"2023-11-06T01:32:27.221583+0000","last_became_active":"2023-11-06T01:32:26.116687+0000","last_became_peered":"2023-11-06T01:32:26.116687+0000","last_unstale":"2023-11-06T01:32:27.221583+0000","last_undegraded":"2023-11-06T01:32:27.221583+0000","last_fullsized":"2023-11-06T01:32:27.221583+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:30.422426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:11.076938+0000","last_change":"2023-11-06T01:32:25.682991+0000","last_active":"2023-11-06T01:33:11.076938+0000","last_peered":"2023-11-06T01:33:11.076938+0000","last_clean":"2023-11-06T01:33:11.076938+0000","last_became_active":"2023-11-06T01:32:25.682687+0000","last_became_peered":"2023-11-06T01:32:25.682687+0000","last_unstale":"2023-11-06T01:33:11.076938+0000","last_undegraded":"2023-11-06T01:33:11.076938+0000","last_fullsized":"2023-11-06T01:33:11.076938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:12:54.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221983+0000","last_change":"2023-11-06T01:32:26.126350+0000","last_active":"2023-11-06T01:32:27.221983+0000","last_peered":"2023-11-06T01:32:27.221983+0000","last_clean":"2023-11-06T01:32:27.221983+0000","last_became_active":"2023-11-06T01:32:26.124703+0000","last_became_peered":"2023-11-06T01:32:26.124703+0000","last_unstale":"2023-11-06T01:32:27.221983+0000","last_undegraded":"2023-11-06T01:32:27.221983+0000","last_fullsized":"2023-11-06T01:32:27.221983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:08:25.866045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222007+0000","last_change":"2023-11-06T01:32:25.680184+0000","last_active":"2023-11-06T01:32:27.222007+0000","last_peered":"2023-11-06T01:32:27.222007+0000","last_clean":"2023-11-06T01:32:27.222007+0000","last_became_active":"2023-11-06T01:32:25.679859+0000","last_became_peered":"2023-11-06T01:32:25.679859+0000","last_unstale":"2023-11-06T01:32:27.222007+0000","last_undegraded":"2023-11-06T01:32:27.222007+0000","last_fullsized":"2023-11-06T01:32:27.222007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:04:34.153562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.232017+0000","last_change":"2023-11-06T01:32:26.114511+0000","last_active":"2023-11-06T01:32:27.232017+0000","last_peered":"2023-11-06T01:32:27.232017+0000","last_clean":"2023-11-06T01:32:27.232017+0000","last_became_active":"2023-11-06T01:32:26.114182+0000","last_became_peered":"2023-11-06T01:32:26.114182+0000","last_unstale":"2023-11-06T01:32:27.232017+0000","last_undegraded":"2023-11-06T01:32:27.232017+0000","last_fullsized":"2023-11-06T01:32:27.232017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:49:10.694358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:33.017084+0000","last_change":"2023-11-06T01:32:25.681453+0000","last_active":"2023-11-06T01:32:33.017084+0000","last_peered":"2023-11-06T01:32:33.017084+0000","last_clean":"2023-11-06T01:32:33.017084+0000","last_became_active":"2023-11-06T01:32:25.681180+0000","last_became_peered":"2023-11-06T01:32:25.681180+0000","last_unstale":"2023-11-06T01:32:33.017084+0000","last_undegraded":"2023-11-06T01:32:33.017084+0000","last_fullsized":"2023-11-06T01:32:33.017084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:39:19.738479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222542+0000","last_change":"2023-11-06T01:32:26.118748+0000","last_active":"2023-11-06T01:32:27.222542+0000","last_peered":"2023-11-06T01:32:27.222542+0000","last_clean":"2023-11-06T01:32:27.222542+0000","last_became_active":"2023-11-06T01:32:26.118449+0000","last_became_peered":"2023-11-06T01:32:26.118449+0000","last_unstale":"2023-11-06T01:32:27.222542+0000","last_undegraded":"2023-11-06T01:32:27.222542+0000","last_fullsized":"2023-11-06T01:32:27.222542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:28.134553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.012440+0000","last_change":"2023-11-06T01:32:25.680967+0000","last_active":"2023-11-06T01:32:32.012440+0000","last_peered":"2023-11-06T01:32:32.012440+0000","last_clean":"2023-11-06T01:32:32.012440+0000","last_became_active":"2023-11-06T01:32:25.678829+0000","last_became_peered":"2023-11-06T01:32:25.678829+0000","last_unstale":"2023-11-06T01:32:32.012440+0000","last_undegraded":"2023-11-06T01:32:32.012440+0000","last_fullsized":"2023-11-06T01:32:32.012440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:40:07.691852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121396+0000","last_change":"2023-11-06T01:32:26.122692+0000","last_active":"2023-11-06T01:32:27.121396+0000","last_peered":"2023-11-06T01:32:27.121396+0000","last_clean":"2023-11-06T01:32:27.121396+0000","last_became_active":"2023-11-06T01:32:26.121960+0000","last_became_peered":"2023-11-06T01:32:26.121960+0000","last_unstale":"2023-11-06T01:32:27.121396+0000","last_undegraded":"2023-11-06T01:32:27.121396+0000","last_fullsized":"2023-11-06T01:32:27.121396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:48.010467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:08.055353+0000","last_change":"2023-11-06T01:32:25.680404+0000","last_active":"2023-11-06T01:33:08.055353+0000","last_peered":"2023-11-06T01:33:08.055353+0000","last_clean":"2023-11-06T01:33:08.055353+0000","last_became_active":"2023-11-06T01:32:25.680060+0000","last_became_peered":"2023-11-06T01:32:25.680060+0000","last_unstale":"2023-11-06T01:33:08.055353+0000","last_undegraded":"2023-11-06T01:33:08.055353+0000","last_fullsized":"2023-11-06T01:33:08.055353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:55:00.476395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.119288+0000","last_change":"2023-11-06T01:32:26.116839+0000","last_active":"2023-11-06T01:32:26.119288+0000","last_peered":"2023-11-06T01:32:26.119288+0000","last_clean":"2023-11-06T01:32:26.119288+0000","last_became_active":"2023-11-06T01:32:26.116535+0000","last_became_peered":"2023-11-06T01:32:26.116535+0000","last_unstale":"2023-11-06T01:32:26.119288+0000","last_undegraded":"2023-11-06T01:32:26.119288+0000","last_fullsized":"2023-11-06T01:32:26.119288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:11.219626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.005598+0000","last_change":"2023-11-06T01:32:25.678426+0000","last_active":"2023-11-06T01:32:31.005598+0000","last_peered":"2023-11-06T01:32:31.005598+0000","last_clean":"2023-11-06T01:32:31.005598+0000","last_became_active":"2023-11-06T01:32:25.678072+0000","last_became_peered":"2023-11-06T01:32:25.678072+0000","last_unstale":"2023-11-06T01:32:31.005598+0000","last_undegraded":"2023-11-06T01:32:31.005598+0000","last_fullsized":"2023-11-06T01:32:31.005598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:09:41.630098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121277+0000","last_change":"2023-11-06T01:32:26.119184+0000","last_active":"2023-11-06T01:32:27.121277+0000","last_peered":"2023-11-06T01:32:27.121277+0000","last_clean":"2023-11-06T01:32:27.121277+0000","last_became_active":"2023-11-06T01:32:26.118896+0000","last_became_peered":"2023-11-06T01:32:26.118896+0000","last_unstale":"2023-11-06T01:32:27.121277+0000","last_undegraded":"2023-11-06T01:32:27.121277+0000","last_fullsized":"2023-11-06T01:32:27.121277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:09:17.351315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.456790+0000","last_change":"2023-11-06T01:32:25.685621+0000","last_active":"2023-11-06T01:32:30.456790+0000","last_peered":"2023-11-06T01:32:30.456790+0000","last_clean":"2023-11-06T01:32:30.456790+0000","last_became_active":"2023-11-06T01:32:25.684965+0000","last_became_peered":"2023-11-06T01:32:25.684965+0000","last_unstale":"2023-11-06T01:32:30.456790+0000","last_undegraded":"2023-11-06T01:32:30.456790+0000","last_fullsized":"2023-11-06T01:32:30.456790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:28:00.922680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221174+0000","last_change":"2023-11-06T01:32:26.122750+0000","last_active":"2023-11-06T01:32:27.221174+0000","last_peered":"2023-11-06T01:32:27.221174+0000","last_clean":"2023-11-06T01:32:27.221174+0000","last_became_active":"2023-11-06T01:32:26.122241+0000","last_became_peered":"2023-11-06T01:32:26.122241+0000","last_unstale":"2023-11-06T01:32:27.221174+0000","last_undegraded":"2023-11-06T01:32:27.221174+0000","last_fullsized":"2023-11-06T01:32:27.221174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:04.402197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.017051+0000","last_change":"2023-11-06T01:32:25.681594+0000","last_active":"2023-11-06T01:32:32.017051+0000","last_peered":"2023-11-06T01:32:32.017051+0000","last_clean":"2023-11-06T01:32:32.017051+0000","last_became_active":"2023-11-06T01:32:25.681303+0000","last_became_peered":"2023-11-06T01:32:25.681303+0000","last_unstale":"2023-11-06T01:32:32.017051+0000","last_undegraded":"2023-11-06T01:32:32.017051+0000","last_fullsized":"2023-11-06T01:32:32.017051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:33.141440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221062+0000","last_change":"2023-11-06T01:32:26.117234+0000","last_active":"2023-11-06T01:32:27.221062+0000","last_peered":"2023-11-06T01:32:27.221062+0000","last_clean":"2023-11-06T01:32:27.221062+0000","last_became_active":"2023-11-06T01:32:26.116915+0000","last_became_peered":"2023-11-06T01:32:26.116915+0000","last_unstale":"2023-11-06T01:32:27.221062+0000","last_undegraded":"2023-11-06T01:32:27.221062+0000","last_fullsized":"2023-11-06T01:32:27.221062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:05:27.521552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":92,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:09.056100+0000","last_change":"2023-11-06T01:32:25.682827+0000","last_active":"2023-11-06T01:33:09.056100+0000","last_peered":"2023-11-06T01:33:09.056100+0000","last_clean":"2023-11-06T01:33:09.056100+0000","last_became_active":"2023-11-06T01:32:25.682470+0000","last_became_peered":"2023-11-06T01:32:25.682470+0000","last_unstale":"2023-11-06T01:33:09.056100+0000","last_undegraded":"2023-11-06T01:33:09.056100+0000","last_fullsized":"2023-11-06T01:33:09.056100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:43.632070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222801+0000","last_change":"2023-11-06T01:32:26.123918+0000","last_active":"2023-11-06T01:32:27.222801+0000","last_peered":"2023-11-06T01:32:27.222801+0000","last_clean":"2023-11-06T01:32:27.222801+0000","last_became_active":"2023-11-06T01:32:26.123450+0000","last_became_peered":"2023-11-06T01:32:26.123450+0000","last_unstale":"2023-11-06T01:32:27.222801+0000","last_undegraded":"2023-11-06T01:32:27.222801+0000","last_fullsized":"2023-11-06T01:32:27.222801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:38.227311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.013421+0000","last_change":"2023-11-06T01:32:25.680051+0000","last_active":"2023-11-06T01:32:32.013421+0000","last_peered":"2023-11-06T01:32:32.013421+0000","last_clean":"2023-11-06T01:32:32.013421+0000","last_became_active":"2023-11-06T01:32:25.679705+0000","last_became_peered":"2023-11-06T01:32:25.679705+0000","last_unstale":"2023-11-06T01:32:32.013421+0000","last_undegraded":"2023-11-06T01:32:32.013421+0000","last_fullsized":"2023-11-06T01:32:32.013421+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:18:22.620036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.232007+0000","last_change":"2023-11-06T01:32:26.115101+0000","last_active":"2023-11-06T01:32:27.232007+0000","last_peered":"2023-11-06T01:32:27.232007+0000","last_clean":"2023-11-06T01:32:27.232007+0000","last_became_active":"2023-11-06T01:32:26.114480+0000","last_became_peered":"2023-11-06T01:32:26.114480+0000","last_unstale":"2023-11-06T01:32:27.232007+0000","last_undegraded":"2023-11-06T01:32:27.232007+0000","last_fullsized":"2023-11-06T01:32:27.232007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:17.770085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.998972+0000","last_change":"2023-11-06T01:32:25.678689+0000","last_active":"2023-11-06T01:32:30.998972+0000","last_peered":"2023-11-06T01:32:30.998972+0000","last_clean":"2023-11-06T01:32:30.998972+0000","last_became_active":"2023-11-06T01:32:25.678355+0000","last_became_peered":"2023-11-06T01:32:25.678355+0000","last_unstale":"2023-11-06T01:32:30.998972+0000","last_undegraded":"2023-11-06T01:32:30.998972+0000","last_fullsized":"2023-11-06T01:32:30.998972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:23.007596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222655+0000","last_change":"2023-11-06T01:32:26.118120+0000","last_active":"2023-11-06T01:32:27.222655+0000","last_peered":"2023-11-06T01:32:27.222655+0000","last_clean":"2023-11-06T01:32:27.222655+0000","last_became_active":"2023-11-06T01:32:26.117762+0000","last_became_peered":"2023-11-06T01:32:26.117762+0000","last_unstale":"2023-11-06T01:32:27.222655+0000","last_undegraded":"2023-11-06T01:32:27.222655+0000","last_fullsized":"2023-11-06T01:32:27.222655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:28:42.417463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'27","reported_seq":123,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:09.056235+0000","last_change":"2023-11-06T01:32:25.676036+0000","last_active":"2023-11-06T01:33:09.056235+0000","last_peered":"2023-11-06T01:33:09.056235+0000","last_clean":"2023-11-06T01:33:09.056235+0000","last_became_active":"2023-11-06T01:32:25.675672+0000","last_became_peered":"2023-11-06T01:32:25.675672+0000","last_unstale":"2023-11-06T01:33:09.056235+0000","last_undegraded":"2023-11-06T01:33:09.056235+0000","last_fullsized":"2023-11-06T01:33:09.056235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:30:54.060926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54039,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":84,"num_read_kb":94,"num_write":29,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.120038+0000","last_change":"2023-11-06T01:32:26.118832+0000","last_active":"2023-11-06T01:32:26.120038+0000","last_peered":"2023-11-06T01:32:26.120038+0000","last_clean":"2023-11-06T01:32:26.120038+0000","last_became_active":"2023-11-06T01:32:26.118551+0000","last_became_peered":"2023-11-06T01:32:26.118551+0000","last_unstale":"2023-11-06T01:32:26.120038+0000","last_undegraded":"2023-11-06T01:32:26.120038+0000","last_fullsized":"2023-11-06T01:32:26.120038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:23.031203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.013630+0000","last_change":"2023-11-06T01:32:25.679433+0000","last_active":"2023-11-06T01:32:32.013630+0000","last_peered":"2023-11-06T01:32:32.013630+0000","last_clean":"2023-11-06T01:32:32.013630+0000","last_became_active":"2023-11-06T01:32:25.679090+0000","last_became_peered":"2023-11-06T01:32:25.679090+0000","last_unstale":"2023-11-06T01:32:32.013630+0000","last_undegraded":"2023-11-06T01:32:32.013630+0000","last_fullsized":"2023-11-06T01:32:32.013630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:14:57.795474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097850+0000","last_change":"2023-11-06T01:32:26.113421+0000","last_active":"2023-11-06T01:32:27.097850+0000","last_peered":"2023-11-06T01:32:27.097850+0000","last_clean":"2023-11-06T01:32:27.097850+0000","last_became_active":"2023-11-06T01:32:26.113085+0000","last_became_peered":"2023-11-06T01:32:26.113085+0000","last_unstale":"2023-11-06T01:32:27.097850+0000","last_undegraded":"2023-11-06T01:32:27.097850+0000","last_fullsized":"2023-11-06T01:32:27.097850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:15:04.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.016512+0000","last_change":"2023-11-06T01:32:25.680293+0000","last_active":"2023-11-06T01:32:32.016512+0000","last_peered":"2023-11-06T01:32:32.016512+0000","last_clean":"2023-11-06T01:32:32.016512+0000","last_became_active":"2023-11-06T01:32:25.679803+0000","last_became_peered":"2023-11-06T01:32:25.679803+0000","last_unstale":"2023-11-06T01:32:32.016512+0000","last_undegraded":"2023-11-06T01:32:32.016512+0000","last_fullsized":"2023-11-06T01:32:32.016512+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:28.562797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221634+0000","last_change":"2023-11-06T01:32:26.115996+0000","last_active":"2023-11-06T01:32:27.221634+0000","last_peered":"2023-11-06T01:32:27.221634+0000","last_clean":"2023-11-06T01:32:27.221634+0000","last_became_active":"2023-11-06T01:32:26.115407+0000","last_became_peered":"2023-11-06T01:32:26.115407+0000","last_unstale":"2023-11-06T01:32:27.221634+0000","last_undegraded":"2023-11-06T01:32:27.221634+0000","last_fullsized":"2023-11-06T01:32:27.221634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:08.069698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:11.059592+0000","last_change":"2023-11-06T01:32:25.680958+0000","last_active":"2023-11-06T01:33:11.059592+0000","last_peered":"2023-11-06T01:33:11.059592+0000","last_clean":"2023-11-06T01:33:11.059592+0000","last_became_active":"2023-11-06T01:32:25.680616+0000","last_became_peered":"2023-11-06T01:32:25.680616+0000","last_unstale":"2023-11-06T01:33:11.059592+0000","last_undegraded":"2023-11-06T01:33:11.059592+0000","last_fullsized":"2023-11-06T01:33:11.059592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:02.631755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221186+0000","last_change":"2023-11-06T01:32:26.115464+0000","last_active":"2023-11-06T01:32:27.221186+0000","last_peered":"2023-11-06T01:32:27.221186+0000","last_clean":"2023-11-06T01:32:27.221186+0000","last_became_active":"2023-11-06T01:32:26.115090+0000","last_became_peered":"2023-11-06T01:32:26.115090+0000","last_unstale":"2023-11-06T01:32:27.221186+0000","last_undegraded":"2023-11-06T01:32:27.221186+0000","last_fullsized":"2023-11-06T01:32:27.221186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:08.018044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.004305+0000","last_change":"2023-11-06T01:32:25.677787+0000","last_active":"2023-11-06T01:32:31.004305+0000","last_peered":"2023-11-06T01:32:31.004305+0000","last_clean":"2023-11-06T01:32:31.004305+0000","last_became_active":"2023-11-06T01:32:25.677527+0000","last_became_peered":"2023-11-06T01:32:25.677527+0000","last_unstale":"2023-11-06T01:32:31.004305+0000","last_undegraded":"2023-11-06T01:32:31.004305+0000","last_fullsized":"2023-11-06T01:32:31.004305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:34:09.452965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121118+0000","last_change":"2023-11-06T01:32:26.117792+0000","last_active":"2023-11-06T01:32:27.121118+0000","last_peered":"2023-11-06T01:32:27.121118+0000","last_clean":"2023-11-06T01:32:27.121118+0000","last_became_active":"2023-11-06T01:32:26.117469+0000","last_became_peered":"2023-11-06T01:32:26.117469+0000","last_unstale":"2023-11-06T01:32:27.121118+0000","last_undegraded":"2023-11-06T01:32:27.121118+0000","last_fullsized":"2023-11-06T01:32:27.121118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:44:54.409476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.993990+0000","last_change":"2023-11-06T01:32:25.677257+0000","last_active":"2023-11-06T01:32:30.993990+0000","last_peered":"2023-11-06T01:32:30.993990+0000","last_clean":"2023-11-06T01:32:30.993990+0000","last_became_active":"2023-11-06T01:32:25.676536+0000","last_became_peered":"2023-11-06T01:32:25.676536+0000","last_unstale":"2023-11-06T01:32:30.993990+0000","last_undegraded":"2023-11-06T01:32:30.993990+0000","last_fullsized":"2023-11-06T01:32:30.993990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:53:33.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408512+0000","last_change":"2023-11-06T01:32:26.111447+0000","last_active":"2023-11-06T01:32:26.408512+0000","last_peered":"2023-11-06T01:32:26.408512+0000","last_clean":"2023-11-06T01:32:26.408512+0000","last_became_active":"2023-11-06T01:32:26.111132+0000","last_became_peered":"2023-11-06T01:32:26.111132+0000","last_unstale":"2023-11-06T01:32:26.408512+0000","last_undegraded":"2023-11-06T01:32:26.408512+0000","last_fullsized":"2023-11-06T01:32:26.408512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:55.385003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.717374+0000","last_change":"2023-11-06T01:32:25.682562+0000","last_active":"2023-11-06T01:32:27.717374+0000","last_peered":"2023-11-06T01:32:27.717374+0000","last_clean":"2023-11-06T01:32:27.717374+0000","last_became_active":"2023-11-06T01:32:25.682227+0000","last_became_peered":"2023-11-06T01:32:25.682227+0000","last_unstale":"2023-11-06T01:32:27.717374+0000","last_undegraded":"2023-11-06T01:32:27.717374+0000","last_fullsized":"2023-11-06T01:32:27.717374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:40.113429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.990952+0000","last_change":"2023-11-06T01:32:26.110589+0000","last_active":"2023-11-06T01:32:29.990952+0000","last_peered":"2023-11-06T01:32:29.990952+0000","last_clean":"2023-11-06T01:32:29.990952+0000","last_became_active":"2023-11-06T01:32:26.107413+0000","last_became_peered":"2023-11-06T01:32:26.107413+0000","last_unstale":"2023-11-06T01:32:29.990952+0000","last_undegraded":"2023-11-06T01:32:29.990952+0000","last_fullsized":"2023-11-06T01:32:29.990952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:39:08.613713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":55,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:08.074150+0000","last_change":"2023-11-06T01:32:25.688176+0000","last_active":"2023-11-06T01:33:08.074150+0000","last_peered":"2023-11-06T01:33:08.074150+0000","last_clean":"2023-11-06T01:33:08.074150+0000","last_became_active":"2023-11-06T01:32:25.687745+0000","last_became_peered":"2023-11-06T01:32:25.687745+0000","last_unstale":"2023-11-06T01:33:08.074150+0000","last_undegraded":"2023-11-06T01:33:08.074150+0000","last_fullsized":"2023-11-06T01:33:08.074150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:21:23.566389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221764+0000","last_change":"2023-11-06T01:32:26.111700+0000","last_active":"2023-11-06T01:32:27.221764+0000","last_peered":"2023-11-06T01:32:27.221764+0000","last_clean":"2023-11-06T01:32:27.221764+0000","last_became_active":"2023-11-06T01:32:26.111380+0000","last_became_peered":"2023-11-06T01:32:26.111380+0000","last_unstale":"2023-11-06T01:32:27.221764+0000","last_undegraded":"2023-11-06T01:32:27.221764+0000","last_fullsized":"2023-11-06T01:32:27.221764+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:56:47.874663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121155+0000","last_change":"2023-11-06T01:32:26.121906+0000","last_active":"2023-11-06T01:32:27.121155+0000","last_peered":"2023-11-06T01:32:27.121155+0000","last_clean":"2023-11-06T01:32:27.121155+0000","last_became_active":"2023-11-06T01:32:26.121292+0000","last_became_peered":"2023-11-06T01:32:26.121292+0000","last_unstale":"2023-11-06T01:32:27.121155+0000","last_undegraded":"2023-11-06T01:32:27.121155+0000","last_fullsized":"2023-11-06T01:32:27.121155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.532098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:07.053366+0000","last_change":"2023-11-06T01:32:25.685814+0000","last_active":"2023-11-06T01:33:07.053366+0000","last_peered":"2023-11-06T01:33:07.053366+0000","last_clean":"2023-11-06T01:33:07.053366+0000","last_became_active":"2023-11-06T01:32:25.684890+0000","last_became_peered":"2023-11-06T01:32:25.684890+0000","last_unstale":"2023-11-06T01:33:07.053366+0000","last_undegraded":"2023-11-06T01:33:07.053366+0000","last_fullsized":"2023-11-06T01:33:07.053366+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:21.950945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222017+0000","last_change":"2023-11-06T01:32:26.121752+0000","last_active":"2023-11-06T01:32:27.222017+0000","last_peered":"2023-11-06T01:32:27.222017+0000","last_clean":"2023-11-06T01:32:27.222017+0000","last_became_active":"2023-11-06T01:32:26.121447+0000","last_became_peered":"2023-11-06T01:32:26.121447+0000","last_unstale":"2023-11-06T01:32:27.222017+0000","last_undegraded":"2023-11-06T01:32:27.222017+0000","last_fullsized":"2023-11-06T01:32:27.222017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:51:19.727335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.017046+0000","last_change":"2023-11-06T01:32:25.682727+0000","last_active":"2023-11-06T01:32:32.017046+0000","last_peered":"2023-11-06T01:32:32.017046+0000","last_clean":"2023-11-06T01:32:32.017046+0000","last_became_active":"2023-11-06T01:32:25.682377+0000","last_became_peered":"2023-11-06T01:32:25.682377+0000","last_unstale":"2023-11-06T01:32:32.017046+0000","last_undegraded":"2023-11-06T01:32:32.017046+0000","last_fullsized":"2023-11-06T01:32:32.017046+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:17:38.614350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":325,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.100201+0000","last_change":"2023-11-06T01:31:27.931076+0000","last_active":"2023-11-06T01:32:26.100201+0000","last_peered":"2023-11-06T01:32:26.100201+0000","last_clean":"2023-11-06T01:32:26.100201+0000","last_became_active":"2023-11-06T01:31:27.930669+0000","last_became_peered":"2023-11-06T01:31:27.930669+0000","last_unstale":"2023-11-06T01:32:26.100201+0000","last_undegraded":"2023-11-06T01:32:26.100201+0000","last_fullsized":"2023-11-06T01:32:26.100201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_clean_scrub_stamp":"2023-11-06T01:28:36.252549+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:47:48.729325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097629+0000","last_change":"2023-11-06T01:32:26.110055+0000","last_active":"2023-11-06T01:32:27.097629+0000","last_peered":"2023-11-06T01:32:27.097629+0000","last_clean":"2023-11-06T01:32:27.097629+0000","last_became_active":"2023-11-06T01:32:26.109640+0000","last_became_peered":"2023-11-06T01:32:26.109640+0000","last_unstale":"2023-11-06T01:32:27.097629+0000","last_undegraded":"2023-11-06T01:32:27.097629+0000","last_fullsized":"2023-11-06T01:32:27.097629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:45:43.076910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.455308+0000","last_change":"2023-11-06T01:32:25.679885+0000","last_active":"2023-11-06T01:32:30.455308+0000","last_peered":"2023-11-06T01:32:30.455308+0000","last_clean":"2023-11-06T01:32:30.455308+0000","last_became_active":"2023-11-06T01:32:25.679518+0000","last_became_peered":"2023-11-06T01:32:25.679518+0000","last_unstale":"2023-11-06T01:32:30.455308+0000","last_undegraded":"2023-11-06T01:32:30.455308+0000","last_fullsized":"2023-11-06T01:32:30.455308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:44.558534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.005421+0000","last_change":"2023-11-06T01:32:25.682016+0000","last_active":"2023-11-06T01:32:31.005421+0000","last_peered":"2023-11-06T01:32:31.005421+0000","last_clean":"2023-11-06T01:32:31.005421+0000","last_became_active":"2023-11-06T01:32:25.681753+0000","last_became_peered":"2023-11-06T01:32:25.681753+0000","last_unstale":"2023-11-06T01:32:31.005421+0000","last_undegraded":"2023-11-06T01:32:31.005421+0000","last_fullsized":"2023-11-06T01:32:31.005421+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:08.575728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097816+0000","last_change":"2023-11-06T01:32:26.117109+0000","last_active":"2023-11-06T01:32:27.097816+0000","last_peered":"2023-11-06T01:32:27.097816+0000","last_clean":"2023-11-06T01:32:27.097816+0000","last_became_active":"2023-11-06T01:32:26.116769+0000","last_became_peered":"2023-11-06T01:32:26.116769+0000","last_unstale":"2023-11-06T01:32:27.097816+0000","last_undegraded":"2023-11-06T01:32:27.097816+0000","last_fullsized":"2023-11-06T01:32:27.097816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:48:04.188806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:34.453895+0000","last_change":"2023-11-06T01:32:25.681013+0000","last_active":"2023-11-06T01:32:34.453895+0000","last_peered":"2023-11-06T01:32:34.453895+0000","last_clean":"2023-11-06T01:32:34.453895+0000","last_became_active":"2023-11-06T01:32:25.680699+0000","last_became_peered":"2023-11-06T01:32:25.680699+0000","last_unstale":"2023-11-06T01:32:34.453895+0000","last_undegraded":"2023-11-06T01:32:34.453895+0000","last_fullsized":"2023-11-06T01:32:34.453895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:31.755527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221905+0000","last_change":"2023-11-06T01:32:26.116351+0000","last_active":"2023-11-06T01:32:27.221905+0000","last_peered":"2023-11-06T01:32:27.221905+0000","last_clean":"2023-11-06T01:32:27.221905+0000","last_became_active":"2023-11-06T01:32:26.116044+0000","last_became_peered":"2023-11-06T01:32:26.116044+0000","last_unstale":"2023-11-06T01:32:27.221905+0000","last_undegraded":"2023-11-06T01:32:27.221905+0000","last_fullsized":"2023-11-06T01:32:27.221905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:20.816412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.016794+0000","last_change":"2023-11-06T01:32:25.676764+0000","last_active":"2023-11-06T01:32:32.016794+0000","last_peered":"2023-11-06T01:32:32.016794+0000","last_clean":"2023-11-06T01:32:32.016794+0000","last_became_active":"2023-11-06T01:32:25.676418+0000","last_became_peered":"2023-11-06T01:32:25.676418+0000","last_unstale":"2023-11-06T01:32:32.016794+0000","last_undegraded":"2023-11-06T01:32:32.016794+0000","last_fullsized":"2023-11-06T01:32:32.016794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:49.320532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231997+0000","last_change":"2023-11-06T01:32:26.121107+0000","last_active":"2023-11-06T01:32:27.231997+0000","last_peered":"2023-11-06T01:32:27.231997+0000","last_clean":"2023-11-06T01:32:27.231997+0000","last_became_active":"2023-11-06T01:32:26.120790+0000","last_became_peered":"2023-11-06T01:32:26.120790+0000","last_unstale":"2023-11-06T01:32:27.231997+0000","last_undegraded":"2023-11-06T01:32:27.231997+0000","last_fullsized":"2023-11-06T01:32:27.231997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:34:46.660301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.447727+0000","last_change":"2023-11-06T01:32:25.678959+0000","last_active":"2023-11-06T01:32:29.447727+0000","last_peered":"2023-11-06T01:32:29.447727+0000","last_clean":"2023-11-06T01:32:29.447727+0000","last_became_active":"2023-11-06T01:32:25.678564+0000","last_became_peered":"2023-11-06T01:32:25.678564+0000","last_unstale":"2023-11-06T01:32:29.447727+0000","last_undegraded":"2023-11-06T01:32:29.447727+0000","last_fullsized":"2023-11-06T01:32:29.447727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:30:06.304889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.114597+0000","last_change":"2023-11-06T01:32:26.113222+0000","last_active":"2023-11-06T01:32:26.114597+0000","last_peered":"2023-11-06T01:32:26.114597+0000","last_clean":"2023-11-06T01:32:26.114597+0000","last_became_active":"2023-11-06T01:32:26.112926+0000","last_became_peered":"2023-11-06T01:32:26.112926+0000","last_unstale":"2023-11-06T01:32:26.114597+0000","last_undegraded":"2023-11-06T01:32:26.114597+0000","last_fullsized":"2023-11-06T01:32:26.114597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:09:51.042469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.013258+0000","last_change":"2023-11-06T01:32:25.684985+0000","last_active":"2023-11-06T01:32:32.013258+0000","last_peered":"2023-11-06T01:32:32.013258+0000","last_clean":"2023-11-06T01:32:32.013258+0000","last_became_active":"2023-11-06T01:32:25.684656+0000","last_became_peered":"2023-11-06T01:32:25.684656+0000","last_unstale":"2023-11-06T01:32:32.013258+0000","last_undegraded":"2023-11-06T01:32:32.013258+0000","last_fullsized":"2023-11-06T01:32:32.013258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:42:29.406803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408460+0000","last_change":"2023-11-06T01:32:26.107484+0000","last_active":"2023-11-06T01:32:26.408460+0000","last_peered":"2023-11-06T01:32:26.408460+0000","last_clean":"2023-11-06T01:32:26.408460+0000","last_became_active":"2023-11-06T01:32:26.106092+0000","last_became_peered":"2023-11-06T01:32:26.106092+0000","last_unstale":"2023-11-06T01:32:26.408460+0000","last_undegraded":"2023-11-06T01:32:26.408460+0000","last_fullsized":"2023-11-06T01:32:26.408460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:43:39.481290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:34.892606+0000","last_change":"2023-11-06T01:32:25.679531+0000","last_active":"2023-11-06T01:32:34.892606+0000","last_peered":"2023-11-06T01:32:34.892606+0000","last_clean":"2023-11-06T01:32:34.892606+0000","last_became_active":"2023-11-06T01:32:25.679076+0000","last_became_peered":"2023-11-06T01:32:25.679076+0000","last_unstale":"2023-11-06T01:32:34.892606+0000","last_undegraded":"2023-11-06T01:32:34.892606+0000","last_fullsized":"2023-11-06T01:32:34.892606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:37.381547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222136+0000","last_change":"2023-11-06T01:32:26.118584+0000","last_active":"2023-11-06T01:32:27.222136+0000","last_peered":"2023-11-06T01:32:27.222136+0000","last_clean":"2023-11-06T01:32:27.222136+0000","last_became_active":"2023-11-06T01:32:26.118285+0000","last_became_peered":"2023-11-06T01:32:26.118285+0000","last_unstale":"2023-11-06T01:32:27.222136+0000","last_undegraded":"2023-11-06T01:32:27.222136+0000","last_fullsized":"2023-11-06T01:32:27.222136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:52:33.057377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.447858+0000","last_change":"2023-11-06T01:32:25.681214+0000","last_active":"2023-11-06T01:32:29.447858+0000","last_peered":"2023-11-06T01:32:29.447858+0000","last_clean":"2023-11-06T01:32:29.447858+0000","last_became_active":"2023-11-06T01:32:25.680907+0000","last_became_peered":"2023-11-06T01:32:25.680907+0000","last_unstale":"2023-11-06T01:32:29.447858+0000","last_undegraded":"2023-11-06T01:32:29.447858+0000","last_fullsized":"2023-11-06T01:32:29.447858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:04:58.248787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221859+0000","last_change":"2023-11-06T01:32:26.120862+0000","last_active":"2023-11-06T01:32:27.221859+0000","last_peered":"2023-11-06T01:32:27.221859+0000","last_clean":"2023-11-06T01:32:27.221859+0000","last_became_active":"2023-11-06T01:32:26.120307+0000","last_became_peered":"2023-11-06T01:32:26.120307+0000","last_unstale":"2023-11-06T01:32:27.221859+0000","last_undegraded":"2023-11-06T01:32:27.221859+0000","last_fullsized":"2023-11-06T01:32:27.221859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:45:35.611541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:08.075152+0000","last_change":"2023-11-06T01:32:25.675607+0000","last_active":"2023-11-06T01:33:08.075152+0000","last_peered":"2023-11-06T01:33:08.075152+0000","last_clean":"2023-11-06T01:33:08.075152+0000","last_became_active":"2023-11-06T01:32:25.675293+0000","last_became_peered":"2023-11-06T01:32:25.675293+0000","last_unstale":"2023-11-06T01:33:08.075152+0000","last_undegraded":"2023-11-06T01:33:08.075152+0000","last_fullsized":"2023-11-06T01:33:08.075152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:05.190844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221522+0000","last_change":"2023-11-06T01:32:26.116987+0000","last_active":"2023-11-06T01:32:27.221522+0000","last_peered":"2023-11-06T01:32:27.221522+0000","last_clean":"2023-11-06T01:32:27.221522+0000","last_became_active":"2023-11-06T01:32:26.116756+0000","last_became_peered":"2023-11-06T01:32:26.116756+0000","last_unstale":"2023-11-06T01:32:27.221522+0000","last_undegraded":"2023-11-06T01:32:27.221522+0000","last_fullsized":"2023-11-06T01:32:27.221522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:44.039520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.005114+0000","last_change":"2023-11-06T01:32:25.675483+0000","last_active":"2023-11-06T01:32:31.005114+0000","last_peered":"2023-11-06T01:32:31.005114+0000","last_clean":"2023-11-06T01:32:31.005114+0000","last_became_active":"2023-11-06T01:32:25.675137+0000","last_became_peered":"2023-11-06T01:32:25.675137+0000","last_unstale":"2023-11-06T01:32:31.005114+0000","last_undegraded":"2023-11-06T01:32:31.005114+0000","last_fullsized":"2023-11-06T01:32:31.005114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:42.415165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.984742+0000","last_change":"2023-11-06T01:32:26.115575+0000","last_active":"2023-11-06T01:32:29.984742+0000","last_peered":"2023-11-06T01:32:29.984742+0000","last_clean":"2023-11-06T01:32:29.984742+0000","last_became_active":"2023-11-06T01:32:26.115142+0000","last_became_peered":"2023-11-06T01:32:26.115142+0000","last_unstale":"2023-11-06T01:32:29.984742+0000","last_undegraded":"2023-11-06T01:32:29.984742+0000","last_fullsized":"2023-11-06T01:32:29.984742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:52.859340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.012610+0000","last_change":"2023-11-06T01:32:25.676972+0000","last_active":"2023-11-06T01:32:32.012610+0000","last_peered":"2023-11-06T01:32:32.012610+0000","last_clean":"2023-11-06T01:32:32.012610+0000","last_became_active":"2023-11-06T01:32:25.676685+0000","last_became_peered":"2023-11-06T01:32:25.676685+0000","last_unstale":"2023-11-06T01:32:32.012610+0000","last_undegraded":"2023-11-06T01:32:32.012610+0000","last_fullsized":"2023-11-06T01:32:32.012610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:30:21.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221207+0000","last_change":"2023-11-06T01:32:26.115164+0000","last_active":"2023-11-06T01:32:27.221207+0000","last_peered":"2023-11-06T01:32:27.221207+0000","last_clean":"2023-11-06T01:32:27.221207+0000","last_became_active":"2023-11-06T01:32:26.114868+0000","last_became_peered":"2023-11-06T01:32:26.114868+0000","last_unstale":"2023-11-06T01:32:27.221207+0000","last_undegraded":"2023-11-06T01:32:27.221207+0000","last_fullsized":"2023-11-06T01:32:27.221207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:01:34.149871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408478+0000","last_change":"2023-11-06T01:32:26.115600+0000","last_active":"2023-11-06T01:32:26.408478+0000","last_peered":"2023-11-06T01:32:26.408478+0000","last_clean":"2023-11-06T01:32:26.408478+0000","last_became_active":"2023-11-06T01:32:26.115290+0000","last_became_peered":"2023-11-06T01:32:26.115290+0000","last_unstale":"2023-11-06T01:32:26.408478+0000","last_undegraded":"2023-11-06T01:32:26.408478+0000","last_fullsized":"2023-11-06T01:32:26.408478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:04:06.393671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221461+0000","last_change":"2023-11-06T01:32:26.115336+0000","last_active":"2023-11-06T01:32:27.221461+0000","last_peered":"2023-11-06T01:32:27.221461+0000","last_clean":"2023-11-06T01:32:27.221461+0000","last_became_active":"2023-11-06T01:32:26.114946+0000","last_became_peered":"2023-11-06T01:32:26.114946+0000","last_unstale":"2023-11-06T01:32:27.221461+0000","last_undegraded":"2023-11-06T01:32:27.221461+0000","last_fullsized":"2023-11-06T01:32:27.221461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:19:13.651494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221633+0000","last_change":"2023-11-06T01:32:26.117794+0000","last_active":"2023-11-06T01:32:27.221633+0000","last_peered":"2023-11-06T01:32:27.221633+0000","last_clean":"2023-11-06T01:32:27.221633+0000","last_became_active":"2023-11-06T01:32:26.117425+0000","last_became_peered":"2023-11-06T01:32:26.117425+0000","last_unstale":"2023-11-06T01:32:27.221633+0000","last_undegraded":"2023-11-06T01:32:27.221633+0000","last_fullsized":"2023-11-06T01:32:27.221633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:00:24.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231970+0000","last_change":"2023-11-06T01:32:26.113662+0000","last_active":"2023-11-06T01:32:27.231970+0000","last_peered":"2023-11-06T01:32:27.231970+0000","last_clean":"2023-11-06T01:32:27.231970+0000","last_became_active":"2023-11-06T01:32:26.113124+0000","last_became_peered":"2023-11-06T01:32:26.113124+0000","last_unstale":"2023-11-06T01:32:27.231970+0000","last_undegraded":"2023-11-06T01:32:27.231970+0000","last_fullsized":"2023-11-06T01:32:27.231970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:50:24.349521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097929+0000","last_change":"2023-11-06T01:32:26.109910+0000","last_active":"2023-11-06T01:32:27.097929+0000","last_peered":"2023-11-06T01:32:27.097929+0000","last_clean":"2023-11-06T01:32:27.097929+0000","last_became_active":"2023-11-06T01:32:26.109568+0000","last_became_peered":"2023-11-06T01:32:26.109568+0000","last_unstale":"2023-11-06T01:32:27.097929+0000","last_undegraded":"2023-11-06T01:32:27.097929+0000","last_fullsized":"2023-11-06T01:32:27.097929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:11.049749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408501+0000","last_change":"2023-11-06T01:32:26.114824+0000","last_active":"2023-11-06T01:32:26.408501+0000","last_peered":"2023-11-06T01:32:26.408501+0000","last_clean":"2023-11-06T01:32:26.408501+0000","last_became_active":"2023-11-06T01:32:26.114539+0000","last_became_peered":"2023-11-06T01:32:26.114539+0000","last_unstale":"2023-11-06T01:32:26.408501+0000","last_undegraded":"2023-11-06T01:32:26.408501+0000","last_fullsized":"2023-11-06T01:32:26.408501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:03:02.018769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221808+0000","last_change":"2023-11-06T01:32:26.114743+0000","last_active":"2023-11-06T01:32:27.221808+0000","last_peered":"2023-11-06T01:32:27.221808+0000","last_clean":"2023-11-06T01:32:27.221808+0000","last_became_active":"2023-11-06T01:32:26.114432+0000","last_became_peered":"2023-11-06T01:32:26.114432+0000","last_unstale":"2023-11-06T01:32:27.221808+0000","last_undegraded":"2023-11-06T01:32:27.221808+0000","last_fullsized":"2023-11-06T01:32:27.221808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:27:18.209810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221801+0000","last_change":"2023-11-06T01:32:26.125724+0000","last_active":"2023-11-06T01:32:27.221801+0000","last_peered":"2023-11-06T01:32:27.221801+0000","last_clean":"2023-11-06T01:32:27.221801+0000","last_became_active":"2023-11-06T01:32:26.125455+0000","last_became_peered":"2023-11-06T01:32:26.125455+0000","last_unstale":"2023-11-06T01:32:27.221801+0000","last_undegraded":"2023-11-06T01:32:27.221801+0000","last_fullsized":"2023-11-06T01:32:27.221801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:49.099308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121127+0000","last_change":"2023-11-06T01:32:26.119328+0000","last_active":"2023-11-06T01:32:27.121127+0000","last_peered":"2023-11-06T01:32:27.121127+0000","last_clean":"2023-11-06T01:32:27.121127+0000","last_became_active":"2023-11-06T01:32:26.119042+0000","last_became_peered":"2023-11-06T01:32:26.119042+0000","last_unstale":"2023-11-06T01:32:27.121127+0000","last_undegraded":"2023-11-06T01:32:27.121127+0000","last_fullsized":"2023-11-06T01:32:27.121127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:36.143788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221676+0000","last_change":"2023-11-06T01:32:26.126538+0000","last_active":"2023-11-06T01:32:27.221676+0000","last_peered":"2023-11-06T01:32:27.221676+0000","last_clean":"2023-11-06T01:32:27.221676+0000","last_became_active":"2023-11-06T01:32:26.124898+0000","last_became_peered":"2023-11-06T01:32:26.124898+0000","last_unstale":"2023-11-06T01:32:27.221676+0000","last_undegraded":"2023-11-06T01:32:27.221676+0000","last_fullsized":"2023-11-06T01:32:27.221676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:33.019326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222542+0000","last_change":"2023-11-06T01:32:26.118751+0000","last_active":"2023-11-06T01:32:27.222542+0000","last_peered":"2023-11-06T01:32:27.222542+0000","last_clean":"2023-11-06T01:32:27.222542+0000","last_became_active":"2023-11-06T01:32:26.118359+0000","last_became_peered":"2023-11-06T01:32:26.118359+0000","last_unstale":"2023-11-06T01:32:27.222542+0000","last_undegraded":"2023-11-06T01:32:27.222542+0000","last_fullsized":"2023-11-06T01:32:27.222542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:55.960400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.098195+0000","last_change":"2023-11-06T01:32:26.110133+0000","last_active":"2023-11-06T01:32:27.098195+0000","last_peered":"2023-11-06T01:32:27.098195+0000","last_clean":"2023-11-06T01:32:27.098195+0000","last_became_active":"2023-11-06T01:32:26.109793+0000","last_became_peered":"2023-11-06T01:32:26.109793+0000","last_unstale":"2023-11-06T01:32:27.098195+0000","last_undegraded":"2023-11-06T01:32:27.098195+0000","last_fullsized":"2023-11-06T01:32:27.098195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:00:03.956947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408491+0000","last_change":"2023-11-06T01:32:26.109828+0000","last_active":"2023-11-06T01:32:26.408491+0000","last_peered":"2023-11-06T01:32:26.408491+0000","last_clean":"2023-11-06T01:32:26.408491+0000","last_became_active":"2023-11-06T01:32:26.109526+0000","last_became_peered":"2023-11-06T01:32:26.109526+0000","last_unstale":"2023-11-06T01:32:26.408491+0000","last_undegraded":"2023-11-06T01:32:26.408491+0000","last_fullsized":"2023-11-06T01:32:26.408491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:36.561908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250075,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60662,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":424,"num_read_kb":279,"num_write":227,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":181986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481813,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":419251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56499999999999995}]},{"osd":1,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.41899999999999998}]},{"osd":2,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.34599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":3.4350000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.60199999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.438}]},{"osd":6,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.503},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.47099999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58899999999999997}]},{"osd":10,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.61799999999999999}]}]},{"osd":10,"up_from":70,"seq":300647710745,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":1063994,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.70299999999999996}]},{"osd":3,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.51500000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.45200000000000001}]},{"osd":5,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.65500000000000003}]},{"osd":6,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.52900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.76900000000000002}]},{"osd":9,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.499}]},{"osd":11,"last update":"Mon Nov 6 01:32:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.75}]}]},{"osd":9,"up_from":65,"seq":279172874270,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":419229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.49199999999999999,"15min":0.50700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.433},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.9780000000000002,"5min":3.9780000000000002,"15min":3.9780000000000002},"last":0.45900000000000002}]},{"osd":1,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.66900000000000004,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77800000000000002,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":6.6210000000000004,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":0.58899999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.67700000000000005,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.476,"5min":6.476,"15min":6.476},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59799999999999998,"15min":0.58499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":4.0270000000000001,"5min":4.0270000000000001,"15min":4.0270000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82199999999999995,"5min":0.87,"15min":0.87},"last":0.56899999999999995}]},{"osd":4,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.70899999999999996,"15min":0.68000000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.623},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77200000000000002,"15min":0.77100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":4.1589999999999998,"5min":4.1589999999999998,"15min":4.1589999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.79300000000000004,"15min":0.81499999999999995},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72499999999999998}]},{"osd":6,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.80100000000000005,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":6.056,"5min":6.056,"15min":6.056},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.3700000000000001,"5min":6.3700000000000001,"15min":6.3700000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85999999999999999,"15min":0.86199999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.97799999999999998,"5min":1.603,"15min":1.603},"last":0.76400000000000001}]},{"osd":8,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.83899999999999997,"15min":0.81599999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.524,"5min":6.524,"15min":6.524},"last":0.752},{"interface":"front","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.85999999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":4.1900000000000004,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":0.77400000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.69999999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":419276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.627},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57099999999999995,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71299999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59499999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46800000000000003,"15min":0.47599999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66100000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55900000000000005}]},{"osd":4,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.82399999999999995}]},{"osd":5,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.34399999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71499999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.55700000000000005,"15min":0.56499999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79900000000000004,"5min":1.036,"15min":1.036},"last":0.58399999999999996}]},{"osd":6,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67700000000000005,"15min":0.68600000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83699999999999997,"5min":1,"15min":1},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.873,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.66900000000000004}]},{"osd":7,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.51400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.83699999999999997}]},{"osd":9,"last update":"Mon Nov 6 01:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.873},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.85699999999999998}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.92200000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":1011989,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.873,"5min":1.157,"15min":1.157},"last":0.376}]},{"osd":1,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.32900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.45600000000000002,"15min":0.44800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40799999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57899999999999996,"15min":0.56799999999999995},"min":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.69599999999999995}]},{"osd":3,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.432},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85899999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.58299999999999996}]},{"osd":4,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.501},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.51300000000000001,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.53200000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76700000000000002}]},{"osd":6,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97099999999999997,"5min":1.232,"15min":1.232},"last":0.55900000000000005}]},{"osd":8,"last update":"Mon Nov 6 01:32:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.76800000000000002,"15min":0.748},"min":{"1min":0.55200000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":0.78400000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.79400000000000004,"15min":0.78200000000000003},"min":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.747}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.86299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":418904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.443},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.39000000000000001}]},{"osd":1,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81100000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.52500000000000002}]},{"osd":2,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.69699999999999995}]},{"osd":3,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.372},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.48899999999999999}]},{"osd":4,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69699999999999995,"15min":0.69399999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.58799999999999997}]},{"osd":5,"last update":"Mon Nov 6 01:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.71499999999999997,"15min":0.69999999999999996},"min":{"1min":0.55900000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63200000000000001}]},{"osd":7,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.625},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56699999999999995}]},{"osd":8,"last update":"Mon Nov 6 01:32:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.82099999999999995,"15min":0.84099999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.014,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.77200000000000002,"15min":0.79600000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88200000000000001,"5min":1.014,"15min":1.014},"last":0.61199999999999999}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.42999999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88900000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.82399999999999995,"15min":0.84799999999999998},"min":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.56100000000000005}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.70799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":473850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.51300000000000001,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.752}]},{"osd":2,"last update":"Mon Nov 6 01:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56000000000000005,"15min":0.52100000000000002},"min":{"1min":0.439,"5min":0.308,"15min":0.308},"max":{"1min":1.121,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.53800000000000003,"15min":0.505},"min":{"1min":0.41299999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67400000000000004}]},{"osd":3,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.61699999999999999,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1459999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.82599999999999996}]},{"osd":4,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.47799999999999998,"15min":0.46100000000000002},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.53500000000000003,"15min":0.51000000000000001},"min":{"1min":0.36899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Mon Nov 6 01:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.61199999999999999,"15min":0.58799999999999997},"min":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.622}]},{"osd":6,"last update":"Mon Nov 6 01:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.495,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.66000000000000003}]},{"osd":7,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.60199999999999998,"15min":0.56799999999999995},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.61399999999999999,"15min":0.57199999999999995},"min":{"1min":0.50700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.84199999999999997}]},{"osd":8,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.71099999999999997,"15min":0.68999999999999995},"min":{"1min":0.55000000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.75800000000000001,"15min":0.73799999999999999},"min":{"1min":0.60999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.71999999999999997}]},{"osd":9,"last update":"Mon Nov 6 01:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.78500000000000003}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.874},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.90800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":420237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50700000000000001},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.37},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.504},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.39100000000000001}]},{"osd":2,"last update":"Mon Nov 6 01:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.432,"5min":0.314,"15min":0.314},"max":{"1min":0.84299999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57299999999999995,"15min":0.56000000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.502}]},{"osd":3,"last update":"Mon Nov 6 01:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44,"15min":0.437},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.34899999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.41499999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.45900000000000002}]},{"osd":5,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73499999999999999,"5min":1.198,"15min":1.198},"last":0.54300000000000004}]},{"osd":6,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52800000000000002,"15min":0.55300000000000005},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66300000000000003,"5min":1.075,"15min":1.075},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.33500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.68000000000000005,"5min":1.222,"15min":1.222},"last":0.53300000000000003}]},{"osd":7,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.745,"15min":0.76800000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80900000000000005,"5min":1.292,"15min":1.292},"last":0.625},{"interface":"front","average":{"1min":0.629,"5min":0.74399999999999999,"15min":0.76300000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81699999999999995,"5min":1.341,"15min":1.341},"last":0.57399999999999995}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.66000000000000003,"15min":0.67500000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":1.123,"15min":1.123},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76000000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.58299999999999996}]},{"osd":10,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.71499999999999997}]},{"osd":11,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.64000000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":474415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.35899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.53000000000000003,"15min":0.504},"min":{"1min":0.40400000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54000000000000004}]},{"osd":1,"last update":"Mon Nov 6 01:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.56000000000000005,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.89800000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60399999999999998,"15min":0.57599999999999996},"min":{"1min":0.443,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.66900000000000004}]},{"osd":4,"last update":"Mon Nov 6 01:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.79500000000000004}]},{"osd":5,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.45500000000000002,"15min":0.439},"min":{"1min":0.317,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.53500000000000003,"15min":0.51800000000000002},"min":{"1min":0.377,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85299999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.753}]},{"osd":6,"last update":"Mon Nov 6 01:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.46200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.97199999999999998}]},{"osd":7,"last update":"Mon Nov 6 01:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.54700000000000004}]},{"osd":8,"last update":"Mon Nov 6 01:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80800000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.83899999999999997}]},{"osd":9,"last update":"Mon Nov 6 01:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76500000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.998}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.022},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.86399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":418783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.36399999999999999},"min":{"1min":0.27300000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51500000000000001,"5min":0.627,"15min":0.627},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.31,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.41399999999999998}]},{"osd":1,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":1.119,"15min":1.119},"last":0.498},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54500000000000004,"15min":0.496},"min":{"1min":0.42099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.52200000000000002}]},{"osd":2,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60399999999999998,"15min":0.55800000000000005},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83099999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61299999999999999,"15min":0.59299999999999997},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.72499999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.72599999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.502,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.98799999999999999,"5min":1.159,"15min":1.159},"last":0.46300000000000002}]},{"osd":5,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.60299999999999998},"min":{"1min":0.38900000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.754,"5min":1.042,"15min":1.042},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.442,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.874,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.432},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89800000000000002,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.59199999999999997}]},{"osd":7,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.66500000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.67600000000000005,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.622}]},{"osd":8,"last update":"Mon Nov 6 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.622},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.0430000000000001,"5min":6.0430000000000001,"15min":6.0430000000000001},"last":0.76500000000000001}]},{"osd":9,"last update":"Mon Nov 6 01:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.74199999999999999,"15min":0.76900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85899999999999999,"5min":1.319,"15min":1.319},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.46400000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82199999999999995,"5min":1.214,"15min":1.214},"last":0.56599999999999995}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.65100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":418759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.45300000000000001},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.49399999999999999,"15min":0.47799999999999998},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.749,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42999999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.377,"15min":0.35799999999999998},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.36599999999999999},"min":{"1min":0.27000000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.52600000000000002}]},{"osd":2,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.44900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.68300000000000005}]},{"osd":3,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.51400000000000001,"15min":0.49099999999999999},"min":{"1min":0.38900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.41199999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.72799999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.59899999999999998,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.45300000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.75600000000000001}]},{"osd":6,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.68999999999999995,"15min":0.64800000000000002},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":5.3949999999999996,"5min":5.3949999999999996,"15min":5.3949999999999996},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.127,"5min":0.73399999999999999,"15min":0.66800000000000004},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.77800000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.85999999999999999}]},{"osd":8,"last update":"Mon Nov 6 01:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.56699999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.60799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.90000000000000002}]},{"osd":9,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":0.82999999999999996,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.0659999999999998,"5min":7.0659999999999998,"15min":7.0659999999999998},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.73699999999999999,"15min":0.71599999999999997},"min":{"1min":0.63900000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.91300000000000003}]},{"osd":10,"last update":"Mon Nov 6 01:32:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.88600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":1011432,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.34399999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.499,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.46999999999999997}]},{"osd":1,"last update":"Mon Nov 6 01:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73399999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.71099999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.38500000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58799999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.505},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.39600000000000002,"15min":0.375},"min":{"1min":0.28199999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44900000000000001}]},{"osd":3,"last update":"Mon Nov 6 01:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.62},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60599999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.76000000000000001}]},{"osd":6,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86099999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.80400000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69199999999999995}]},{"osd":7,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73799999999999999}]},{"osd":8,"last update":"Mon Nov 6 01:32:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.80100000000000005}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.85899999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.79000000000000004}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":55743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":55727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T01:33:14.753 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph pg dump --format=json 2023-11-06T01:33:15.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:15 smithi053 ceph-mon[99472]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:15.081 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:15 smithi053 ceph-mon[99472]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:15.313 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:15.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:15 smithi062 ceph-mon[103924]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:15.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:15 smithi062 ceph-mon[103924]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:15.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:15 smithi163 ceph-mon[104416]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:15.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:15 smithi163 ceph-mon[104416]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.695+0000 7fc7eae7b700 1 -- 172.21.15.53:0/2889102136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7e4101010 msgr2=0x7fc7e41013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.695+0000 7fc7eae7b700 1 --2- 172.21.15.53:0/2889102136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7e4101010 0x7fc7e41013f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009990 tx=0x7fc7d8009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.695+0000 7fc7eae7b700 1 -- 172.21.15.53:0/2889102136 shutdown_connections 2023-11-06T01:33:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.695+0000 7fc7eae7b700 1 --2- 172.21.15.53:0/2889102136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7e41060f0 0x7fc7e410dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.695+0000 7fc7eae7b700 1 --2- 172.21.15.53:0/2889102136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7e4101010 0x7fc7e41013f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.695+0000 7fc7eae7b700 1 --2- 172.21.15.53:0/2889102136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e41019c0 0x7fc7e4105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.695+0000 7fc7eae7b700 1 -- 172.21.15.53:0/2889102136 >> 172.21.15.53:0/2889102136 conn(0x7fc7e40fc880 msgr2=0x7fc7e40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.695+0000 7fc7eae7b700 1 -- 172.21.15.53:0/2889102136 shutdown_connections 2023-11-06T01:33:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.695+0000 7fc7eae7b700 1 -- 172.21.15.53:0/2889102136 wait complete. 2023-11-06T01:33:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.695+0000 7fc7eae7b700 1 Processor -- start 2023-11-06T01:33:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.695+0000 7fc7eae7b700 1 -- start start 2023-11-06T01:33:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7eae7b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7e4101010 0x7fc7e4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7eae7b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e41019c0 0x7fc7e40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7eae7b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7e41060f0 0x7fc7e4073b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7eae7b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41a5310 con 0x7fc7e4101010 2023-11-06T01:33:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7eae7b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41a5480 con 0x7fc7e41060f0 2023-11-06T01:33:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7eae7b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41a5640 con 0x7fc7e41019c0 2023-11-06T01:33:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e8c17700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7e4101010 0x7fc7e4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e8c17700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7e4101010 0x7fc7e4073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38586/0 (socket says 172.21.15.53:38586) 2023-11-06T01:33:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e8c17700 1 -- 172.21.15.53:0/3898602077 learned_addr learned my addr 172.21.15.53:0/3898602077 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e3fff700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e41019c0 0x7fc7e40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e9418700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7e41060f0 0x7fc7e4073b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:16.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e8c17700 1 -- 172.21.15.53:0/3898602077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e41019c0 msgr2=0x7fc7e40735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:16.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e8c17700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e41019c0 0x7fc7e40735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:16.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e8c17700 1 -- 172.21.15.53:0/3898602077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7e41060f0 msgr2=0x7fc7e4073b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:16.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e8c17700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7e41060f0 0x7fc7e4073b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:16.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e8c17700 1 -- 172.21.15.53:0/3898602077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d80095f0 con 0x7fc7e4101010 2023-11-06T01:33:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e3fff700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e41019c0 0x7fc7e40735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e8c17700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7e4101010 0x7fc7e4073090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8000c00 tx=0x7fc7d800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.696+0000 7fc7e1ffb700 1 -- 172.21.15.53:0/3898602077 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d801d070 con 0x7fc7e4101010 2023-11-06T01:33:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.697+0000 7fc7e1ffb700 1 -- 172.21.15.53:0/3898602077 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7d800fc30 con 0x7fc7e4101010 2023-11-06T01:33:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.697+0000 7fc7eae7b700 1 -- 172.21.15.53:0/3898602077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e41a58c0 con 0x7fc7e4101010 2023-11-06T01:33:16.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.697+0000 7fc7e1ffb700 1 -- 172.21.15.53:0/3898602077 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8017770 con 0x7fc7e4101010 2023-11-06T01:33:16.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.697+0000 7fc7eae7b700 1 -- 172.21.15.53:0/3898602077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e41a5d30 con 0x7fc7e4101010 2023-11-06T01:33:16.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.698+0000 7fc7eae7b700 1 -- 172.21.15.53:0/3898602077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e404ea90 con 0x7fc7e4101010 2023-11-06T01:33:16.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.700+0000 7fc7e1ffb700 1 -- 172.21.15.53:0/3898602077 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc7d8022a30 con 0x7fc7e4101010 2023-11-06T01:33:16.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.701+0000 7fc7e1ffb700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7d40a0630 0x7fc7d40a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:16.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.701+0000 7fc7e3fff700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7d40a0630 0x7fc7d40a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:16.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.701+0000 7fc7e1ffb700 1 -- 172.21.15.53:0/3898602077 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc7d80ccf10 con 0x7fc7e4101010 2023-11-06T01:33:16.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.701+0000 7fc7e3fff700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7d40a0630 0x7fc7d40a2af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4074880 tx=0x7fc7d0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:16.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.703+0000 7fc7e1ffb700 1 -- 172.21.15.53:0/3898602077 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc7d8096cb0 con 0x7fc7e4101010 2023-11-06T01:33:16.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.896+0000 7fc7eae7b700 1 -- 172.21.15.53:0/3898602077 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7e41051a0 con 0x7fc7d40a0630 2023-11-06T01:33:16.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.907+0000 7fc7e1ffb700 1 -- 172.21.15.53:0/3898602077 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339660 (secure 0 0 0) 0x7fc7e41051a0 con 0x7fc7d40a0630 2023-11-06T01:33:16.914 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:16.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 -- 172.21.15.53:0/3898602077 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7d40a0630 msgr2=0x7fc7d40a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:16.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7d40a0630 0x7fc7d40a2af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4074880 tx=0x7fc7d0005f50 comp rx=0 tx=0).stop 2023-11-06T01:33:16.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 -- 172.21.15.53:0/3898602077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7e4101010 msgr2=0x7fc7e4073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:16.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7e4101010 0x7fc7e4073090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8000c00 tx=0x7fc7d800f790 comp rx=0 tx=0).stop 2023-11-06T01:33:16.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 -- 172.21.15.53:0/3898602077 shutdown_connections 2023-11-06T01:33:16.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7d40a0630 0x7fc7d40a2af0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:16.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7e41060f0 0x7fc7e4073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:16.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7e4101010 0x7fc7e4073090 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:16.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 --2- 172.21.15.53:0/3898602077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e41019c0 0x7fc7e40735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:16.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 -- 172.21.15.53:0/3898602077 >> 172.21.15.53:0/3898602077 conn(0x7fc7e40fc880 msgr2=0x7fc7e4104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:16.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 -- 172.21.15.53:0/3898602077 shutdown_connections 2023-11-06T01:33:16.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:16.911+0000 7fc7eae7b700 1 -- 172.21.15.53:0/3898602077 wait complete. 2023-11-06T01:33:16.919 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-06T01:33:17.309 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:17 smithi053 ceph-mon[99472]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-11-06T01:33:17.376 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2023-11-06T01:33:15.488256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651276,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":590,"num_read_kb":408,"num_write":410,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338,"ondisk_log_size":338,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334344,"kb_used_data":10920,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607480,"statfs":{"total":1151940427776,"available":1151598059520,"internally_reserved":0,"allocated":11182080,"data_stored":6970119,"data_compressed":108579,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003677"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408521+0000","last_change":"2023-11-06T01:32:26.110793+0000","last_active":"2023-11-06T01:32:26.408521+0000","last_peered":"2023-11-06T01:32:26.408521+0000","last_clean":"2023-11-06T01:32:26.408521+0000","last_became_active":"2023-11-06T01:32:26.110489+0000","last_became_peered":"2023-11-06T01:32:26.110489+0000","last_unstale":"2023-11-06T01:32:26.408521+0000","last_undegraded":"2023-11-06T01:32:26.408521+0000","last_fullsized":"2023-11-06T01:32:26.408521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:01:08.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231988+0000","last_change":"2023-11-06T01:32:26.107273+0000","last_active":"2023-11-06T01:32:27.231988+0000","last_peered":"2023-11-06T01:32:27.231988+0000","last_clean":"2023-11-06T01:32:27.231988+0000","last_became_active":"2023-11-06T01:32:26.106961+0000","last_became_peered":"2023-11-06T01:32:26.106961+0000","last_unstale":"2023-11-06T01:32:27.231988+0000","last_undegraded":"2023-11-06T01:32:27.231988+0000","last_fullsized":"2023-11-06T01:32:27.231988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:40:39.073926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231979+0000","last_change":"2023-11-06T01:32:26.121316+0000","last_active":"2023-11-06T01:32:27.231979+0000","last_peered":"2023-11-06T01:32:27.231979+0000","last_clean":"2023-11-06T01:32:27.231979+0000","last_became_active":"2023-11-06T01:32:26.120756+0000","last_became_peered":"2023-11-06T01:32:26.120756+0000","last_unstale":"2023-11-06T01:32:27.231979+0000","last_undegraded":"2023-11-06T01:32:27.231979+0000","last_fullsized":"2023-11-06T01:32:27.231979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:02:45.814577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.100053+0000","last_change":"2023-11-06T01:32:26.121951+0000","last_active":"2023-11-06T01:32:27.100053+0000","last_peered":"2023-11-06T01:32:27.100053+0000","last_clean":"2023-11-06T01:32:27.100053+0000","last_became_active":"2023-11-06T01:32:26.121636+0000","last_became_peered":"2023-11-06T01:32:26.121636+0000","last_unstale":"2023-11-06T01:32:27.100053+0000","last_undegraded":"2023-11-06T01:32:27.100053+0000","last_fullsized":"2023-11-06T01:32:27.100053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:02.381355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.119944+0000","last_change":"2023-11-06T01:32:26.117788+0000","last_active":"2023-11-06T01:32:26.119944+0000","last_peered":"2023-11-06T01:32:26.119944+0000","last_clean":"2023-11-06T01:32:26.119944+0000","last_became_active":"2023-11-06T01:32:26.117161+0000","last_became_peered":"2023-11-06T01:32:26.117161+0000","last_unstale":"2023-11-06T01:32:26.119944+0000","last_undegraded":"2023-11-06T01:32:26.119944+0000","last_fullsized":"2023-11-06T01:32:26.119944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:31.665690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221591+0000","last_change":"2023-11-06T01:32:26.109532+0000","last_active":"2023-11-06T01:32:27.221591+0000","last_peered":"2023-11-06T01:32:27.221591+0000","last_clean":"2023-11-06T01:32:27.221591+0000","last_became_active":"2023-11-06T01:32:26.106144+0000","last_became_peered":"2023-11-06T01:32:26.106144+0000","last_unstale":"2023-11-06T01:32:27.221591+0000","last_undegraded":"2023-11-06T01:32:27.221591+0000","last_fullsized":"2023-11-06T01:32:27.221591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:54.220515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221306+0000","last_change":"2023-11-06T01:32:26.116280+0000","last_active":"2023-11-06T01:32:27.221306+0000","last_peered":"2023-11-06T01:32:27.221306+0000","last_clean":"2023-11-06T01:32:27.221306+0000","last_became_active":"2023-11-06T01:32:26.115957+0000","last_became_peered":"2023-11-06T01:32:26.115957+0000","last_unstale":"2023-11-06T01:32:27.221306+0000","last_undegraded":"2023-11-06T01:32:27.221306+0000","last_fullsized":"2023-11-06T01:32:27.221306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:25.956902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221244+0000","last_change":"2023-11-06T01:32:26.124818+0000","last_active":"2023-11-06T01:32:27.221244+0000","last_peered":"2023-11-06T01:32:27.221244+0000","last_clean":"2023-11-06T01:32:27.221244+0000","last_became_active":"2023-11-06T01:32:26.124511+0000","last_became_peered":"2023-11-06T01:32:26.124511+0000","last_unstale":"2023-11-06T01:32:27.221244+0000","last_undegraded":"2023-11-06T01:32:27.221244+0000","last_fullsized":"2023-11-06T01:32:27.221244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:35.853898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222751+0000","last_change":"2023-11-06T01:32:26.114243+0000","last_active":"2023-11-06T01:32:27.222751+0000","last_peered":"2023-11-06T01:32:27.222751+0000","last_clean":"2023-11-06T01:32:27.222751+0000","last_became_active":"2023-11-06T01:32:26.113923+0000","last_became_peered":"2023-11-06T01:32:26.113923+0000","last_unstale":"2023-11-06T01:32:27.222751+0000","last_undegraded":"2023-11-06T01:32:27.222751+0000","last_fullsized":"2023-11-06T01:32:27.222751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:48.197329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221817+0000","last_change":"2023-11-06T01:32:26.113819+0000","last_active":"2023-11-06T01:32:27.221817+0000","last_peered":"2023-11-06T01:32:27.221817+0000","last_clean":"2023-11-06T01:32:27.221817+0000","last_became_active":"2023-11-06T01:32:26.113506+0000","last_became_peered":"2023-11-06T01:32:26.113506+0000","last_unstale":"2023-11-06T01:32:27.221817+0000","last_undegraded":"2023-11-06T01:32:27.221817+0000","last_fullsized":"2023-11-06T01:32:27.221817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:33.122100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221622+0000","last_change":"2023-11-06T01:32:26.116504+0000","last_active":"2023-11-06T01:32:27.221622+0000","last_peered":"2023-11-06T01:32:27.221622+0000","last_clean":"2023-11-06T01:32:27.221622+0000","last_became_active":"2023-11-06T01:32:26.115220+0000","last_became_peered":"2023-11-06T01:32:26.115220+0000","last_unstale":"2023-11-06T01:32:27.221622+0000","last_undegraded":"2023-11-06T01:32:27.221622+0000","last_fullsized":"2023-11-06T01:32:27.221622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:34.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.100025+0000","last_change":"2023-11-06T01:32:26.111596+0000","last_active":"2023-11-06T01:32:27.100025+0000","last_peered":"2023-11-06T01:32:27.100025+0000","last_clean":"2023-11-06T01:32:27.100025+0000","last_became_active":"2023-11-06T01:32:26.111309+0000","last_became_peered":"2023-11-06T01:32:26.111309+0000","last_unstale":"2023-11-06T01:32:27.100025+0000","last_undegraded":"2023-11-06T01:32:27.100025+0000","last_fullsized":"2023-11-06T01:32:27.100025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:34.917068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.098303+0000","last_change":"2023-11-06T01:32:26.114536+0000","last_active":"2023-11-06T01:32:27.098303+0000","last_peered":"2023-11-06T01:32:27.098303+0000","last_clean":"2023-11-06T01:32:27.098303+0000","last_became_active":"2023-11-06T01:32:26.114124+0000","last_became_peered":"2023-11-06T01:32:26.114124+0000","last_unstale":"2023-11-06T01:32:27.098303+0000","last_undegraded":"2023-11-06T01:32:27.098303+0000","last_fullsized":"2023-11-06T01:32:27.098303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:05:10.855240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231958+0000","last_change":"2023-11-06T01:32:26.111139+0000","last_active":"2023-11-06T01:32:27.231958+0000","last_peered":"2023-11-06T01:32:27.231958+0000","last_clean":"2023-11-06T01:32:27.231958+0000","last_became_active":"2023-11-06T01:32:26.110764+0000","last_became_peered":"2023-11-06T01:32:26.110764+0000","last_unstale":"2023-11-06T01:32:27.231958+0000","last_undegraded":"2023-11-06T01:32:27.231958+0000","last_fullsized":"2023-11-06T01:32:27.231958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:46.422701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231938+0000","last_change":"2023-11-06T01:32:26.116096+0000","last_active":"2023-11-06T01:32:27.231938+0000","last_peered":"2023-11-06T01:32:27.231938+0000","last_clean":"2023-11-06T01:32:27.231938+0000","last_became_active":"2023-11-06T01:32:26.115754+0000","last_became_peered":"2023-11-06T01:32:26.115754+0000","last_unstale":"2023-11-06T01:32:27.231938+0000","last_undegraded":"2023-11-06T01:32:27.231938+0000","last_fullsized":"2023-11-06T01:32:27.231938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:15:58.109435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.220995+0000","last_change":"2023-11-06T01:32:26.118169+0000","last_active":"2023-11-06T01:32:27.220995+0000","last_peered":"2023-11-06T01:32:27.220995+0000","last_clean":"2023-11-06T01:32:27.220995+0000","last_became_active":"2023-11-06T01:32:26.117837+0000","last_became_peered":"2023-11-06T01:32:26.117837+0000","last_unstale":"2023-11-06T01:32:27.220995+0000","last_undegraded":"2023-11-06T01:32:27.220995+0000","last_fullsized":"2023-11-06T01:32:27.220995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:46.838927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408530+0000","last_change":"2023-11-06T01:32:26.110020+0000","last_active":"2023-11-06T01:32:26.408530+0000","last_peered":"2023-11-06T01:32:26.408530+0000","last_clean":"2023-11-06T01:32:26.408530+0000","last_became_active":"2023-11-06T01:32:26.109707+0000","last_became_peered":"2023-11-06T01:32:26.109707+0000","last_unstale":"2023-11-06T01:32:26.408530+0000","last_undegraded":"2023-11-06T01:32:26.408530+0000","last_fullsized":"2023-11-06T01:32:26.408530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:00.016258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222625+0000","last_change":"2023-11-06T01:32:26.119949+0000","last_active":"2023-11-06T01:32:27.222625+0000","last_peered":"2023-11-06T01:32:27.222625+0000","last_clean":"2023-11-06T01:32:27.222625+0000","last_became_active":"2023-11-06T01:32:26.117594+0000","last_became_peered":"2023-11-06T01:32:26.117594+0000","last_unstale":"2023-11-06T01:32:27.222625+0000","last_undegraded":"2023-11-06T01:32:27.222625+0000","last_fullsized":"2023-11-06T01:32:27.222625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:17:22.757846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221583+0000","last_change":"2023-11-06T01:32:26.123176+0000","last_active":"2023-11-06T01:32:27.221583+0000","last_peered":"2023-11-06T01:32:27.221583+0000","last_clean":"2023-11-06T01:32:27.221583+0000","last_became_active":"2023-11-06T01:32:26.116687+0000","last_became_peered":"2023-11-06T01:32:26.116687+0000","last_unstale":"2023-11-06T01:32:27.221583+0000","last_undegraded":"2023-11-06T01:32:27.221583+0000","last_fullsized":"2023-11-06T01:32:27.221583+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:30.422426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:11.076938+0000","last_change":"2023-11-06T01:32:25.682991+0000","last_active":"2023-11-06T01:33:11.076938+0000","last_peered":"2023-11-06T01:33:11.076938+0000","last_clean":"2023-11-06T01:33:11.076938+0000","last_became_active":"2023-11-06T01:32:25.682687+0000","last_became_peered":"2023-11-06T01:32:25.682687+0000","last_unstale":"2023-11-06T01:33:11.076938+0000","last_undegraded":"2023-11-06T01:33:11.076938+0000","last_fullsized":"2023-11-06T01:33:11.076938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:12:54.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221983+0000","last_change":"2023-11-06T01:32:26.126350+0000","last_active":"2023-11-06T01:32:27.221983+0000","last_peered":"2023-11-06T01:32:27.221983+0000","last_clean":"2023-11-06T01:32:27.221983+0000","last_became_active":"2023-11-06T01:32:26.124703+0000","last_became_peered":"2023-11-06T01:32:26.124703+0000","last_unstale":"2023-11-06T01:32:27.221983+0000","last_undegraded":"2023-11-06T01:32:27.221983+0000","last_fullsized":"2023-11-06T01:32:27.221983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:08:25.866045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222007+0000","last_change":"2023-11-06T01:32:25.680184+0000","last_active":"2023-11-06T01:32:27.222007+0000","last_peered":"2023-11-06T01:32:27.222007+0000","last_clean":"2023-11-06T01:32:27.222007+0000","last_became_active":"2023-11-06T01:32:25.679859+0000","last_became_peered":"2023-11-06T01:32:25.679859+0000","last_unstale":"2023-11-06T01:32:27.222007+0000","last_undegraded":"2023-11-06T01:32:27.222007+0000","last_fullsized":"2023-11-06T01:32:27.222007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:04:34.153562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.232017+0000","last_change":"2023-11-06T01:32:26.114511+0000","last_active":"2023-11-06T01:32:27.232017+0000","last_peered":"2023-11-06T01:32:27.232017+0000","last_clean":"2023-11-06T01:32:27.232017+0000","last_became_active":"2023-11-06T01:32:26.114182+0000","last_became_peered":"2023-11-06T01:32:26.114182+0000","last_unstale":"2023-11-06T01:32:27.232017+0000","last_undegraded":"2023-11-06T01:32:27.232017+0000","last_fullsized":"2023-11-06T01:32:27.232017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:49:10.694358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:33.017084+0000","last_change":"2023-11-06T01:32:25.681453+0000","last_active":"2023-11-06T01:32:33.017084+0000","last_peered":"2023-11-06T01:32:33.017084+0000","last_clean":"2023-11-06T01:32:33.017084+0000","last_became_active":"2023-11-06T01:32:25.681180+0000","last_became_peered":"2023-11-06T01:32:25.681180+0000","last_unstale":"2023-11-06T01:32:33.017084+0000","last_undegraded":"2023-11-06T01:32:33.017084+0000","last_fullsized":"2023-11-06T01:32:33.017084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:39:19.738479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222542+0000","last_change":"2023-11-06T01:32:26.118748+0000","last_active":"2023-11-06T01:32:27.222542+0000","last_peered":"2023-11-06T01:32:27.222542+0000","last_clean":"2023-11-06T01:32:27.222542+0000","last_became_active":"2023-11-06T01:32:26.118449+0000","last_became_peered":"2023-11-06T01:32:26.118449+0000","last_unstale":"2023-11-06T01:32:27.222542+0000","last_undegraded":"2023-11-06T01:32:27.222542+0000","last_fullsized":"2023-11-06T01:32:27.222542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:28.134553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.012440+0000","last_change":"2023-11-06T01:32:25.680967+0000","last_active":"2023-11-06T01:32:32.012440+0000","last_peered":"2023-11-06T01:32:32.012440+0000","last_clean":"2023-11-06T01:32:32.012440+0000","last_became_active":"2023-11-06T01:32:25.678829+0000","last_became_peered":"2023-11-06T01:32:25.678829+0000","last_unstale":"2023-11-06T01:32:32.012440+0000","last_undegraded":"2023-11-06T01:32:32.012440+0000","last_fullsized":"2023-11-06T01:32:32.012440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:40:07.691852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121396+0000","last_change":"2023-11-06T01:32:26.122692+0000","last_active":"2023-11-06T01:32:27.121396+0000","last_peered":"2023-11-06T01:32:27.121396+0000","last_clean":"2023-11-06T01:32:27.121396+0000","last_became_active":"2023-11-06T01:32:26.121960+0000","last_became_peered":"2023-11-06T01:32:26.121960+0000","last_unstale":"2023-11-06T01:32:27.121396+0000","last_undegraded":"2023-11-06T01:32:27.121396+0000","last_fullsized":"2023-11-06T01:32:27.121396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:48.010467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:13.062640+0000","last_change":"2023-11-06T01:32:25.680404+0000","last_active":"2023-11-06T01:33:13.062640+0000","last_peered":"2023-11-06T01:33:13.062640+0000","last_clean":"2023-11-06T01:33:13.062640+0000","last_became_active":"2023-11-06T01:32:25.680060+0000","last_became_peered":"2023-11-06T01:32:25.680060+0000","last_unstale":"2023-11-06T01:33:13.062640+0000","last_undegraded":"2023-11-06T01:33:13.062640+0000","last_fullsized":"2023-11-06T01:33:13.062640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:55:00.476395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.119288+0000","last_change":"2023-11-06T01:32:26.116839+0000","last_active":"2023-11-06T01:32:26.119288+0000","last_peered":"2023-11-06T01:32:26.119288+0000","last_clean":"2023-11-06T01:32:26.119288+0000","last_became_active":"2023-11-06T01:32:26.116535+0000","last_became_peered":"2023-11-06T01:32:26.116535+0000","last_unstale":"2023-11-06T01:32:26.119288+0000","last_undegraded":"2023-11-06T01:32:26.119288+0000","last_fullsized":"2023-11-06T01:32:26.119288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:11.219626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.005598+0000","last_change":"2023-11-06T01:32:25.678426+0000","last_active":"2023-11-06T01:32:31.005598+0000","last_peered":"2023-11-06T01:32:31.005598+0000","last_clean":"2023-11-06T01:32:31.005598+0000","last_became_active":"2023-11-06T01:32:25.678072+0000","last_became_peered":"2023-11-06T01:32:25.678072+0000","last_unstale":"2023-11-06T01:32:31.005598+0000","last_undegraded":"2023-11-06T01:32:31.005598+0000","last_fullsized":"2023-11-06T01:32:31.005598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:09:41.630098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121277+0000","last_change":"2023-11-06T01:32:26.119184+0000","last_active":"2023-11-06T01:32:27.121277+0000","last_peered":"2023-11-06T01:32:27.121277+0000","last_clean":"2023-11-06T01:32:27.121277+0000","last_became_active":"2023-11-06T01:32:26.118896+0000","last_became_peered":"2023-11-06T01:32:26.118896+0000","last_unstale":"2023-11-06T01:32:27.121277+0000","last_undegraded":"2023-11-06T01:32:27.121277+0000","last_fullsized":"2023-11-06T01:32:27.121277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:09:17.351315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.456790+0000","last_change":"2023-11-06T01:32:25.685621+0000","last_active":"2023-11-06T01:32:30.456790+0000","last_peered":"2023-11-06T01:32:30.456790+0000","last_clean":"2023-11-06T01:32:30.456790+0000","last_became_active":"2023-11-06T01:32:25.684965+0000","last_became_peered":"2023-11-06T01:32:25.684965+0000","last_unstale":"2023-11-06T01:32:30.456790+0000","last_undegraded":"2023-11-06T01:32:30.456790+0000","last_fullsized":"2023-11-06T01:32:30.456790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:28:00.922680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221174+0000","last_change":"2023-11-06T01:32:26.122750+0000","last_active":"2023-11-06T01:32:27.221174+0000","last_peered":"2023-11-06T01:32:27.221174+0000","last_clean":"2023-11-06T01:32:27.221174+0000","last_became_active":"2023-11-06T01:32:26.122241+0000","last_became_peered":"2023-11-06T01:32:26.122241+0000","last_unstale":"2023-11-06T01:32:27.221174+0000","last_undegraded":"2023-11-06T01:32:27.221174+0000","last_fullsized":"2023-11-06T01:32:27.221174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:04.402197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.017051+0000","last_change":"2023-11-06T01:32:25.681594+0000","last_active":"2023-11-06T01:32:32.017051+0000","last_peered":"2023-11-06T01:32:32.017051+0000","last_clean":"2023-11-06T01:32:32.017051+0000","last_became_active":"2023-11-06T01:32:25.681303+0000","last_became_peered":"2023-11-06T01:32:25.681303+0000","last_unstale":"2023-11-06T01:32:32.017051+0000","last_undegraded":"2023-11-06T01:32:32.017051+0000","last_fullsized":"2023-11-06T01:32:32.017051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:33.141440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221062+0000","last_change":"2023-11-06T01:32:26.117234+0000","last_active":"2023-11-06T01:32:27.221062+0000","last_peered":"2023-11-06T01:32:27.221062+0000","last_clean":"2023-11-06T01:32:27.221062+0000","last_became_active":"2023-11-06T01:32:26.116915+0000","last_became_peered":"2023-11-06T01:32:26.116915+0000","last_unstale":"2023-11-06T01:32:27.221062+0000","last_undegraded":"2023-11-06T01:32:27.221062+0000","last_fullsized":"2023-11-06T01:32:27.221062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:05:27.521552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":102,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:14.063501+0000","last_change":"2023-11-06T01:32:25.682827+0000","last_active":"2023-11-06T01:33:14.063501+0000","last_peered":"2023-11-06T01:33:14.063501+0000","last_clean":"2023-11-06T01:33:14.063501+0000","last_became_active":"2023-11-06T01:32:25.682470+0000","last_became_peered":"2023-11-06T01:32:25.682470+0000","last_unstale":"2023-11-06T01:33:14.063501+0000","last_undegraded":"2023-11-06T01:33:14.063501+0000","last_fullsized":"2023-11-06T01:33:14.063501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:43.632070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222801+0000","last_change":"2023-11-06T01:32:26.123918+0000","last_active":"2023-11-06T01:32:27.222801+0000","last_peered":"2023-11-06T01:32:27.222801+0000","last_clean":"2023-11-06T01:32:27.222801+0000","last_became_active":"2023-11-06T01:32:26.123450+0000","last_became_peered":"2023-11-06T01:32:26.123450+0000","last_unstale":"2023-11-06T01:32:27.222801+0000","last_undegraded":"2023-11-06T01:32:27.222801+0000","last_fullsized":"2023-11-06T01:32:27.222801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:38.227311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.013421+0000","last_change":"2023-11-06T01:32:25.680051+0000","last_active":"2023-11-06T01:32:32.013421+0000","last_peered":"2023-11-06T01:32:32.013421+0000","last_clean":"2023-11-06T01:32:32.013421+0000","last_became_active":"2023-11-06T01:32:25.679705+0000","last_became_peered":"2023-11-06T01:32:25.679705+0000","last_unstale":"2023-11-06T01:32:32.013421+0000","last_undegraded":"2023-11-06T01:32:32.013421+0000","last_fullsized":"2023-11-06T01:32:32.013421+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:18:22.620036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.232007+0000","last_change":"2023-11-06T01:32:26.115101+0000","last_active":"2023-11-06T01:32:27.232007+0000","last_peered":"2023-11-06T01:32:27.232007+0000","last_clean":"2023-11-06T01:32:27.232007+0000","last_became_active":"2023-11-06T01:32:26.114480+0000","last_became_peered":"2023-11-06T01:32:26.114480+0000","last_unstale":"2023-11-06T01:32:27.232007+0000","last_undegraded":"2023-11-06T01:32:27.232007+0000","last_fullsized":"2023-11-06T01:32:27.232007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:17.770085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.998972+0000","last_change":"2023-11-06T01:32:25.678689+0000","last_active":"2023-11-06T01:32:30.998972+0000","last_peered":"2023-11-06T01:32:30.998972+0000","last_clean":"2023-11-06T01:32:30.998972+0000","last_became_active":"2023-11-06T01:32:25.678355+0000","last_became_peered":"2023-11-06T01:32:25.678355+0000","last_unstale":"2023-11-06T01:32:30.998972+0000","last_undegraded":"2023-11-06T01:32:30.998972+0000","last_fullsized":"2023-11-06T01:32:30.998972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:23.007596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222655+0000","last_change":"2023-11-06T01:32:26.118120+0000","last_active":"2023-11-06T01:32:27.222655+0000","last_peered":"2023-11-06T01:32:27.222655+0000","last_clean":"2023-11-06T01:32:27.222655+0000","last_became_active":"2023-11-06T01:32:26.117762+0000","last_became_peered":"2023-11-06T01:32:26.117762+0000","last_unstale":"2023-11-06T01:32:27.222655+0000","last_undegraded":"2023-11-06T01:32:27.222655+0000","last_fullsized":"2023-11-06T01:32:27.222655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:28:42.417463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'27","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:14.063460+0000","last_change":"2023-11-06T01:32:25.676036+0000","last_active":"2023-11-06T01:33:14.063460+0000","last_peered":"2023-11-06T01:33:14.063460+0000","last_clean":"2023-11-06T01:33:14.063460+0000","last_became_active":"2023-11-06T01:32:25.675672+0000","last_became_peered":"2023-11-06T01:32:25.675672+0000","last_unstale":"2023-11-06T01:33:14.063460+0000","last_undegraded":"2023-11-06T01:33:14.063460+0000","last_fullsized":"2023-11-06T01:33:14.063460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:30:54.060926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54039,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":99,"num_write":29,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.120038+0000","last_change":"2023-11-06T01:32:26.118832+0000","last_active":"2023-11-06T01:32:26.120038+0000","last_peered":"2023-11-06T01:32:26.120038+0000","last_clean":"2023-11-06T01:32:26.120038+0000","last_became_active":"2023-11-06T01:32:26.118551+0000","last_became_peered":"2023-11-06T01:32:26.118551+0000","last_unstale":"2023-11-06T01:32:26.120038+0000","last_undegraded":"2023-11-06T01:32:26.120038+0000","last_fullsized":"2023-11-06T01:32:26.120038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:23.031203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.013630+0000","last_change":"2023-11-06T01:32:25.679433+0000","last_active":"2023-11-06T01:32:32.013630+0000","last_peered":"2023-11-06T01:32:32.013630+0000","last_clean":"2023-11-06T01:32:32.013630+0000","last_became_active":"2023-11-06T01:32:25.679090+0000","last_became_peered":"2023-11-06T01:32:25.679090+0000","last_unstale":"2023-11-06T01:32:32.013630+0000","last_undegraded":"2023-11-06T01:32:32.013630+0000","last_fullsized":"2023-11-06T01:32:32.013630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:14:57.795474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097850+0000","last_change":"2023-11-06T01:32:26.113421+0000","last_active":"2023-11-06T01:32:27.097850+0000","last_peered":"2023-11-06T01:32:27.097850+0000","last_clean":"2023-11-06T01:32:27.097850+0000","last_became_active":"2023-11-06T01:32:26.113085+0000","last_became_peered":"2023-11-06T01:32:26.113085+0000","last_unstale":"2023-11-06T01:32:27.097850+0000","last_undegraded":"2023-11-06T01:32:27.097850+0000","last_fullsized":"2023-11-06T01:32:27.097850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:15:04.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.016512+0000","last_change":"2023-11-06T01:32:25.680293+0000","last_active":"2023-11-06T01:32:32.016512+0000","last_peered":"2023-11-06T01:32:32.016512+0000","last_clean":"2023-11-06T01:32:32.016512+0000","last_became_active":"2023-11-06T01:32:25.679803+0000","last_became_peered":"2023-11-06T01:32:25.679803+0000","last_unstale":"2023-11-06T01:32:32.016512+0000","last_undegraded":"2023-11-06T01:32:32.016512+0000","last_fullsized":"2023-11-06T01:32:32.016512+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:28.562797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221634+0000","last_change":"2023-11-06T01:32:26.115996+0000","last_active":"2023-11-06T01:32:27.221634+0000","last_peered":"2023-11-06T01:32:27.221634+0000","last_clean":"2023-11-06T01:32:27.221634+0000","last_became_active":"2023-11-06T01:32:26.115407+0000","last_became_peered":"2023-11-06T01:32:26.115407+0000","last_unstale":"2023-11-06T01:32:27.221634+0000","last_undegraded":"2023-11-06T01:32:27.221634+0000","last_fullsized":"2023-11-06T01:32:27.221634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:08.069698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:11.059592+0000","last_change":"2023-11-06T01:32:25.680958+0000","last_active":"2023-11-06T01:33:11.059592+0000","last_peered":"2023-11-06T01:33:11.059592+0000","last_clean":"2023-11-06T01:33:11.059592+0000","last_became_active":"2023-11-06T01:32:25.680616+0000","last_became_peered":"2023-11-06T01:32:25.680616+0000","last_unstale":"2023-11-06T01:33:11.059592+0000","last_undegraded":"2023-11-06T01:33:11.059592+0000","last_fullsized":"2023-11-06T01:33:11.059592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:02.631755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221186+0000","last_change":"2023-11-06T01:32:26.115464+0000","last_active":"2023-11-06T01:32:27.221186+0000","last_peered":"2023-11-06T01:32:27.221186+0000","last_clean":"2023-11-06T01:32:27.221186+0000","last_became_active":"2023-11-06T01:32:26.115090+0000","last_became_peered":"2023-11-06T01:32:26.115090+0000","last_unstale":"2023-11-06T01:32:27.221186+0000","last_undegraded":"2023-11-06T01:32:27.221186+0000","last_fullsized":"2023-11-06T01:32:27.221186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:08.018044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.004305+0000","last_change":"2023-11-06T01:32:25.677787+0000","last_active":"2023-11-06T01:32:31.004305+0000","last_peered":"2023-11-06T01:32:31.004305+0000","last_clean":"2023-11-06T01:32:31.004305+0000","last_became_active":"2023-11-06T01:32:25.677527+0000","last_became_peered":"2023-11-06T01:32:25.677527+0000","last_unstale":"2023-11-06T01:32:31.004305+0000","last_undegraded":"2023-11-06T01:32:31.004305+0000","last_fullsized":"2023-11-06T01:32:31.004305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:34:09.452965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121118+0000","last_change":"2023-11-06T01:32:26.117792+0000","last_active":"2023-11-06T01:32:27.121118+0000","last_peered":"2023-11-06T01:32:27.121118+0000","last_clean":"2023-11-06T01:32:27.121118+0000","last_became_active":"2023-11-06T01:32:26.117469+0000","last_became_peered":"2023-11-06T01:32:26.117469+0000","last_unstale":"2023-11-06T01:32:27.121118+0000","last_undegraded":"2023-11-06T01:32:27.121118+0000","last_fullsized":"2023-11-06T01:32:27.121118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:44:54.409476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.993990+0000","last_change":"2023-11-06T01:32:25.677257+0000","last_active":"2023-11-06T01:32:30.993990+0000","last_peered":"2023-11-06T01:32:30.993990+0000","last_clean":"2023-11-06T01:32:30.993990+0000","last_became_active":"2023-11-06T01:32:25.676536+0000","last_became_peered":"2023-11-06T01:32:25.676536+0000","last_unstale":"2023-11-06T01:32:30.993990+0000","last_undegraded":"2023-11-06T01:32:30.993990+0000","last_fullsized":"2023-11-06T01:32:30.993990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:53:33.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408512+0000","last_change":"2023-11-06T01:32:26.111447+0000","last_active":"2023-11-06T01:32:26.408512+0000","last_peered":"2023-11-06T01:32:26.408512+0000","last_clean":"2023-11-06T01:32:26.408512+0000","last_became_active":"2023-11-06T01:32:26.111132+0000","last_became_peered":"2023-11-06T01:32:26.111132+0000","last_unstale":"2023-11-06T01:32:26.408512+0000","last_undegraded":"2023-11-06T01:32:26.408512+0000","last_fullsized":"2023-11-06T01:32:26.408512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:55.385003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.717374+0000","last_change":"2023-11-06T01:32:25.682562+0000","last_active":"2023-11-06T01:32:27.717374+0000","last_peered":"2023-11-06T01:32:27.717374+0000","last_clean":"2023-11-06T01:32:27.717374+0000","last_became_active":"2023-11-06T01:32:25.682227+0000","last_became_peered":"2023-11-06T01:32:25.682227+0000","last_unstale":"2023-11-06T01:32:27.717374+0000","last_undegraded":"2023-11-06T01:32:27.717374+0000","last_fullsized":"2023-11-06T01:32:27.717374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:40.113429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.990952+0000","last_change":"2023-11-06T01:32:26.110589+0000","last_active":"2023-11-06T01:32:29.990952+0000","last_peered":"2023-11-06T01:32:29.990952+0000","last_clean":"2023-11-06T01:32:29.990952+0000","last_became_active":"2023-11-06T01:32:26.107413+0000","last_became_peered":"2023-11-06T01:32:26.107413+0000","last_unstale":"2023-11-06T01:32:29.990952+0000","last_undegraded":"2023-11-06T01:32:29.990952+0000","last_fullsized":"2023-11-06T01:32:29.990952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:39:08.613713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:13.079735+0000","last_change":"2023-11-06T01:32:25.688176+0000","last_active":"2023-11-06T01:33:13.079735+0000","last_peered":"2023-11-06T01:33:13.079735+0000","last_clean":"2023-11-06T01:33:13.079735+0000","last_became_active":"2023-11-06T01:32:25.687745+0000","last_became_peered":"2023-11-06T01:32:25.687745+0000","last_unstale":"2023-11-06T01:33:13.079735+0000","last_undegraded":"2023-11-06T01:33:13.079735+0000","last_fullsized":"2023-11-06T01:33:13.079735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:21:23.566389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221764+0000","last_change":"2023-11-06T01:32:26.111700+0000","last_active":"2023-11-06T01:32:27.221764+0000","last_peered":"2023-11-06T01:32:27.221764+0000","last_clean":"2023-11-06T01:32:27.221764+0000","last_became_active":"2023-11-06T01:32:26.111380+0000","last_became_peered":"2023-11-06T01:32:26.111380+0000","last_unstale":"2023-11-06T01:32:27.221764+0000","last_undegraded":"2023-11-06T01:32:27.221764+0000","last_fullsized":"2023-11-06T01:32:27.221764+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:56:47.874663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121155+0000","last_change":"2023-11-06T01:32:26.121906+0000","last_active":"2023-11-06T01:32:27.121155+0000","last_peered":"2023-11-06T01:32:27.121155+0000","last_clean":"2023-11-06T01:32:27.121155+0000","last_became_active":"2023-11-06T01:32:26.121292+0000","last_became_peered":"2023-11-06T01:32:26.121292+0000","last_unstale":"2023-11-06T01:32:27.121155+0000","last_undegraded":"2023-11-06T01:32:27.121155+0000","last_fullsized":"2023-11-06T01:32:27.121155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.532098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:12.060259+0000","last_change":"2023-11-06T01:32:25.685814+0000","last_active":"2023-11-06T01:33:12.060259+0000","last_peered":"2023-11-06T01:33:12.060259+0000","last_clean":"2023-11-06T01:33:12.060259+0000","last_became_active":"2023-11-06T01:32:25.684890+0000","last_became_peered":"2023-11-06T01:32:25.684890+0000","last_unstale":"2023-11-06T01:33:12.060259+0000","last_undegraded":"2023-11-06T01:33:12.060259+0000","last_fullsized":"2023-11-06T01:33:12.060259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:21.950945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222017+0000","last_change":"2023-11-06T01:32:26.121752+0000","last_active":"2023-11-06T01:32:27.222017+0000","last_peered":"2023-11-06T01:32:27.222017+0000","last_clean":"2023-11-06T01:32:27.222017+0000","last_became_active":"2023-11-06T01:32:26.121447+0000","last_became_peered":"2023-11-06T01:32:26.121447+0000","last_unstale":"2023-11-06T01:32:27.222017+0000","last_undegraded":"2023-11-06T01:32:27.222017+0000","last_fullsized":"2023-11-06T01:32:27.222017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:51:19.727335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.017046+0000","last_change":"2023-11-06T01:32:25.682727+0000","last_active":"2023-11-06T01:32:32.017046+0000","last_peered":"2023-11-06T01:32:32.017046+0000","last_clean":"2023-11-06T01:32:32.017046+0000","last_became_active":"2023-11-06T01:32:25.682377+0000","last_became_peered":"2023-11-06T01:32:25.682377+0000","last_unstale":"2023-11-06T01:32:32.017046+0000","last_undegraded":"2023-11-06T01:32:32.017046+0000","last_fullsized":"2023-11-06T01:32:32.017046+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:17:38.614350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":325,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.100201+0000","last_change":"2023-11-06T01:31:27.931076+0000","last_active":"2023-11-06T01:32:26.100201+0000","last_peered":"2023-11-06T01:32:26.100201+0000","last_clean":"2023-11-06T01:32:26.100201+0000","last_became_active":"2023-11-06T01:31:27.930669+0000","last_became_peered":"2023-11-06T01:31:27.930669+0000","last_unstale":"2023-11-06T01:32:26.100201+0000","last_undegraded":"2023-11-06T01:32:26.100201+0000","last_fullsized":"2023-11-06T01:32:26.100201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_clean_scrub_stamp":"2023-11-06T01:28:36.252549+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:47:48.729325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097629+0000","last_change":"2023-11-06T01:32:26.110055+0000","last_active":"2023-11-06T01:32:27.097629+0000","last_peered":"2023-11-06T01:32:27.097629+0000","last_clean":"2023-11-06T01:32:27.097629+0000","last_became_active":"2023-11-06T01:32:26.109640+0000","last_became_peered":"2023-11-06T01:32:26.109640+0000","last_unstale":"2023-11-06T01:32:27.097629+0000","last_undegraded":"2023-11-06T01:32:27.097629+0000","last_fullsized":"2023-11-06T01:32:27.097629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:45:43.076910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:30.455308+0000","last_change":"2023-11-06T01:32:25.679885+0000","last_active":"2023-11-06T01:32:30.455308+0000","last_peered":"2023-11-06T01:32:30.455308+0000","last_clean":"2023-11-06T01:32:30.455308+0000","last_became_active":"2023-11-06T01:32:25.679518+0000","last_became_peered":"2023-11-06T01:32:25.679518+0000","last_unstale":"2023-11-06T01:32:30.455308+0000","last_undegraded":"2023-11-06T01:32:30.455308+0000","last_fullsized":"2023-11-06T01:32:30.455308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:44.558534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.005421+0000","last_change":"2023-11-06T01:32:25.682016+0000","last_active":"2023-11-06T01:32:31.005421+0000","last_peered":"2023-11-06T01:32:31.005421+0000","last_clean":"2023-11-06T01:32:31.005421+0000","last_became_active":"2023-11-06T01:32:25.681753+0000","last_became_peered":"2023-11-06T01:32:25.681753+0000","last_unstale":"2023-11-06T01:32:31.005421+0000","last_undegraded":"2023-11-06T01:32:31.005421+0000","last_fullsized":"2023-11-06T01:32:31.005421+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:08.575728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097816+0000","last_change":"2023-11-06T01:32:26.117109+0000","last_active":"2023-11-06T01:32:27.097816+0000","last_peered":"2023-11-06T01:32:27.097816+0000","last_clean":"2023-11-06T01:32:27.097816+0000","last_became_active":"2023-11-06T01:32:26.116769+0000","last_became_peered":"2023-11-06T01:32:26.116769+0000","last_unstale":"2023-11-06T01:32:27.097816+0000","last_undegraded":"2023-11-06T01:32:27.097816+0000","last_fullsized":"2023-11-06T01:32:27.097816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:48:04.188806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:34.453895+0000","last_change":"2023-11-06T01:32:25.681013+0000","last_active":"2023-11-06T01:32:34.453895+0000","last_peered":"2023-11-06T01:32:34.453895+0000","last_clean":"2023-11-06T01:32:34.453895+0000","last_became_active":"2023-11-06T01:32:25.680699+0000","last_became_peered":"2023-11-06T01:32:25.680699+0000","last_unstale":"2023-11-06T01:32:34.453895+0000","last_undegraded":"2023-11-06T01:32:34.453895+0000","last_fullsized":"2023-11-06T01:32:34.453895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:31.755527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221905+0000","last_change":"2023-11-06T01:32:26.116351+0000","last_active":"2023-11-06T01:32:27.221905+0000","last_peered":"2023-11-06T01:32:27.221905+0000","last_clean":"2023-11-06T01:32:27.221905+0000","last_became_active":"2023-11-06T01:32:26.116044+0000","last_became_peered":"2023-11-06T01:32:26.116044+0000","last_unstale":"2023-11-06T01:32:27.221905+0000","last_undegraded":"2023-11-06T01:32:27.221905+0000","last_fullsized":"2023-11-06T01:32:27.221905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:20.816412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.016794+0000","last_change":"2023-11-06T01:32:25.676764+0000","last_active":"2023-11-06T01:32:32.016794+0000","last_peered":"2023-11-06T01:32:32.016794+0000","last_clean":"2023-11-06T01:32:32.016794+0000","last_became_active":"2023-11-06T01:32:25.676418+0000","last_became_peered":"2023-11-06T01:32:25.676418+0000","last_unstale":"2023-11-06T01:32:32.016794+0000","last_undegraded":"2023-11-06T01:32:32.016794+0000","last_fullsized":"2023-11-06T01:32:32.016794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:49.320532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231997+0000","last_change":"2023-11-06T01:32:26.121107+0000","last_active":"2023-11-06T01:32:27.231997+0000","last_peered":"2023-11-06T01:32:27.231997+0000","last_clean":"2023-11-06T01:32:27.231997+0000","last_became_active":"2023-11-06T01:32:26.120790+0000","last_became_peered":"2023-11-06T01:32:26.120790+0000","last_unstale":"2023-11-06T01:32:27.231997+0000","last_undegraded":"2023-11-06T01:32:27.231997+0000","last_fullsized":"2023-11-06T01:32:27.231997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:34:46.660301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.447727+0000","last_change":"2023-11-06T01:32:25.678959+0000","last_active":"2023-11-06T01:32:29.447727+0000","last_peered":"2023-11-06T01:32:29.447727+0000","last_clean":"2023-11-06T01:32:29.447727+0000","last_became_active":"2023-11-06T01:32:25.678564+0000","last_became_peered":"2023-11-06T01:32:25.678564+0000","last_unstale":"2023-11-06T01:32:29.447727+0000","last_undegraded":"2023-11-06T01:32:29.447727+0000","last_fullsized":"2023-11-06T01:32:29.447727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:30:06.304889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.114597+0000","last_change":"2023-11-06T01:32:26.113222+0000","last_active":"2023-11-06T01:32:26.114597+0000","last_peered":"2023-11-06T01:32:26.114597+0000","last_clean":"2023-11-06T01:32:26.114597+0000","last_became_active":"2023-11-06T01:32:26.112926+0000","last_became_peered":"2023-11-06T01:32:26.112926+0000","last_unstale":"2023-11-06T01:32:26.114597+0000","last_undegraded":"2023-11-06T01:32:26.114597+0000","last_fullsized":"2023-11-06T01:32:26.114597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:09:51.042469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.013258+0000","last_change":"2023-11-06T01:32:25.684985+0000","last_active":"2023-11-06T01:32:32.013258+0000","last_peered":"2023-11-06T01:32:32.013258+0000","last_clean":"2023-11-06T01:32:32.013258+0000","last_became_active":"2023-11-06T01:32:25.684656+0000","last_became_peered":"2023-11-06T01:32:25.684656+0000","last_unstale":"2023-11-06T01:32:32.013258+0000","last_undegraded":"2023-11-06T01:32:32.013258+0000","last_fullsized":"2023-11-06T01:32:32.013258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:42:29.406803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408460+0000","last_change":"2023-11-06T01:32:26.107484+0000","last_active":"2023-11-06T01:32:26.408460+0000","last_peered":"2023-11-06T01:32:26.408460+0000","last_clean":"2023-11-06T01:32:26.408460+0000","last_became_active":"2023-11-06T01:32:26.106092+0000","last_became_peered":"2023-11-06T01:32:26.106092+0000","last_unstale":"2023-11-06T01:32:26.408460+0000","last_undegraded":"2023-11-06T01:32:26.408460+0000","last_fullsized":"2023-11-06T01:32:26.408460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:43:39.481290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:34.892606+0000","last_change":"2023-11-06T01:32:25.679531+0000","last_active":"2023-11-06T01:32:34.892606+0000","last_peered":"2023-11-06T01:32:34.892606+0000","last_clean":"2023-11-06T01:32:34.892606+0000","last_became_active":"2023-11-06T01:32:25.679076+0000","last_became_peered":"2023-11-06T01:32:25.679076+0000","last_unstale":"2023-11-06T01:32:34.892606+0000","last_undegraded":"2023-11-06T01:32:34.892606+0000","last_fullsized":"2023-11-06T01:32:34.892606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:37.381547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222136+0000","last_change":"2023-11-06T01:32:26.118584+0000","last_active":"2023-11-06T01:32:27.222136+0000","last_peered":"2023-11-06T01:32:27.222136+0000","last_clean":"2023-11-06T01:32:27.222136+0000","last_became_active":"2023-11-06T01:32:26.118285+0000","last_became_peered":"2023-11-06T01:32:26.118285+0000","last_unstale":"2023-11-06T01:32:27.222136+0000","last_undegraded":"2023-11-06T01:32:27.222136+0000","last_fullsized":"2023-11-06T01:32:27.222136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:52:33.057377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.447858+0000","last_change":"2023-11-06T01:32:25.681214+0000","last_active":"2023-11-06T01:32:29.447858+0000","last_peered":"2023-11-06T01:32:29.447858+0000","last_clean":"2023-11-06T01:32:29.447858+0000","last_became_active":"2023-11-06T01:32:25.680907+0000","last_became_peered":"2023-11-06T01:32:25.680907+0000","last_unstale":"2023-11-06T01:32:29.447858+0000","last_undegraded":"2023-11-06T01:32:29.447858+0000","last_fullsized":"2023-11-06T01:32:29.447858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:04:58.248787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221859+0000","last_change":"2023-11-06T01:32:26.120862+0000","last_active":"2023-11-06T01:32:27.221859+0000","last_peered":"2023-11-06T01:32:27.221859+0000","last_clean":"2023-11-06T01:32:27.221859+0000","last_became_active":"2023-11-06T01:32:26.120307+0000","last_became_peered":"2023-11-06T01:32:26.120307+0000","last_unstale":"2023-11-06T01:32:27.221859+0000","last_undegraded":"2023-11-06T01:32:27.221859+0000","last_fullsized":"2023-11-06T01:32:27.221859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:45:35.611541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:33:13.081350+0000","last_change":"2023-11-06T01:32:25.675607+0000","last_active":"2023-11-06T01:33:13.081350+0000","last_peered":"2023-11-06T01:33:13.081350+0000","last_clean":"2023-11-06T01:33:13.081350+0000","last_became_active":"2023-11-06T01:32:25.675293+0000","last_became_peered":"2023-11-06T01:32:25.675293+0000","last_unstale":"2023-11-06T01:33:13.081350+0000","last_undegraded":"2023-11-06T01:33:13.081350+0000","last_fullsized":"2023-11-06T01:33:13.081350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:05.190844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221522+0000","last_change":"2023-11-06T01:32:26.116987+0000","last_active":"2023-11-06T01:32:27.221522+0000","last_peered":"2023-11-06T01:32:27.221522+0000","last_clean":"2023-11-06T01:32:27.221522+0000","last_became_active":"2023-11-06T01:32:26.116756+0000","last_became_peered":"2023-11-06T01:32:26.116756+0000","last_unstale":"2023-11-06T01:32:27.221522+0000","last_undegraded":"2023-11-06T01:32:27.221522+0000","last_fullsized":"2023-11-06T01:32:27.221522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:44.039520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:31.005114+0000","last_change":"2023-11-06T01:32:25.675483+0000","last_active":"2023-11-06T01:32:31.005114+0000","last_peered":"2023-11-06T01:32:31.005114+0000","last_clean":"2023-11-06T01:32:31.005114+0000","last_became_active":"2023-11-06T01:32:25.675137+0000","last_became_peered":"2023-11-06T01:32:25.675137+0000","last_unstale":"2023-11-06T01:32:31.005114+0000","last_undegraded":"2023-11-06T01:32:31.005114+0000","last_fullsized":"2023-11-06T01:32:31.005114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:42.415165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:29.984742+0000","last_change":"2023-11-06T01:32:26.115575+0000","last_active":"2023-11-06T01:32:29.984742+0000","last_peered":"2023-11-06T01:32:29.984742+0000","last_clean":"2023-11-06T01:32:29.984742+0000","last_became_active":"2023-11-06T01:32:26.115142+0000","last_became_peered":"2023-11-06T01:32:26.115142+0000","last_unstale":"2023-11-06T01:32:29.984742+0000","last_undegraded":"2023-11-06T01:32:29.984742+0000","last_fullsized":"2023-11-06T01:32:29.984742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:52.859340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:32.012610+0000","last_change":"2023-11-06T01:32:25.676972+0000","last_active":"2023-11-06T01:32:32.012610+0000","last_peered":"2023-11-06T01:32:32.012610+0000","last_clean":"2023-11-06T01:32:32.012610+0000","last_became_active":"2023-11-06T01:32:25.676685+0000","last_became_peered":"2023-11-06T01:32:25.676685+0000","last_unstale":"2023-11-06T01:32:32.012610+0000","last_undegraded":"2023-11-06T01:32:32.012610+0000","last_fullsized":"2023-11-06T01:32:32.012610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:30:21.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221207+0000","last_change":"2023-11-06T01:32:26.115164+0000","last_active":"2023-11-06T01:32:27.221207+0000","last_peered":"2023-11-06T01:32:27.221207+0000","last_clean":"2023-11-06T01:32:27.221207+0000","last_became_active":"2023-11-06T01:32:26.114868+0000","last_became_peered":"2023-11-06T01:32:26.114868+0000","last_unstale":"2023-11-06T01:32:27.221207+0000","last_undegraded":"2023-11-06T01:32:27.221207+0000","last_fullsized":"2023-11-06T01:32:27.221207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:01:34.149871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408478+0000","last_change":"2023-11-06T01:32:26.115600+0000","last_active":"2023-11-06T01:32:26.408478+0000","last_peered":"2023-11-06T01:32:26.408478+0000","last_clean":"2023-11-06T01:32:26.408478+0000","last_became_active":"2023-11-06T01:32:26.115290+0000","last_became_peered":"2023-11-06T01:32:26.115290+0000","last_unstale":"2023-11-06T01:32:26.408478+0000","last_undegraded":"2023-11-06T01:32:26.408478+0000","last_fullsized":"2023-11-06T01:32:26.408478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:04:06.393671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221461+0000","last_change":"2023-11-06T01:32:26.115336+0000","last_active":"2023-11-06T01:32:27.221461+0000","last_peered":"2023-11-06T01:32:27.221461+0000","last_clean":"2023-11-06T01:32:27.221461+0000","last_became_active":"2023-11-06T01:32:26.114946+0000","last_became_peered":"2023-11-06T01:32:26.114946+0000","last_unstale":"2023-11-06T01:32:27.221461+0000","last_undegraded":"2023-11-06T01:32:27.221461+0000","last_fullsized":"2023-11-06T01:32:27.221461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:19:13.651494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221633+0000","last_change":"2023-11-06T01:32:26.117794+0000","last_active":"2023-11-06T01:32:27.221633+0000","last_peered":"2023-11-06T01:32:27.221633+0000","last_clean":"2023-11-06T01:32:27.221633+0000","last_became_active":"2023-11-06T01:32:26.117425+0000","last_became_peered":"2023-11-06T01:32:26.117425+0000","last_unstale":"2023-11-06T01:32:27.221633+0000","last_undegraded":"2023-11-06T01:32:27.221633+0000","last_fullsized":"2023-11-06T01:32:27.221633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:00:24.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.231970+0000","last_change":"2023-11-06T01:32:26.113662+0000","last_active":"2023-11-06T01:32:27.231970+0000","last_peered":"2023-11-06T01:32:27.231970+0000","last_clean":"2023-11-06T01:32:27.231970+0000","last_became_active":"2023-11-06T01:32:26.113124+0000","last_became_peered":"2023-11-06T01:32:26.113124+0000","last_unstale":"2023-11-06T01:32:27.231970+0000","last_undegraded":"2023-11-06T01:32:27.231970+0000","last_fullsized":"2023-11-06T01:32:27.231970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:50:24.349521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.097929+0000","last_change":"2023-11-06T01:32:26.109910+0000","last_active":"2023-11-06T01:32:27.097929+0000","last_peered":"2023-11-06T01:32:27.097929+0000","last_clean":"2023-11-06T01:32:27.097929+0000","last_became_active":"2023-11-06T01:32:26.109568+0000","last_became_peered":"2023-11-06T01:32:26.109568+0000","last_unstale":"2023-11-06T01:32:27.097929+0000","last_undegraded":"2023-11-06T01:32:27.097929+0000","last_fullsized":"2023-11-06T01:32:27.097929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:11.049749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408501+0000","last_change":"2023-11-06T01:32:26.114824+0000","last_active":"2023-11-06T01:32:26.408501+0000","last_peered":"2023-11-06T01:32:26.408501+0000","last_clean":"2023-11-06T01:32:26.408501+0000","last_became_active":"2023-11-06T01:32:26.114539+0000","last_became_peered":"2023-11-06T01:32:26.114539+0000","last_unstale":"2023-11-06T01:32:26.408501+0000","last_undegraded":"2023-11-06T01:32:26.408501+0000","last_fullsized":"2023-11-06T01:32:26.408501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:03:02.018769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221808+0000","last_change":"2023-11-06T01:32:26.114743+0000","last_active":"2023-11-06T01:32:27.221808+0000","last_peered":"2023-11-06T01:32:27.221808+0000","last_clean":"2023-11-06T01:32:27.221808+0000","last_became_active":"2023-11-06T01:32:26.114432+0000","last_became_peered":"2023-11-06T01:32:26.114432+0000","last_unstale":"2023-11-06T01:32:27.221808+0000","last_undegraded":"2023-11-06T01:32:27.221808+0000","last_fullsized":"2023-11-06T01:32:27.221808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:27:18.209810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221801+0000","last_change":"2023-11-06T01:32:26.125724+0000","last_active":"2023-11-06T01:32:27.221801+0000","last_peered":"2023-11-06T01:32:27.221801+0000","last_clean":"2023-11-06T01:32:27.221801+0000","last_became_active":"2023-11-06T01:32:26.125455+0000","last_became_peered":"2023-11-06T01:32:26.125455+0000","last_unstale":"2023-11-06T01:32:27.221801+0000","last_undegraded":"2023-11-06T01:32:27.221801+0000","last_fullsized":"2023-11-06T01:32:27.221801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:49.099308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.121127+0000","last_change":"2023-11-06T01:32:26.119328+0000","last_active":"2023-11-06T01:32:27.121127+0000","last_peered":"2023-11-06T01:32:27.121127+0000","last_clean":"2023-11-06T01:32:27.121127+0000","last_became_active":"2023-11-06T01:32:26.119042+0000","last_became_peered":"2023-11-06T01:32:26.119042+0000","last_unstale":"2023-11-06T01:32:27.121127+0000","last_undegraded":"2023-11-06T01:32:27.121127+0000","last_fullsized":"2023-11-06T01:32:27.121127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:36.143788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.221676+0000","last_change":"2023-11-06T01:32:26.126538+0000","last_active":"2023-11-06T01:32:27.221676+0000","last_peered":"2023-11-06T01:32:27.221676+0000","last_clean":"2023-11-06T01:32:27.221676+0000","last_became_active":"2023-11-06T01:32:26.124898+0000","last_became_peered":"2023-11-06T01:32:26.124898+0000","last_unstale":"2023-11-06T01:32:27.221676+0000","last_undegraded":"2023-11-06T01:32:27.221676+0000","last_fullsized":"2023-11-06T01:32:27.221676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:33.019326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.222542+0000","last_change":"2023-11-06T01:32:26.118751+0000","last_active":"2023-11-06T01:32:27.222542+0000","last_peered":"2023-11-06T01:32:27.222542+0000","last_clean":"2023-11-06T01:32:27.222542+0000","last_became_active":"2023-11-06T01:32:26.118359+0000","last_became_peered":"2023-11-06T01:32:26.118359+0000","last_unstale":"2023-11-06T01:32:27.222542+0000","last_undegraded":"2023-11-06T01:32:27.222542+0000","last_fullsized":"2023-11-06T01:32:27.222542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:55.960400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T01:32:27.098195+0000","last_change":"2023-11-06T01:32:26.110133+0000","last_active":"2023-11-06T01:32:27.098195+0000","last_peered":"2023-11-06T01:32:27.098195+0000","last_clean":"2023-11-06T01:32:27.098195+0000","last_became_active":"2023-11-06T01:32:26.109793+0000","last_became_peered":"2023-11-06T01:32:26.109793+0000","last_unstale":"2023-11-06T01:32:27.098195+0000","last_undegraded":"2023-11-06T01:32:27.098195+0000","last_fullsized":"2023-11-06T01:32:27.098195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:00:03.956947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T01:32:26.408491+0000","last_change":"2023-11-06T01:32:26.109828+0000","last_active":"2023-11-06T01:32:26.408491+0000","last_peered":"2023-11-06T01:32:26.408491+0000","last_clean":"2023-11-06T01:32:26.408491+0000","last_became_active":"2023-11-06T01:32:26.109526+0000","last_became_peered":"2023-11-06T01:32:26.109526+0000","last_unstale":"2023-11-06T01:32:26.408491+0000","last_undegraded":"2023-11-06T01:32:26.408491+0000","last_fullsized":"2023-11-06T01:32:26.408491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:36.561908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250075,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60662,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":464,"num_read_kb":299,"num_write":227,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":181986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481813,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":419251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56499999999999995}]},{"osd":1,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.41899999999999998}]},{"osd":2,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.34599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":3.4350000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.60199999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.438}]},{"osd":6,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.503},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.47099999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58899999999999997}]},{"osd":10,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.61799999999999999}]}]},{"osd":10,"up_from":70,"seq":300647710746,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":1063994,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":1,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.54900000000000004}]},{"osd":2,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.873},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.83999999999999997}]},{"osd":3,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.76900000000000002}]},{"osd":4,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.93899999999999995}]},{"osd":9,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.749}]},{"osd":11,"last update":"Mon Nov 6 01:32:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.0009999999999999}]}]},{"osd":9,"up_from":65,"seq":279172874270,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":419229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.49199999999999999,"15min":0.50700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.433},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.9780000000000002,"5min":3.9780000000000002,"15min":3.9780000000000002},"last":0.45900000000000002}]},{"osd":1,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.66900000000000004,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77800000000000002,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":6.6210000000000004,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":0.58899999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.67700000000000005,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.476,"5min":6.476,"15min":6.476},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59799999999999998,"15min":0.58499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":4.0270000000000001,"5min":4.0270000000000001,"15min":4.0270000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82199999999999995,"5min":0.87,"15min":0.87},"last":0.56899999999999995}]},{"osd":4,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.70899999999999996,"15min":0.68000000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.623},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77200000000000002,"15min":0.77100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":4.1589999999999998,"5min":4.1589999999999998,"15min":4.1589999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.79300000000000004,"15min":0.81499999999999995},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72499999999999998}]},{"osd":6,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.80100000000000005,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":6.056,"5min":6.056,"15min":6.056},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.3700000000000001,"5min":6.3700000000000001,"15min":6.3700000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85999999999999999,"15min":0.86199999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.97799999999999998,"5min":1.603,"15min":1.603},"last":0.76400000000000001}]},{"osd":8,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.83899999999999997,"15min":0.81599999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.524,"5min":6.524,"15min":6.524},"last":0.752},{"interface":"front","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.85999999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":4.1900000000000004,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":0.77400000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.69999999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":419276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.627},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57099999999999995,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71299999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59499999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46800000000000003,"15min":0.47599999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66100000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55900000000000005}]},{"osd":4,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.82399999999999995}]},{"osd":5,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.34399999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71499999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.55700000000000005,"15min":0.56499999999999995},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79900000000000004,"5min":1.036,"15min":1.036},"last":0.58399999999999996}]},{"osd":6,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67700000000000005,"15min":0.68600000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83699999999999997,"5min":1,"15min":1},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.873,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.66900000000000004}]},{"osd":7,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.51400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.83699999999999997}]},{"osd":9,"last update":"Mon Nov 6 01:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.873},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.85699999999999998}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.92200000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":1011989,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.873,"5min":1.157,"15min":1.157},"last":0.376}]},{"osd":1,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.32900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.5,"5min":0.45600000000000002,"15min":0.44800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42299999999999999}]},{"osd":2,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.628},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57899999999999996,"15min":0.56799999999999995},"min":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.44700000000000001}]},{"osd":3,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85899999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.66400000000000003}]},{"osd":4,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.51300000000000001,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.495}]},{"osd":5,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.53200000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.68400000000000005}]},{"osd":6,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97099999999999997,"5min":1.232,"15min":1.232},"last":0.84499999999999997}]},{"osd":8,"last update":"Mon Nov 6 01:32:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.76800000000000002,"15min":0.748},"min":{"1min":0.55200000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":0.81299999999999994}]},{"osd":9,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.79400000000000004,"15min":0.78200000000000003},"min":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.77100000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.85999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":418904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.52100000000000002,"15min":0.52600000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.47199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.68899999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.60699999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70899999999999996,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51600000000000001}]},{"osd":3,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.57399999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.376},{"interface":"front","average":{"1min":0.502,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.48199999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.61199999999999999,"15min":0.62},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78800000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86799999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63900000000000001}]},{"osd":8,"last update":"Mon Nov 6 01:32:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.82099999999999995,"15min":0.84099999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.014,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.77200000000000002,"15min":0.79600000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88200000000000001,"5min":1.014,"15min":1.014},"last":0.68799999999999994}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.42999999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88900000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.82399999999999995,"15min":0.84799999999999998},"min":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.56699999999999995}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.72499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":473850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.51300000000000001,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.58299999999999996}]},{"osd":2,"last update":"Mon Nov 6 01:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56000000000000005,"15min":0.52100000000000002},"min":{"1min":0.439,"5min":0.308,"15min":0.308},"max":{"1min":1.121,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.53800000000000003,"15min":0.505},"min":{"1min":0.41299999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.78700000000000003}]},{"osd":3,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.61699999999999999,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1459999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.82399999999999995}]},{"osd":4,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.47799999999999998,"15min":0.46100000000000002},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.53500000000000003,"15min":0.51000000000000001},"min":{"1min":0.36899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.93400000000000005}]},{"osd":5,"last update":"Mon Nov 6 01:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.61199999999999999,"15min":0.58799999999999997},"min":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.85799999999999998}]},{"osd":6,"last update":"Mon Nov 6 01:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.64700000000000002,"15min":0.61499999999999999},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96699999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.874},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.69899999999999995,"15min":0.66600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.96899999999999997}]},{"osd":7,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.60199999999999998,"15min":0.56799999999999995},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.61399999999999999,"15min":0.57199999999999995},"min":{"1min":0.50700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.0800000000000001}]},{"osd":8,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.71099999999999997,"15min":0.68999999999999995},"min":{"1min":0.55000000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":1.157},{"interface":"front","average":{"1min":0.874,"5min":0.75800000000000001,"15min":0.73799999999999999},"min":{"1min":0.60999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.101}]},{"osd":9,"last update":"Mon Nov 6 01:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.45300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.82799999999999996,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.127}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.91700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":420237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50700000000000001},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.37},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.504},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.39100000000000001}]},{"osd":2,"last update":"Mon Nov 6 01:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.432,"5min":0.314,"15min":0.314},"max":{"1min":0.84299999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57299999999999995,"15min":0.56000000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.502}]},{"osd":3,"last update":"Mon Nov 6 01:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44,"15min":0.437},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.34899999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.41499999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.45900000000000002}]},{"osd":5,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73499999999999999,"5min":1.198,"15min":1.198},"last":0.54300000000000004}]},{"osd":6,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52800000000000002,"15min":0.55300000000000005},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66300000000000003,"5min":1.075,"15min":1.075},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.33500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.68000000000000005,"5min":1.222,"15min":1.222},"last":0.53300000000000003}]},{"osd":7,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.745,"15min":0.76800000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80900000000000005,"5min":1.292,"15min":1.292},"last":0.625},{"interface":"front","average":{"1min":0.629,"5min":0.74399999999999999,"15min":0.76300000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81699999999999995,"5min":1.341,"15min":1.341},"last":0.57399999999999995}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.66000000000000003,"15min":0.67500000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":1.123,"15min":1.123},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76000000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.58299999999999996}]},{"osd":10,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.71499999999999997}]},{"osd":11,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.64000000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":474415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.35899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.53000000000000003,"15min":0.504},"min":{"1min":0.40400000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.48399999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.56000000000000005,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58799999999999997}]},{"osd":3,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60399999999999998,"15min":0.57599999999999996},"min":{"1min":0.443,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57299999999999995}]},{"osd":4,"last update":"Mon Nov 6 01:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60299999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.45500000000000002,"15min":0.439},"min":{"1min":0.317,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.53500000000000003,"15min":0.51800000000000002},"min":{"1min":0.377,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85299999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67600000000000005}]},{"osd":6,"last update":"Mon Nov 6 01:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.46200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.73199999999999998}]},{"osd":7,"last update":"Mon Nov 6 01:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.54700000000000004}]},{"osd":8,"last update":"Mon Nov 6 01:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80800000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.64300000000000002}]},{"osd":9,"last update":"Mon Nov 6 01:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76500000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.78200000000000003}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.76600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":418783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.36399999999999999},"min":{"1min":0.27300000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51500000000000001,"5min":0.627,"15min":0.627},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.31,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.307}]},{"osd":1,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":1.119,"15min":1.119},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54500000000000004,"15min":0.496},"min":{"1min":0.42099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.52700000000000002}]},{"osd":2,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60399999999999998,"15min":0.55800000000000005},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83099999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61299999999999999,"15min":0.59299999999999997},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.72499999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.59399999999999997}]},{"osd":4,"last update":"Mon Nov 6 01:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.502,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.98799999999999999,"5min":1.159,"15min":1.159},"last":0.46300000000000002}]},{"osd":5,"last update":"Mon Nov 6 01:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.60299999999999998},"min":{"1min":0.38900000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.754,"5min":1.042,"15min":1.042},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.442,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.874,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.58399999999999996}]},{"osd":6,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89800000000000002,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.44400000000000001}]},{"osd":7,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.66500000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.67600000000000005,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.502}]},{"osd":8,"last update":"Mon Nov 6 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.622},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.0430000000000001,"5min":6.0430000000000001,"15min":6.0430000000000001},"last":0.76500000000000001}]},{"osd":9,"last update":"Mon Nov 6 01:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.74199999999999999,"15min":0.76900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85899999999999999,"5min":1.319,"15min":1.319},"last":0.38},{"interface":"front","average":{"1min":0.628,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.46400000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82199999999999995,"5min":1.214,"15min":1.214},"last":0.48599999999999999}]},{"osd":10,"last update":"Mon Nov 6 01:32:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.53400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":418759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.45300000000000001},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.49399999999999999,"15min":0.47799999999999998},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.749,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42999999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.377,"15min":0.35799999999999998},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.36599999999999999},"min":{"1min":0.27000000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.52600000000000002}]},{"osd":2,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.44900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.68300000000000005}]},{"osd":3,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.51400000000000001,"15min":0.49099999999999999},"min":{"1min":0.38900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.41199999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.72799999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.59899999999999998,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.45300000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.75600000000000001}]},{"osd":6,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.68999999999999995,"15min":0.64800000000000002},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":5.3949999999999996,"5min":5.3949999999999996,"15min":5.3949999999999996},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.127,"5min":0.73399999999999999,"15min":0.66800000000000004},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.77800000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.85999999999999999}]},{"osd":8,"last update":"Mon Nov 6 01:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.56699999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.60799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.90000000000000002}]},{"osd":9,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":0.82999999999999996,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.0659999999999998,"5min":7.0659999999999998,"15min":7.0659999999999998},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.73699999999999999,"15min":0.71599999999999997},"min":{"1min":0.63900000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.91300000000000003}]},{"osd":10,"last update":"Mon Nov 6 01:32:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.88600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724591,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":1011432,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.34399999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.76400000000000001}]},{"osd":1,"last update":"Mon Nov 6 01:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73399999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.84199999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.38500000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58799999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.39600000000000002,"15min":0.375},"min":{"1min":0.28199999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.621}]},{"osd":3,"last update":"Mon Nov 6 01:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.74199999999999999}]},{"osd":4,"last update":"Mon Nov 6 01:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.874}]},{"osd":6,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86099999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.80400000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.86099999999999999}]},{"osd":7,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.92100000000000004}]},{"osd":8,"last update":"Mon Nov 6 01:32:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.93100000000000005}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.85899999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.90100000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.96799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":55743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":55727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T01:33:17.384 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-11-06T01:33:17.385 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-06T01:33:17.385 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-11-06T01:33:17.385 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph health --format=json 2023-11-06T01:33:17.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:17 smithi062 ceph-mon[103924]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-11-06T01:33:17.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:17 smithi163 ceph-mon[104416]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-11-06T01:33:17.949 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:18.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:18 smithi053 ceph-mon[99472]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:18.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:18 smithi062 ceph-mon[103924]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:18.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:18 smithi163 ceph-mon[104416]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.277+0000 7f9e844f2700 1 -- 172.21.15.53:0/766587594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e7c0ffcd0 msgr2=0x7f9e7c100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.277+0000 7f9e844f2700 1 --2- 172.21.15.53:0/766587594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e7c0ffcd0 0x7f9e7c100150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009990 tx=0x7f9e70009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.277+0000 7f9e844f2700 1 -- 172.21.15.53:0/766587594 shutdown_connections 2023-11-06T01:33:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.277+0000 7f9e844f2700 1 --2- 172.21.15.53:0/766587594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c105d80 0x7f9e7c106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.277+0000 7f9e844f2700 1 --2- 172.21.15.53:0/766587594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7c100690 0x7f9e7c10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:19.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.277+0000 7f9e844f2700 1 --2- 172.21.15.53:0/766587594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e7c0ffcd0 0x7f9e7c100150 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:19.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.277+0000 7f9e844f2700 1 -- 172.21.15.53:0/766587594 >> 172.21.15.53:0/766587594 conn(0x7f9e7c076d90 msgr2=0x7f9e7c0771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:19.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.277+0000 7f9e844f2700 1 -- 172.21.15.53:0/766587594 shutdown_connections 2023-11-06T01:33:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.277+0000 7f9e844f2700 1 -- 172.21.15.53:0/766587594 wait complete. 2023-11-06T01:33:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e844f2700 1 Processor -- start 2023-11-06T01:33:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e844f2700 1 -- start start 2023-11-06T01:33:19.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e844f2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7c0ffcd0 0x7f9e7c197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:19.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e844f2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e7c100690 0x7f9e7c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:19.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e844f2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c105d80 0x7f9e7c1a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:19.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e844f2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c198b60 con 0x7f9e7c100690 2023-11-06T01:33:19.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e844f2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c1a32d0 con 0x7f9e7c105d80 2023-11-06T01:33:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e844f2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c1a3440 con 0x7f9e7c0ffcd0 2023-11-06T01:33:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e81a8d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e7c100690 0x7f9e7c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e81a8d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e7c100690 0x7f9e7c198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38602/0 (socket says 172.21.15.53:38602) 2023-11-06T01:33:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e81a8d700 1 -- 172.21.15.53:0/3529232135 learned_addr learned my addr 172.21.15.53:0/3529232135 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:19.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e8228e700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7c0ffcd0 0x7f9e7c197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e81a8d700 1 -- 172.21.15.53:0/3529232135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7c0ffcd0 msgr2=0x7f9e7c197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e81a8d700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7c0ffcd0 0x7f9e7c197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e81a8d700 1 -- 172.21.15.53:0/3529232135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c105d80 msgr2=0x7f9e7c1a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.278+0000 7f9e82a8f700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c105d80 0x7f9e7c1a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:19.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.279+0000 7f9e81a8d700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c105d80 0x7f9e7c1a2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:19.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.279+0000 7f9e81a8d700 1 -- 172.21.15.53:0/3529232135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e700095f0 con 0x7f9e7c100690 2023-11-06T01:33:19.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.279+0000 7f9e81a8d700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e7c100690 0x7f9e7c198250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9e700095c0 tx=0x7f9e7000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:19.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.279+0000 7f9e6f7fe700 1 -- 172.21.15.53:0/3529232135 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7001d070 con 0x7f9e7c100690 2023-11-06T01:33:19.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.279+0000 7f9e6f7fe700 1 -- 172.21.15.53:0/3529232135 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e700054f0 con 0x7f9e7c100690 2023-11-06T01:33:19.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.279+0000 7f9e6f7fe700 1 -- 172.21.15.53:0/3529232135 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70004bd0 con 0x7f9e7c100690 2023-11-06T01:33:19.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.279+0000 7f9e844f2700 1 -- 172.21.15.53:0/3529232135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7c1a36c0 con 0x7f9e7c100690 2023-11-06T01:33:19.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.279+0000 7f9e844f2700 1 -- 172.21.15.53:0/3529232135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7c1a3b60 con 0x7f9e7c100690 2023-11-06T01:33:19.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.280+0000 7f9e844f2700 1 -- 172.21.15.53:0/3529232135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7c04ea90 con 0x7f9e7c100690 2023-11-06T01:33:19.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.281+0000 7f9e6f7fe700 1 -- 172.21.15.53:0/3529232135 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f9e7000fc30 con 0x7f9e7c100690 2023-11-06T01:33:19.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.282+0000 7f9e6f7fe700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e680a0630 0x7f9e680a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:19.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.282+0000 7f9e8228e700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e680a0630 0x7f9e680a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:19.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.282+0000 7f9e6f7fe700 1 -- 172.21.15.53:0/3529232135 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9e700cce30 con 0x7f9e7c100690 2023-11-06T01:33:19.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.282+0000 7f9e8228e700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e680a0630 0x7f9e680a2af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e78006fd0 tx=0x7f9e78008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:19.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.285+0000 7f9e6f7fe700 1 -- 172.21.15.53:0/3529232135 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9e700969b0 con 0x7f9e7c100690 2023-11-06T01:33:19.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:19 smithi053 ceph-mon[99472]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:19.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:19 smithi062 ceph-mon[103924]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:19.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:19 smithi163 ceph-mon[104416]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:19.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.533+0000 7f9e844f2700 1 -- 172.21.15.53:0/3529232135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9e7c068e40 con 0x7f9e7c100690 2023-11-06T01:33:19.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.533+0000 7f9e6f7fe700 1 -- 172.21.15.53:0/3529232135 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9e70099fd0 con 0x7f9e7c100690 2023-11-06T01:33:19.535 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:19.535 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-06T01:33:19.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 -- 172.21.15.53:0/3529232135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e680a0630 msgr2=0x7f9e680a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:19.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e680a0630 0x7f9e680a2af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e78006fd0 tx=0x7f9e78008040 comp rx=0 tx=0).stop 2023-11-06T01:33:19.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 -- 172.21.15.53:0/3529232135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e7c100690 msgr2=0x7f9e7c198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:19.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e7c100690 0x7f9e7c198250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9e700095c0 tx=0x7f9e7000f740 comp rx=0 tx=0).stop 2023-11-06T01:33:19.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 -- 172.21.15.53:0/3529232135 shutdown_connections 2023-11-06T01:33:19.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e680a0630 0x7f9e680a2af0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c105d80 0x7f9e7c1a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7c0ffcd0 0x7f9e7c197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 --2- 172.21.15.53:0/3529232135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e7c100690 0x7f9e7c198250 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 -- 172.21.15.53:0/3529232135 >> 172.21.15.53:0/3529232135 conn(0x7f9e7c076d90 msgr2=0x7f9e7c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 -- 172.21.15.53:0/3529232135 shutdown_connections 2023-11-06T01:33:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:19.535+0000 7f9e844f2700 1 -- 172.21.15.53:0/3529232135 wait complete. 2023-11-06T01:33:19.999 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-11-06T01:33:20.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:33:20.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.133+0000 7effff959700 1 -- 172.21.15.53:0/547598953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efff8106ab0 msgr2=0x7efff8108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.133+0000 7effff959700 1 --2- 172.21.15.53:0/547598953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efff8106ab0 0x7efff8108ec0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efff000b0a0 tx=0x7efff000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.133+0000 7effff959700 1 -- 172.21.15.53:0/547598953 shutdown_connections 2023-11-06T01:33:20.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.133+0000 7effff959700 1 --2- 172.21.15.53:0/547598953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8101850 0x7efff8103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.133+0000 7effff959700 1 --2- 172.21.15.53:0/547598953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efff8106ab0 0x7efff8108ec0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.133+0000 7effff959700 1 --2- 172.21.15.53:0/547598953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efff8104180 0x7efff8106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.133+0000 7effff959700 1 -- 172.21.15.53:0/547598953 >> 172.21.15.53:0/547598953 conn(0x7efff80fb1c0 msgr2=0x7efff80fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:20.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.133+0000 7effff959700 1 -- 172.21.15.53:0/547598953 shutdown_connections 2023-11-06T01:33:20.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.133+0000 7effff959700 1 -- 172.21.15.53:0/547598953 wait complete. 2023-11-06T01:33:20.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.133+0000 7effff959700 1 Processor -- start 2023-11-06T01:33:20.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7effff959700 1 -- start start 2023-11-06T01:33:20.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7effff959700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8101850 0x7efff8196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7effff959700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efff8104180 0x7efff8197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7effff959700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efff8106ab0 0x7efff819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7effff959700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff819e6d0 con 0x7efff8104180 2023-11-06T01:33:20.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7effff959700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff819e810 con 0x7efff8101850 2023-11-06T01:33:20.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7effff959700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff819e9a0 con 0x7efff8106ab0 2023-11-06T01:33:20.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffcef4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efff8104180 0x7efff8197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffcef4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efff8104180 0x7efff8197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38616/0 (socket says 172.21.15.53:38616) 2023-11-06T01:33:20.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffcef4700 1 -- 172.21.15.53:0/3729649421 learned_addr learned my addr 172.21.15.53:0/3729649421 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:20.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffdef6700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efff8106ab0 0x7efff819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffd6f5700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8101850 0x7efff8196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffcef4700 1 -- 172.21.15.53:0/3729649421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efff8106ab0 msgr2=0x7efff819e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffcef4700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efff8106ab0 0x7efff819e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffcef4700 1 -- 172.21.15.53:0/3729649421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8101850 msgr2=0x7efff8196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffcef4700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8101850 0x7efff8196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffcef4700 1 -- 172.21.15.53:0/3729649421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff0009d00 con 0x7efff8104180 2023-11-06T01:33:20.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffdef6700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efff8106ab0 0x7efff819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:20.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7efffcef4700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efff8104180 0x7efff8197090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7effe800b530 tx=0x7effe800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:20.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.134+0000 7effee7fc700 1 -- 172.21.15.53:0/3729649421 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8011830 con 0x7efff8104180 2023-11-06T01:33:20.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.135+0000 7effee7fc700 1 -- 172.21.15.53:0/3729649421 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effe8011e50 con 0x7efff8104180 2023-11-06T01:33:20.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.135+0000 7effff959700 1 -- 172.21.15.53:0/3729649421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff819ec80 con 0x7efff8104180 2023-11-06T01:33:20.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.135+0000 7effee7fc700 1 -- 172.21.15.53:0/3729649421 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe800f2e0 con 0x7efff8104180 2023-11-06T01:33:20.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.135+0000 7effff959700 1 -- 172.21.15.53:0/3729649421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff819f190 con 0x7efff8104180 2023-11-06T01:33:20.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.135+0000 7effff959700 1 -- 172.21.15.53:0/3729649421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff80fcd40 con 0x7efff8104180 2023-11-06T01:33:20.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.137+0000 7effee7fc700 1 -- 172.21.15.53:0/3729649421 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7effe8010580 con 0x7efff8104180 2023-11-06T01:33:20.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.137+0000 7effee7fc700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effe40a0670 0x7effe40a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.137+0000 7efffd6f5700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effe40a0670 0x7effe40a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.137+0000 7effee7fc700 1 -- 172.21.15.53:0/3729649421 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7effe80cba10 con 0x7efff8104180 2023-11-06T01:33:20.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.138+0000 7efffd6f5700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effe40a0670 0x7effe40a2b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efff4005950 tx=0x7efff40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:20.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.141+0000 7effee7fc700 1 -- 172.21.15.53:0/3729649421 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7effe8095610 con 0x7efff8104180 2023-11-06T01:33:20.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:20 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3529232135' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-06T01:33:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.346+0000 7effff959700 1 -- 172.21.15.53:0/3729649421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efff802cec0 con 0x7efff8104180 2023-11-06T01:33:20.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.348+0000 7effee7fc700 1 -- 172.21.15.53:0/3729649421 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22071 (secure 0 0 0) 0x7effe8098c30 con 0x7efff8104180 2023-11-06T01:33:20.349 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:20.349 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:32:27.094591+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:33:20.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 -- 172.21.15.53:0/3729649421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effe40a0670 msgr2=0x7effe40a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effe40a0670 0x7effe40a2b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efff4005950 tx=0x7efff40058e0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 -- 172.21.15.53:0/3729649421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efff8104180 msgr2=0x7efff8197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efff8104180 0x7efff8197090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7effe800b530 tx=0x7effe800b8f0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 -- 172.21.15.53:0/3729649421 shutdown_connections 2023-11-06T01:33:20.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effe40a0670 0x7effe40a2b30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8101850 0x7efff8196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efff8106ab0 0x7efff819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 --2- 172.21.15.53:0/3729649421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efff8104180 0x7efff8197090 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 -- 172.21.15.53:0/3729649421 >> 172.21.15.53:0/3729649421 conn(0x7efff80fb1c0 msgr2=0x7efff81024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:20.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 -- 172.21.15.53:0/3729649421 shutdown_connections 2023-11-06T01:33:20.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.350+0000 7effff959700 1 -- 172.21.15.53:0/3729649421 wait complete. 2023-11-06T01:33:20.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T01:32:24.604601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T01:32:25.457741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:33:20.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T01:33:20.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:20 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3529232135' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-06T01:33:20.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:20 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3529232135' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-06T01:33:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.494+0000 7fbe72a70700 1 -- 172.21.15.53:0/367275433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0ffcb0 msgr2=0x7fbe6c1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.494+0000 7fbe72a70700 1 --2- 172.21.15.53:0/367275433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0ffcb0 0x7fbe6c1000d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbe600099c0 tx=0x7fbe60009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.494+0000 7fbe72a70700 1 -- 172.21.15.53:0/367275433 shutdown_connections 2023-11-06T01:33:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.494+0000 7fbe72a70700 1 --2- 172.21.15.53:0/367275433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c101840 0x7fbe6c1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.494+0000 7fbe72a70700 1 --2- 172.21.15.53:0/367275433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0ffcb0 0x7fbe6c1000d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.494+0000 7fbe72a70700 1 --2- 172.21.15.53:0/367275433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe6c100ea0 0x7fbe6c101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.494+0000 7fbe72a70700 1 -- 172.21.15.53:0/367275433 >> 172.21.15.53:0/367275433 conn(0x7fbe6c0fb220 msgr2=0x7fbe6c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:20.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.494+0000 7fbe72a70700 1 -- 172.21.15.53:0/367275433 shutdown_connections 2023-11-06T01:33:20.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.494+0000 7fbe72a70700 1 -- 172.21.15.53:0/367275433 wait complete. 2023-11-06T01:33:20.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.494+0000 7fbe72a70700 1 Processor -- start 2023-11-06T01:33:20.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe72a70700 1 -- start start 2023-11-06T01:33:20.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe72a70700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0ffcb0 0x7fbe6c195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe72a70700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe6c100ea0 0x7fbe6c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe72a70700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c101840 0x7fbe6c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe72a70700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c19d4a0 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe72a70700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c068ec0 con 0x7fbe6c101840 2023-11-06T01:33:20.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe72a70700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c069050 con 0x7fbe6c100ea0 2023-11-06T01:33:20.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe7080c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0ffcb0 0x7fbe6c195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe7080c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0ffcb0 0x7fbe6c195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38628/0 (socket says 172.21.15.53:38628) 2023-11-06T01:33:20.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe7080c700 1 -- 172.21.15.53:0/805466784 learned_addr learned my addr 172.21.15.53:0/805466784 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:20.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe6bfff700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe6c100ea0 0x7fbe6c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe7100d700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c101840 0x7fbe6c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe7080c700 1 -- 172.21.15.53:0/805466784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe6c100ea0 msgr2=0x7fbe6c195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe7080c700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe6c100ea0 0x7fbe6c195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe7080c700 1 -- 172.21.15.53:0/805466784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c101840 msgr2=0x7fbe6c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe7080c700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c101840 0x7fbe6c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe7080c700 1 -- 172.21.15.53:0/805466784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe600095f0 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe6bfff700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe6c100ea0 0x7fbe6c195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe7100d700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c101840 0x7fbe6c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe7080c700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0ffcb0 0x7fbe6c195920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009990 tx=0x7fbe60005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.495+0000 7fbe69ffb700 1 -- 172.21.15.53:0/805466784 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60025070 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.496+0000 7fbe69ffb700 1 -- 172.21.15.53:0/805466784 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe60003ff0 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.496+0000 7fbe72a70700 1 -- 172.21.15.53:0/805466784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6c0692d0 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.496+0000 7fbe69ffb700 1 -- 172.21.15.53:0/805466784 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6000fc60 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.496+0000 7fbe72a70700 1 -- 172.21.15.53:0/805466784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6c0697d0 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.497+0000 7fbe72a70700 1 -- 172.21.15.53:0/805466784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6c10b4e0 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.498+0000 7fbe69ffb700 1 -- 172.21.15.53:0/805466784 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fbe60029070 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.498+0000 7fbe69ffb700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbe5c0a0630 0x7fbe5c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.499+0000 7fbe6bfff700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbe5c0a0630 0x7fbe5c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.499+0000 7fbe69ffb700 1 -- 172.21.15.53:0/805466784 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbe600d5b50 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.499+0000 7fbe6bfff700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbe5c0a0630 0x7fbe5c0a2af0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800bcf0 tx=0x7fbe5800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.502+0000 7fbe69ffb700 1 -- 172.21.15.53:0/805466784 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbe6009f6d0 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.705+0000 7fbe72a70700 1 -- 172.21.15.53:0/805466784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe6c106be0 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.705+0000 7fbe69ffb700 1 -- 172.21.15.53:0/805466784 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe600a2cf0 con 0x7fbe6c0ffcb0 2023-11-06T01:33:20.706 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:33:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 -- 172.21.15.53:0/805466784 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbe5c0a0630 msgr2=0x7fbe5c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbe5c0a0630 0x7fbe5c0a2af0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800bcf0 tx=0x7fbe5800b2c0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 -- 172.21.15.53:0/805466784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0ffcb0 msgr2=0x7fbe6c195920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0ffcb0 0x7fbe6c195920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009990 tx=0x7fbe60005e80 comp rx=0 tx=0).stop 2023-11-06T01:33:20.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 -- 172.21.15.53:0/805466784 shutdown_connections 2023-11-06T01:33:20.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbe5c0a0630 0x7fbe5c0a2af0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c101840 0x7fbe6c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0ffcb0 0x7fbe6c195920 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 --2- 172.21.15.53:0/805466784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe6c100ea0 0x7fbe6c195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 -- 172.21.15.53:0/805466784 >> 172.21.15.53:0/805466784 conn(0x7fbe6c0fb220 msgr2=0x7fbe6c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 -- 172.21.15.53:0/805466784 shutdown_connections 2023-11-06T01:33:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.707+0000 7fbe72a70700 1 -- 172.21.15.53:0/805466784 wait complete. 2023-11-06T01:33:20.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T01:33:20.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.849+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3914828856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c041018a0 msgr2=0x7f8c04103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.849+0000 7f8c0ac78700 1 --2- 172.21.15.53:0/3914828856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c041018a0 0x7f8c04103c90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009990 tx=0x7f8bf8009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.849+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3914828856 shutdown_connections 2023-11-06T01:33:20.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.849+0000 7f8c0ac78700 1 --2- 172.21.15.53:0/3914828856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c04106b00 0x7f8c04108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.849+0000 7f8c0ac78700 1 --2- 172.21.15.53:0/3914828856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c041018a0 0x7f8c04103c90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.849+0000 7f8c0ac78700 1 --2- 172.21.15.53:0/3914828856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041041d0 0x7f8c041065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.849+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3914828856 >> 172.21.15.53:0/3914828856 conn(0x7f8c040fb1f0 msgr2=0x7f8c040fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:20.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.849+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3914828856 shutdown_connections 2023-11-06T01:33:20.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.849+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3914828856 wait complete. 2023-11-06T01:33:20.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c0ac78700 1 Processor -- start 2023-11-06T01:33:20.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c0ac78700 1 -- start start 2023-11-06T01:33:20.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c0ac78700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c041018a0 0x7f8c04196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c0ac78700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041041d0 0x7f8c04196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c0ac78700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c04106b00 0x7f8c0419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c0ac78700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419e600 con 0x7f8c04106b00 2023-11-06T01:33:20.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c0ac78700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419e740 con 0x7f8c041041d0 2023-11-06T01:33:20.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c0ac78700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419e8d0 con 0x7f8c041018a0 2023-11-06T01:33:20.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c09215700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c04106b00 0x7f8c0419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c09215700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c04106b00 0x7f8c0419e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38646/0 (socket says 172.21.15.53:38646) 2023-11-06T01:33:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c09215700 1 -- 172.21.15.53:0/3741375026 learned_addr learned my addr 172.21.15.53:0/3741375026 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c08a14700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c041018a0 0x7f8c04196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c03fff700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041041d0 0x7f8c04196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c09215700 1 -- 172.21.15.53:0/3741375026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c041018a0 msgr2=0x7f8c04196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c09215700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c041018a0 0x7f8c04196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c09215700 1 -- 172.21.15.53:0/3741375026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041041d0 msgr2=0x7f8c04196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c09215700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041041d0 0x7f8c04196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c09215700 1 -- 172.21.15.53:0/3741375026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80095f0 con 0x7f8c04106b00 2023-11-06T01:33:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c08a14700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c041018a0 0x7f8c04196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.850+0000 7f8c09215700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c04106b00 0x7f8c0419e030 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc007ee0 tx=0x7f8bfc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:20.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.851+0000 7f8c01ffb700 1 -- 172.21.15.53:0/3741375026 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc017070 con 0x7f8c04106b00 2023-11-06T01:33:20.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.851+0000 7f8c01ffb700 1 -- 172.21.15.53:0/3741375026 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bfc00f040 con 0x7f8c04106b00 2023-11-06T01:33:20.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.851+0000 7f8c01ffb700 1 -- 172.21.15.53:0/3741375026 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc005740 con 0x7f8c04106b00 2023-11-06T01:33:20.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.851+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3741375026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0419ebb0 con 0x7f8c04106b00 2023-11-06T01:33:20.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.851+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3741375026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0419f070 con 0x7f8c04106b00 2023-11-06T01:33:20.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.852+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3741375026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c040fcd70 con 0x7f8c04106b00 2023-11-06T01:33:20.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.853+0000 7f8c01ffb700 1 -- 172.21.15.53:0/3741375026 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f8bfc0058a0 con 0x7f8c04106b00 2023-11-06T01:33:20.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.853+0000 7f8c01ffb700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8bf40a05b0 0x7f8bf40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:20.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.853+0000 7f8c01ffb700 1 -- 172.21.15.53:0/3741375026 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8bfc0cb250 con 0x7f8c04106b00 2023-11-06T01:33:20.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.853+0000 7f8c08a14700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8bf40a05b0 0x7f8bf40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:20.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.854+0000 7f8c08a14700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8bf40a05b0 0x7f8bf40a2a70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800b2e0 tx=0x7f8bf8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:20.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:20.857+0000 7f8c01ffb700 1 -- 172.21.15.53:0/3741375026 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8bfc094e40 con 0x7f8c04106b00 2023-11-06T01:33:21.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.061+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3741375026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c0402cc70 con 0x7f8c04106b00 2023-11-06T01:33:21.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.062+0000 7f8c01ffb700 1 -- 172.21.15.53:0/3741375026 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8bfc098460 con 0x7f8c04106b00 2023-11-06T01:33:21.063 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-06T01:33:21.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3741375026 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8bf40a05b0 msgr2=0x7f8bf40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8bf40a05b0 0x7f8bf40a2a70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800b2e0 tx=0x7f8bf8005fb0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3741375026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c04106b00 msgr2=0x7f8c0419e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c04106b00 0x7f8c0419e030 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc007ee0 tx=0x7f8bfc00d3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3741375026 shutdown_connections 2023-11-06T01:33:21.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8bf40a05b0 0x7f8bf40a2a70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c04106b00 0x7f8c0419e030 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c041018a0 0x7f8c04196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 --2- 172.21.15.53:0/3741375026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041041d0 0x7f8c04196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3741375026 >> 172.21.15.53:0/3741375026 conn(0x7f8c040fb1f0 msgr2=0x7f8c04102510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:21.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3741375026 shutdown_connections 2023-11-06T01:33:21.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.064+0000 7f8c0ac78700 1 -- 172.21.15.53:0/3741375026 wait complete. 2023-11-06T01:33:21.077 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T01:33:21.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.206+0000 7f2b6b808700 1 -- 172.21.15.53:0/263492418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b640ffcb0 msgr2=0x7f2b641000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.206+0000 7f2b6b808700 1 --2- 172.21.15.53:0/263492418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b640ffcb0 0x7f2b641000d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b600099c0 tx=0x7f2b60009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.206+0000 7f2b6b808700 1 -- 172.21.15.53:0/263492418 shutdown_connections 2023-11-06T01:33:21.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.206+0000 7f2b6b808700 1 --2- 172.21.15.53:0/263492418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b64101840 0x7f2b641080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.206+0000 7f2b6b808700 1 --2- 172.21.15.53:0/263492418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b640ffcb0 0x7f2b641000d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.206+0000 7f2b6b808700 1 --2- 172.21.15.53:0/263492418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b64100ea0 0x7f2b64101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.206+0000 7f2b6b808700 1 -- 172.21.15.53:0/263492418 >> 172.21.15.53:0/263492418 conn(0x7f2b640fb220 msgr2=0x7f2b640fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:21.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.206+0000 7f2b6b808700 1 -- 172.21.15.53:0/263492418 shutdown_connections 2023-11-06T01:33:21.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.206+0000 7f2b6b808700 1 -- 172.21.15.53:0/263492418 wait complete. 2023-11-06T01:33:21.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b6b808700 1 Processor -- start 2023-11-06T01:33:21.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b6b808700 1 -- start start 2023-11-06T01:33:21.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b6b808700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b640ffcb0 0x7f2b64103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b6b808700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b64100ea0 0x7f2b64102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b6b808700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b64101840 0x7f2b641027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b6b808700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64102f60 con 0x7f2b64100ea0 2023-11-06T01:33:21.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b6b808700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b641030d0 con 0x7f2b64101840 2023-11-06T01:33:21.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b6b808700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b641a0a80 con 0x7f2b640ffcb0 2023-11-06T01:33:21.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b68da3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b64100ea0 0x7f2b64102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:21.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b68da3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b64100ea0 0x7f2b64102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38658/0 (socket says 172.21.15.53:38658) 2023-11-06T01:33:21.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b68da3700 1 -- 172.21.15.53:0/2487153733 learned_addr learned my addr 172.21.15.53:0/2487153733 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:21.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.207+0000 7f2b69da5700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b64101840 0x7f2b641027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:21.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.208+0000 7f2b68da3700 1 -- 172.21.15.53:0/2487153733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b640ffcb0 msgr2=0x7f2b64103c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.208+0000 7f2b68da3700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b640ffcb0 0x7f2b64103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.208+0000 7f2b68da3700 1 -- 172.21.15.53:0/2487153733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b64101840 msgr2=0x7f2b641027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.208+0000 7f2b68da3700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b64101840 0x7f2b641027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.208+0000 7f2b68da3700 1 -- 172.21.15.53:0/2487153733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b600095f0 con 0x7f2b64100ea0 2023-11-06T01:33:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.208+0000 7f2b68da3700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b64100ea0 0x7f2b64102280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c920 tx=0x7f2b5400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.208+0000 7f2b5a7fc700 1 -- 172.21.15.53:0/2487153733 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54007b00 con 0x7f2b64100ea0 2023-11-06T01:33:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.208+0000 7f2b5a7fc700 1 -- 172.21.15.53:0/2487153733 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b54004cf0 con 0x7f2b64100ea0 2023-11-06T01:33:21.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.208+0000 7f2b6b808700 1 -- 172.21.15.53:0/2487153733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b641a0d60 con 0x7f2b64100ea0 2023-11-06T01:33:21.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.208+0000 7f2b5a7fc700 1 -- 172.21.15.53:0/2487153733 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54005610 con 0x7f2b64100ea0 2023-11-06T01:33:21.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.208+0000 7f2b6b808700 1 -- 172.21.15.53:0/2487153733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b641a1210 con 0x7f2b64100ea0 2023-11-06T01:33:21.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.209+0000 7f2b6b808700 1 -- 172.21.15.53:0/2487153733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6410b4c0 con 0x7f2b64100ea0 2023-11-06T01:33:21.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.210+0000 7f2b5a7fc700 1 -- 172.21.15.53:0/2487153733 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f2b54004750 con 0x7f2b64100ea0 2023-11-06T01:33:21.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.211+0000 7f2b5a7fc700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2b500a05b0 0x7f2b500a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.211+0000 7f2b695a4700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2b500a05b0 0x7f2b500a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:21.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.211+0000 7f2b5a7fc700 1 -- 172.21.15.53:0/2487153733 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2b540cba50 con 0x7f2b64100ea0 2023-11-06T01:33:21.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.211+0000 7f2b695a4700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2b500a05b0 0x7f2b500a2a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2b600036f0 tx=0x7f2b60000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:21.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.215+0000 7f2b5a7fc700 1 -- 172.21.15.53:0/2487153733 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2b54095540 con 0x7f2b64100ea0 2023-11-06T01:33:21.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:21 smithi053 ceph-mon[99472]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-11-06T01:33:21.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:21 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3729649421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:21.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:21 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/805466784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:21.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:21 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3741375026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:21.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:21 smithi062 ceph-mon[103924]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-11-06T01:33:21.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:21 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3729649421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:21.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:21 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/805466784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:21.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:21 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3741375026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:21.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.417+0000 7f2b6b808700 1 -- 172.21.15.53:0/2487153733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2b641a1880 con 0x7f2b64100ea0 2023-11-06T01:33:21.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.418+0000 7f2b5a7fc700 1 -- 172.21.15.53:0/2487153733 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2b5401d020 con 0x7f2b64100ea0 2023-11-06T01:33:21.419 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-06T01:33:21.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 -- 172.21.15.53:0/2487153733 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2b500a05b0 msgr2=0x7f2b500a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2b500a05b0 0x7f2b500a2a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2b600036f0 tx=0x7f2b60000bc0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 -- 172.21.15.53:0/2487153733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b64100ea0 msgr2=0x7f2b64102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b64100ea0 0x7f2b64102280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c920 tx=0x7f2b5400cce0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 -- 172.21.15.53:0/2487153733 shutdown_connections 2023-11-06T01:33:21.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f2b500a05b0 0x7f2b500a2a70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b64101840 0x7f2b641027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b640ffcb0 0x7f2b64103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 --2- 172.21.15.53:0/2487153733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b64100ea0 0x7f2b64102280 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 -- 172.21.15.53:0/2487153733 >> 172.21.15.53:0/2487153733 conn(0x7f2b640fb220 msgr2=0x7f2b64068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:21.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 -- 172.21.15.53:0/2487153733 shutdown_connections 2023-11-06T01:33:21.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.420+0000 7f2b6b808700 1 -- 172.21.15.53:0/2487153733 wait complete. 2023-11-06T01:33:21.433 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:33:21.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:21 smithi163 ceph-mon[104416]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-11-06T01:33:21.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:21 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3729649421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:21.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:21 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/805466784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:21.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:21 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3741375026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:21.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.559+0000 7fc808638700 1 -- 172.21.15.53:0/624243336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800100d90 msgr2=0x7fc8001011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.559+0000 7fc808638700 1 --2- 172.21.15.53:0/624243336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800100d90 0x7fc8001011b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0099c0 tx=0x7fc7fc009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.559+0000 7fc808638700 1 -- 172.21.15.53:0/624243336 shutdown_connections 2023-11-06T01:33:21.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.559+0000 7fc808638700 1 --2- 172.21.15.53:0/624243336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc800102920 0x7fc8001091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.559+0000 7fc808638700 1 --2- 172.21.15.53:0/624243336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800100d90 0x7fc8001011b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.559+0000 7fc808638700 1 --2- 172.21.15.53:0/624243336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc800101f80 0x7fc8001023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.559+0000 7fc808638700 1 -- 172.21.15.53:0/624243336 >> 172.21.15.53:0/624243336 conn(0x7fc8000fc2e0 msgr2=0x7fc8000fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:21.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.559+0000 7fc808638700 1 -- 172.21.15.53:0/624243336 shutdown_connections 2023-11-06T01:33:21.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.559+0000 7fc808638700 1 -- 172.21.15.53:0/624243336 wait complete. 2023-11-06T01:33:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.559+0000 7fc808638700 1 Processor -- start 2023-11-06T01:33:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.559+0000 7fc808638700 1 -- start start 2023-11-06T01:33:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc808638700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800100d90 0x7fc800074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc808638700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc800101f80 0x7fc800073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc808638700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc800102920 0x7fc8000736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc808638700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc800073e60 con 0x7fc800100d90 2023-11-06T01:33:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc808638700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc800073fd0 con 0x7fc800101f80 2023-11-06T01:33:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc8063d4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800100d90 0x7fc800074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc808638700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8001a0ab0 con 0x7fc800102920 2023-11-06T01:33:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc8063d4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800100d90 0x7fc800074b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38670/0 (socket says 172.21.15.53:38670) 2023-11-06T01:33:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc8063d4700 1 -- 172.21.15.53:0/3205657197 learned_addr learned my addr 172.21.15.53:0/3205657197 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc805bd3700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc800101f80 0x7fc800073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc806bd5700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc800102920 0x7fc8000736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc8063d4700 1 -- 172.21.15.53:0/3205657197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc800102920 msgr2=0x7fc8000736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc8063d4700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc800102920 0x7fc8000736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc8063d4700 1 -- 172.21.15.53:0/3205657197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc800101f80 msgr2=0x7fc800073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc8063d4700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc800101f80 0x7fc800073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc8063d4700 1 -- 172.21.15.53:0/3205657197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc0095f0 con 0x7fc800100d90 2023-11-06T01:33:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc8063d4700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800100d90 0x7fc800074b40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0036f0 tx=0x7fc7fc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc7f77fe700 1 -- 172.21.15.53:0/3205657197 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc01d070 con 0x7fc800100d90 2023-11-06T01:33:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc7f77fe700 1 -- 172.21.15.53:0/3205657197 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7fc004110 con 0x7fc800100d90 2023-11-06T01:33:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc808638700 1 -- 172.21.15.53:0/3205657197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8001a0d30 con 0x7fc800100d90 2023-11-06T01:33:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc7f77fe700 1 -- 172.21.15.53:0/3205657197 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc021410 con 0x7fc800100d90 2023-11-06T01:33:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.560+0000 7fc808638700 1 -- 172.21.15.53:0/3205657197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8001a1150 con 0x7fc800100d90 2023-11-06T01:33:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.561+0000 7fc808638700 1 -- 172.21.15.53:0/3205657197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80010c660 con 0x7fc800100d90 2023-11-06T01:33:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.562+0000 7fc7f77fe700 1 -- 172.21.15.53:0/3205657197 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc7fc00f660 con 0x7fc800100d90 2023-11-06T01:33:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.563+0000 7fc7f77fe700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7ec0a05f0 0x7fc7ec0a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.563+0000 7fc805bd3700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7ec0a05f0 0x7fc7ec0a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.563+0000 7fc7f77fe700 1 -- 172.21.15.53:0/3205657197 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc7fc0cca80 con 0x7fc800100d90 2023-11-06T01:33:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.563+0000 7fc805bd3700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7ec0a05f0 0x7fc7ec0a2ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0005950 tx=0x7fc7f00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.567+0000 7fc7f77fe700 1 -- 172.21.15.53:0/3205657197 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc7fc096600 con 0x7fc800100d90 2023-11-06T01:33:21.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.818+0000 7fc808638700 1 -- 172.21.15.53:0/3205657197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc800107c40 con 0x7fc800100d90 2023-11-06T01:33:21.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.818+0000 7fc7f77fe700 1 -- 172.21.15.53:0/3205657197 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10815 (secure 0 0 0) 0x7fc7fc099c20 con 0x7fc800100d90 2023-11-06T01:33:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:21.820 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-06T01:32:26.093194+0000","modified":"2023-11-06T01:32:33.006369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24472,"mds_2":24464,"mds_3":24422,"mds_4":24395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24395":{"gid":24395,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T01:33:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 -- 172.21.15.53:0/3205657197 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7ec0a05f0 msgr2=0x7fc7ec0a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7ec0a05f0 0x7fc7ec0a2ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0005950 tx=0x7fc7f00058e0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 -- 172.21.15.53:0/3205657197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800100d90 msgr2=0x7fc800074b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800100d90 0x7fc800074b40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0036f0 tx=0x7fc7fc009580 comp rx=0 tx=0).stop 2023-11-06T01:33:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 -- 172.21.15.53:0/3205657197 shutdown_connections 2023-11-06T01:33:21.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7ec0a05f0 0x7fc7ec0a2ab0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc800102920 0x7fc8000736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800100d90 0x7fc800074b40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 --2- 172.21.15.53:0/3205657197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc800101f80 0x7fc800073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 -- 172.21.15.53:0/3205657197 >> 172.21.15.53:0/3205657197 conn(0x7fc8000fc2e0 msgr2=0x7fc8000fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 -- 172.21.15.53:0/3205657197 shutdown_connections 2023-11-06T01:33:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.821+0000 7fc808638700 1 -- 172.21.15.53:0/3205657197 wait complete. 2023-11-06T01:33:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-11-06T01:33:21.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:33:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.959+0000 7f443e85c700 1 -- 172.21.15.53:0/2315714339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4438102940 msgr2=0x7f44381091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.959+0000 7f443e85c700 1 --2- 172.21.15.53:0/2315714339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4438102940 0x7f44381091f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f443000b0d0 tx=0x7f443000b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.960+0000 7f443e85c700 1 -- 172.21.15.53:0/2315714339 shutdown_connections 2023-11-06T01:33:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.960+0000 7f443e85c700 1 --2- 172.21.15.53:0/2315714339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4438100db0 0x7f44381011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.960+0000 7f443e85c700 1 --2- 172.21.15.53:0/2315714339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4438102940 0x7f44381091f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.960+0000 7f443e85c700 1 --2- 172.21.15.53:0/2315714339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4438101fa0 0x7f4438102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.960+0000 7f443e85c700 1 -- 172.21.15.53:0/2315714339 >> 172.21.15.53:0/2315714339 conn(0x7f44380fc300 msgr2=0x7f44380fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.960+0000 7f443e85c700 1 -- 172.21.15.53:0/2315714339 shutdown_connections 2023-11-06T01:33:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.960+0000 7f443e85c700 1 -- 172.21.15.53:0/2315714339 wait complete. 2023-11-06T01:33:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.960+0000 7f443e85c700 1 Processor -- start 2023-11-06T01:33:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.960+0000 7f443e85c700 1 -- start start 2023-11-06T01:33:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f443e85c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4438100db0 0x7f4438196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f443e85c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4438101fa0 0x7f4438196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f443e85c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4438102940 0x7f443819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f443e85c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819e5d0 con 0x7f4438101fa0 2023-11-06T01:33:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f443e85c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819e710 con 0x7f4438102940 2023-11-06T01:33:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f443e85c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819e8a0 con 0x7f4438100db0 2023-11-06T01:33:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44377fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4438101fa0 0x7f4438196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44377fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4438101fa0 0x7f4438196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38686/0 (socket says 172.21.15.53:38686) 2023-11-06T01:33:21.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f4437fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4438100db0 0x7f4438196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:21.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44377fe700 1 -- 172.21.15.53:0/1499204433 learned_addr learned my addr 172.21.15.53:0/1499204433 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:21.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f443cdf9700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4438102940 0x7f443819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44377fe700 1 -- 172.21.15.53:0/1499204433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4438100db0 msgr2=0x7f4438196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44377fe700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4438100db0 0x7f4438196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44377fe700 1 -- 172.21.15.53:0/1499204433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4438102940 msgr2=0x7f443819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44377fe700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4438102940 0x7f443819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44377fe700 1 -- 172.21.15.53:0/1499204433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4430009d00 con 0x7f4438101fa0 2023-11-06T01:33:21.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f443cdf9700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4438102940 0x7f443819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:21.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44377fe700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4438101fa0 0x7f4438196f90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f442800d980 tx=0x7f442800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:21.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44357fa700 1 -- 172.21.15.53:0/1499204433 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428004220 con 0x7f4438101fa0 2023-11-06T01:33:21.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44357fa700 1 -- 172.21.15.53:0/1499204433 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4428007640 con 0x7f4438101fa0 2023-11-06T01:33:21.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f443e85c700 1 -- 172.21.15.53:0/1499204433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443819eb80 con 0x7f4438101fa0 2023-11-06T01:33:21.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f44357fa700 1 -- 172.21.15.53:0/1499204433 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428005610 con 0x7f4438101fa0 2023-11-06T01:33:21.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.961+0000 7f443e85c700 1 -- 172.21.15.53:0/1499204433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443819f0e0 con 0x7f4438101fa0 2023-11-06T01:33:21.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.962+0000 7f443e85c700 1 -- 172.21.15.53:0/1499204433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443810c5d0 con 0x7f4438101fa0 2023-11-06T01:33:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.964+0000 7f44357fa700 1 -- 172.21.15.53:0/1499204433 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f4428004750 con 0x7f4438101fa0 2023-11-06T01:33:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.964+0000 7f44357fa700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44200a0670 0x7f44200a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.964+0000 7f44357fa700 1 -- 172.21.15.53:0/1499204433 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f44280cbc10 con 0x7f4438101fa0 2023-11-06T01:33:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.964+0000 7f4437fff700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44200a0670 0x7f44200a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.965+0000 7f4437fff700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44200a0670 0x7f44200a2b30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4438101e00 tx=0x7f442c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:21.968+0000 7f44357fa700 1 -- 172.21.15.53:0/1499204433 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4428095800 con 0x7f4438101fa0 2023-11-06T01:33:22.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.178+0000 7f443e85c700 1 -- 172.21.15.53:0/1499204433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4438107b80 con 0x7f4438101fa0 2023-11-06T01:33:22.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.180+0000 7f44357fa700 1 -- 172.21.15.53:0/1499204433 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f4428016020 con 0x7f4438101fa0 2023-11-06T01:33:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:22.182 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:32:27.094591+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:33:22.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 -- 172.21.15.53:0/1499204433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44200a0670 msgr2=0x7f44200a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44200a0670 0x7f44200a2b30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4438101e00 tx=0x7f442c00a3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 -- 172.21.15.53:0/1499204433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4438101fa0 msgr2=0x7f4438196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4438101fa0 0x7f4438196f90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f442800d980 tx=0x7f442800dc90 comp rx=0 tx=0).stop 2023-11-06T01:33:22.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 -- 172.21.15.53:0/1499204433 shutdown_connections 2023-11-06T01:33:22.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44200a0670 0x7f44200a2b30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4438102940 0x7f443819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4438100db0 0x7f4438196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 --2- 172.21.15.53:0/1499204433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4438101fa0 0x7f4438196f90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 -- 172.21.15.53:0/1499204433 >> 172.21.15.53:0/1499204433 conn(0x7f44380fc300 msgr2=0x7f44380fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:22.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 -- 172.21.15.53:0/1499204433 shutdown_connections 2023-11-06T01:33:22.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.182+0000 7f443e85c700 1 -- 172.21.15.53:0/1499204433 wait complete. 2023-11-06T01:33:22.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T01:32:24.604601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T01:32:25.457741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:33:22.196 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T01:33:22.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.324+0000 7fc12577b700 1 -- 172.21.15.53:0/801091912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1200ffc80 msgr2=0x7fc1201000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.324+0000 7fc12577b700 1 --2- 172.21.15.53:0/801091912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1200ffc80 0x7fc1201000a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc1100099c0 tx=0x7fc110009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.324+0000 7fc12577b700 1 -- 172.21.15.53:0/801091912 shutdown_connections 2023-11-06T01:33:22.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.324+0000 7fc12577b700 1 --2- 172.21.15.53:0/801091912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc120101810 0x7fc1201080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.324+0000 7fc12577b700 1 --2- 172.21.15.53:0/801091912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1200ffc80 0x7fc1201000a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.324+0000 7fc12577b700 1 --2- 172.21.15.53:0/801091912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc120100e70 0x7fc1201012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.324+0000 7fc12577b700 1 -- 172.21.15.53:0/801091912 >> 172.21.15.53:0/801091912 conn(0x7fc1200fb1f0 msgr2=0x7fc1200fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:22.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.324+0000 7fc12577b700 1 -- 172.21.15.53:0/801091912 shutdown_connections 2023-11-06T01:33:22.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.324+0000 7fc12577b700 1 -- 172.21.15.53:0/801091912 wait complete. 2023-11-06T01:33:22.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc12577b700 1 Processor -- start 2023-11-06T01:33:22.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc12577b700 1 -- start start 2023-11-06T01:33:22.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc12577b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1200ffc80 0x7fc120068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:22.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc12577b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc120100e70 0x7fc12019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:22.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc12577b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc120101810 0x7fc12019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:22.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc12577b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12019e590 con 0x7fc120100e70 2023-11-06T01:33:22.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc12577b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12019e700 con 0x7fc1200ffc80 2023-11-06T01:33:22.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc12577b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12019e8c0 con 0x7fc120101810 2023-11-06T01:33:22.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc11e7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc120100e70 0x7fc12019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:22.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc11e7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc120100e70 0x7fc12019b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38700/0 (socket says 172.21.15.53:38700) 2023-11-06T01:33:22.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc11e7fc700 1 -- 172.21.15.53:0/1543057169 learned_addr learned my addr 172.21.15.53:0/1543057169 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:22.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.325+0000 7fc11effd700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1200ffc80 0x7fc120068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:22.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc11f7fe700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc120101810 0x7fc12019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:22.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc11e7fc700 1 -- 172.21.15.53:0/1543057169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc120101810 msgr2=0x7fc12019ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc11e7fc700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc120101810 0x7fc12019ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc11e7fc700 1 -- 172.21.15.53:0/1543057169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1200ffc80 msgr2=0x7fc120068ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc11e7fc700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1200ffc80 0x7fc120068ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc11e7fc700 1 -- 172.21.15.53:0/1543057169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1100095f0 con 0x7fc120100e70 2023-11-06T01:33:22.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc11f7fe700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc120101810 0x7fc12019ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:22.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc11e7fc700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc120100e70 0x7fc12019b760 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc10800e9a0 tx=0x7fc10800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:22.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc117fff700 1 -- 172.21.15.53:0/1543057169 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108009790 con 0x7fc120100e70 2023-11-06T01:33:22.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc117fff700 1 -- 172.21.15.53:0/1543057169 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc10800cbd0 con 0x7fc120100e70 2023-11-06T01:33:22.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc12577b700 1 -- 172.21.15.53:0/1543057169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12019eba0 con 0x7fc120100e70 2023-11-06T01:33:22.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc117fff700 1 -- 172.21.15.53:0/1543057169 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108018680 con 0x7fc120100e70 2023-11-06T01:33:22.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.326+0000 7fc12577b700 1 -- 172.21.15.53:0/1543057169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12019f0b0 con 0x7fc120100e70 2023-11-06T01:33:22.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.327+0000 7fc12577b700 1 -- 172.21.15.53:0/1543057169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc120107260 con 0x7fc120100e70 2023-11-06T01:33:22.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.328+0000 7fc117fff700 1 -- 172.21.15.53:0/1543057169 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc10800cd40 con 0x7fc120100e70 2023-11-06T01:33:22.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.329+0000 7fc117fff700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc10c0a05b0 0x7fc10c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:22.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.329+0000 7fc117fff700 1 -- 172.21.15.53:0/1543057169 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc108014070 con 0x7fc120100e70 2023-11-06T01:33:22.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.329+0000 7fc11effd700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc10c0a05b0 0x7fc10c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:22.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.329+0000 7fc11effd700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc10c0a05b0 0x7fc10c0a2a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc110009990 tx=0x7fc110005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:22.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.333+0000 7fc117fff700 1 -- 172.21.15.53:0/1543057169 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc1080520e0 con 0x7fc120100e70 2023-11-06T01:33:22.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:22 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2487153733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:22.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:22 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3205657197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:22.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:22 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2487153733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:22.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:22 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3205657197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:22.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.534+0000 7fc12577b700 1 -- 172.21.15.53:0/1543057169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc12002cec0 con 0x7fc120100e70 2023-11-06T01:33:22.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.534+0000 7fc117fff700 1 -- 172.21.15.53:0/1543057169 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1080957e0 con 0x7fc120100e70 2023-11-06T01:33:22.536 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:33:22.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 -- 172.21.15.53:0/1543057169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc10c0a05b0 msgr2=0x7fc10c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc10c0a05b0 0x7fc10c0a2a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc110009990 tx=0x7fc110005fb0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 -- 172.21.15.53:0/1543057169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc120100e70 msgr2=0x7fc12019b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc120100e70 0x7fc12019b760 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc10800e9a0 tx=0x7fc10800ecb0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 -- 172.21.15.53:0/1543057169 shutdown_connections 2023-11-06T01:33:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc10c0a05b0 0x7fc10c0a2a70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1200ffc80 0x7fc120068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc120101810 0x7fc12019ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 --2- 172.21.15.53:0/1543057169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc120100e70 0x7fc12019b760 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 -- 172.21.15.53:0/1543057169 >> 172.21.15.53:0/1543057169 conn(0x7fc1200fb1f0 msgr2=0x7fc1201043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 -- 172.21.15.53:0/1543057169 shutdown_connections 2023-11-06T01:33:22.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.537+0000 7fc12577b700 1 -- 172.21.15.53:0/1543057169 wait complete. 2023-11-06T01:33:22.549 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:22 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2487153733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:22.549 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:22 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3205657197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:22.550 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T01:33:22.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.679+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/3144684232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c4100e70 msgr2=0x7fe2c41012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.679+0000 7fe2c9ba7700 1 --2- 172.21.15.53:0/3144684232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c4100e70 0x7fe2c41012d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80099c0 tx=0x7fe2b8009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.679+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/3144684232 shutdown_connections 2023-11-06T01:33:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.679+0000 7fe2c9ba7700 1 --2- 172.21.15.53:0/3144684232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2c4101810 0x7fe2c41080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.679+0000 7fe2c9ba7700 1 --2- 172.21.15.53:0/3144684232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c40ffc80 0x7fe2c41000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.679+0000 7fe2c9ba7700 1 --2- 172.21.15.53:0/3144684232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c4100e70 0x7fe2c41012d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.679+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/3144684232 >> 172.21.15.53:0/3144684232 conn(0x7fe2c40fb1f0 msgr2=0x7fe2c40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.679+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/3144684232 shutdown_connections 2023-11-06T01:33:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.679+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/3144684232 wait complete. 2023-11-06T01:33:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c9ba7700 1 Processor -- start 2023-11-06T01:33:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c9ba7700 1 -- start start 2023-11-06T01:33:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c9ba7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c40ffc80 0x7fe2c4103fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c9ba7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c4100e70 0x7fe2c4102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c9ba7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2c4101810 0x7fe2c4102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c9ba7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c41032e0 con 0x7fe2c4100e70 2023-11-06T01:33:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c9ba7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c4103450 con 0x7fe2c4101810 2023-11-06T01:33:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c9ba7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c41a0ae0 con 0x7fe2c40ffc80 2023-11-06T01:33:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c4100e70 0x7fe2c4102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c4100e70 0x7fe2c4102600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38714/0 (socket says 172.21.15.53:38714) 2023-11-06T01:33:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c2ffd700 1 -- 172.21.15.53:0/1644859509 learned_addr learned my addr 172.21.15.53:0/1644859509 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c37fe700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c40ffc80 0x7fe2c4103fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.680+0000 7fe2c3fff700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2c4101810 0x7fe2c4102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.681+0000 7fe2c2ffd700 1 -- 172.21.15.53:0/1644859509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c40ffc80 msgr2=0x7fe2c4103fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.681+0000 7fe2c2ffd700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c40ffc80 0x7fe2c4103fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.681+0000 7fe2c2ffd700 1 -- 172.21.15.53:0/1644859509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2c4101810 msgr2=0x7fe2c4102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.681+0000 7fe2c2ffd700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2c4101810 0x7fe2c4102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.681+0000 7fe2c2ffd700 1 -- 172.21.15.53:0/1644859509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b80095f0 con 0x7fe2c4100e70 2023-11-06T01:33:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.681+0000 7fe2c2ffd700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c4100e70 0x7fe2c4102600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009fd0 tx=0x7fe2b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:22.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.681+0000 7fe2c0ff9700 1 -- 172.21.15.53:0/1644859509 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b801d070 con 0x7fe2c4100e70 2023-11-06T01:33:22.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.681+0000 7fe2c0ff9700 1 -- 172.21.15.53:0/1644859509 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2b8005030 con 0x7fe2c4100e70 2023-11-06T01:33:22.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.681+0000 7fe2c0ff9700 1 -- 172.21.15.53:0/1644859509 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8004b30 con 0x7fe2c4100e70 2023-11-06T01:33:22.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.681+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/1644859509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c41a0d60 con 0x7fe2c4100e70 2023-11-06T01:33:22.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.681+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/1644859509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c41a1180 con 0x7fe2c4100e70 2023-11-06T01:33:22.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.682+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/1644859509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c4107e00 con 0x7fe2c4100e70 2023-11-06T01:33:22.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.683+0000 7fe2c0ff9700 1 -- 172.21.15.53:0/1644859509 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fe2b8005410 con 0x7fe2c4100e70 2023-11-06T01:33:22.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.684+0000 7fe2c0ff9700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2ac0a0630 0x7fe2ac0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:22.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.684+0000 7fe2c37fe700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2ac0a0630 0x7fe2ac0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.684+0000 7fe2c0ff9700 1 -- 172.21.15.53:0/1644859509 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe2b80ccb10 con 0x7fe2c4100e70 2023-11-06T01:33:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.684+0000 7fe2c37fe700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2ac0a0630 0x7fe2ac0a2af0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4100cd0 tx=0x7fe2b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.687+0000 7fe2c0ff9700 1 -- 172.21.15.53:0/1644859509 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe2b8096790 con 0x7fe2c4100e70 2023-11-06T01:33:22.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.889+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/1644859509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2c410b500 con 0x7fe2c4100e70 2023-11-06T01:33:22.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.890+0000 7fe2c0ff9700 1 -- 172.21.15.53:0/1644859509 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2b8099db0 con 0x7fe2c4100e70 2023-11-06T01:33:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-06T01:33:22.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/1644859509 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2ac0a0630 msgr2=0x7fe2ac0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2ac0a0630 0x7fe2ac0a2af0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4100cd0 tx=0x7fe2b400b410 comp rx=0 tx=0).stop 2023-11-06T01:33:22.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/1644859509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c4100e70 msgr2=0x7fe2c4102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:22.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c4100e70 0x7fe2c4102600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009fd0 tx=0x7fe2b800f690 comp rx=0 tx=0).stop 2023-11-06T01:33:22.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/1644859509 shutdown_connections 2023-11-06T01:33:22.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2ac0a0630 0x7fe2ac0a2af0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2c4101810 0x7fe2c4102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c40ffc80 0x7fe2c4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 --2- 172.21.15.53:0/1644859509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c4100e70 0x7fe2c4102600 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:22.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/1644859509 >> 172.21.15.53:0/1644859509 conn(0x7fe2c40fb1f0 msgr2=0x7fe2c40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:22.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/1644859509 shutdown_connections 2023-11-06T01:33:22.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:22.892+0000 7fe2c9ba7700 1 -- 172.21.15.53:0/1644859509 wait complete. 2023-11-06T01:33:22.905 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T01:33:23.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.033+0000 7fdd68190700 1 -- 172.21.15.53:0/1578869021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd60100190 msgr2=0x7fdd601005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.033+0000 7fdd68190700 1 --2- 172.21.15.53:0/1578869021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd60100190 0x7fdd601005b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009990 tx=0x7fdd5c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.033+0000 7fdd68190700 1 -- 172.21.15.53:0/1578869021 shutdown_connections 2023-11-06T01:33:23.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.033+0000 7fdd68190700 1 --2- 172.21.15.53:0/1578869021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60101860 0x7fdd601080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.033+0000 7fdd68190700 1 --2- 172.21.15.53:0/1578869021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd60100190 0x7fdd601005b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.033+0000 7fdd68190700 1 --2- 172.21.15.53:0/1578869021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd60100ea0 0x7fdd60101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.033+0000 7fdd68190700 1 -- 172.21.15.53:0/1578869021 >> 172.21.15.53:0/1578869021 conn(0x7fdd600fa960 msgr2=0x7fdd600fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:23.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.033+0000 7fdd68190700 1 -- 172.21.15.53:0/1578869021 shutdown_connections 2023-11-06T01:33:23.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.033+0000 7fdd68190700 1 -- 172.21.15.53:0/1578869021 wait complete. 2023-11-06T01:33:23.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd68190700 1 Processor -- start 2023-11-06T01:33:23.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd68190700 1 -- start start 2023-11-06T01:33:23.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd68190700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60100190 0x7fdd601959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd68190700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd60100ea0 0x7fdd60195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd68190700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd60101860 0x7fdd6019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd68190700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6019d560 con 0x7fdd60100ea0 2023-11-06T01:33:23.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd68190700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6019d6a0 con 0x7fdd60101860 2023-11-06T01:33:23.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd68190700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6019d830 con 0x7fdd60100190 2023-11-06T01:33:23.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd6572b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd60100ea0 0x7fdd60195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd6572b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd60100ea0 0x7fdd60195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38728/0 (socket says 172.21.15.53:38728) 2023-11-06T01:33:23.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd6572b700 1 -- 172.21.15.53:0/3828597739 learned_addr learned my addr 172.21.15.53:0/3828597739 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:23.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd65f2c700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60100190 0x7fdd601959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd6672d700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd60101860 0x7fdd6019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd6572b700 1 -- 172.21.15.53:0/3828597739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60100190 msgr2=0x7fdd601959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd6572b700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60100190 0x7fdd601959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd6572b700 1 -- 172.21.15.53:0/3828597739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd60101860 msgr2=0x7fdd6019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.034+0000 7fdd6572b700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd60101860 0x7fdd6019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.035+0000 7fdd6572b700 1 -- 172.21.15.53:0/3828597739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0095f0 con 0x7fdd60100ea0 2023-11-06T01:33:23.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.035+0000 7fdd6672d700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd60101860 0x7fdd6019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:23.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.035+0000 7fdd6572b700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd60100ea0 0x7fdd60195f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000d980 tx=0x7fdd5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:23.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.035+0000 7fdd56ffd700 1 -- 172.21.15.53:0/3828597739 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5000b950 con 0x7fdd60100ea0 2023-11-06T01:33:23.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.035+0000 7fdd56ffd700 1 -- 172.21.15.53:0/3828597739 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd50007620 con 0x7fdd60100ea0 2023-11-06T01:33:23.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.035+0000 7fdd68190700 1 -- 172.21.15.53:0/3828597739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6019db10 con 0x7fdd60100ea0 2023-11-06T01:33:23.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.035+0000 7fdd56ffd700 1 -- 172.21.15.53:0/3828597739 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5000f3c0 con 0x7fdd60100ea0 2023-11-06T01:33:23.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.035+0000 7fdd68190700 1 -- 172.21.15.53:0/3828597739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6019e020 con 0x7fdd60100ea0 2023-11-06T01:33:23.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.036+0000 7fdd68190700 1 -- 172.21.15.53:0/3828597739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6010b4a0 con 0x7fdd60100ea0 2023-11-06T01:33:23.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.037+0000 7fdd56ffd700 1 -- 172.21.15.53:0/3828597739 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fdd50010640 con 0x7fdd60100ea0 2023-11-06T01:33:23.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.038+0000 7fdd56ffd700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdd4c0a05b0 0x7fdd4c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.038+0000 7fdd56ffd700 1 -- 172.21.15.53:0/3828597739 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdd500cc0e0 con 0x7fdd60100ea0 2023-11-06T01:33:23.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.038+0000 7fdd65f2c700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdd4c0a05b0 0x7fdd4c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.038+0000 7fdd65f2c700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdd4c0a05b0 0x7fdd4c0a2a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c006010 tx=0x7fdd5c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:23.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.041+0000 7fdd56ffd700 1 -- 172.21.15.53:0/3828597739 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdd50095cd0 con 0x7fdd60100ea0 2023-11-06T01:33:23.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.243+0000 7fdd68190700 1 -- 172.21.15.53:0/3828597739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd60106c30 con 0x7fdd60100ea0 2023-11-06T01:33:23.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.244+0000 7fdd56ffd700 1 -- 172.21.15.53:0/3828597739 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd500992f0 con 0x7fdd60100ea0 2023-11-06T01:33:23.245 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-06T01:33:23.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 -- 172.21.15.53:0/3828597739 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdd4c0a05b0 msgr2=0x7fdd4c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdd4c0a05b0 0x7fdd4c0a2a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c006010 tx=0x7fdd5c005960 comp rx=0 tx=0).stop 2023-11-06T01:33:23.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 -- 172.21.15.53:0/3828597739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd60100ea0 msgr2=0x7fdd60195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd60100ea0 0x7fdd60195f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000d980 tx=0x7fdd5000dc90 comp rx=0 tx=0).stop 2023-11-06T01:33:23.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 -- 172.21.15.53:0/3828597739 shutdown_connections 2023-11-06T01:33:23.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdd4c0a05b0 0x7fdd4c0a2a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd60101860 0x7fdd6019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60100190 0x7fdd601959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 --2- 172.21.15.53:0/3828597739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd60100ea0 0x7fdd60195f20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 -- 172.21.15.53:0/3828597739 >> 172.21.15.53:0/3828597739 conn(0x7fdd600fa960 msgr2=0x7fdd600fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:23.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 -- 172.21.15.53:0/3828597739 shutdown_connections 2023-11-06T01:33:23.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.246+0000 7fdd68190700 1 -- 172.21.15.53:0/3828597739 wait complete. 2023-11-06T01:33:23.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:23 smithi053 ceph-mon[99472]: pgmap v280: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-11-06T01:33:23.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:23 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1499204433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:23.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:23 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1543057169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:23.259 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:23 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1644859509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:23.260 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:33:23.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.385+0000 7f7b451cc700 1 -- 172.21.15.53:0/3706800215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b40101800 msgr2=0x7f7b401080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.385+0000 7f7b451cc700 1 --2- 172.21.15.53:0/3706800215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b40101800 0x7f7b401080b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800b0d0 tx=0x7f7b3800b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.385+0000 7f7b451cc700 1 -- 172.21.15.53:0/3706800215 shutdown_connections 2023-11-06T01:33:23.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.385+0000 7f7b451cc700 1 --2- 172.21.15.53:0/3706800215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b40101800 0x7f7b401080b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.385+0000 7f7b451cc700 1 --2- 172.21.15.53:0/3706800215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b400ffc70 0x7f7b40100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.385+0000 7f7b451cc700 1 --2- 172.21.15.53:0/3706800215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b40100e60 0x7f7b401012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.385+0000 7f7b451cc700 1 -- 172.21.15.53:0/3706800215 >> 172.21.15.53:0/3706800215 conn(0x7f7b400fb1c0 msgr2=0x7f7b400fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:23.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.385+0000 7f7b451cc700 1 -- 172.21.15.53:0/3706800215 shutdown_connections 2023-11-06T01:33:23.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.385+0000 7f7b451cc700 1 -- 172.21.15.53:0/3706800215 wait complete. 2023-11-06T01:33:23.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b451cc700 1 Processor -- start 2023-11-06T01:33:23.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b451cc700 1 -- start start 2023-11-06T01:33:23.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b451cc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b400ffc70 0x7f7b40196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b451cc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b40100e60 0x7f7b401970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b451cc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b40101800 0x7f7b4019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b451cc700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4019e720 con 0x7f7b40101800 2023-11-06T01:33:23.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b451cc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4019e860 con 0x7f7b400ffc70 2023-11-06T01:33:23.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b451cc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4019e9f0 con 0x7f7b40100e60 2023-11-06T01:33:23.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b3f59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b40101800 0x7f7b4019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b3f59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b40101800 0x7f7b4019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38744/0 (socket says 172.21.15.53:38744) 2023-11-06T01:33:23.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b3f59e700 1 -- 172.21.15.53:0/3126811127 learned_addr learned my addr 172.21.15.53:0/3126811127 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:23.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b3ed9d700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b400ffc70 0x7f7b40196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.386+0000 7f7b3e59c700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b40100e60 0x7f7b401970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.387+0000 7f7b3f59e700 1 -- 172.21.15.53:0/3126811127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b40100e60 msgr2=0x7f7b401970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.387+0000 7f7b3f59e700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b40100e60 0x7f7b401970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.387+0000 7f7b3f59e700 1 -- 172.21.15.53:0/3126811127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b400ffc70 msgr2=0x7f7b40196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.387+0000 7f7b3f59e700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b400ffc70 0x7f7b40196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.387+0000 7f7b3f59e700 1 -- 172.21.15.53:0/3126811127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b38009d00 con 0x7f7b40101800 2023-11-06T01:33:23.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.387+0000 7f7b3f59e700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b40101800 0x7f7b4019e150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b38008f10 tx=0x7f7b38008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:23.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.387+0000 7f7b27fff700 1 -- 172.21.15.53:0/3126811127 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800e040 con 0x7f7b40101800 2023-11-06T01:33:23.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.387+0000 7f7b27fff700 1 -- 172.21.15.53:0/3126811127 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b38006080 con 0x7f7b40101800 2023-11-06T01:33:23.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.387+0000 7f7b451cc700 1 -- 172.21.15.53:0/3126811127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4019ec70 con 0x7f7b40101800 2023-11-06T01:33:23.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.387+0000 7f7b27fff700 1 -- 172.21.15.53:0/3126811127 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3801b6c0 con 0x7f7b40101800 2023-11-06T01:33:23.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.387+0000 7f7b451cc700 1 -- 172.21.15.53:0/3126811127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4019f090 con 0x7f7b40101800 2023-11-06T01:33:23.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.388+0000 7f7b451cc700 1 -- 172.21.15.53:0/3126811127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4010b500 con 0x7f7b40101800 2023-11-06T01:33:23.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.389+0000 7f7b27fff700 1 -- 172.21.15.53:0/3126811127 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f7b38003a00 con 0x7f7b40101800 2023-11-06T01:33:23.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.390+0000 7f7b27fff700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b280a0670 0x7f7b280a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.390+0000 7f7b3ed9d700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b280a0670 0x7f7b280a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.390+0000 7f7b27fff700 1 -- 172.21.15.53:0/3126811127 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7b380cd050 con 0x7f7b40101800 2023-11-06T01:33:23.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.390+0000 7f7b3ed9d700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b280a0670 0x7f7b280a2b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b40100cc0 tx=0x7f7b30007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:23.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:23 smithi062 ceph-mon[103924]: pgmap v280: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-11-06T01:33:23.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:23 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1499204433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:23.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:23 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1543057169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:23.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:23 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1644859509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:23.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.393+0000 7f7b27fff700 1 -- 172.21.15.53:0/3126811127 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7b38096df0 con 0x7f7b40101800 2023-11-06T01:33:23.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:23 smithi163 ceph-mon[104416]: pgmap v280: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-11-06T01:33:23.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:23 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1499204433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:23.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:23 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1543057169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:23.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:23 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1644859509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:23.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.645+0000 7f7b451cc700 1 -- 172.21.15.53:0/3126811127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b4019f640 con 0x7f7b40101800 2023-11-06T01:33:23.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.646+0000 7f7b27fff700 1 -- 172.21.15.53:0/3126811127 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10815 (secure 0 0 0) 0x7f7b3809a410 con 0x7f7b40101800 2023-11-06T01:33:23.647 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:23.647 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-06T01:32:26.093194+0000","modified":"2023-11-06T01:32:33.006369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24472,"mds_2":24464,"mds_3":24422,"mds_4":24395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24395":{"gid":24395,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T01:33:23.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 -- 172.21.15.53:0/3126811127 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b280a0670 msgr2=0x7f7b280a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b280a0670 0x7f7b280a2b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b40100cc0 tx=0x7f7b30007400 comp rx=0 tx=0).stop 2023-11-06T01:33:23.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 -- 172.21.15.53:0/3126811127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b40101800 msgr2=0x7f7b4019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b40101800 0x7f7b4019e150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b38008f10 tx=0x7f7b38008f40 comp rx=0 tx=0).stop 2023-11-06T01:33:23.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 -- 172.21.15.53:0/3126811127 shutdown_connections 2023-11-06T01:33:23.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b280a0670 0x7f7b280a2b30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b400ffc70 0x7f7b40196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b40101800 0x7f7b4019e150 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 --2- 172.21.15.53:0/3126811127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b40100e60 0x7f7b401970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 -- 172.21.15.53:0/3126811127 >> 172.21.15.53:0/3126811127 conn(0x7f7b400fb1c0 msgr2=0x7f7b40067f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:23.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 -- 172.21.15.53:0/3126811127 shutdown_connections 2023-11-06T01:33:23.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.648+0000 7f7b451cc700 1 -- 172.21.15.53:0/3126811127 wait complete. 2023-11-06T01:33:23.651 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-11-06T01:33:23.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:33:23.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.786+0000 7f4bad52c700 1 -- 172.21.15.53:0/1183214554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba81008e0 msgr2=0x7f4ba8108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.786+0000 7f4bad52c700 1 --2- 172.21.15.53:0/1183214554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba81008e0 0x7f4ba8108180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000b0a0 tx=0x7f4ba000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.786+0000 7f4bad52c700 1 -- 172.21.15.53:0/1183214554 shutdown_connections 2023-11-06T01:33:23.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.786+0000 7f4bad52c700 1 --2- 172.21.15.53:0/1183214554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba81008e0 0x7f4ba8108180 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.786+0000 7f4bad52c700 1 --2- 172.21.15.53:0/1183214554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ba81012d0 0x7f4ba81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.786+0000 7f4bad52c700 1 --2- 172.21.15.53:0/1183214554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ba80fff20 0x7f4ba81003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.786+0000 7f4bad52c700 1 -- 172.21.15.53:0/1183214554 >> 172.21.15.53:0/1183214554 conn(0x7f4ba80fa9f0 msgr2=0x7f4ba80fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:23.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.786+0000 7f4bad52c700 1 -- 172.21.15.53:0/1183214554 shutdown_connections 2023-11-06T01:33:23.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.786+0000 7f4bad52c700 1 -- 172.21.15.53:0/1183214554 wait complete. 2023-11-06T01:33:23.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.787+0000 7f4bad52c700 1 Processor -- start 2023-11-06T01:33:23.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.787+0000 7f4bad52c700 1 -- start start 2023-11-06T01:33:23.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.787+0000 7f4bad52c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ba80fff20 0x7f4ba80ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.787+0000 7f4bad52c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ba81008e0 0x7f4ba819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.787+0000 7f4bad52c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba81012d0 0x7f4ba819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.787+0000 7f4bad52c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba819e5a0 con 0x7f4ba81012d0 2023-11-06T01:33:23.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.787+0000 7f4bad52c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba819e710 con 0x7f4ba81008e0 2023-11-06T01:33:23.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.787+0000 7f4bad52c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba819e8d0 con 0x7f4ba80fff20 2023-11-06T01:33:23.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.787+0000 7f4ba77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba81012d0 0x7f4ba819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.787+0000 7f4ba77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba81012d0 0x7f4ba819ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38758/0 (socket says 172.21.15.53:38758) 2023-11-06T01:33:23.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4ba77fe700 1 -- 172.21.15.53:0/1696108142 learned_addr learned my addr 172.21.15.53:0/1696108142 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:23.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4ba6ffd700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ba80fff20 0x7f4ba80ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4ba67fc700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ba81008e0 0x7f4ba819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4ba77fe700 1 -- 172.21.15.53:0/1696108142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ba80fff20 msgr2=0x7f4ba80ff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4ba77fe700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ba80fff20 0x7f4ba80ff9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4ba77fe700 1 -- 172.21.15.53:0/1696108142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ba81008e0 msgr2=0x7f4ba819b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:23.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4ba77fe700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ba81008e0 0x7f4ba819b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:23.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4ba77fe700 1 -- 172.21.15.53:0/1696108142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba0009d00 con 0x7f4ba81012d0 2023-11-06T01:33:23.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4ba77fe700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba81012d0 0x7f4ba819ddd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00062a0 tx=0x7f4ba0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:23.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4b8ffff700 1 -- 172.21.15.53:0/1696108142 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000e040 con 0x7f4ba81012d0 2023-11-06T01:33:23.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4b8ffff700 1 -- 172.21.15.53:0/1696108142 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ba000cec0 con 0x7f4ba81012d0 2023-11-06T01:33:23.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4bad52c700 1 -- 172.21.15.53:0/1696108142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba819eb50 con 0x7f4ba81012d0 2023-11-06T01:33:23.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4b8ffff700 1 -- 172.21.15.53:0/1696108142 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba001baf0 con 0x7f4ba81012d0 2023-11-06T01:33:23.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.788+0000 7f4bad52c700 1 -- 172.21.15.53:0/1696108142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba819ef80 con 0x7f4ba81012d0 2023-11-06T01:33:23.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.789+0000 7f4bad52c700 1 -- 172.21.15.53:0/1696108142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba810b4c0 con 0x7f4ba81012d0 2023-11-06T01:33:23.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.791+0000 7f4b8ffff700 1 -- 172.21.15.53:0/1696108142 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f4ba001bc50 con 0x7f4ba81012d0 2023-11-06T01:33:23.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.791+0000 7f4b8ffff700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b900a05f0 0x7f4b900a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:23.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.791+0000 7f4ba6ffd700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b900a05f0 0x7f4b900a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:23.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.791+0000 7f4b8ffff700 1 -- 172.21.15.53:0/1696108142 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4ba00cda70 con 0x7f4ba81012d0 2023-11-06T01:33:23.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.792+0000 7f4ba6ffd700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b900a05f0 0x7f4b900a2ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8068f70 tx=0x7f4b9800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:23.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.795+0000 7f4b8ffff700 1 -- 172.21.15.53:0/1696108142 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4ba00976f0 con 0x7f4ba81012d0 2023-11-06T01:33:23.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.994+0000 7f4bad52c700 1 -- 172.21.15.53:0/1696108142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ba8106af0 con 0x7f4ba81012d0 2023-11-06T01:33:23.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.996+0000 7f4b8ffff700 1 -- 172.21.15.53:0/1696108142 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f4ba009ad10 con 0x7f4ba81012d0 2023-11-06T01:33:23.997 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:23.998 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:32:27.094591+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:33:24.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 -- 172.21.15.53:0/1696108142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b900a05f0 msgr2=0x7f4b900a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b900a05f0 0x7f4b900a2ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8068f70 tx=0x7f4b9800b410 comp rx=0 tx=0).stop 2023-11-06T01:33:24.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 -- 172.21.15.53:0/1696108142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba81012d0 msgr2=0x7f4ba819ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba81012d0 0x7f4ba819ddd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00062a0 tx=0x7f4ba0004d00 comp rx=0 tx=0).stop 2023-11-06T01:33:24.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 -- 172.21.15.53:0/1696108142 shutdown_connections 2023-11-06T01:33:24.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b900a05f0 0x7f4b900a2ab0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba81012d0 0x7f4ba819ddd0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ba80fff20 0x7f4ba80ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 --2- 172.21.15.53:0/1696108142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ba81008e0 0x7f4ba819b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 -- 172.21.15.53:0/1696108142 >> 172.21.15.53:0/1696108142 conn(0x7f4ba80fa9f0 msgr2=0x7f4ba80fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 -- 172.21.15.53:0/1696108142 shutdown_connections 2023-11-06T01:33:24.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:23.998+0000 7f4bad52c700 1 -- 172.21.15.53:0/1696108142 wait complete. 2023-11-06T01:33:24.011 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:33:24.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.138+0000 7f567e844700 1 -- 172.21.15.53:0/3935598802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5678101800 msgr2=0x7f56781080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.138+0000 7f567e844700 1 --2- 172.21.15.53:0/3935598802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5678101800 0x7f56781080b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f567000b0d0 tx=0x7f567000b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.138+0000 7f567e844700 1 -- 172.21.15.53:0/3935598802 shutdown_connections 2023-11-06T01:33:24.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.138+0000 7f567e844700 1 --2- 172.21.15.53:0/3935598802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56780ffc70 0x7f5678100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.138+0000 7f567e844700 1 --2- 172.21.15.53:0/3935598802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5678101800 0x7f56781080b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.138+0000 7f567e844700 1 --2- 172.21.15.53:0/3935598802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5678100e60 0x7f56781012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.138+0000 7f567e844700 1 -- 172.21.15.53:0/3935598802 >> 172.21.15.53:0/3935598802 conn(0x7f56780fb1c0 msgr2=0x7f56780fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.138+0000 7f567e844700 1 -- 172.21.15.53:0/3935598802 shutdown_connections 2023-11-06T01:33:24.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.138+0000 7f567e844700 1 -- 172.21.15.53:0/3935598802 wait complete. 2023-11-06T01:33:24.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.138+0000 7f567e844700 1 Processor -- start 2023-11-06T01:33:24.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.138+0000 7f567e844700 1 -- start start 2023-11-06T01:33:24.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f567e844700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56780ffc70 0x7f5678068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f567e844700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5678100e60 0x7f567819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f567e844700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5678101800 0x7f567819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f567e844700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567819e630 con 0x7f5678100e60 2023-11-06T01:33:24.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f567e844700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567819e7a0 con 0x7f56780ffc70 2023-11-06T01:33:24.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f567e844700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567819e960 con 0x7f5678101800 2023-11-06T01:33:24.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f56777fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5678100e60 0x7f567819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f56777fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5678100e60 0x7f567819b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38770/0 (socket says 172.21.15.53:38770) 2023-11-06T01:33:24.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f56777fe700 1 -- 172.21.15.53:0/3822321107 learned_addr learned my addr 172.21.15.53:0/3822321107 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f567cde1700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5678101800 0x7f567819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f5677fff700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56780ffc70 0x7f5678068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f56777fe700 1 -- 172.21.15.53:0/3822321107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5678101800 msgr2=0x7f567819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f56777fe700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5678101800 0x7f567819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f56777fe700 1 -- 172.21.15.53:0/3822321107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56780ffc70 msgr2=0x7f5678068a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f56777fe700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56780ffc70 0x7f5678068a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f56777fe700 1 -- 172.21.15.53:0/3822321107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5670009d00 con 0x7f5678100e60 2023-11-06T01:33:24.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f5677fff700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56780ffc70 0x7f5678068a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.139+0000 7f56777fe700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5678100e60 0x7f567819b810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c920 tx=0x7f566c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.140+0000 7f56757fa700 1 -- 172.21.15.53:0/3822321107 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c007b00 con 0x7f5678100e60 2023-11-06T01:33:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.140+0000 7f56757fa700 1 -- 172.21.15.53:0/3822321107 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f566c00ce70 con 0x7f5678100e60 2023-11-06T01:33:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.140+0000 7f567e844700 1 -- 172.21.15.53:0/3822321107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567819ec40 con 0x7f5678100e60 2023-11-06T01:33:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.140+0000 7f56757fa700 1 -- 172.21.15.53:0/3822321107 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c0054e0 con 0x7f5678100e60 2023-11-06T01:33:24.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.140+0000 7f567e844700 1 -- 172.21.15.53:0/3822321107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567819f150 con 0x7f5678100e60 2023-11-06T01:33:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.141+0000 7f567e844700 1 -- 172.21.15.53:0/3822321107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5678107160 con 0x7f5678100e60 2023-11-06T01:33:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.142+0000 7f56757fa700 1 -- 172.21.15.53:0/3822321107 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f566c004750 con 0x7f5678100e60 2023-11-06T01:33:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.142+0000 7f56757fa700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56600a05f0 0x7f56600a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.143+0000 7f56757fa700 1 -- 172.21.15.53:0/3822321107 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f566c0cb860 con 0x7f5678100e60 2023-11-06T01:33:24.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.143+0000 7f5677fff700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56600a05f0 0x7f56600a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.143+0000 7f5677fff700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56600a05f0 0x7f56600a2ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5678100cc0 tx=0x7f5668009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.146+0000 7f56757fa700 1 -- 172.21.15.53:0/3822321107 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f566c095450 con 0x7f5678100e60 2023-11-06T01:33:24.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:24 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3828597739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:24.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:24 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3126811127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:24.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:24 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1696108142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:24.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.392+0000 7f567e844700 1 -- 172.21.15.53:0/3822321107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f567802cc70 con 0x7f5678100e60 2023-11-06T01:33:24.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.392+0000 7f56757fa700 1 -- 172.21.15.53:0/3822321107 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10815 (secure 0 0 0) 0x7f566c01d020 con 0x7f5678100e60 2023-11-06T01:33:24.393 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:24.394 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-06T01:32:26.093194+0000","modified":"2023-11-06T01:32:33.006369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24472,"mds_2":24464,"mds_3":24422,"mds_4":24395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24395":{"gid":24395,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T01:33:24.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:24 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3828597739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:24.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:24 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3126811127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:24.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:24 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1696108142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:24.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.394+0000 7f567e844700 1 -- 172.21.15.53:0/3822321107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56600a05f0 msgr2=0x7f56600a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.394+0000 7f567e844700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56600a05f0 0x7f56600a2ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5678100cc0 tx=0x7f5668009670 comp rx=0 tx=0).stop 2023-11-06T01:33:24.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.394+0000 7f567e844700 1 -- 172.21.15.53:0/3822321107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5678100e60 msgr2=0x7f567819b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.394+0000 7f567e844700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5678100e60 0x7f567819b810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c920 tx=0x7f566c00cc30 comp rx=0 tx=0).stop 2023-11-06T01:33:24.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.395+0000 7f567e844700 1 -- 172.21.15.53:0/3822321107 shutdown_connections 2023-11-06T01:33:24.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.395+0000 7f567e844700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56600a05f0 0x7f56600a2ab0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.395+0000 7f567e844700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56780ffc70 0x7f5678068a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.395+0000 7f567e844700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5678101800 0x7f567819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.395+0000 7f567e844700 1 --2- 172.21.15.53:0/3822321107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5678100e60 0x7f567819b810 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.395+0000 7f567e844700 1 -- 172.21.15.53:0/3822321107 >> 172.21.15.53:0/3822321107 conn(0x7f56780fb1c0 msgr2=0x7f56781042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.395+0000 7f567e844700 1 -- 172.21.15.53:0/3822321107 shutdown_connections 2023-11-06T01:33:24.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.395+0000 7f567e844700 1 -- 172.21.15.53:0/3822321107 wait complete. 2023-11-06T01:33:24.400 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-11-06T01:33:24.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-06T01:32:26.093194+0000', 'modified': '2023-11-06T01:32:33.006369+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24487, 'mds_1': 24472, 'mds_2': 24464, 'mds_3': 24422, 'mds_4': 24395}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6839/1353368498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 1353368498}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 1353368498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6841/1972292821', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6840', 'nonce': 1972292821}, {'type': 'v1', 'addr': '172.21.15.53:6841', 'nonce': 1972292821}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24395': {'gid': 24395, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.62:6833/585581012', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 585581012}, {'type': 'v1', 'addr': '172.21.15.62:6833', 'nonce': 585581012}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.62:6835/3901057483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 3901057483}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 3901057483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6833/731722016', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6832', 'nonce': 731722016}, {'type': 'v1', 'addr': '172.21.15.163:6833', 'nonce': 731722016}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/644941927', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 644941927}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 644941927}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6835/392449784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 392449784}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 392449784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.62:6839/1997633670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 1997633670}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 1997633670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.163:6837/1759271265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 1759271265}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 1759271265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.163:6839/3554773881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6838', 'nonce': 3554773881}, {'type': 'v1', 'addr': '172.21.15.163:6839', 'nonce': 3554773881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T01:33:24.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-11-06T01:33:24.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-11-06T01:33:24.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:24 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3828597739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:24.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:24 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3126811127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:24.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:24 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1696108142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:24.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.535+0000 7f4a046a2700 1 -- 172.21.15.53:0/121415977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc1005a0 msgr2=0x7f49fc100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.535+0000 7f4a046a2700 1 --2- 172.21.15.53:0/121415977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc1005a0 0x7f49fc100a00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0099c0 tx=0x7f49ec009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.535+0000 7f4a046a2700 1 -- 172.21.15.53:0/121415977 shutdown_connections 2023-11-06T01:33:24.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.535+0000 7f4a046a2700 1 --2- 172.21.15.53:0/121415977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49fc100f40 0x7f49fc1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.535+0000 7f4a046a2700 1 --2- 172.21.15.53:0/121415977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49fc0ff3b0 0x7f49fc0ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.535+0000 7f4a046a2700 1 --2- 172.21.15.53:0/121415977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc1005a0 0x7f49fc100a00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.535+0000 7f4a046a2700 1 -- 172.21.15.53:0/121415977 >> 172.21.15.53:0/121415977 conn(0x7f49fc0fa920 msgr2=0x7f49fc0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a046a2700 1 -- 172.21.15.53:0/121415977 shutdown_connections 2023-11-06T01:33:24.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a046a2700 1 -- 172.21.15.53:0/121415977 wait complete. 2023-11-06T01:33:24.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a046a2700 1 Processor -- start 2023-11-06T01:33:24.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a046a2700 1 -- start start 2023-11-06T01:33:24.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a046a2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0ff3b0 0x7f49fc103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a046a2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49fc1005a0 0x7f49fc101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a046a2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49fc100f40 0x7f49fc101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a046a2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc102660 con 0x7f49fc0ff3b0 2023-11-06T01:33:24.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a046a2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc1027d0 con 0x7f49fc1005a0 2023-11-06T01:33:24.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a046a2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc1a0ab0 con 0x7f49fc100f40 2023-11-06T01:33:24.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a0243e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0ff3b0 0x7f49fc103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a0243e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0ff3b0 0x7f49fc103340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38782/0 (socket says 172.21.15.53:38782) 2023-11-06T01:33:24.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a0243e700 1 -- 172.21.15.53:0/3575784520 learned_addr learned my addr 172.21.15.53:0/3575784520 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.536+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49fc1005a0 0x7f49fc101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f4a02c3f700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49fc100f40 0x7f49fc101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f4a0243e700 1 -- 172.21.15.53:0/3575784520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49fc100f40 msgr2=0x7f49fc101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f4a0243e700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49fc100f40 0x7f49fc101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f4a0243e700 1 -- 172.21.15.53:0/3575784520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49fc1005a0 msgr2=0x7f49fc101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f4a0243e700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49fc1005a0 0x7f49fc101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f4a0243e700 1 -- 172.21.15.53:0/3575784520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec0095f0 con 0x7f49fc0ff3b0 2023-11-06T01:33:24.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f4a0243e700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0ff3b0 0x7f49fc103340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f49f800ddf0 tx=0x7f49f800a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f49f37fe700 1 -- 172.21.15.53:0/3575784520 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f800de20 con 0x7f49fc0ff3b0 2023-11-06T01:33:24.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f4a046a2700 1 -- 172.21.15.53:0/3575784520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49fc1a0d90 con 0x7f49fc0ff3b0 2023-11-06T01:33:24.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f49f37fe700 1 -- 172.21.15.53:0/3575784520 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f800b740 con 0x7f49fc0ff3b0 2023-11-06T01:33:24.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f49f37fe700 1 -- 172.21.15.53:0/3575784520 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f800e450 con 0x7f49fc0ff3b0 2023-11-06T01:33:24.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.537+0000 7f4a046a2700 1 -- 172.21.15.53:0/3575784520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc1a1210 con 0x7f49fc0ff3b0 2023-11-06T01:33:24.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.539+0000 7f49f37fe700 1 -- 172.21.15.53:0/3575784520 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f49f8003680 con 0x7f49fc0ff3b0 2023-11-06T01:33:24.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.540+0000 7f49f37fe700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49e80a05b0 0x7f49e80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.540+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49e80a05b0 0x7f49e80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.540+0000 7f49f37fe700 1 -- 172.21.15.53:0/3575784520 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f49f80cb780 con 0x7f49fc0ff3b0 2023-11-06T01:33:24.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.540+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49e80a05b0 0x7f49e80a2a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00b580 tx=0x7f49ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a10832b80 1 Processor -- start 2023-11-06T01:33:24.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a10832b80 1 -- start start 2023-11-06T01:33:24.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1940a0 0x5628dd1944e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd194a20 0x5628dd192e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1933a0 0x5628dd193820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a10832b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dd0ea9c0 con 0x5628dd194a20 2023-11-06T01:33:24.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a10832b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dd0e8380 con 0x5628dd1940a0 2023-11-06T01:33:24.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a10832b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dd0e3c70 con 0x5628dd1933a0 2023-11-06T01:33:24.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a01c3d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd194a20 0x5628dd192e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a01c3d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd194a20 0x5628dd192e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38792/0 (socket says 172.21.15.53:38792) 2023-11-06T01:33:24.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a01c3d700 1 -- 172.21.15.53:0/3100002444 learned_addr learned my addr 172.21.15.53:0/3100002444 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a0243e700 1 --2- 172.21.15.53:0/3100002444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1940a0 0x5628dd1944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.541+0000 7f4a02c3f700 1 --2- 172.21.15.53:0/3100002444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1933a0 0x5628dd193820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.542+0000 7f4a01c3d700 1 -- 172.21.15.53:0/3100002444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1933a0 msgr2=0x5628dd193820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.542+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/3100002444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1933a0 0x5628dd193820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.542+0000 7f4a01c3d700 1 -- 172.21.15.53:0/3100002444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1940a0 msgr2=0x5628dd1944e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.542+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/3100002444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1940a0 0x5628dd1944e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.542+0000 7f4a01c3d700 1 -- 172.21.15.53:0/3100002444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628dcf11d90 con 0x5628dd194a20 2023-11-06T01:33:24.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.542+0000 7f4a02c3f700 1 --2- 172.21.15.53:0/3100002444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1933a0 0x5628dd193820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:24.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.542+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/3100002444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd194a20 0x5628dd192e10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f49ec015000 tx=0x7f49ec01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff8ccd92ec20a03 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.542+0000 7f49f0ff9700 1 -- 172.21.15.53:0/3100002444 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec029070 con 0x5628dd194a20 2023-11-06T01:33:24.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.542+0000 7f49f0ff9700 1 -- 172.21.15.53:0/3100002444 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49ec01e040 con 0x5628dd194a20 2023-11-06T01:33:24.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.542+0000 7f49f0ff9700 1 -- 172.21.15.53:0/3100002444 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00f7c0 con 0x5628dd194a20 2023-11-06T01:33:24.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.543+0000 7f4a10832b80 1 -- 172.21.15.53:0/3100002444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd194a20 msgr2=0x5628dd192e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.543+0000 7f4a10832b80 1 --2- 172.21.15.53:0/3100002444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd194a20 0x5628dd192e10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f49ec015000 tx=0x7f49ec01f000 comp rx=0 tx=0).stop 2023-11-06T01:33:24.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.543+0000 7f4a10832b80 1 -- 172.21.15.53:0/3100002444 shutdown_connections 2023-11-06T01:33:24.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.543+0000 7f4a10832b80 1 --2- 172.21.15.53:0/3100002444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1940a0 0x5628dd1944e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.543+0000 7f4a10832b80 1 --2- 172.21.15.53:0/3100002444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1933a0 0x5628dd193820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.543+0000 7f4a10832b80 1 --2- 172.21.15.53:0/3100002444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd194a20 0x5628dd192e10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.543+0000 7f4a10832b80 1 -- 172.21.15.53:0/3100002444 >> 172.21.15.53:0/3100002444 conn(0x5628dd0f2080 msgr2=0x5628dd18d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.543+0000 7f4a10832b80 1 -- 172.21.15.53:0/3100002444 shutdown_connections 2023-11-06T01:33:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.543+0000 7f4a10832b80 1 -- 172.21.15.53:0/3100002444 wait complete. 2023-11-06T01:33:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.543+0000 7f4a10832b80 1 -- start start 2023-11-06T01:33:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.543+0000 7f4a10832b80 1 Processor -- start 2023-11-06T01:33:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1933a0 0x5628dd19c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd194a20 0x5628dd1a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a10832b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dd0804c0 con 0x5628dd1940a0 2023-11-06T01:33:24.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a10832b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dd080160 con 0x5628dd194a20 2023-11-06T01:33:24.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a10832b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dcf678d0 con 0x5628dd1933a0 2023-11-06T01:33:24.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a10832b80 10 client.0 _ll_register_callbacks cb 0x5628dcfcda30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:24.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a01c3d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a01c3d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38798/0 (socket says 172.21.15.53:38798) 2023-11-06T01:33:24.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a01c3d700 1 -- 172.21.15.53:0/461221922 learned_addr learned my addr 172.21.15.53:0/461221922 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a02c3f700 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd194a20 0x5628dd1a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a0243e700 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1933a0 0x5628dd19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a01c3d700 1 -- 172.21.15.53:0/461221922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1933a0 msgr2=0x5628dd19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1933a0 0x5628dd19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a01c3d700 1 -- 172.21.15.53:0/461221922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd194a20 msgr2=0x5628dd1a3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd194a20 0x5628dd1a3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a01c3d700 1 -- 172.21.15.53:0/461221922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc108140 con 0x5628dd1940a0 2023-11-06T01:33:24.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a02c3f700 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd194a20 0x5628dd1a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:24.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd19cdb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f49ec02f040 tx=0x7f49ec00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3097cd783a0681d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f49e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:24.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.544+0000 7f49e5ffb700 1 -- 172.21.15.53:0/461221922 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec029040 con 0x5628dd1940a0 2023-11-06T01:33:24.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f49e5ffb700 1 -- 172.21.15.53:0/461221922 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f49ec0037d0 con 0x5628dd1940a0 2023-11-06T01:33:24.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f4a10832b80 1 -- 172.21.15.53:0/461221922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628dd10ecc0 con 0x5628dd1940a0 2023-11-06T01:33:24.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f49e5ffb700 1 -- 172.21.15.53:0/461221922 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49ec003990 con 0x5628dd1940a0 2023-11-06T01:33:24.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f4a10832b80 10 client.15036 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:24.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f4a10832b80 1 -- 172.21.15.53:0/461221922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5628dcefac10 con 0x5628dd1940a0 2023-11-06T01:33:24.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f49e5ffb700 1 -- 172.21.15.53:0/461221922 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5628dd10ecc0 con 0x5628dd1940a0 2023-11-06T01:33:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f49e5ffb700 1 -- 172.21.15.53:0/461221922 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f49ec037400 con 0x5628dd1940a0 2023-11-06T01:33:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f4a10832b80 10 client.15036 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f4a10832b80 10 client.15036 resolve_mds: resolved name 'k' to daemon mds.k 2023-11-06T01:33:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f4a10832b80 20 client.15036 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f4a10832b80 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x5628dd1a7c10 0x5628dd1a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f4a10832b80 4 client.15036 mds_command: new command op to 24395 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f4a10832b80 1 -- 172.21.15.53:0/461221922 --> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628dd161dc0 con 0x5628dd1a7c10 2023-11-06T01:33:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.545+0000 7f4a0243e700 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x5628dd1a7c10 0x5628dd1a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.546+0000 7f49e5ffb700 10 client.15036 ms_handle_connect on v2:172.21.15.62:6832/585581012 2023-11-06T01:33:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.546+0000 7f4a0243e700 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x5628dd1a7c10 0x5628dd1a6d20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dfa9358a3d52cf72 server_cookie=bed5f5322654890a in_seq=0 out_seq=0 2023-11-06T01:33:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f49e5ffb700 1 -- 172.21.15.53:0/461221922 <== mds.4 v2:172.21.15.62:6832/585581012 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5628dd161dc0 con 0x5628dd1a7c10 2023-11-06T01:33:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f49e5ffb700 10 client.15036 handle_command_reply: tid=0 2023-11-06T01:33:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 1 client.15036 shutdown 2023-11-06T01:33:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 1 -- 172.21.15.53:0/461221922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 msgr2=0x5628dd19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd19cdb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f49ec02f040 tx=0x7f49ec00fa80 comp rx=0 tx=0).stop 2023-11-06T01:33:24.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 1 -- 172.21.15.53:0/461221922 shutdown_connections 2023-11-06T01:33:24.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x5628dd1a7c10 0x5628dd1a6d20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd194a20 0x5628dd1a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f49e5ffb700 0 client.15036 ms_handle_reset on v2:172.21.15.62:6832/585581012 2023-11-06T01:33:24.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1933a0 0x5628dd19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 1 --2- 172.21.15.53:0/461221922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd19cdb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 1 -- 172.21.15.53:0/461221922 >> 172.21.15.53:0/461221922 conn(0x5628dd0f2080 msgr2=0x5628dd18d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 1 -- 172.21.15.53:0/461221922 shutdown_connections 2023-11-06T01:33:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 1 -- 172.21.15.53:0/461221922 wait complete. 2023-11-06T01:33:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.547+0000 7f4a10832b80 20 client.15036 trim_cache size 0 max 16384 2023-11-06T01:33:24.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a10832b80 1 Processor -- start 2023-11-06T01:33:24.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a10832b80 1 -- start start 2023-11-06T01:33:24.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd0f2080 0x5628dd19a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 0x5628dd1b0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1940a0 0x5628dd19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a10832b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dd1a73a0 con 0x5628dd0f2080 2023-11-06T01:33:24.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a10832b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dd161dc0 con 0x5628dd1933a0 2023-11-06T01:33:24.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a10832b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dd08d830 con 0x5628dd1940a0 2023-11-06T01:33:24.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a0243e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd0f2080 0x5628dd19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a0243e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd0f2080 0x5628dd19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38804/0 (socket says 172.21.15.53:38804) 2023-11-06T01:33:24.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a0243e700 1 -- 172.21.15.53:0/2621255409 learned_addr learned my addr 172.21.15.53:0/2621255409 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/2621255409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 0x5628dd1b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a0243e700 1 -- 172.21.15.53:0/2621255409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1940a0 msgr2=0x5628dd19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a02c3f700 1 --2- 172.21.15.53:0/2621255409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1940a0 0x5628dd19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a0243e700 1 --2- 172.21.15.53:0/2621255409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1940a0 0x5628dd19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a0243e700 1 -- 172.21.15.53:0/2621255409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 msgr2=0x5628dd1b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a0243e700 1 --2- 172.21.15.53:0/2621255409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 0x5628dd1b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a0243e700 1 -- 172.21.15.53:0/2621255409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628dcf38200 con 0x5628dd0f2080 2023-11-06T01:33:24.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a02c3f700 1 --2- 172.21.15.53:0/2621255409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1940a0 0x5628dd19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:24.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.564+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/2621255409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 0x5628dd1b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:24.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f4a0243e700 1 --2- 172.21.15.53:0/2621255409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd0f2080 0x5628dd19a740 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49f8000c00 tx=0x7f49f8010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f8960db0a49d83 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f49f17fa700 1 -- 172.21.15.53:0/2621255409 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f80974e0 con 0x5628dd0f2080 2023-11-06T01:33:24.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f49f17fa700 1 -- 172.21.15.53:0/2621255409 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f8098630 con 0x5628dd0f2080 2023-11-06T01:33:24.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f49f17fa700 1 -- 172.21.15.53:0/2621255409 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f80980a0 con 0x5628dd0f2080 2023-11-06T01:33:24.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f4a10832b80 1 -- 172.21.15.53:0/2621255409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd0f2080 msgr2=0x5628dd19a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f4a10832b80 1 --2- 172.21.15.53:0/2621255409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd0f2080 0x5628dd19a740 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49f8000c00 tx=0x7f49f8010040 comp rx=0 tx=0).stop 2023-11-06T01:33:24.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f4a10832b80 1 -- 172.21.15.53:0/2621255409 shutdown_connections 2023-11-06T01:33:24.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f4a10832b80 1 --2- 172.21.15.53:0/2621255409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd1940a0 0x5628dd19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f4a10832b80 1 --2- 172.21.15.53:0/2621255409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd0f2080 0x5628dd19a740 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f4a10832b80 1 --2- 172.21.15.53:0/2621255409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 0x5628dd1b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f4a10832b80 1 -- 172.21.15.53:0/2621255409 >> 172.21.15.53:0/2621255409 conn(0x5628dd18dc20 msgr2=0x5628dd1972c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f4a10832b80 1 -- 172.21.15.53:0/2621255409 shutdown_connections 2023-11-06T01:33:24.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.565+0000 7f4a10832b80 1 -- 172.21.15.53:0/2621255409 wait complete. 2023-11-06T01:33:24.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a10832b80 1 -- start start 2023-11-06T01:33:24.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a10832b80 1 Processor -- start 2023-11-06T01:33:24.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd0f2080 0x5628dd1ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 0x5628dd1f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a10832b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd1f9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a10832b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dd1f9b70 con 0x5628dd1940a0 2023-11-06T01:33:24.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a10832b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dd1f9ce0 con 0x5628dd1933a0 2023-11-06T01:33:24.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a10832b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628dd1f9ea0 con 0x5628dd0f2080 2023-11-06T01:33:24.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a10832b80 10 client.0 _ll_register_callbacks cb 0x7f49f8094d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:24.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a02c3f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd1f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a02c3f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd1f9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38814/0 (socket says 172.21.15.53:38814) 2023-11-06T01:33:24.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a02c3f700 1 -- 172.21.15.53:0/1157147445 learned_addr learned my addr 172.21.15.53:0/1157147445 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 0x5628dd1f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a0243e700 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd0f2080 0x5628dd1ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a02c3f700 1 -- 172.21.15.53:0/1157147445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd0f2080 msgr2=0x5628dd1ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a02c3f700 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd0f2080 0x5628dd1ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a02c3f700 1 -- 172.21.15.53:0/1157147445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 msgr2=0x5628dd1f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a02c3f700 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 0x5628dd1f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a02c3f700 1 -- 172.21.15.53:0/1157147445 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc1a1210 con 0x5628dd1940a0 2023-11-06T01:33:24.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a01c3d700 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 0x5628dd1f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:24.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.566+0000 7f4a0243e700 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd0f2080 0x5628dd1ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:24.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f4a02c3f700 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd1f9630 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f49f4008e50 tx=0x7f49f4008f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787e938c9a194ff4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f49e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:24.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f49e5ffb700 1 -- 172.21.15.53:0/1157147445 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f400e030 con 0x5628dd1940a0 2023-11-06T01:33:24.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f49e5ffb700 1 -- 172.21.15.53:0/1157147445 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f49f4007dd0 con 0x5628dd1940a0 2023-11-06T01:33:24.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f4a10832b80 1 -- 172.21.15.53:0/1157147445 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49ec01e040 con 0x5628dd1940a0 2023-11-06T01:33:24.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f49e5ffb700 1 -- 172.21.15.53:0/1157147445 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49f4017440 con 0x5628dd1940a0 2023-11-06T01:33:24.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f4a10832b80 10 client.15042 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:24.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f4a10832b80 1 -- 172.21.15.53:0/1157147445 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5628dcf10260 con 0x5628dd1940a0 2023-11-06T01:33:24.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f49e5ffb700 1 -- 172.21.15.53:0/1157147445 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49ec01e040 con 0x5628dd1940a0 2023-11-06T01:33:24.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f49e5ffb700 1 -- 172.21.15.53:0/1157147445 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f49f4014050 con 0x5628dd1940a0 2023-11-06T01:33:24.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f4a10832b80 10 client.15042 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:24.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f4a10832b80 10 client.15042 resolve_mds: resolved name 'k' to daemon mds.k 2023-11-06T01:33:24.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f4a10832b80 20 client.15042 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:24.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f4a10832b80 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x5628dd194a20 0x5628dd1fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f4a10832b80 4 client.15042 mds_command: new command op to 24395 tid=0[{"prefix": "status"}] 2023-11-06T01:33:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.567+0000 7f4a10832b80 1 -- 172.21.15.53:0/1157147445 --> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5628dd0dccf0 con 0x5628dd194a20 2023-11-06T01:33:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a0243e700 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x5628dd194a20 0x5628dd1fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f49e5ffb700 10 client.15042 ms_handle_connect on v2:172.21.15.62:6832/585581012 2023-11-06T01:33:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a0243e700 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x5628dd194a20 0x5628dd1fb5e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2be0436bf1547186 server_cookie=8e4548af943b2eb5 in_seq=0 out_seq=0 2023-11-06T01:33:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f49e5ffb700 1 -- 172.21.15.53:0/1157147445 <== mds.4 v2:172.21.15.62:6832/585581012 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5628dd0dccf0 con 0x5628dd194a20 2023-11-06T01:33:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f49e5ffb700 10 client.15042 handle_command_reply: tid=0 2023-11-06T01:33:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 1 client.15042 shutdown 2023-11-06T01:33:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 1 -- 172.21.15.53:0/1157147445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 msgr2=0x5628dd1f9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd1f9630 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f49f4008e50 tx=0x7f49f4008f30 comp rx=0 tx=0).stop 2023-11-06T01:33:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 1 -- 172.21.15.53:0/1157147445 shutdown_connections 2023-11-06T01:33:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628dd1940a0 0x5628dd1f9630 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5628dd0f2080 0x5628dd1ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x5628dd194a20 0x5628dd1fb5e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 1 --2- 172.21.15.53:0/1157147445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628dd1933a0 0x5628dd1f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f49e5ffb700 0 client.15042 ms_handle_reset on v2:172.21.15.62:6832/585581012 2023-11-06T01:33:24.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 1 -- 172.21.15.53:0/1157147445 >> 172.21.15.53:0/1157147445 conn(0x5628dd18dc20 msgr2=0x5628dd19a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 1 -- 172.21.15.53:0/1157147445 shutdown_connections 2023-11-06T01:33:24.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 1 -- 172.21.15.53:0/1157147445 wait complete. 2023-11-06T01:33:24.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.568+0000 7f4a10832b80 20 client.15042 trim_cache size 0 max 16384 2023-11-06T01:33:24.593 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:24.593 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:33:24.593 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 4, 2023-11-06T01:33:24.593 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24395, 2023-11-06T01:33:24.593 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-06T01:33:24.593 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-06T01:33:24.594 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-06T01:33:24.594 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 52.565309847000002, 2023-11-06T01:33:24.594 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 27, 2023-11-06T01:33:24.594 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 82, 2023-11-06T01:33:24.594 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 82, 2023-11-06T01:33:24.595 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 85.701428312000004 2023-11-06T01:33:24.595 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:24.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 -- 172.21.15.53:0/3575784520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49e80a05b0 msgr2=0x7f49e80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49e80a05b0 0x7f49e80a2a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00b580 tx=0x7f49ec005fb0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 -- 172.21.15.53:0/3575784520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0ff3b0 msgr2=0x7f49fc103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0ff3b0 0x7f49fc103340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f49f800ddf0 tx=0x7f49f800a530 comp rx=0 tx=0).stop 2023-11-06T01:33:24.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 -- 172.21.15.53:0/3575784520 shutdown_connections 2023-11-06T01:33:24.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49e80a05b0 0x7f49e80a2a70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49fc100f40 0x7f49fc101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0ff3b0 0x7f49fc103340 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 --2- 172.21.15.53:0/3575784520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49fc1005a0 0x7f49fc101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 -- 172.21.15.53:0/3575784520 >> 172.21.15.53:0/3575784520 conn(0x7f49fc0fa920 msgr2=0x7f49fc105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 -- 172.21.15.53:0/3575784520 shutdown_connections 2023-11-06T01:33:24.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.569+0000 7f4a046a2700 1 -- 172.21.15.53:0/3575784520 wait complete. 2023-11-06T01:33:24.599 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-11-06T01:33:24.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.725+0000 7f50cf3bb700 1 -- 172.21.15.53:0/1931220723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c8100e70 msgr2=0x7f50c81012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.725+0000 7f50cf3bb700 1 --2- 172.21.15.53:0/1931220723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c8100e70 0x7f50c81012d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f50c40099c0 tx=0x7f50c4009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.725+0000 7f50cf3bb700 1 -- 172.21.15.53:0/1931220723 shutdown_connections 2023-11-06T01:33:24.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.725+0000 7f50cf3bb700 1 --2- 172.21.15.53:0/1931220723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8101810 0x7f50c81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.725+0000 7f50cf3bb700 1 --2- 172.21.15.53:0/1931220723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c80ffc80 0x7f50c81000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.725+0000 7f50cf3bb700 1 --2- 172.21.15.53:0/1931220723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c8100e70 0x7f50c81012d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.725+0000 7f50cf3bb700 1 -- 172.21.15.53:0/1931220723 >> 172.21.15.53:0/1931220723 conn(0x7f50c80fb1f0 msgr2=0x7f50c80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.725+0000 7f50cf3bb700 1 -- 172.21.15.53:0/1931220723 shutdown_connections 2023-11-06T01:33:24.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.725+0000 7f50cf3bb700 1 -- 172.21.15.53:0/1931220723 wait complete. 2023-11-06T01:33:24.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cf3bb700 1 Processor -- start 2023-11-06T01:33:24.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cf3bb700 1 -- start start 2023-11-06T01:33:24.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cf3bb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c80ffc80 0x7f50c8196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cf3bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8100e70 0x7f50c8197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cf3bb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8101810 0x7f50c819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cf3bb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c819e650 con 0x7f50c80ffc80 2023-11-06T01:33:24.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cf3bb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c819e790 con 0x7f50c8100e70 2023-11-06T01:33:24.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cf3bb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c819e920 con 0x7f50c8101810 2023-11-06T01:33:24.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cd157700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c80ffc80 0x7f50c8196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cd157700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c80ffc80 0x7f50c8196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38824/0 (socket says 172.21.15.53:38824) 2023-11-06T01:33:24.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cd157700 1 -- 172.21.15.53:0/2407177498 learned_addr learned my addr 172.21.15.53:0/2407177498 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cd958700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8101810 0x7f50c819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cc956700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8100e70 0x7f50c8197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cd157700 1 -- 172.21.15.53:0/2407177498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8101810 msgr2=0x7f50c819e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cd157700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8101810 0x7f50c819e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cd157700 1 -- 172.21.15.53:0/2407177498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8100e70 msgr2=0x7f50c8197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cd157700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8100e70 0x7f50c8197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cd157700 1 -- 172.21.15.53:0/2407177498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c40095f0 con 0x7f50c80ffc80 2023-11-06T01:33:24.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cc956700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8100e70 0x7f50c8197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:24.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.726+0000 7f50cd157700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c80ffc80 0x7f50c8196ad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00d980 tx=0x7f50bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.727+0000 7f50ba7fc700 1 -- 172.21.15.53:0/2407177498 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc00b950 con 0x7f50c80ffc80 2023-11-06T01:33:24.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.727+0000 7f50ba7fc700 1 -- 172.21.15.53:0/2407177498 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50bc007620 con 0x7f50c80ffc80 2023-11-06T01:33:24.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.727+0000 7f50cf3bb700 1 -- 172.21.15.53:0/2407177498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c819ec00 con 0x7f50c80ffc80 2023-11-06T01:33:24.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.727+0000 7f50ba7fc700 1 -- 172.21.15.53:0/2407177498 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc00f3c0 con 0x7f50c80ffc80 2023-11-06T01:33:24.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.727+0000 7f50cf3bb700 1 -- 172.21.15.53:0/2407177498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c819f160 con 0x7f50c80ffc80 2023-11-06T01:33:24.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50db54bb80 1 Processor -- start 2023-11-06T01:33:24.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50db54bb80 1 -- start start 2023-11-06T01:33:24.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50ba7fc700 1 -- 172.21.15.53:0/2407177498 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f50bc010640 con 0x7f50c80ffc80 2023-11-06T01:33:24.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10e0a0 0x56383c10e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c10ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10d3a0 0x56383c10d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50db54bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383c0649c0 con 0x56383c10ea20 2023-11-06T01:33:24.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50db54bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383c062380 con 0x56383c10d3a0 2023-11-06T01:33:24.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50db54bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383c05dc70 con 0x56383c10e0a0 2023-11-06T01:33:24.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50cd157700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c10ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50cd157700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c10ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38832/0 (socket says 172.21.15.53:38832) 2023-11-06T01:33:24.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50cd157700 1 -- 172.21.15.53:0/1471672773 learned_addr learned my addr 172.21.15.53:0/1471672773 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50cd958700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10e0a0 0x56383c10e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50cc956700 1 --2- 172.21.15.53:0/1471672773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10d3a0 0x56383c10d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50cd157700 1 -- 172.21.15.53:0/1471672773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10e0a0 msgr2=0x56383c10e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50cd157700 1 --2- 172.21.15.53:0/1471672773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10e0a0 0x56383c10e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50cd157700 1 -- 172.21.15.53:0/1471672773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10d3a0 msgr2=0x56383c10d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50cd157700 1 --2- 172.21.15.53:0/1471672773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10d3a0 0x56383c10d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50cd157700 1 -- 172.21.15.53:0/1471672773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56383be8bd90 con 0x56383c10ea20 2023-11-06T01:33:24.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.731+0000 7f50cc956700 1 --2- 172.21.15.53:0/1471672773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10d3a0 0x56383c10d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:24.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.732+0000 7f50ba7fc700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f50b40a0630 0x7f50b40a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.732+0000 7f50cd157700 1 --2- 172.21.15.53:0/1471672773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c10ce10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0c9260 tx=0x7f50bc0d1a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4621af5502767c1b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.732+0000 7f50cd958700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f50b40a0630 0x7f50b40a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.732+0000 7f50b37fe700 1 -- 172.21.15.53:0/1471672773 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0953b0 con 0x56383c10ea20 2023-11-06T01:33:24.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.732+0000 7f50b37fe700 1 -- 172.21.15.53:0/1471672773 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50bc0cee90 con 0x56383c10ea20 2023-11-06T01:33:24.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.732+0000 7f50b37fe700 1 -- 172.21.15.53:0/1471672773 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0229c0 con 0x56383c10ea20 2023-11-06T01:33:24.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.732+0000 7f50ba7fc700 1 -- 172.21.15.53:0/2407177498 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f50bc0cb7c0 con 0x7f50c80ffc80 2023-11-06T01:33:24.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 -- 172.21.15.53:0/1471672773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 msgr2=0x56383c10ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50cd958700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f50b40a0630 0x7f50b40a2af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009790 tx=0x7f50c0009130 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 --2- 172.21.15.53:0/1471672773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c10ce10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0c9260 tx=0x7f50bc0d1a60 comp rx=0 tx=0).stop 2023-11-06T01:33:24.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 -- 172.21.15.53:0/1471672773 shutdown_connections 2023-11-06T01:33:24.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 --2- 172.21.15.53:0/1471672773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10d3a0 0x56383c10d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 --2- 172.21.15.53:0/1471672773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10e0a0 0x56383c10e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 --2- 172.21.15.53:0/1471672773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c10ce10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 -- 172.21.15.53:0/1471672773 >> 172.21.15.53:0/1471672773 conn(0x56383c06c080 msgr2=0x56383c1078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 -- 172.21.15.53:0/1471672773 shutdown_connections 2023-11-06T01:33:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 -- 172.21.15.53:0/1471672773 wait complete. 2023-11-06T01:33:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 -- start start 2023-11-06T01:33:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 Processor -- start 2023-11-06T01:33:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c1167b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.733+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10e0a0 0x56383c116cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10ea20 0x56383c11dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50db54bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383bffa4c0 con 0x56383c10d3a0 2023-11-06T01:33:24.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50db54bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383bffa160 con 0x56383c10e0a0 2023-11-06T01:33:24.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50db54bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383bee18d0 con 0x56383c10ea20 2023-11-06T01:33:24.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50db54bb80 10 client.0 _ll_register_callbacks cb 0x56383bf47a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:24.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cd157700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c1167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cd157700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c1167b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38838/0 (socket says 172.21.15.53:38838) 2023-11-06T01:33:24.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cd157700 1 -- 172.21.15.53:0/589894576 learned_addr learned my addr 172.21.15.53:0/589894576 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cd958700 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10ea20 0x56383c11dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cc956700 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10e0a0 0x56383c116cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cd157700 1 -- 172.21.15.53:0/589894576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10ea20 msgr2=0x56383c11dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cd157700 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10ea20 0x56383c11dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cd157700 1 -- 172.21.15.53:0/589894576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10e0a0 msgr2=0x56383c116cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cd157700 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10e0a0 0x56383c116cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cd157700 1 -- 172.21.15.53:0/589894576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c819f160 con 0x56383c10d3a0 2023-11-06T01:33:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cd958700 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10ea20 0x56383c11dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50cd157700 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c1167b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f50bc095ae0 tx=0x7f50bc0cbb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470526dbaa60dd65 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50b0ff9700 1 -- 172.21.15.53:0/589894576 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0d16d0 con 0x56383c10d3a0 2023-11-06T01:33:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50b0ff9700 1 -- 172.21.15.53:0/589894576 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f50bc010460 con 0x56383c10d3a0 2023-11-06T01:33:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50db54bb80 1 -- 172.21.15.53:0/589894576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56383c088cc0 con 0x56383c10d3a0 2023-11-06T01:33:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50b0ff9700 1 -- 172.21.15.53:0/589894576 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50bc0953b0 con 0x56383c10d3a0 2023-11-06T01:33:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.734+0000 7f50db54bb80 10 client.15054 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.735+0000 7f50db54bb80 1 -- 172.21.15.53:0/589894576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56383be74c10 con 0x56383c10d3a0 2023-11-06T01:33:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.735+0000 7f50b0ff9700 1 -- 172.21.15.53:0/589894576 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56383c088cc0 con 0x56383c10d3a0 2023-11-06T01:33:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.735+0000 7f50b0ff9700 1 -- 172.21.15.53:0/589894576 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f50bc0dc020 con 0x56383c10d3a0 2023-11-06T01:33:24.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.735+0000 7f50db54bb80 10 client.15054 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:24.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.735+0000 7f50db54bb80 10 client.15054 resolve_mds: resolved name 'e' to daemon mds.e 2023-11-06T01:33:24.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.735+0000 7f50db54bb80 20 client.15054 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:24.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.735+0000 7f50db54bb80 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x56383c1214a0 0x56383c120c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.735+0000 7f50db54bb80 4 client.15054 mds_command: new command op to 24422 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:24.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.735+0000 7f50db54bb80 1 -- 172.21.15.53:0/589894576 --> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56383c0dbdc0 con 0x56383c1214a0 2023-11-06T01:33:24.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.735+0000 7f50cc956700 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x56383c1214a0 0x56383c120c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.736+0000 7f50b0ff9700 10 client.15054 ms_handle_connect on v2:172.21.15.62:6834/3901057483 2023-11-06T01:33:24.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.736+0000 7f50cc956700 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x56383c1214a0 0x56383c120c60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=43a8606237ff4e53 server_cookie=c3334b7c2c779e8f in_seq=0 out_seq=0 2023-11-06T01:33:24.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.736+0000 7f50b0ff9700 1 -- 172.21.15.53:0/589894576 <== mds.3 v2:172.21.15.62:6834/3901057483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56383c0dbdc0 con 0x56383c1214a0 2023-11-06T01:33:24.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.736+0000 7f50b0ff9700 10 client.15054 handle_command_reply: tid=0 2023-11-06T01:33:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.736+0000 7f50db54bb80 1 client.15054 shutdown 2023-11-06T01:33:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50db54bb80 1 -- 172.21.15.53:0/589894576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 msgr2=0x56383c1167b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50db54bb80 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c1167b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f50bc095ae0 tx=0x7f50bc0cbb40 comp rx=0 tx=0).stop 2023-11-06T01:33:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50db54bb80 1 -- 172.21.15.53:0/589894576 shutdown_connections 2023-11-06T01:33:24.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50db54bb80 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x56383c1214a0 0x56383c120c60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50db54bb80 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10ea20 0x56383c11dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50db54bb80 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c1167b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50db54bb80 1 --2- 172.21.15.53:0/589894576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10e0a0 0x56383c116cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50b0ff9700 0 client.15054 ms_handle_reset on v2:172.21.15.62:6834/3901057483 2023-11-06T01:33:24.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50db54bb80 1 -- 172.21.15.53:0/589894576 >> 172.21.15.53:0/589894576 conn(0x56383c06c080 msgr2=0x56383c107060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50db54bb80 1 -- 172.21.15.53:0/589894576 shutdown_connections 2023-11-06T01:33:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50db54bb80 1 -- 172.21.15.53:0/589894576 wait complete. 2023-11-06T01:33:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.737+0000 7f50db54bb80 20 client.15054 trim_cache size 0 max 16384 2023-11-06T01:33:24.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.754+0000 7f50db54bb80 1 Processor -- start 2023-11-06T01:33:24.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.754+0000 7f50db54bb80 1 -- start start 2023-11-06T01:33:24.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c10ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10e0a0 0x56383c12a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10ea20 0x56383c116560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50db54bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383c108530 con 0x56383c10d3a0 2023-11-06T01:33:24.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50db54bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383c0dbdc0 con 0x56383c10e0a0 2023-11-06T01:33:24.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50db54bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383c088cc0 con 0x56383c10ea20 2023-11-06T01:33:24.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50cd157700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c10ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50cd157700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c10ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38846/0 (socket says 172.21.15.53:38846) 2023-11-06T01:33:24.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50cd157700 1 -- 172.21.15.53:0/1296069302 learned_addr learned my addr 172.21.15.53:0/1296069302 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50cc956700 1 --2- 172.21.15.53:0/1296069302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10e0a0 0x56383c12a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50cd157700 1 -- 172.21.15.53:0/1296069302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10ea20 msgr2=0x56383c116560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50cd157700 1 --2- 172.21.15.53:0/1296069302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10ea20 0x56383c116560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50cd157700 1 -- 172.21.15.53:0/1296069302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10e0a0 msgr2=0x56383c12a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50cd157700 1 --2- 172.21.15.53:0/1296069302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10e0a0 0x56383c12a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50cd157700 1 -- 172.21.15.53:0/1296069302 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56383beb2200 con 0x56383c10d3a0 2023-11-06T01:33:24.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50cc956700 1 --2- 172.21.15.53:0/1296069302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10e0a0 0x56383c12a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:24.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50cd157700 1 --2- 172.21.15.53:0/1296069302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c10ce10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f50bc098d70 tx=0x7f50bc095990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1dcea27b5b14dac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50b3fff700 1 -- 172.21.15.53:0/1296069302 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0953b0 con 0x56383c10d3a0 2023-11-06T01:33:24.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50b3fff700 1 -- 172.21.15.53:0/1296069302 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50bc0989d0 con 0x56383c10d3a0 2023-11-06T01:33:24.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.755+0000 7f50b3fff700 1 -- 172.21.15.53:0/1296069302 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc099770 con 0x56383c10d3a0 2023-11-06T01:33:24.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.756+0000 7f50db54bb80 1 -- 172.21.15.53:0/1296069302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 msgr2=0x56383c10ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.756+0000 7f50db54bb80 1 --2- 172.21.15.53:0/1296069302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c10ce10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f50bc098d70 tx=0x7f50bc095990 comp rx=0 tx=0).stop 2023-11-06T01:33:24.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.756+0000 7f50db54bb80 1 -- 172.21.15.53:0/1296069302 shutdown_connections 2023-11-06T01:33:24.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.756+0000 7f50db54bb80 1 --2- 172.21.15.53:0/1296069302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10ea20 0x56383c116560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.756+0000 7f50db54bb80 1 --2- 172.21.15.53:0/1296069302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10d3a0 0x56383c10ce10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.756+0000 7f50db54bb80 1 --2- 172.21.15.53:0/1296069302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10e0a0 0x56383c12a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.756+0000 7f50db54bb80 1 -- 172.21.15.53:0/1296069302 >> 172.21.15.53:0/1296069302 conn(0x56383c06c080 msgr2=0x56383c1146b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.756+0000 7f50db54bb80 1 -- 172.21.15.53:0/1296069302 shutdown_connections 2023-11-06T01:33:24.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.756+0000 7f50db54bb80 1 -- 172.21.15.53:0/1296069302 wait complete. 2023-11-06T01:33:24.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.756+0000 7f50db54bb80 1 -- start start 2023-11-06T01:33:24.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.756+0000 7f50db54bb80 1 Processor -- start 2023-11-06T01:33:24.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10d3a0 0x56383c124e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10e0a0 0x56383c125360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50db54bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c172f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50db54bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383c12a390 con 0x56383c10ea20 2023-11-06T01:33:24.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50db54bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383c12a500 con 0x56383c10d3a0 2023-11-06T01:33:24.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50db54bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383c1734d0 con 0x56383c10e0a0 2023-11-06T01:33:24.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50db54bb80 10 client.0 _ll_register_callbacks cb 0x7f50c0009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:24.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50cd958700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c172f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50cd958700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c172f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38856/0 (socket says 172.21.15.53:38856) 2023-11-06T01:33:24.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50cd157700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10d3a0 0x56383c124e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50cd958700 1 -- 172.21.15.53:0/2557056016 learned_addr learned my addr 172.21.15.53:0/2557056016 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50cc956700 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10e0a0 0x56383c125360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50cd958700 1 -- 172.21.15.53:0/2557056016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10e0a0 msgr2=0x56383c125360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50cd958700 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10e0a0 0x56383c125360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50cd958700 1 -- 172.21.15.53:0/2557056016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10d3a0 msgr2=0x56383c124e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50cd958700 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10d3a0 0x56383c124e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50cd958700 1 -- 172.21.15.53:0/2557056016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56383be74c10 con 0x56383c10ea20 2023-11-06T01:33:24.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50cd958700 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c172f90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f50c0018040 tx=0x7f50c0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a1af301b62f143 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:24.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50b0ff9700 1 -- 172.21.15.53:0/2557056016 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0016e00 con 0x56383c10ea20 2023-11-06T01:33:24.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.757+0000 7f50b0ff9700 1 -- 172.21.15.53:0/2557056016 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f50c0007420 con 0x56383c10ea20 2023-11-06T01:33:24.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50db54bb80 1 -- 172.21.15.53:0/2557056016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50bc0cee90 con 0x56383c10ea20 2023-11-06T01:33:24.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50b0ff9700 1 -- 172.21.15.53:0/2557056016 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c00075e0 con 0x56383c10ea20 2023-11-06T01:33:24.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50db54bb80 10 client.15060 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:24.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50db54bb80 1 -- 172.21.15.53:0/2557056016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56383be8a260 con 0x56383c10ea20 2023-11-06T01:33:24.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50b0ff9700 1 -- 172.21.15.53:0/2557056016 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50bc0cee90 con 0x56383c10ea20 2023-11-06T01:33:24.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50b0ff9700 1 -- 172.21.15.53:0/2557056016 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f50c002c8c0 con 0x56383c10ea20 2023-11-06T01:33:24.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50db54bb80 10 client.15060 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:24.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50db54bb80 10 client.15060 resolve_mds: resolved name 'e' to daemon mds.e 2023-11-06T01:33:24.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50db54bb80 20 client.15060 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:24.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50db54bb80 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x56383c178140 0x56383c174b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50db54bb80 4 client.15060 mds_command: new command op to 24422 tid=0[{"prefix": "status"}] 2023-11-06T01:33:24.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50db54bb80 1 -- 172.21.15.53:0/2557056016 --> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56383c056cf0 con 0x56383c178140 2023-11-06T01:33:24.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.758+0000 7f50cd157700 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x56383c178140 0x56383c174b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50b0ff9700 10 client.15060 ms_handle_connect on v2:172.21.15.62:6834/3901057483 2023-11-06T01:33:24.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50cd157700 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x56383c178140 0x56383c174b30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5cff680a1412d887 server_cookie=d9eb3fe1dfb48d59 in_seq=0 out_seq=0 2023-11-06T01:33:24.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50b0ff9700 1 -- 172.21.15.53:0/2557056016 <== mds.3 v2:172.21.15.62:6834/3901057483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56383c056cf0 con 0x56383c178140 2023-11-06T01:33:24.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50b0ff9700 10 client.15060 handle_command_reply: tid=0 2023-11-06T01:33:24.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50db54bb80 1 client.15060 shutdown 2023-11-06T01:33:24.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50db54bb80 1 -- 172.21.15.53:0/2557056016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 msgr2=0x56383c172f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50db54bb80 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c172f90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f50c0018040 tx=0x7f50c0004b40 comp rx=0 tx=0).stop 2023-11-06T01:33:24.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50db54bb80 1 -- 172.21.15.53:0/2557056016 shutdown_connections 2023-11-06T01:33:24.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50db54bb80 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x56383c178140 0x56383c174b30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50db54bb80 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56383c10d3a0 0x56383c124e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50db54bb80 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56383c10ea20 0x56383c172f90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50db54bb80 1 --2- 172.21.15.53:0/2557056016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56383c10e0a0 0x56383c125360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50b0ff9700 0 client.15060 ms_handle_reset on v2:172.21.15.62:6834/3901057483 2023-11-06T01:33:24.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50db54bb80 1 -- 172.21.15.53:0/2557056016 >> 172.21.15.53:0/2557056016 conn(0x56383c06c080 msgr2=0x56383c116560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50db54bb80 1 -- 172.21.15.53:0/2557056016 shutdown_connections 2023-11-06T01:33:24.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.759+0000 7f50db54bb80 1 -- 172.21.15.53:0/2557056016 wait complete. 2023-11-06T01:33:24.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50db54bb80 20 client.15060 trim_cache size 0 max 16384 2023-11-06T01:33:24.780 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:24.781 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:33:24.781 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 3, 2023-11-06T01:33:24.781 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24422, 2023-11-06T01:33:24.781 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-06T01:33:24.781 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-06T01:33:24.782 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-06T01:33:24.782 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 53.766136797999998, 2023-11-06T01:33:24.782 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 27, 2023-11-06T01:33:24.782 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 82, 2023-11-06T01:33:24.782 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 82, 2023-11-06T01:33:24.782 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 79.119675107999996 2023-11-06T01:33:24.783 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:24.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 -- 172.21.15.53:0/2407177498 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f50b40a0630 msgr2=0x7f50b40a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f50b40a0630 0x7f50b40a2af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009790 tx=0x7f50c0009130 comp rx=0 tx=0).stop 2023-11-06T01:33:24.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 -- 172.21.15.53:0/2407177498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c80ffc80 msgr2=0x7f50c8196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c80ffc80 0x7f50c8196ad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00d980 tx=0x7f50bc00dc90 comp rx=0 tx=0).stop 2023-11-06T01:33:24.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 -- 172.21.15.53:0/2407177498 shutdown_connections 2023-11-06T01:33:24.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f50b40a0630 0x7f50b40a2af0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8101810 0x7f50c819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c80ffc80 0x7f50c8196ad0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 --2- 172.21.15.53:0/2407177498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8100e70 0x7f50c8197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 -- 172.21.15.53:0/2407177498 >> 172.21.15.53:0/2407177498 conn(0x7f50c80fb1f0 msgr2=0x7f50c80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 -- 172.21.15.53:0/2407177498 shutdown_connections 2023-11-06T01:33:24.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.760+0000 7f50cf3bb700 1 -- 172.21.15.53:0/2407177498 wait complete. 2023-11-06T01:33:24.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-06T01:33:24.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.913+0000 7f70b6287700 1 -- 172.21.15.53:0/1391040561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70b0104130 msgr2=0x7f70b0106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.913+0000 7f70b6287700 1 --2- 172.21.15.53:0/1391040561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70b0104130 0x7f70b0106520 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009990 tx=0x7f70a4009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.913+0000 7f70b6287700 1 -- 172.21.15.53:0/1391040561 shutdown_connections 2023-11-06T01:33:24.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.913+0000 7f70b6287700 1 --2- 172.21.15.53:0/1391040561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0106a60 0x7f70b0108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.913+0000 7f70b6287700 1 --2- 172.21.15.53:0/1391040561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b0101800 0x7f70b0103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.913+0000 7f70b6287700 1 --2- 172.21.15.53:0/1391040561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70b0104130 0x7f70b0106520 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.913+0000 7f70b6287700 1 -- 172.21.15.53:0/1391040561 >> 172.21.15.53:0/1391040561 conn(0x7f70b00fb1f0 msgr2=0x7f70b00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.913+0000 7f70b6287700 1 -- 172.21.15.53:0/1391040561 shutdown_connections 2023-11-06T01:33:24.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.913+0000 7f70b6287700 1 -- 172.21.15.53:0/1391040561 wait complete. 2023-11-06T01:33:24.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.913+0000 7f70b6287700 1 Processor -- start 2023-11-06T01:33:24.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70b6287700 1 -- start start 2023-11-06T01:33:24.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70b6287700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70b0101800 0x7f70b0196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70b6287700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0104130 0x7f70b0196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70b6287700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b0106a60 0x7f70b019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70b6287700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b019e620 con 0x7f70b0101800 2023-11-06T01:33:24.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70b6287700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b019e760 con 0x7f70b0104130 2023-11-06T01:33:24.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70b6287700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b019e8f0 con 0x7f70b0106a60 2023-11-06T01:33:24.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70affff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70b0101800 0x7f70b0196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70affff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70b0101800 0x7f70b0196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38866/0 (socket says 172.21.15.53:38866) 2023-11-06T01:33:24.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70affff700 1 -- 172.21.15.53:0/1127771335 learned_addr learned my addr 172.21.15.53:0/1127771335 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70b4824700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b0106a60 0x7f70b019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70af7fe700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0104130 0x7f70b0196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70affff700 1 -- 172.21.15.53:0/1127771335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b0106a60 msgr2=0x7f70b019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70affff700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b0106a60 0x7f70b019e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70affff700 1 -- 172.21.15.53:0/1127771335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0104130 msgr2=0x7f70b0196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70affff700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0104130 0x7f70b0196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70affff700 1 -- 172.21.15.53:0/1127771335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a40095f0 con 0x7f70b0101800 2023-11-06T01:33:24.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.914+0000 7f70af7fe700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0104130 0x7f70b0196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:24.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.915+0000 7f70affff700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70b0101800 0x7f70b0196aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f70a000d950 tx=0x7f70a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.915+0000 7f70ad7fa700 1 -- 172.21.15.53:0/1127771335 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0009940 con 0x7f70b0101800 2023-11-06T01:33:24.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.915+0000 7f70ad7fa700 1 -- 172.21.15.53:0/1127771335 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70a0010460 con 0x7f70b0101800 2023-11-06T01:33:24.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.915+0000 7f70b6287700 1 -- 172.21.15.53:0/1127771335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b019ebd0 con 0x7f70b0101800 2023-11-06T01:33:24.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.915+0000 7f70ad7fa700 1 -- 172.21.15.53:0/1127771335 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a000f620 con 0x7f70b0101800 2023-11-06T01:33:24.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.915+0000 7f70b6287700 1 -- 172.21.15.53:0/1127771335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b019f0e0 con 0x7f70b0101800 2023-11-06T01:33:24.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.918+0000 7f70ad7fa700 1 -- 172.21.15.53:0/1127771335 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f70a00108d0 con 0x7f70b0101800 2023-11-06T01:33:24.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.918+0000 7f70ad7fa700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70980a05b0 0x7f70980a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.918+0000 7f70ad7fa700 1 -- 172.21.15.53:0/1127771335 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f70a00cb7f0 con 0x7f70b0101800 2023-11-06T01:33:24.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.918+0000 7f70af7fe700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70980a05b0 0x7f70980a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70af7fe700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70980a05b0 0x7f70980a2a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f70a4005f50 tx=0x7f70a4005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70c2417b80 1 Processor -- start 2023-11-06T01:33:24.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70c2417b80 1 -- start start 2023-11-06T01:33:24.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d16b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 0x55d48d169e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d16a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70c2417b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48d0c19c0 con 0x55d48d16b0a0 2023-11-06T01:33:24.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70c2417b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48d0bf380 con 0x55d48d16ba20 2023-11-06T01:33:24.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70c2417b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48d0bac70 con 0x55d48d16a3a0 2023-11-06T01:33:24.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70affff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d16b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70affff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d16b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38874/0 (socket says 172.21.15.53:38874) 2023-11-06T01:33:24.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70affff700 1 -- 172.21.15.53:0/944241418 learned_addr learned my addr 172.21.15.53:0/944241418 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70affff700 1 -- 172.21.15.53:0/944241418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 msgr2=0x55d48d16a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70affff700 1 --2- 172.21.15.53:0/944241418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d16a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70affff700 1 -- 172.21.15.53:0/944241418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 msgr2=0x55d48d169e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70af7fe700 1 --2- 172.21.15.53:0/944241418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 0x55d48d169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70affff700 1 --2- 172.21.15.53:0/944241418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 0x55d48d169e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70affff700 1 -- 172.21.15.53:0/944241418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d48cee8d90 con 0x55d48d16b0a0 2023-11-06T01:33:24.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70af7fe700 1 --2- 172.21.15.53:0/944241418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 0x55d48d169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:24.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.919+0000 7f70affff700 1 --2- 172.21.15.53:0/944241418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d16b4e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f70a00a0be0 tx=0x7f70a0098b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35b5b3e581cc4db server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70967fc700 1 -- 172.21.15.53:0/944241418 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a00a1110 con 0x55d48d16b0a0 2023-11-06T01:33:24.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70967fc700 1 -- 172.21.15.53:0/944241418 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70a0099300 con 0x55d48d16b0a0 2023-11-06T01:33:24.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70967fc700 1 -- 172.21.15.53:0/944241418 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a00a0e80 con 0x55d48d16b0a0 2023-11-06T01:33:24.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70c2417b80 1 -- 172.21.15.53:0/944241418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 msgr2=0x55d48d16b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70c2417b80 1 --2- 172.21.15.53:0/944241418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d16b4e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f70a00a0be0 tx=0x7f70a0098b00 comp rx=0 tx=0).stop 2023-11-06T01:33:24.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70c2417b80 1 -- 172.21.15.53:0/944241418 shutdown_connections 2023-11-06T01:33:24.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70c2417b80 1 --2- 172.21.15.53:0/944241418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d16a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70c2417b80 1 --2- 172.21.15.53:0/944241418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d16b4e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70c2417b80 1 --2- 172.21.15.53:0/944241418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 0x55d48d169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70c2417b80 1 -- 172.21.15.53:0/944241418 >> 172.21.15.53:0/944241418 conn(0x55d48d0c9080 msgr2=0x55d48d1648f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70c2417b80 1 -- 172.21.15.53:0/944241418 shutdown_connections 2023-11-06T01:33:24.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.920+0000 7f70c2417b80 1 -- 172.21.15.53:0/944241418 wait complete. 2023-11-06T01:33:24.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70c2417b80 1 -- start start 2023-11-06T01:33:24.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70c2417b80 1 Processor -- start 2023-11-06T01:33:24.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16a3a0 0x55d48d1738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16b0a0 0x55d48d173e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16ba20 0x55d48d17ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70c2417b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48d0574c0 con 0x55d48d16a3a0 2023-11-06T01:33:24.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70c2417b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48d057160 con 0x55d48d16b0a0 2023-11-06T01:33:24.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70c2417b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48cf3e8d0 con 0x55d48d16ba20 2023-11-06T01:33:24.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70c2417b80 10 client.0 _ll_register_callbacks cb 0x55d48cfa4a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:24.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70affff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16a3a0 0x55d48d1738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70affff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16a3a0 0x55d48d1738c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38880/0 (socket says 172.21.15.53:38880) 2023-11-06T01:33:24.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.921+0000 7f70affff700 1 -- 172.21.15.53:0/739063185 learned_addr learned my addr 172.21.15.53:0/739063185 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70af7fe700 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16b0a0 0x55d48d173e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70affff700 1 -- 172.21.15.53:0/739063185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16ba20 msgr2=0x55d48d17ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70affff700 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16ba20 0x55d48d17ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70affff700 1 -- 172.21.15.53:0/739063185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16b0a0 msgr2=0x55d48d173e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70affff700 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16b0a0 0x55d48d173e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70affff700 1 -- 172.21.15.53:0/739063185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b019f0e0 con 0x55d48d16a3a0 2023-11-06T01:33:24.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70affff700 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16a3a0 0x55d48d1738c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f70a0099030 tx=0x7f70a0010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab6ffb63dc36d08 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f708f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:24.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f708f7fe700 1 -- 172.21.15.53:0/739063185 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0097440 con 0x55d48d16a3a0 2023-11-06T01:33:24.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f708f7fe700 1 -- 172.21.15.53:0/739063185 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f70a00a0e90 con 0x55d48d16a3a0 2023-11-06T01:33:24.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70c2417b80 1 -- 172.21.15.53:0/739063185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d48d0e5cc0 con 0x55d48d16a3a0 2023-11-06T01:33:24.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f708f7fe700 1 -- 172.21.15.53:0/739063185 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70a00acb60 con 0x55d48d16a3a0 2023-11-06T01:33:24.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70c2417b80 10 client.15072 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:24.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70c2417b80 1 -- 172.21.15.53:0/739063185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d48ced1c10 con 0x55d48d16a3a0 2023-11-06T01:33:24.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f708f7fe700 1 -- 172.21.15.53:0/739063185 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d48d0e5cc0 con 0x55d48d16a3a0 2023-11-06T01:33:24.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f708f7fe700 1 -- 172.21.15.53:0/739063185 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f70a00b1280 con 0x55d48d16a3a0 2023-11-06T01:33:24.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70c2417b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:24.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70c2417b80 10 client.15072 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-06T01:33:24.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.922+0000 7f70c2417b80 20 client.15072 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:24.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.923+0000 7f70c2417b80 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x55d48d17e6e0 0x55d48d17dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.923+0000 7f70c2417b80 4 client.15072 mds_command: new command op to 24464 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:24.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.923+0000 7f70c2417b80 1 -- 172.21.15.53:0/739063185 --> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d48d138dc0 con 0x55d48d17e6e0 2023-11-06T01:33:24.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.923+0000 7f70af7fe700 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x55d48d17e6e0 0x55d48d17dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.923+0000 7f708f7fe700 10 client.15072 ms_handle_connect on v2:172.21.15.62:6838/1997633670 2023-11-06T01:33:24.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.923+0000 7f70af7fe700 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x55d48d17e6e0 0x55d48d17dce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=455549b92f624831 server_cookie=104e1c702f21dac0 in_seq=0 out_seq=0 2023-11-06T01:33:24.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f708f7fe700 1 -- 172.21.15.53:0/739063185 <== mds.2 v2:172.21.15.62:6838/1997633670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d48d138dc0 con 0x55d48d17e6e0 2023-11-06T01:33:24.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f708f7fe700 10 client.15072 handle_command_reply: tid=0 2023-11-06T01:33:24.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 1 client.15072 shutdown 2023-11-06T01:33:24.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 1 -- 172.21.15.53:0/739063185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16a3a0 msgr2=0x55d48d1738c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16a3a0 0x55d48d1738c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f70a0099030 tx=0x7f70a0010cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 1 -- 172.21.15.53:0/739063185 shutdown_connections 2023-11-06T01:33:24.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x55d48d17e6e0 0x55d48d17dce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16ba20 0x55d48d17ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f708f7fe700 0 client.15072 ms_handle_reset on v2:172.21.15.62:6838/1997633670 2023-11-06T01:33:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16a3a0 0x55d48d1738c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 1 --2- 172.21.15.53:0/739063185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16b0a0 0x55d48d173e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 1 -- 172.21.15.53:0/739063185 >> 172.21.15.53:0/739063185 conn(0x55d48d0c9080 msgr2=0x55d48d164060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 1 -- 172.21.15.53:0/739063185 shutdown_connections 2023-11-06T01:33:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 1 -- 172.21.15.53:0/739063185 wait complete. 2023-11-06T01:33:24.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.924+0000 7f70c2417b80 20 client.15072 trim_cache size 0 max 16384 2023-11-06T01:33:24.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70c2417b80 1 Processor -- start 2023-11-06T01:33:24.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70c2417b80 1 -- start start 2023-11-06T01:33:24.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d169e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d173670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 0x55d48d173bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70c2417b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48d164ec0 con 0x55d48d16b0a0 2023-11-06T01:33:24.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70c2417b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48d16ac20 con 0x55d48d16ba20 2023-11-06T01:33:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70c2417b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48d138dc0 con 0x55d48d16a3a0 2023-11-06T01:33:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70af7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d173670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70af7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d173670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38890/0 (socket says 172.21.15.53:38890) 2023-11-06T01:33:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70af7fe700 1 -- 172.21.15.53:0/577258909 learned_addr learned my addr 172.21.15.53:0/577258909 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70b4824700 1 --2- 172.21.15.53:0/577258909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 0x55d48d173bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70affff700 1 --2- 172.21.15.53:0/577258909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70af7fe700 1 -- 172.21.15.53:0/577258909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 msgr2=0x55d48d169e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70af7fe700 1 --2- 172.21.15.53:0/577258909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d169e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70af7fe700 1 -- 172.21.15.53:0/577258909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 msgr2=0x55d48d173bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70af7fe700 1 --2- 172.21.15.53:0/577258909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 0x55d48d173bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70af7fe700 1 -- 172.21.15.53:0/577258909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d48cf0f200 con 0x55d48d16b0a0 2023-11-06T01:33:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70b4824700 1 --2- 172.21.15.53:0/577258909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 0x55d48d173bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.942+0000 7f70affff700 1 --2- 172.21.15.53:0/577258909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f70af7fe700 1 --2- 172.21.15.53:0/577258909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d173670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c010 tx=0x7f70a401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19683b652e86778e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f7096ffd700 1 -- 172.21.15.53:0/577258909 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400b7d0 con 0x55d48d16b0a0 2023-11-06T01:33:24.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f7096ffd700 1 -- 172.21.15.53:0/577258909 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70a401d040 con 0x55d48d16b0a0 2023-11-06T01:33:24.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f7096ffd700 1 -- 172.21.15.53:0/577258909 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400fd90 con 0x55d48d16b0a0 2023-11-06T01:33:24.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f70c2417b80 1 -- 172.21.15.53:0/577258909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 msgr2=0x55d48d173670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f70c2417b80 1 --2- 172.21.15.53:0/577258909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d173670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c010 tx=0x7f70a401e000 comp rx=0 tx=0).stop 2023-11-06T01:33:24.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f70c2417b80 1 -- 172.21.15.53:0/577258909 shutdown_connections 2023-11-06T01:33:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f70c2417b80 1 --2- 172.21.15.53:0/577258909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16ba20 0x55d48d173bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f70c2417b80 1 --2- 172.21.15.53:0/577258909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f70c2417b80 1 --2- 172.21.15.53:0/577258909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16b0a0 0x55d48d173670 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f70c2417b80 1 -- 172.21.15.53:0/577258909 >> 172.21.15.53:0/577258909 conn(0x55d48d0c9080 msgr2=0x55d48d16e300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f70c2417b80 1 -- 172.21.15.53:0/577258909 shutdown_connections 2023-11-06T01:33:24.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.943+0000 7f70c2417b80 1 -- 172.21.15.53:0/577258909 wait complete. 2023-11-06T01:33:24.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70c2417b80 1 -- start start 2023-11-06T01:33:24.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70c2417b80 1 Processor -- start 2023-11-06T01:33:24.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d1d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16b0a0 0x55d48d1d2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70c2417b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16ba20 0x55d48d1d4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70c2417b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48d1d5210 con 0x55d48d16ba20 2023-11-06T01:33:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70c2417b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48d1d5380 con 0x55d48d16b0a0 2023-11-06T01:33:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70c2417b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48d1d5540 con 0x55d48d16a3a0 2023-11-06T01:33:24.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70c2417b80 10 client.0 _ll_register_callbacks cb 0x7f70a0098a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:24.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70b4824700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16ba20 0x55d48d1d4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70b4824700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16ba20 0x55d48d1d4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38898/0 (socket says 172.21.15.53:38898) 2023-11-06T01:33:24.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70b4824700 1 -- 172.21.15.53:0/3035871278 learned_addr learned my addr 172.21.15.53:0/3035871278 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:24.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70affff700 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d1d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70af7fe700 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16b0a0 0x55d48d1d2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.944+0000 7f70b4824700 1 -- 172.21.15.53:0/3035871278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 msgr2=0x55d48d1d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f70b4824700 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d1d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f70b4824700 1 -- 172.21.15.53:0/3035871278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16b0a0 msgr2=0x55d48d1d2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:24.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f70b4824700 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16b0a0 0x55d48d1d2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f70b4824700 1 -- 172.21.15.53:0/3035871278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b0109ac0 con 0x55d48d16ba20 2023-11-06T01:33:24.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f70b4824700 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16ba20 0x55d48d1d4c90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009ea0 tx=0x7f70a8004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d2eb553f66541 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:24.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f708f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:24.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f708f7fe700 1 -- 172.21.15.53:0/3035871278 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8007910 con 0x55d48d16ba20 2023-11-06T01:33:24.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f708f7fe700 1 -- 172.21.15.53:0/3035871278 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f70a8007a70 con 0x55d48d16ba20 2023-11-06T01:33:24.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f70c2417b80 1 -- 172.21.15.53:0/3035871278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70a0099300 con 0x55d48d16ba20 2023-11-06T01:33:24.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f708f7fe700 1 -- 172.21.15.53:0/3035871278 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70a80048f0 con 0x55d48d16ba20 2023-11-06T01:33:24.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f70c2417b80 10 client.15078 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:24.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f70c2417b80 1 -- 172.21.15.53:0/3035871278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d48cee7260 con 0x55d48d16ba20 2023-11-06T01:33:24.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f708f7fe700 1 -- 172.21.15.53:0/3035871278 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70a8027020 con 0x55d48d16ba20 2023-11-06T01:33:24.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f708f7fe700 1 -- 172.21.15.53:0/3035871278 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f70a8014040 con 0x55d48d16ba20 2023-11-06T01:33:24.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f70c2417b80 10 client.15078 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:24.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f70c2417b80 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-06T01:33:24.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.945+0000 7f70c2417b80 20 client.15078 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:24.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.946+0000 7f70c2417b80 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x55d48d1da360 0x55d48d1d6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:24.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.946+0000 7f70c2417b80 4 client.15078 mds_command: new command op to 24464 tid=0[{"prefix": "status"}] 2023-11-06T01:33:24.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.946+0000 7f70c2417b80 1 -- 172.21.15.53:0/3035871278 --> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d48d0b3cf0 con 0x55d48d1da360 2023-11-06T01:33:24.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.946+0000 7f70affff700 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x55d48d1da360 0x55d48d1d6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:24.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.946+0000 7f708f7fe700 10 client.15078 ms_handle_connect on v2:172.21.15.62:6838/1997633670 2023-11-06T01:33:24.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.946+0000 7f70affff700 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x55d48d1da360 0x55d48d1d6cf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1381eb191edc15e3 server_cookie=9e0f2c5d74191933 in_seq=0 out_seq=0 2023-11-06T01:33:24.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.946+0000 7f708f7fe700 1 -- 172.21.15.53:0/3035871278 <== mds.2 v2:172.21.15.62:6838/1997633670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d48d0b3cf0 con 0x55d48d1da360 2023-11-06T01:33:24.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.946+0000 7f708f7fe700 10 client.15078 handle_command_reply: tid=0 2023-11-06T01:33:24.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.946+0000 7f70c2417b80 1 client.15078 shutdown 2023-11-06T01:33:24.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70c2417b80 1 -- 172.21.15.53:0/3035871278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16ba20 msgr2=0x55d48d1d4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70c2417b80 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16ba20 0x55d48d1d4c90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009ea0 tx=0x7f70a8004320 comp rx=0 tx=0).stop 2023-11-06T01:33:24.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70c2417b80 1 -- 172.21.15.53:0/3035871278 shutdown_connections 2023-11-06T01:33:24.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70c2417b80 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x55d48d1da360 0x55d48d1d6cf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70c2417b80 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d48d16ba20 0x55d48d1d4c90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70c2417b80 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d48d16a3a0 0x55d48d1d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70c2417b80 1 --2- 172.21.15.53:0/3035871278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d48d16b0a0 0x55d48d1d2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70c2417b80 1 -- 172.21.15.53:0/3035871278 >> 172.21.15.53:0/3035871278 conn(0x55d48d0c9080 msgr2=0x55d48d173670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f708f7fe700 0 client.15078 ms_handle_reset on v2:172.21.15.62:6838/1997633670 2023-11-06T01:33:24.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70c2417b80 1 -- 172.21.15.53:0/3035871278 shutdown_connections 2023-11-06T01:33:24.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70c2417b80 1 -- 172.21.15.53:0/3035871278 wait complete. 2023-11-06T01:33:24.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70c2417b80 20 client.15078 trim_cache size 0 max 16384 2023-11-06T01:33:24.971 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:33:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 2, 2023-11-06T01:33:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24464, 2023-11-06T01:33:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-06T01:33:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-06T01:33:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-06T01:33:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 54.500033766000001, 2023-11-06T01:33:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 27, 2023-11-06T01:33:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 82, 2023-11-06T01:33:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 82, 2023-11-06T01:33:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 63.676576627000003 2023-11-06T01:33:24.974 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:24.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70b6287700 1 -- 172.21.15.53:0/1127771335 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70980a05b0 msgr2=0x7f70980a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70b6287700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70980a05b0 0x7f70980a2a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f70a4005f50 tx=0x7f70a4005e80 comp rx=0 tx=0).stop 2023-11-06T01:33:24.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70b6287700 1 -- 172.21.15.53:0/1127771335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70b0101800 msgr2=0x7f70b0196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:24.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70b6287700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70b0101800 0x7f70b0196aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f70a000d950 tx=0x7f70a000dc60 comp rx=0 tx=0).stop 2023-11-06T01:33:24.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70b6287700 1 -- 172.21.15.53:0/1127771335 shutdown_connections 2023-11-06T01:33:24.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70b6287700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70980a05b0 0x7f70980a2a70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70b6287700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b0106a60 0x7f70b019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70b6287700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70b0101800 0x7f70b0196aa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70b6287700 1 --2- 172.21.15.53:0/1127771335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0104130 0x7f70b0196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:24.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.947+0000 7f70b6287700 1 -- 172.21.15.53:0/1127771335 >> 172.21.15.53:0/1127771335 conn(0x7f70b00fb1f0 msgr2=0x7f70b01024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:24.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.948+0000 7f70b6287700 1 -- 172.21.15.53:0/1127771335 shutdown_connections 2023-11-06T01:33:24.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:24.948+0000 7f70b6287700 1 -- 172.21.15.53:0/1127771335 wait complete. 2023-11-06T01:33:24.978 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T01:33:25.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.105+0000 7f5d91169700 1 -- 172.21.15.53:0/519903907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d8c101200 msgr2=0x7f5d8c101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.105+0000 7f5d91169700 1 --2- 172.21.15.53:0/519903907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d8c101200 0x7f5d8c101620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009990 tx=0x7f5d7c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.105+0000 7f5d91169700 1 -- 172.21.15.53:0/519903907 shutdown_connections 2023-11-06T01:33:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.105+0000 7f5d91169700 1 --2- 172.21.15.53:0/519903907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d8c1008d0 0x7f5d8c108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.105+0000 7f5d91169700 1 --2- 172.21.15.53:0/519903907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d8c101200 0x7f5d8c101620 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.105+0000 7f5d91169700 1 --2- 172.21.15.53:0/519903907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d8c0fff30 0x7f5d8c100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.105+0000 7f5d91169700 1 -- 172.21.15.53:0/519903907 >> 172.21.15.53:0/519903907 conn(0x7f5d8c0fa960 msgr2=0x7f5d8c0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.105+0000 7f5d91169700 1 -- 172.21.15.53:0/519903907 shutdown_connections 2023-11-06T01:33:25.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.105+0000 7f5d91169700 1 -- 172.21.15.53:0/519903907 wait complete. 2023-11-06T01:33:25.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.106+0000 7f5d91169700 1 Processor -- start 2023-11-06T01:33:25.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.106+0000 7f5d91169700 1 -- start start 2023-11-06T01:33:25.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.106+0000 7f5d91169700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d8c0fff30 0x7f5d8c10a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d91169700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d8c1008d0 0x7f5d8c108bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d91169700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d8c101200 0x7f5d8c109100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d91169700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1098a0 con 0x7f5d8c101200 2023-11-06T01:33:25.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d91169700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c109a10 con 0x7f5d8c1008d0 2023-11-06T01:33:25.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d91169700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1a0b00 con 0x7f5d8c0fff30 2023-11-06T01:33:25.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d8b59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d8c101200 0x7f5d8c109100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d8b59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d8c101200 0x7f5d8c109100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38912/0 (socket says 172.21.15.53:38912) 2023-11-06T01:33:25.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d8b59e700 1 -- 172.21.15.53:0/1247080447 learned_addr learned my addr 172.21.15.53:0/1247080447 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:25.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d8a59c700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d8c1008d0 0x7f5d8c108bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d8c0fff30 0x7f5d8c10a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d8b59e700 1 -- 172.21.15.53:0/1247080447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d8c0fff30 msgr2=0x7f5d8c10a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d8b59e700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d8c0fff30 0x7f5d8c10a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d8b59e700 1 -- 172.21.15.53:0/1247080447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d8c1008d0 msgr2=0x7f5d8c108bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d8b59e700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d8c1008d0 0x7f5d8c108bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d8b59e700 1 -- 172.21.15.53:0/1247080447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0095f0 con 0x7f5d8c101200 2023-11-06T01:33:25.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.107+0000 7f5d8b59e700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d8c101200 0x7f5d8c109100 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d84007ee0 tx=0x7f5d8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.109+0000 7f5d73fff700 1 -- 172.21.15.53:0/1247080447 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84017070 con 0x7f5d8c101200 2023-11-06T01:33:25.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.109+0000 7f5d73fff700 1 -- 172.21.15.53:0/1247080447 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d8400f040 con 0x7f5d8c101200 2023-11-06T01:33:25.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.109+0000 7f5d91169700 1 -- 172.21.15.53:0/1247080447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c1a0de0 con 0x7f5d8c101200 2023-11-06T01:33:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.109+0000 7f5d73fff700 1 -- 172.21.15.53:0/1247080447 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84005580 con 0x7f5d8c101200 2023-11-06T01:33:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.109+0000 7f5d91169700 1 -- 172.21.15.53:0/1247080447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c1a1210 con 0x7f5d8c101200 2023-11-06T01:33:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.111+0000 7f5d73fff700 1 -- 172.21.15.53:0/1247080447 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f5d84005760 con 0x7f5d8c101200 2023-11-06T01:33:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.112+0000 7f5d73fff700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5d740a0630 0x7f5d740a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.112+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5d740a0630 0x7f5d740a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.112+0000 7f5d73fff700 1 -- 172.21.15.53:0/1247080447 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5d840cb230 con 0x7f5d8c101200 2023-11-06T01:33:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.112+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5d740a0630 0x7f5d740a2af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00b5c0 tx=0x7f5d7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.112+0000 7f5d9d2f9b80 1 Processor -- start 2023-11-06T01:33:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d9d2f9b80 1 -- start start 2023-11-06T01:33:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f99024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f9900e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f9901820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f98589c0 con 0x5602f99020a0 2023-11-06T01:33:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f9856380 con 0x5602f9902a20 2023-11-06T01:33:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f9851c70 con 0x5602f99013a0 2023-11-06T01:33:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d8ad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f99024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d8ad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f99024e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38916/0 (socket says 172.21.15.53:38916) 2023-11-06T01:33:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d8ad9d700 1 -- 172.21.15.53:0/1614716503 learned_addr learned my addr 172.21.15.53:0/1614716503 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d8b59e700 1 --2- 172.21.15.53:0/1614716503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f9901820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d8a59c700 1 --2- 172.21.15.53:0/1614716503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f9900e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d8ad9d700 1 -- 172.21.15.53:0/1614716503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 msgr2=0x5602f9901820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/1614716503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f9901820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d8ad9d700 1 -- 172.21.15.53:0/1614716503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 msgr2=0x5602f9900e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/1614716503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f9900e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d8ad9d700 1 -- 172.21.15.53:0/1614716503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602f967fd90 con 0x5602f99020a0 2023-11-06T01:33:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.113+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/1614716503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f99024e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c015000 tx=0x7f5d7c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34aa3f8657f08f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d717fa700 1 -- 172.21.15.53:0/1614716503 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c02a070 con 0x5602f99020a0 2023-11-06T01:33:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d717fa700 1 -- 172.21.15.53:0/1614716503 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d7c01e040 con 0x5602f99020a0 2023-11-06T01:33:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d717fa700 1 -- 172.21.15.53:0/1614716503 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c023bc0 con 0x5602f99020a0 2023-11-06T01:33:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/1614716503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 msgr2=0x5602f99024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/1614716503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f99024e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c015000 tx=0x7f5d7c020000 comp rx=0 tx=0).stop 2023-11-06T01:33:25.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/1614716503 shutdown_connections 2023-11-06T01:33:25.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/1614716503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f9901820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/1614716503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f99024e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/1614716503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f9900e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/1614716503 >> 172.21.15.53:0/1614716503 conn(0x5602f9860080 msgr2=0x5602f98fb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/1614716503 shutdown_connections 2023-11-06T01:33:25.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.114+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/1614716503 wait complete. 2023-11-06T01:33:25.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d9d2f9b80 1 -- start start 2023-11-06T01:33:25.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d9d2f9b80 1 Processor -- start 2023-11-06T01:33:25.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f990a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f990adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f9911df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f97ee4c0 con 0x5602f99020a0 2023-11-06T01:33:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f97ee160 con 0x5602f9902a20 2023-11-06T01:33:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f96d58d0 con 0x5602f99013a0 2023-11-06T01:33:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d9d2f9b80 10 client.0 _ll_register_callbacks cb 0x5602f973ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d8a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f990adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d8a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f990adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38924/0 (socket says 172.21.15.53:38924) 2023-11-06T01:33:25.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d8a59c700 1 -- 172.21.15.53:0/4132784667 learned_addr learned my addr 172.21.15.53:0/4132784667 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:25.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f990a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d8b59e700 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f9911df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d8a59c700 1 -- 172.21.15.53:0/4132784667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 msgr2=0x5602f990a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d8a59c700 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f990a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d8a59c700 1 -- 172.21.15.53:0/4132784667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 msgr2=0x5602f9911df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d8a59c700 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f9911df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d8a59c700 1 -- 172.21.15.53:0/4132784667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c0fee20 con 0x5602f99020a0 2023-11-06T01:33:25.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.115+0000 7f5d8b59e700 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f9911df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:25.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d8a59c700 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f990adb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000b890 tx=0x7f5d8000bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d790f2e97c4e77a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:25.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d6eff5700 1 -- 172.21.15.53:0/4132784667 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80003d90 con 0x5602f99020a0 2023-11-06T01:33:25.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d6eff5700 1 -- 172.21.15.53:0/4132784667 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5d8001b440 con 0x5602f99020a0 2023-11-06T01:33:25.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/4132784667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602f987ccc0 con 0x5602f99020a0 2023-11-06T01:33:25.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d6eff5700 1 -- 172.21.15.53:0/4132784667 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d8001b600 con 0x5602f99020a0 2023-11-06T01:33:25.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d9d2f9b80 10 client.15090 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:25.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/4132784667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5602f9668c10 con 0x5602f99020a0 2023-11-06T01:33:25.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d6eff5700 1 -- 172.21.15.53:0/4132784667 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602f987ccc0 con 0x5602f99020a0 2023-11-06T01:33:25.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d6eff5700 1 -- 172.21.15.53:0/4132784667 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f5d80010440 con 0x5602f99020a0 2023-11-06T01:33:25.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d9d2f9b80 10 client.15090 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:25.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d9d2f9b80 10 client.15090 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T01:33:25.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d9d2f9b80 20 client.15090 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:25.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5602f9915ba0 0x5602f9914fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d9d2f9b80 4 client.15090 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:25.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.116+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/4132784667 --> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602f98cfdc0 con 0x5602f9915ba0 2023-11-06T01:33:25.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.117+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5602f9915ba0 0x5602f9914fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.117+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5602f9915ba0 0x5602f9914fa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98e2fa7757c9f569 server_cookie=1b84a35753ed5bac in_seq=0 out_seq=0 2023-11-06T01:33:25.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.117+0000 7f5d6eff5700 10 client.15090 ms_handle_connect on v2:172.21.15.163:6836/1759271265 2023-11-06T01:33:25.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d6eff5700 1 -- 172.21.15.53:0/4132784667 <== mds.1 v2:172.21.15.163:6836/1759271265 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5602f98cfdc0 con 0x5602f9915ba0 2023-11-06T01:33:25.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d6eff5700 10 client.15090 handle_command_reply: tid=0 2023-11-06T01:33:25.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 1 client.15090 shutdown 2023-11-06T01:33:25.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/4132784667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 msgr2=0x5602f990adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f990adb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000b890 tx=0x7f5d8000bba0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/4132784667 shutdown_connections 2023-11-06T01:33:25.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5602f9915ba0 0x5602f9914fa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f9911df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f990a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/4132784667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f990adb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d6eff5700 0 client.15090 ms_handle_reset on v2:172.21.15.163:6836/1759271265 2023-11-06T01:33:25.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/4132784667 >> 172.21.15.53:0/4132784667 conn(0x5602f9860080 msgr2=0x5602f98fb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/4132784667 shutdown_connections 2023-11-06T01:33:25.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/4132784667 wait complete. 2023-11-06T01:33:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.118+0000 7f5d9d2f9b80 20 client.15090 trim_cache size 0 max 16384 2023-11-06T01:33:25.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d9d2f9b80 1 Processor -- start 2023-11-06T01:33:25.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d9d2f9b80 1 -- start start 2023-11-06T01:33:25.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f9908710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f9908c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f991e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f990a440 con 0x5602f99020a0 2023-11-06T01:33:25.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f99093f0 con 0x5602f9902a20 2023-11-06T01:33:25.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f98cfdc0 con 0x5602f99013a0 2023-11-06T01:33:25.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f9908c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f9908c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38932/0 (socket says 172.21.15.53:38932) 2023-11-06T01:33:25.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8a59c700 1 -- 172.21.15.53:0/2421483351 learned_addr learned my addr 172.21.15.53:0/2421483351 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:25.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/2421483351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f9908710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8a59c700 1 -- 172.21.15.53:0/2421483351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 msgr2=0x5602f9908710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8a59c700 1 --2- 172.21.15.53:0/2421483351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f9908710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8a59c700 1 -- 172.21.15.53:0/2421483351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 msgr2=0x5602f991e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8b59e700 1 --2- 172.21.15.53:0/2421483351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f991e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8a59c700 1 --2- 172.21.15.53:0/2421483351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f991e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8a59c700 1 -- 172.21.15.53:0/2421483351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602f96a6200 con 0x5602f99020a0 2023-11-06T01:33:25.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/2421483351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f9908710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:25.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.136+0000 7f5d8b59e700 1 --2- 172.21.15.53:0/2421483351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f991e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:25.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d8a59c700 1 --2- 172.21.15.53:0/2421483351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f9908c50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5d80010020 tx=0x7f5d80005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca1f063c4d31857 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d71ffb700 1 -- 172.21.15.53:0/2421483351 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80003680 con 0x5602f99020a0 2023-11-06T01:33:25.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d71ffb700 1 -- 172.21.15.53:0/2421483351 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d80010ac0 con 0x5602f99020a0 2023-11-06T01:33:25.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d71ffb700 1 -- 172.21.15.53:0/2421483351 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8001bb60 con 0x5602f99020a0 2023-11-06T01:33:25.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2421483351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 msgr2=0x5602f9908c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/2421483351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f9908c50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5d80010020 tx=0x7f5d80005b80 comp rx=0 tx=0).stop 2023-11-06T01:33:25.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2421483351 shutdown_connections 2023-11-06T01:33:25.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/2421483351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f9902a20 0x5602f991e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/2421483351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99013a0 0x5602f9908710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/2421483351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f99020a0 0x5602f9908c50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2421483351 >> 172.21.15.53:0/2421483351 conn(0x5602f9860080 msgr2=0x5602f98fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2421483351 shutdown_connections 2023-11-06T01:33:25.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.137+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2421483351 wait complete. 2023-11-06T01:33:25.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d9d2f9b80 1 -- start start 2023-11-06T01:33:25.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d9d2f9b80 1 Processor -- start 2023-11-06T01:33:25.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f99013a0 0x5602f9906ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99020a0 0x5602f9907430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d9d2f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f9902a20 0x5602f9967690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f9907970 con 0x5602f9902a20 2023-11-06T01:33:25.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f9907ae0 con 0x5602f99013a0 2023-11-06T01:33:25.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d9d2f9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602f9967bd0 con 0x5602f99020a0 2023-11-06T01:33:25.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d9d2f9b80 10 client.0 _ll_register_callbacks cb 0x7f5d80004240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:25.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8b59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f9902a20 0x5602f9967690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8b59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f9902a20 0x5602f9967690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38940/0 (socket says 172.21.15.53:38940) 2023-11-06T01:33:25.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8b59e700 1 -- 172.21.15.53:0/2337819006 learned_addr learned my addr 172.21.15.53:0/2337819006 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:25.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f99013a0 0x5602f9906ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8a59c700 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99020a0 0x5602f9907430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8b59e700 1 -- 172.21.15.53:0/2337819006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99020a0 msgr2=0x5602f9907430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8b59e700 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99020a0 0x5602f9907430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8b59e700 1 -- 172.21.15.53:0/2337819006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f99013a0 msgr2=0x5602f9906ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8b59e700 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f99013a0 0x5602f9906ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8b59e700 1 -- 172.21.15.53:0/2337819006 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d80020040 con 0x5602f9902a20 2023-11-06T01:33:25.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8a59c700 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99020a0 0x5602f9907430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:25.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d8b59e700 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f9902a20 0x5602f9967690 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400d700 tx=0x7f5d84094e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca5ba86d65cc820 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.138+0000 7f5d6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:25.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d6eff5700 1 -- 172.21.15.53:0/2337819006 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8409c810 con 0x5602f9902a20 2023-11-06T01:33:25.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d6eff5700 1 -- 172.21.15.53:0/2337819006 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5d840a0a50 con 0x5602f9902a20 2023-11-06T01:33:25.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2337819006 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d7c01e040 con 0x5602f9902a20 2023-11-06T01:33:25.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d6eff5700 1 -- 172.21.15.53:0/2337819006 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d84098c40 con 0x5602f9902a20 2023-11-06T01:33:25.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d9d2f9b80 10 client.15096 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:25.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2337819006 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5602f967e260 con 0x5602f9902a20 2023-11-06T01:33:25.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d6eff5700 1 -- 172.21.15.53:0/2337819006 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d84062070 con 0x5602f9902a20 2023-11-06T01:33:25.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d6eff5700 1 -- 172.21.15.53:0/2337819006 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f5d840a05c0 con 0x5602f9902a20 2023-11-06T01:33:25.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d9d2f9b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:25.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d9d2f9b80 10 client.15096 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T01:33:25.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d9d2f9b80 20 client.15096 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:25.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5602f996c960 0x5602f99692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d9d2f9b80 4 client.15096 mds_command: new command op to 24472 tid=0[{"prefix": "status"}] 2023-11-06T01:33:25.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.139+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2337819006 --> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5602f984acf0 con 0x5602f996c960 2023-11-06T01:33:25.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5602f996c960 0x5602f99692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d8ad9d700 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5602f996c960 0x5602f99692d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6efc44d52c269ea8 server_cookie=9e08ce44301b1b0 in_seq=0 out_seq=0 2023-11-06T01:33:25.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d6eff5700 10 client.15096 ms_handle_connect on v2:172.21.15.163:6836/1759271265 2023-11-06T01:33:25.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d6eff5700 1 -- 172.21.15.53:0/2337819006 <== mds.1 v2:172.21.15.163:6836/1759271265 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5602f984acf0 con 0x5602f996c960 2023-11-06T01:33:25.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d6eff5700 10 client.15096 handle_command_reply: tid=0 2023-11-06T01:33:25.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 1 client.15096 shutdown 2023-11-06T01:33:25.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2337819006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f9902a20 msgr2=0x5602f9967690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f9902a20 0x5602f9967690 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400d700 tx=0x7f5d84094e50 comp rx=0 tx=0).stop 2023-11-06T01:33:25.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2337819006 shutdown_connections 2023-11-06T01:33:25.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5602f996c960 0x5602f99692d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602f99013a0 0x5602f9906ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602f9902a20 0x5602f9967690 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d6eff5700 0 client.15096 ms_handle_reset on v2:172.21.15.163:6836/1759271265 2023-11-06T01:33:25.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 1 --2- 172.21.15.53:0/2337819006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5602f99020a0 0x5602f9907430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2337819006 >> 172.21.15.53:0/2337819006 conn(0x5602f9860080 msgr2=0x5602f9905260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2337819006 shutdown_connections 2023-11-06T01:33:25.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 1 -- 172.21.15.53:0/2337819006 wait complete. 2023-11-06T01:33:25.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.140+0000 7f5d9d2f9b80 20 client.15096 trim_cache size 0 max 16384 2023-11-06T01:33:25.169 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:25.169 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:33:25.169 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 1, 2023-11-06T01:33:25.169 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24472, 2023-11-06T01:33:25.169 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-06T01:33:25.170 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-06T01:33:25.170 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-06T01:33:25.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 55.703324043999999, 2023-11-06T01:33:25.170 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 27, 2023-11-06T01:33:25.170 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 82, 2023-11-06T01:33:25.171 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 82, 2023-11-06T01:33:25.171 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 75.251837520999999 2023-11-06T01:33:25.171 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:25.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 -- 172.21.15.53:0/1247080447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5d740a0630 msgr2=0x7f5d740a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5d740a0630 0x7f5d740a2af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00b5c0 tx=0x7f5d7c005fd0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 -- 172.21.15.53:0/1247080447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d8c101200 msgr2=0x7f5d8c109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d8c101200 0x7f5d8c109100 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d84007ee0 tx=0x7f5d8400d3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 -- 172.21.15.53:0/1247080447 shutdown_connections 2023-11-06T01:33:25.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5d740a0630 0x7f5d740a2af0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d8c101200 0x7f5d8c109100 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d8c0fff30 0x7f5d8c10a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 --2- 172.21.15.53:0/1247080447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d8c1008d0 0x7f5d8c108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 -- 172.21.15.53:0/1247080447 >> 172.21.15.53:0/1247080447 conn(0x7f5d8c0fa960 msgr2=0x7f5d8c0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 -- 172.21.15.53:0/1247080447 shutdown_connections 2023-11-06T01:33:25.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.141+0000 7f5d91169700 1 -- 172.21.15.53:0/1247080447 wait complete. 2023-11-06T01:33:25.175 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-11-06T01:33:25.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.302+0000 7fdeb9c42700 1 -- 172.21.15.53:0/389346904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4106ae0 msgr2=0x7fdeb4108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.302+0000 7fdeb9c42700 1 --2- 172.21.15.53:0/389346904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4106ae0 0x7fdeb4108ed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00b0a0 tx=0x7fdeac00b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.302+0000 7fdeb9c42700 1 -- 172.21.15.53:0/389346904 shutdown_connections 2023-11-06T01:33:25.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.302+0000 7fdeb9c42700 1 --2- 172.21.15.53:0/389346904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb4101880 0x7fdeb4103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.302+0000 7fdeb9c42700 1 --2- 172.21.15.53:0/389346904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4106ae0 0x7fdeb4108ed0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.302+0000 7fdeb9c42700 1 --2- 172.21.15.53:0/389346904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb41041b0 0x7fdeb41065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.302+0000 7fdeb9c42700 1 -- 172.21.15.53:0/389346904 >> 172.21.15.53:0/389346904 conn(0x7fdeb40fb1f0 msgr2=0x7fdeb40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.302+0000 7fdeb9c42700 1 -- 172.21.15.53:0/389346904 shutdown_connections 2023-11-06T01:33:25.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.302+0000 7fdeb9c42700 1 -- 172.21.15.53:0/389346904 wait complete. 2023-11-06T01:33:25.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.302+0000 7fdeb9c42700 1 Processor -- start 2023-11-06T01:33:25.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb9c42700 1 -- start start 2023-11-06T01:33:25.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb9c42700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4101880 0x7fdeb4196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb9c42700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb41041b0 0x7fdeb4197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb9c42700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb4106ae0 0x7fdeb419e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb9c42700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb419e750 con 0x7fdeb4101880 2023-11-06T01:33:25.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb9c42700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb419e890 con 0x7fdeb41041b0 2023-11-06T01:33:25.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb9c42700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb419ea20 con 0x7fdeb4106ae0 2023-11-06T01:33:25.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4101880 0x7fdeb4196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4101880 0x7fdeb4196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38950/0 (socket says 172.21.15.53:38950) 2023-11-06T01:33:25.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb37fe700 1 -- 172.21.15.53:0/239205383 learned_addr learned my addr 172.21.15.53:0/239205383 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:25.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb3fff700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb4106ae0 0x7fdeb419e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb41041b0 0x7fdeb4197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb37fe700 1 -- 172.21.15.53:0/239205383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb4106ae0 msgr2=0x7fdeb419e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb4106ae0 0x7fdeb419e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb37fe700 1 -- 172.21.15.53:0/239205383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb41041b0 msgr2=0x7fdeb4197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb41041b0 0x7fdeb4197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb37fe700 1 -- 172.21.15.53:0/239205383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac009d00 con 0x7fdeb4101880 2023-11-06T01:33:25.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb3fff700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb4106ae0 0x7fdeb419e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:25.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4101880 0x7fdeb4196bd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e9a0 tx=0x7fdea400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.303+0000 7fdeb0ff9700 1 -- 172.21.15.53:0/239205383 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4009790 con 0x7fdeb4101880 2023-11-06T01:33:25.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.304+0000 7fdeb0ff9700 1 -- 172.21.15.53:0/239205383 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea400cbd0 con 0x7fdeb4101880 2023-11-06T01:33:25.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.304+0000 7fdeb9c42700 1 -- 172.21.15.53:0/239205383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb419ed00 con 0x7fdeb4101880 2023-11-06T01:33:25.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.304+0000 7fdeb0ff9700 1 -- 172.21.15.53:0/239205383 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4018680 con 0x7fdeb4101880 2023-11-06T01:33:25.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.304+0000 7fdeb9c42700 1 -- 172.21.15.53:0/239205383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb419f1c0 con 0x7fdeb4101880 2023-11-06T01:33:25.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.306+0000 7fdeb0ff9700 1 -- 172.21.15.53:0/239205383 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fdea400cd40 con 0x7fdeb4101880 2023-11-06T01:33:25.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.307+0000 7fdeb0ff9700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde9c0a05b0 0x7fde9c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.307+0000 7fdeb0ff9700 1 -- 172.21.15.53:0/239205383 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdea4014070 con 0x7fdeb4101880 2023-11-06T01:33:25.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.307+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde9c0a05b0 0x7fde9c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.307+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde9c0a05b0 0x7fde9c0a2a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009af0 tx=0x7fdea80094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdec5dd2b80 1 Processor -- start 2023-11-06T01:33:25.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdec5dd2b80 1 -- start start 2023-11-06T01:33:25.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f20a20 0x556d38f1ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f1f3a0 0x556d38f1f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38e769c0 con 0x556d38f200a0 2023-11-06T01:33:25.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38e74380 con 0x556d38f1f3a0 2023-11-06T01:33:25.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38e6fc70 con 0x556d38f20a20 2023-11-06T01:33:25.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f204e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38958/0 (socket says 172.21.15.53:38958) 2023-11-06T01:33:25.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb37fe700 1 -- 172.21.15.53:0/3072192778 learned_addr learned my addr 172.21.15.53:0/3072192778 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:25.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb3fff700 1 --2- 172.21.15.53:0/3072192778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f1f3a0 0x556d38f1f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/3072192778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f20a20 0x556d38f1ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb37fe700 1 -- 172.21.15.53:0/3072192778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f20a20 msgr2=0x556d38f1ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/3072192778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f20a20 0x556d38f1ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb37fe700 1 -- 172.21.15.53:0/3072192778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f1f3a0 msgr2=0x556d38f1f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/3072192778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f1f3a0 0x556d38f1f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb37fe700 1 -- 172.21.15.53:0/3072192778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d38c9dd90 con 0x556d38f200a0 2023-11-06T01:33:25.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb3fff700 1 --2- 172.21.15.53:0/3072192778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f1f3a0 0x556d38f1f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:25.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.308+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/3072192778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f204e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdea400f010 tx=0x7fdea4099c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2811e2d0be1a2a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fde99ffb700 1 -- 172.21.15.53:0/3072192778 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea409d160 con 0x556d38f200a0 2023-11-06T01:33:25.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fde99ffb700 1 -- 172.21.15.53:0/3072192778 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea40a15d0 con 0x556d38f200a0 2023-11-06T01:33:25.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fde99ffb700 1 -- 172.21.15.53:0/3072192778 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4098cb0 con 0x556d38f200a0 2023-11-06T01:33:25.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3072192778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 msgr2=0x556d38f204e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3072192778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f204e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdea400f010 tx=0x7fdea4099c60 comp rx=0 tx=0).stop 2023-11-06T01:33:25.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3072192778 shutdown_connections 2023-11-06T01:33:25.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3072192778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f1f3a0 0x556d38f1f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3072192778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f204e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3072192778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f20a20 0x556d38f1ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3072192778 >> 172.21.15.53:0/3072192778 conn(0x556d38e7e080 msgr2=0x556d38f198f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3072192778 shutdown_connections 2023-11-06T01:33:25.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.309+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3072192778 wait complete. 2023-11-06T01:33:25.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdec5dd2b80 1 -- start start 2023-11-06T01:33:25.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdec5dd2b80 1 Processor -- start 2023-11-06T01:33:25.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f1f3a0 0x556d38f28a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f28f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f20a20 0x556d38f2ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38e0c4c0 con 0x556d38f200a0 2023-11-06T01:33:25.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38e0c160 con 0x556d38f20a20 2023-11-06T01:33:25.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38cf38d0 con 0x556d38f1f3a0 2023-11-06T01:33:25.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdec5dd2b80 10 client.0 _ll_register_callbacks cb 0x556d38d59a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:25.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdeb2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f28f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdeb2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f28f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38966/0 (socket says 172.21.15.53:38966) 2023-11-06T01:33:25.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/1190319371 learned_addr learned my addr 172.21.15.53:0/1190319371 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:25.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdeb3fff700 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f20a20 0x556d38f2ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f1f3a0 0x556d38f28a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.310+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/1190319371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f1f3a0 msgr2=0x556d38f28a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f1f3a0 0x556d38f28a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/1190319371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f20a20 msgr2=0x556d38f2ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f20a20 0x556d38f2ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/1190319371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb419f1c0 con 0x556d38f200a0 2023-11-06T01:33:25.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f28f50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdea80116f0 tx=0x7fdea8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d3458009a35496 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fde92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:25.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fde92ffd700 1 -- 172.21.15.53:0/1190319371 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea800e8e0 con 0x556d38f200a0 2023-11-06T01:33:25.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fde92ffd700 1 -- 172.21.15.53:0/1190319371 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdea801f830 con 0x556d38f200a0 2023-11-06T01:33:25.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/1190319371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d38e9acc0 con 0x556d38f200a0 2023-11-06T01:33:25.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fde92ffd700 1 -- 172.21.15.53:0/1190319371 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea801f9f0 con 0x556d38f200a0 2023-11-06T01:33:25.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fdec5dd2b80 10 client.15108 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:25.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/1190319371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556d38c86c10 con 0x556d38f200a0 2023-11-06T01:33:25.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fde92ffd700 1 -- 172.21.15.53:0/1190319371 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d38e9acc0 con 0x556d38f200a0 2023-11-06T01:33:25.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.311+0000 7fde92ffd700 1 -- 172.21.15.53:0/1190319371 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fdea80103d0 con 0x556d38f200a0 2023-11-06T01:33:25.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.312+0000 7fdec5dd2b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:25.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.312+0000 7fdec5dd2b80 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-06T01:33:25.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.312+0000 7fdec5dd2b80 20 client.15108 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:25.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.312+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556d38f33d30 0x556d38f330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.312+0000 7fdec5dd2b80 4 client.15108 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:25.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.312+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/1190319371 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d38eeddc0 con 0x556d38f33d30 2023-11-06T01:33:25.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.312+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556d38f33d30 0x556d38f330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.313+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556d38f33d30 0x556d38f330e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef59352d56036f4d server_cookie=2f21ad8e5b8cca56 in_seq=0 out_seq=0 2023-11-06T01:33:25.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.313+0000 7fde92ffd700 10 client.15108 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:25.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.313+0000 7fde92ffd700 1 -- 172.21.15.53:0/1190319371 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556d38eeddc0 con 0x556d38f33d30 2023-11-06T01:33:25.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.313+0000 7fde92ffd700 10 client.15108 handle_command_reply: tid=0 2023-11-06T01:33:25.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.313+0000 7fdec5dd2b80 1 client.15108 shutdown 2023-11-06T01:33:25.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/1190319371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 msgr2=0x556d38f28f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f28f50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdea80116f0 tx=0x7fdea8004750 comp rx=0 tx=0).stop 2023-11-06T01:33:25.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/1190319371 shutdown_connections 2023-11-06T01:33:25.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f20a20 0x556d38f2ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556d38f33d30 0x556d38f330e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f1f3a0 0x556d38f28a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/1190319371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f200a0 0x556d38f28f50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fde92ffd700 0 client.15108 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:25.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/1190319371 >> 172.21.15.53:0/1190319371 conn(0x556d38e7e080 msgr2=0x556d38f19060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/1190319371 shutdown_connections 2023-11-06T01:33:25.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/1190319371 wait complete. 2023-11-06T01:33:25.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.314+0000 7fdec5dd2b80 20 client.15108 trim_cache size 0 max 16384 2023-11-06T01:33:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdec5dd2b80 1 Processor -- start 2023-11-06T01:33:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdec5dd2b80 1 -- start start 2023-11-06T01:33:25.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38e7e080 0x556d38f23360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f200a0 0x556d38f238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38f1fc20 con 0x556d38f1f3a0 2023-11-06T01:33:25.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38f1a410 con 0x556d38e7e080 2023-11-06T01:33:25.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38eeddc0 con 0x556d38f200a0 2023-11-06T01:33:25.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdeb2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdeb2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38974/0 (socket says 172.21.15.53:38974) 2023-11-06T01:33:25.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/3232745558 learned_addr learned my addr 172.21.15.53:0/3232745558 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:25.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.331+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/3232745558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38e7e080 0x556d38f23360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.332+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/3232745558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f200a0 msgr2=0x556d38f238a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.332+0000 7fdeb3fff700 1 --2- 172.21.15.53:0/3232745558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f200a0 0x556d38f238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.332+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/3232745558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f200a0 0x556d38f238a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.332+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/3232745558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38e7e080 msgr2=0x556d38f23360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.332+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/3232745558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38e7e080 0x556d38f23360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.332+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/3232745558 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d38cc4200 con 0x556d38f1f3a0 2023-11-06T01:33:25.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.332+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/3232745558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38e7e080 0x556d38f23360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:33:25.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.332+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/3232745558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3ca80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdea8005a80 tx=0x7fdea800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b569012f2d7f80 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.332+0000 7fde9a7fc700 1 -- 172.21.15.53:0/3232745558 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8011d90 con 0x556d38f1f3a0 2023-11-06T01:33:25.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.332+0000 7fde9a7fc700 1 -- 172.21.15.53:0/3232745558 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea8010e60 con 0x556d38f1f3a0 2023-11-06T01:33:25.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.332+0000 7fde9a7fc700 1 -- 172.21.15.53:0/3232745558 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8027db0 con 0x556d38f1f3a0 2023-11-06T01:33:25.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3232745558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 msgr2=0x556d38f3ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3232745558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3ca80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdea8005a80 tx=0x7fdea800e9a0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3232745558 shutdown_connections 2023-11-06T01:33:25.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3232745558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38e7e080 0x556d38f23360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3232745558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38f200a0 0x556d38f238a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3232745558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3ca80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3232745558 >> 172.21.15.53:0/3232745558 conn(0x556d38f19c20 msgr2=0x556d38f330e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3232745558 shutdown_connections 2023-11-06T01:33:25.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3232745558 wait complete. 2023-11-06T01:33:25.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 -- start start 2023-11-06T01:33:25.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 Processor -- start 2023-11-06T01:33:25.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38e7e080 0x556d38f39bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f200a0 0x556d38f3b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38f3b640 con 0x556d38f1f3a0 2023-11-06T01:33:25.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38f3b7b0 con 0x556d38f200a0 2023-11-06T01:33:25.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d38f3b970 con 0x556d38e7e080 2023-11-06T01:33:25.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdec5dd2b80 10 client.0 _ll_register_callbacks cb 0x7fdea800ede0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:25.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.333+0000 7fdeb2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdeb2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38980/0 (socket says 172.21.15.53:38980) 2023-11-06T01:33:25.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/3565545507 learned_addr learned my addr 172.21.15.53:0/3565545507 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38e7e080 0x556d38f39bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdeb3fff700 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f200a0 0x556d38f3b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/3565545507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38e7e080 msgr2=0x556d38f39bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38e7e080 0x556d38f39bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/3565545507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f200a0 msgr2=0x556d38f3b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f200a0 0x556d38f3b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdeb2ffd700 1 -- 172.21.15.53:0/3565545507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d38c9dd90 con 0x556d38f1f3a0 2023-11-06T01:33:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdeb2ffd700 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3a0f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdea8027980 tx=0x7fdea8005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5ba4ecb5e3b7cc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fde92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fde92ffd700 1 -- 172.21.15.53:0/3565545507 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea800e600 con 0x556d38f1f3a0 2023-11-06T01:33:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fde92ffd700 1 -- 172.21.15.53:0/3565545507 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdea8027ab0 con 0x556d38f1f3a0 2023-11-06T01:33:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3565545507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdea40a15d0 con 0x556d38f1f3a0 2023-11-06T01:33:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fde92ffd700 1 -- 172.21.15.53:0/3565545507 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea8011d90 con 0x556d38f1f3a0 2023-11-06T01:33:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdec5dd2b80 10 client.15114 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3565545507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556d38c86c10 con 0x556d38f1f3a0 2023-11-06T01:33:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.334+0000 7fde92ffd700 1 -- 172.21.15.53:0/3565545507 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea40a15d0 con 0x556d38f1f3a0 2023-11-06T01:33:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.335+0000 7fde92ffd700 1 -- 172.21.15.53:0/3565545507 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fdea804f450 con 0x556d38f1f3a0 2023-11-06T01:33:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.335+0000 7fdec5dd2b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.335+0000 7fdec5dd2b80 10 client.15114 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-06T01:33:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.335+0000 7fdec5dd2b80 20 client.15114 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.335+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556d38f20a20 0x556d38f87810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.335+0000 7fdec5dd2b80 4 client.15114 mds_command: new command op to 24487 tid=0[{"prefix": "status"}] 2023-11-06T01:33:25.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.335+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3565545507 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556d38e68cf0 con 0x556d38f20a20 2023-11-06T01:33:25.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.335+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556d38f20a20 0x556d38f87810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:25.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fde92ffd700 10 client.15114 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:25.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdeb37fe700 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556d38f20a20 0x556d38f87810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9952c279ac352e server_cookie=5959ec97d8118e30 in_seq=0 out_seq=0 2023-11-06T01:33:25.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fde92ffd700 1 -- 172.21.15.53:0/3565545507 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556d38e68cf0 con 0x556d38f20a20 2023-11-06T01:33:25.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fde92ffd700 10 client.15114 handle_command_reply: tid=0 2023-11-06T01:33:25.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 1 client.15114 shutdown 2023-11-06T01:33:25.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3565545507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 msgr2=0x556d38f3a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3a0f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdea8027980 tx=0x7fdea8005bb0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3565545507 shutdown_connections 2023-11-06T01:33:25.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d38f200a0 0x556d38f3b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556d38f20a20 0x556d38f87810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d38e7e080 0x556d38f39bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 1 --2- 172.21.15.53:0/3565545507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d38f1f3a0 0x556d38f3a0f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fde92ffd700 0 client.15114 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:25.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3565545507 >> 172.21.15.53:0/3565545507 conn(0x556d38f19c20 msgr2=0x556d38f330e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3565545507 shutdown_connections 2023-11-06T01:33:25.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 1 -- 172.21.15.53:0/3565545507 wait complete. 2023-11-06T01:33:25.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.336+0000 7fdec5dd2b80 20 client.15114 trim_cache size 0 max 16384 2023-11-06T01:33:25.356 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:33:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-11-06T01:33:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24487, 2023-11-06T01:33:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-06T01:33:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-06T01:33:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-06T01:33:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 59.240239539999997, 2023-11-06T01:33:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 27, 2023-11-06T01:33:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 82, 2023-11-06T01:33:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 82, 2023-11-06T01:33:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 70.885975995999999 2023-11-06T01:33:25.359 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:25.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 -- 172.21.15.53:0/239205383 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde9c0a05b0 msgr2=0x7fde9c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde9c0a05b0 0x7fde9c0a2a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009af0 tx=0x7fdea80094f0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 -- 172.21.15.53:0/239205383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4101880 msgr2=0x7fdeb4196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:25.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4101880 0x7fdeb4196bd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e9a0 tx=0x7fdea400ed60 comp rx=0 tx=0).stop 2023-11-06T01:33:25.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 -- 172.21.15.53:0/239205383 shutdown_connections 2023-11-06T01:33:25.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde9c0a05b0 0x7fde9c0a2a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb4106ae0 0x7fdeb419e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4101880 0x7fdeb4196bd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 --2- 172.21.15.53:0/239205383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb41041b0 0x7fdeb4197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:25.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 -- 172.21.15.53:0/239205383 >> 172.21.15.53:0/239205383 conn(0x7fdeb40fb1f0 msgr2=0x7fdeb4102510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:25.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 -- 172.21.15.53:0/239205383 shutdown_connections 2023-11-06T01:33:25.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:25.337+0000 7fdeb9c42700 1 -- 172.21.15.53:0/239205383 wait complete. 2023-11-06T01:33:25.363 INFO:tasks.cephadm:Setup complete, yielding 2023-11-06T01:33:25.363 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-06T01:33:25.373 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi053.front.sepia.ceph.com 2023-11-06T01:33:25.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- bash -c 'ceph orch status' 2023-11-06T01:33:25.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:25 smithi062 ceph-mon[103924]: pgmap v281: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:25.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:25 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3822321107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:25.398 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:25 smithi053 ceph-mon[99472]: pgmap v281: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:25.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:25 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3822321107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:25.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:25 smithi163 ceph-mon[104416]: pgmap v281: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:25.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:25 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3822321107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:25.932 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:26.374 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-11-06T01:33:26.374 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-11-06T01:33:26.374 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-11-06T01:33:26.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2023-11-06T01:33:26.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2023-11-06T01:33:26.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-11-06T01:33:26.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-11-06T01:33:26.375 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-11-06T01:33:26.376 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-11-06T01:33:26.376 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:26 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2023-11-06T01:33:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:26 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-11-06T01:33:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:26 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-11-06T01:33:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:26 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-11-06T01:33:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:26 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2023-11-06T01:33:26.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:26 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2023-11-06T01:33:26.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:26 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-11-06T01:33:26.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:26 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-11-06T01:33:26.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:26 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-11-06T01:33:26.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:26 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-11-06T01:33:26.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:26 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2023-11-06T01:33:26.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-11-06T01:33:26.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-11-06T01:33:26.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-11-06T01:33:26.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2023-11-06T01:33:26.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2023-11-06T01:33:26.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-11-06T01:33:26.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-11-06T01:33:26.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-11-06T01:33:26.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-11-06T01:33:26.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:26 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2023-11-06T01:33:27.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.145+0000 7f1a871a5700 1 -- 172.21.15.53:0/1484124385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a800ffd20 msgr2=0x7f1a80100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:27.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.145+0000 7f1a871a5700 1 --2- 172.21.15.53:0/1484124385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a800ffd20 0x7f1a80100140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1a740099c0 tx=0x7f1a74009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:27.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.145+0000 7f1a871a5700 1 -- 172.21.15.53:0/1484124385 shutdown_connections 2023-11-06T01:33:27.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.145+0000 7f1a871a5700 1 --2- 172.21.15.53:0/1484124385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a801018c0 0x7f1a80108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:27.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.145+0000 7f1a871a5700 1 --2- 172.21.15.53:0/1484124385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a800ffd20 0x7f1a80100140 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:27.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.145+0000 7f1a871a5700 1 --2- 172.21.15.53:0/1484124385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a80100f20 0x7f1a80101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:27.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.145+0000 7f1a871a5700 1 -- 172.21.15.53:0/1484124385 >> 172.21.15.53:0/1484124385 conn(0x7f1a800fb2a0 msgr2=0x7f1a800fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:27.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.145+0000 7f1a871a5700 1 -- 172.21.15.53:0/1484124385 shutdown_connections 2023-11-06T01:33:27.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.145+0000 7f1a871a5700 1 -- 172.21.15.53:0/1484124385 wait complete. 2023-11-06T01:33:27.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a871a5700 1 Processor -- start 2023-11-06T01:33:27.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a871a5700 1 -- start start 2023-11-06T01:33:27.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a871a5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a800ffd20 0x7f1a800693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:27.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a871a5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a80100f20 0x7f1a8019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:27.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a871a5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a801018c0 0x7f1a8019de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:27.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a871a5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8019e610 con 0x7f1a800ffd20 2023-11-06T01:33:27.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a871a5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8019e780 con 0x7f1a801018c0 2023-11-06T01:33:27.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a871a5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8019e940 con 0x7f1a80100f20 2023-11-06T01:33:27.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a84f41700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a800ffd20 0x7f1a800693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:27.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a84f41700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a800ffd20 0x7f1a800693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38996/0 (socket says 172.21.15.53:38996) 2023-11-06T01:33:27.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a84f41700 1 -- 172.21.15.53:0/3958057135 learned_addr learned my addr 172.21.15.53:0/3958057135 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:27.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a7ffff700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a80100f20 0x7f1a8019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:27.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a85742700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a801018c0 0x7f1a8019de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:27.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a84f41700 1 -- 172.21.15.53:0/3958057135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a80100f20 msgr2=0x7f1a8019b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:27.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a84f41700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a80100f20 0x7f1a8019b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:27.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a84f41700 1 -- 172.21.15.53:0/3958057135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a801018c0 msgr2=0x7f1a8019de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:27.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a84f41700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a801018c0 0x7f1a8019de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:27.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.146+0000 7f1a84f41700 1 -- 172.21.15.53:0/3958057135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a740095f0 con 0x7f1a800ffd20 2023-11-06T01:33:27.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.147+0000 7f1a84f41700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a800ffd20 0x7f1a800693d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009990 tx=0x7f1a74005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:27.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.147+0000 7f1a7dffb700 1 -- 172.21.15.53:0/3958057135 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7401d070 con 0x7f1a800ffd20 2023-11-06T01:33:27.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.147+0000 7f1a7dffb700 1 -- 172.21.15.53:0/3958057135 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a74004200 con 0x7f1a800ffd20 2023-11-06T01:33:27.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.147+0000 7f1a871a5700 1 -- 172.21.15.53:0/3958057135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8019ebc0 con 0x7f1a800ffd20 2023-11-06T01:33:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.147+0000 7f1a7dffb700 1 -- 172.21.15.53:0/3958057135 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74022e30 con 0x7f1a800ffd20 2023-11-06T01:33:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.147+0000 7f1a871a5700 1 -- 172.21.15.53:0/3958057135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8019f0b0 con 0x7f1a800ffd20 2023-11-06T01:33:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.148+0000 7f1a871a5700 1 -- 172.21.15.53:0/3958057135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8010b520 con 0x7f1a800ffd20 2023-11-06T01:33:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.149+0000 7f1a7dffb700 1 -- 172.21.15.53:0/3958057135 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f1a74003d40 con 0x7f1a800ffd20 2023-11-06T01:33:27.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.150+0000 7f1a7dffb700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1a700a0630 0x7f1a700a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:27.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.150+0000 7f1a7ffff700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1a700a0630 0x7f1a700a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:27.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.150+0000 7f1a7dffb700 1 -- 172.21.15.53:0/3958057135 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1a740cd190 con 0x7f1a800ffd20 2023-11-06T01:33:27.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.150+0000 7f1a7ffff700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1a700a0630 0x7f1a700a2af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c005fd0 tx=0x7f1a6c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:27.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.153+0000 7f1a7dffb700 1 -- 172.21.15.53:0/3958057135 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1a74096be0 con 0x7f1a800ffd20 2023-11-06T01:33:27.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: pgmap v282: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-11-06T01:33:27.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-11-06T01:33:27.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-11-06T01:33:27.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-11-06T01:33:27.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2023-11-06T01:33:27.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2023-11-06T01:33:27.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-11-06T01:33:27.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-11-06T01:33:27.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-11-06T01:33:27.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-11-06T01:33:27.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2023-11-06T01:33:27.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:27 smithi053 ceph-mon[99472]: osdmap e83: 12 total, 12 up, 12 in 2023-11-06T01:33:27.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.370+0000 7f1a871a5700 1 -- 172.21.15.53:0/3958057135 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a800611d0 con 0x7f1a700a0630 2023-11-06T01:33:27.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.371+0000 7f1a7dffb700 1 -- 172.21.15.53:0/3958057135 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1a800611d0 con 0x7f1a700a0630 2023-11-06T01:33:27.372 INFO:teuthology.orchestra.run.smithi053.stdout:Backend: cephadm 2023-11-06T01:33:27.372 INFO:teuthology.orchestra.run.smithi053.stdout:Available: Yes 2023-11-06T01:33:27.372 INFO:teuthology.orchestra.run.smithi053.stdout:Paused: No 2023-11-06T01:33:27.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 -- 172.21.15.53:0/3958057135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1a700a0630 msgr2=0x7f1a700a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:27.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1a700a0630 0x7f1a700a2af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c005fd0 tx=0x7f1a6c00a560 comp rx=0 tx=0).stop 2023-11-06T01:33:27.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 -- 172.21.15.53:0/3958057135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a800ffd20 msgr2=0x7f1a800693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:27.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a800ffd20 0x7f1a800693d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009990 tx=0x7f1a74005e80 comp rx=0 tx=0).stop 2023-11-06T01:33:27.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 -- 172.21.15.53:0/3958057135 shutdown_connections 2023-11-06T01:33:27.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1a700a0630 0x7f1a700a2af0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:27.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a801018c0 0x7f1a8019de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:27.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a800ffd20 0x7f1a800693d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:27.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 --2- 172.21.15.53:0/3958057135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a80100f20 0x7f1a8019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:27.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 -- 172.21.15.53:0/3958057135 >> 172.21.15.53:0/3958057135 conn(0x7f1a800fb2a0 msgr2=0x7f1a800fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:27.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 -- 172.21.15.53:0/3958057135 shutdown_connections 2023-11-06T01:33:27.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:27.373+0000 7f1a871a5700 1 -- 172.21.15.53:0/3958057135 wait complete. 2023-11-06T01:33:27.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: pgmap v282: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-11-06T01:33:27.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-11-06T01:33:27.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-11-06T01:33:27.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-11-06T01:33:27.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2023-11-06T01:33:27.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2023-11-06T01:33:27.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-11-06T01:33:27.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-11-06T01:33:27.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-11-06T01:33:27.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-11-06T01:33:27.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2023-11-06T01:33:27.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:27 smithi062 ceph-mon[103924]: osdmap e83: 12 total, 12 up, 12 in 2023-11-06T01:33:27.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: pgmap v282: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-11-06T01:33:27.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-11-06T01:33:27.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-11-06T01:33:27.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-11-06T01:33:27.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2023-11-06T01:33:27.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2023-11-06T01:33:27.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-11-06T01:33:27.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-11-06T01:33:27.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-11-06T01:33:27.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-11-06T01:33:27.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2023-11-06T01:33:27.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:27 smithi163 ceph-mon[104416]: osdmap e83: 12 total, 12 up, 12 in 2023-11-06T01:33:27.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-11-06T01:33:28.382 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:28.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:28 smithi062 ceph-mon[103924]: osdmap e84: 12 total, 12 up, 12 in 2023-11-06T01:33:28.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:28 smithi163 ceph-mon[104416]: osdmap e84: 12 total, 12 up, 12 in 2023-11-06T01:33:28.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:28 smithi053 ceph-mon[99472]: osdmap e84: 12 total, 12 up, 12 in 2023-11-06T01:33:29.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:29 smithi053 ceph-mon[99472]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:29.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:29 smithi053 ceph-mon[99472]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-11-06T01:33:29.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:29 smithi053 ceph-mon[99472]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-11-06T01:33:29.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:29 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:33:29.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:29 smithi062 ceph-mon[103924]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:29.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:29 smithi062 ceph-mon[103924]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-11-06T01:33:29.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:29 smithi062 ceph-mon[103924]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-11-06T01:33:29.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:29 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:33:29.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:29 smithi163 ceph-mon[104416]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:29.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:29 smithi163 ceph-mon[104416]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-11-06T01:33:29.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:29 smithi163 ceph-mon[104416]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-11-06T01:33:29.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:29 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:33:29.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.699+0000 7fb34e2b7700 1 -- 172.21.15.53:0/760046786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb348102970 msgr2=0x7fb348109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:29.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.699+0000 7fb34e2b7700 1 --2- 172.21.15.53:0/760046786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb348102970 0x7fb348109290 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb34000b0a0 tx=0x7fb34000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:29.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.699+0000 7fb34e2b7700 1 -- 172.21.15.53:0/760046786 shutdown_connections 2023-11-06T01:33:29.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.699+0000 7fb34e2b7700 1 --2- 172.21.15.53:0/760046786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb348100e50 0x7fb348101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:29.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.699+0000 7fb34e2b7700 1 --2- 172.21.15.53:0/760046786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb348102970 0x7fb348109290 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:29.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.699+0000 7fb34e2b7700 1 --2- 172.21.15.53:0/760046786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb348101fb0 0x7fb348102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:29.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.699+0000 7fb34e2b7700 1 -- 172.21.15.53:0/760046786 >> 172.21.15.53:0/760046786 conn(0x7fb3480fc410 msgr2=0x7fb3480fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:29.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.699+0000 7fb34e2b7700 1 -- 172.21.15.53:0/760046786 shutdown_connections 2023-11-06T01:33:29.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.699+0000 7fb34e2b7700 1 -- 172.21.15.53:0/760046786 wait complete. 2023-11-06T01:33:29.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb34e2b7700 1 Processor -- start 2023-11-06T01:33:29.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb34e2b7700 1 -- start start 2023-11-06T01:33:29.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb34e2b7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb348100e50 0x7fb348196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:29.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb34e2b7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb348101fb0 0x7fb3481970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:29.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb34e2b7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb348102970 0x7fb34819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:29.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb34e2b7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34819e740 con 0x7fb348102970 2023-11-06T01:33:29.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb34e2b7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34819e880 con 0x7fb348101fb0 2023-11-06T01:33:29.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb34e2b7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34819ea10 con 0x7fb348100e50 2023-11-06T01:33:29.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb34c854700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb348102970 0x7fb34819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:29.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb347fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb348100e50 0x7fb348196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:29.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb347fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb348100e50 0x7fb348196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:53248/0 (socket says 172.21.15.53:53248) 2023-11-06T01:33:29.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb347fff700 1 -- 172.21.15.53:0/3199127033 learned_addr learned my addr 172.21.15.53:0/3199127033 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:29.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb33ffff700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb348101fb0 0x7fb3481970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:29.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb33ffff700 1 -- 172.21.15.53:0/3199127033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb348100e50 msgr2=0x7fb348196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:29.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb33ffff700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb348100e50 0x7fb348196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:29.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb33ffff700 1 -- 172.21.15.53:0/3199127033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb348102970 msgr2=0x7fb34819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:29.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb33ffff700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb348102970 0x7fb34819e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:29.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb33ffff700 1 -- 172.21.15.53:0/3199127033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb340009d00 con 0x7fb348101fb0 2023-11-06T01:33:29.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.700+0000 7fb347fff700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb348100e50 0x7fb348196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:33:29.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.701+0000 7fb33ffff700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb348101fb0 0x7fb3481970f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e9f0 tx=0x7fb33000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:29.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.701+0000 7fb345ffb700 1 -- 172.21.15.53:0/3199127033 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000c9c0 con 0x7fb348101fb0 2023-11-06T01:33:29.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.701+0000 7fb345ffb700 1 -- 172.21.15.53:0/3199127033 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb330010430 con 0x7fb348101fb0 2023-11-06T01:33:29.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.701+0000 7fb345ffb700 1 -- 172.21.15.53:0/3199127033 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330018680 con 0x7fb348101fb0 2023-11-06T01:33:29.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.701+0000 7fb34e2b7700 1 -- 172.21.15.53:0/3199127033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34819ecf0 con 0x7fb348101fb0 2023-11-06T01:33:29.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.701+0000 7fb34e2b7700 1 -- 172.21.15.53:0/3199127033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34819f1a0 con 0x7fb348101fb0 2023-11-06T01:33:29.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.702+0000 7fb34e2b7700 1 -- 172.21.15.53:0/3199127033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb348108330 con 0x7fb348101fb0 2023-11-06T01:33:29.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.703+0000 7fb345ffb700 1 -- 172.21.15.53:0/3199127033 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fb330010860 con 0x7fb348101fb0 2023-11-06T01:33:29.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.703+0000 7fb345ffb700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3340a05b0 0x7fb3340a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:29.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.704+0000 7fb347fff700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3340a05b0 0x7fb3340a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:29.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.704+0000 7fb345ffb700 1 -- 172.21.15.53:0/3199127033 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb330014070 con 0x7fb348101fb0 2023-11-06T01:33:29.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.704+0000 7fb347fff700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3340a05b0 0x7fb3340a2a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb338011550 tx=0x7fb338009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:29.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.707+0000 7fb345ffb700 1 -- 172.21.15.53:0/3199127033 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb330095830 con 0x7fb348101fb0 2023-11-06T01:33:29.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.934+0000 7fb34e2b7700 1 -- 172.21.15.53:0/3199127033 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3480611d0 con 0x7fb3340a05b0 2023-11-06T01:33:29.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.965+0000 7fb345ffb700 1 -- 172.21.15.53:0/3199127033 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fb3480611d0 con 0x7fb3340a05b0 2023-11-06T01:33:29.967 INFO:teuthology.orchestra.run.smithi053.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-11-06T01:33:29.968 INFO:teuthology.orchestra.run.smithi053.stdout:mds.a smithi053 running (71s) 61s ago 70s 16.8M - 18.2.0-1181-gf7e9f9af fe94629dd841 e80f60241caf 2023-11-06T01:33:29.969 INFO:teuthology.orchestra.run.smithi053.stdout:mds.b smithi062 running (69s) 61s ago 68s 14.0M - 18.2.0-1181-gf7e9f9af fe94629dd841 7c14addd1cc1 2023-11-06T01:33:29.969 INFO:teuthology.orchestra.run.smithi053.stdout:mds.c smithi163 running (80s) 61s ago 79s 14.4M - 18.2.0-1181-gf7e9f9af fe94629dd841 90aa7c916ad9 2023-11-06T01:33:29.969 INFO:teuthology.orchestra.run.smithi053.stdout:mds.d smithi053 running (87s) 61s ago 86s 16.9M - 18.2.0-1181-gf7e9f9af fe94629dd841 a74c678039c5 2023-11-06T01:33:29.970 INFO:teuthology.orchestra.run.smithi053.stdout:mds.e smithi062 running (84s) 61s ago 84s 14.1M - 18.2.0-1181-gf7e9f9af fe94629dd841 c9a617deb6b7 2023-11-06T01:33:29.970 INFO:teuthology.orchestra.run.smithi053.stdout:mds.f smithi163 running (82s) 61s ago 82s 15.7M - 18.2.0-1181-gf7e9f9af fe94629dd841 3258437e66a8 2023-11-06T01:33:29.970 INFO:teuthology.orchestra.run.smithi053.stdout:mds.g smithi053 running (73s) 61s ago 73s 15.7M - 18.2.0-1181-gf7e9f9af fe94629dd841 c377542377e7 2023-11-06T01:33:29.972 INFO:teuthology.orchestra.run.smithi053.stdout:mds.h smithi062 running (78s) 61s ago 77s 16.7M - 18.2.0-1181-gf7e9f9af fe94629dd841 b00858d5de9b 2023-11-06T01:33:29.972 INFO:teuthology.orchestra.run.smithi053.stdout:mds.i smithi163 running (75s) 61s ago 75s 15.7M - 18.2.0-1181-gf7e9f9af fe94629dd841 aea540823ced 2023-11-06T01:33:29.972 INFO:teuthology.orchestra.run.smithi053.stdout:mds.j smithi053 running (89s) 61s ago 88s 15.9M - 18.2.0-1181-gf7e9f9af fe94629dd841 add7cb9622f1 2023-11-06T01:33:29.972 INFO:teuthology.orchestra.run.smithi053.stdout:mds.k smithi062 running (91s) 61s ago 91s 14.2M - 18.2.0-1181-gf7e9f9af fe94629dd841 13ea49f7a8bb 2023-11-06T01:33:29.973 INFO:teuthology.orchestra.run.smithi053.stdout:mds.l smithi163 running (93s) 61s ago 93s 16.7M - 18.2.0-1181-gf7e9f9af fe94629dd841 3991491d8cbf 2023-11-06T01:33:29.973 INFO:teuthology.orchestra.run.smithi053.stdout:mgr.x smithi053 *:9283,8765 running (7m) 61s ago 7m 515M - 18.2.0-1181-gf7e9f9af fe94629dd841 48add67455ec 2023-11-06T01:33:29.973 INFO:teuthology.orchestra.run.smithi053.stdout:mgr.y smithi062 *:8443,8765 running (6m) 61s ago 6m 451M - 18.2.0-1181-gf7e9f9af fe94629dd841 3ace7424ff4f 2023-11-06T01:33:29.973 INFO:teuthology.orchestra.run.smithi053.stdout:mgr.z smithi163 *:8443,8765 running (6m) 61s ago 6m 451M - 18.2.0-1181-gf7e9f9af fe94629dd841 4fb48f8bee1a 2023-11-06T01:33:29.974 INFO:teuthology.orchestra.run.smithi053.stdout:mon.a smithi053 running (7m) 61s ago 8m 83.5M 2048M 18.2.0-1181-gf7e9f9af fe94629dd841 9fac24cd4d2a 2023-11-06T01:33:29.974 INFO:teuthology.orchestra.run.smithi053.stdout:mon.b smithi062 running (6m) 61s ago 6m 59.2M 2048M 18.2.0-1181-gf7e9f9af fe94629dd841 28bde106559b 2023-11-06T01:33:29.974 INFO:teuthology.orchestra.run.smithi053.stdout:mon.c smithi163 running (6m) 61s ago 6m 58.9M 2048M 18.2.0-1181-gf7e9f9af fe94629dd841 6690a5c21368 2023-11-06T01:33:29.974 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0 smithi053 running (5m) 61s ago 5m 60.0M 4293M 18.2.0-1181-gf7e9f9af fe94629dd841 d00b1112710a 2023-11-06T01:33:29.975 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1 smithi062 running (5m) 61s ago 5m 65.3M 4292M 18.2.0-1181-gf7e9f9af fe94629dd841 5ade8b20ad1d 2023-11-06T01:33:29.975 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2 smithi163 running (5m) 61s ago 5m 63.0M 4292M 18.2.0-1181-gf7e9f9af fe94629dd841 e77d0329dea0 2023-11-06T01:33:29.975 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3 smithi053 running (4m) 61s ago 4m 57.3M 4293M 18.2.0-1181-gf7e9f9af fe94629dd841 1e14aaa94e2e 2023-11-06T01:33:29.975 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4 smithi062 running (4m) 61s ago 4m 50.3M 4292M 18.2.0-1181-gf7e9f9af fe94629dd841 70d2af0dc412 2023-11-06T01:33:29.976 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5 smithi163 running (3m) 61s ago 3m 55.5M 4292M 18.2.0-1181-gf7e9f9af fe94629dd841 0ce1f6635b91 2023-11-06T01:33:29.976 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6 smithi053 running (3m) 61s ago 3m 51.7M 4293M 18.2.0-1181-gf7e9f9af fe94629dd841 cd7034ee884e 2023-11-06T01:33:29.976 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7 smithi062 running (3m) 61s ago 3m 55.1M 4292M 18.2.0-1181-gf7e9f9af fe94629dd841 236b834ebde4 2023-11-06T01:33:29.976 INFO:teuthology.orchestra.run.smithi053.stdout:osd.8 smithi163 running (2m) 61s ago 2m 52.5M 4292M 18.2.0-1181-gf7e9f9af fe94629dd841 f24d20dcb8d4 2023-11-06T01:33:29.977 INFO:teuthology.orchestra.run.smithi053.stdout:osd.9 smithi053 running (2m) 61s ago 2m 48.4M 4293M 18.2.0-1181-gf7e9f9af fe94629dd841 522da08bc97f 2023-11-06T01:33:29.977 INFO:teuthology.orchestra.run.smithi053.stdout:osd.10 smithi062 running (2m) 61s ago 2m 53.1M 4292M 18.2.0-1181-gf7e9f9af fe94629dd841 c64d9e656f6f 2023-11-06T01:33:29.977 INFO:teuthology.orchestra.run.smithi053.stdout:osd.11 smithi163 running (111s) 61s ago 111s 44.8M 4292M 18.2.0-1181-gf7e9f9af fe94629dd841 e990bbfdc77e 2023-11-06T01:33:29.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 -- 172.21.15.53:0/3199127033 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3340a05b0 msgr2=0x7fb3340a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:29.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3340a05b0 0x7fb3340a2a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb338011550 tx=0x7fb338009260 comp rx=0 tx=0).stop 2023-11-06T01:33:29.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 -- 172.21.15.53:0/3199127033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb348101fb0 msgr2=0x7fb3481970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:29.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb348101fb0 0x7fb3481970f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e9f0 tx=0x7fb33000ed00 comp rx=0 tx=0).stop 2023-11-06T01:33:29.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 -- 172.21.15.53:0/3199127033 shutdown_connections 2023-11-06T01:33:29.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb3340a05b0 0x7fb3340a2a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:29.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb348102970 0x7fb34819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:29.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb348100e50 0x7fb348196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:29.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 --2- 172.21.15.53:0/3199127033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb348101fb0 0x7fb3481970f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:29.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 -- 172.21.15.53:0/3199127033 >> 172.21.15.53:0/3199127033 conn(0x7fb3480fc410 msgr2=0x7fb348105470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:29.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 -- 172.21.15.53:0/3199127033 shutdown_connections 2023-11-06T01:33:29.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:29.967+0000 7fb34e2b7700 1 -- 172.21.15.53:0/3199127033 wait complete. 2023-11-06T01:33:30.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-11-06T01:33:31.005 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:31.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:31 smithi062 ceph-mon[103924]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 204 B/s, 0 keys/s, 1 objects/s recovering 2023-11-06T01:33:31.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:31 smithi062 ceph-mon[103924]: from='client.24791 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:31.421 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:31 smithi053 ceph-mon[99472]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 204 B/s, 0 keys/s, 1 objects/s recovering 2023-11-06T01:33:31.422 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:31 smithi053 ceph-mon[99472]: from='client.24791 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:31.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:31 smithi163 ceph-mon[104416]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 204 B/s, 0 keys/s, 1 objects/s recovering 2023-11-06T01:33:31.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:31 smithi163 ceph-mon[104416]: from='client.24791 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:32.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.265+0000 7f7beb297700 1 -- 172.21.15.53:0/2474767937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7be40ffd20 msgr2=0x7f7be4100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:32.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.265+0000 7f7beb297700 1 --2- 172.21.15.53:0/2474767937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7be40ffd20 0x7f7be4100140 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7be000b0d0 tx=0x7f7be000b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:32.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.265+0000 7f7beb297700 1 -- 172.21.15.53:0/2474767937 shutdown_connections 2023-11-06T01:33:32.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.265+0000 7f7beb297700 1 --2- 172.21.15.53:0/2474767937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7be41018c0 0x7f7be4108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:32.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.265+0000 7f7beb297700 1 --2- 172.21.15.53:0/2474767937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7be40ffd20 0x7f7be4100140 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:32.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.265+0000 7f7beb297700 1 --2- 172.21.15.53:0/2474767937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be4100f20 0x7f7be4101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:32.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.265+0000 7f7beb297700 1 -- 172.21.15.53:0/2474767937 >> 172.21.15.53:0/2474767937 conn(0x7f7be40fb2a0 msgr2=0x7f7be40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:32.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.266+0000 7f7beb297700 1 -- 172.21.15.53:0/2474767937 shutdown_connections 2023-11-06T01:33:32.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.266+0000 7f7beb297700 1 -- 172.21.15.53:0/2474767937 wait complete. 2023-11-06T01:33:32.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.266+0000 7f7beb297700 1 Processor -- start 2023-11-06T01:33:32.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.266+0000 7f7beb297700 1 -- start start 2023-11-06T01:33:32.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.266+0000 7f7beb297700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be40ffd20 0x7f7be40693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:32.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.266+0000 7f7beb297700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7be4100f20 0x7f7be419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:32.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.266+0000 7f7beb297700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7be41018c0 0x7f7be419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:32.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.266+0000 7f7beb297700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be419e780 con 0x7f7be41018c0 2023-11-06T01:33:32.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.266+0000 7f7beb297700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be419e8f0 con 0x7f7be4100f20 2023-11-06T01:33:32.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.266+0000 7f7beb297700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be419eab0 con 0x7f7be40ffd20 2023-11-06T01:33:32.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7be9834700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7be41018c0 0x7f7be419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:32.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7be9834700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7be41018c0 0x7f7be419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39022/0 (socket says 172.21.15.53:39022) 2023-11-06T01:33:32.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7be9834700 1 -- 172.21.15.53:0/2326077300 learned_addr learned my addr 172.21.15.53:0/2326077300 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:32.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7be8832700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7be4100f20 0x7f7be419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:32.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7be9033700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be40ffd20 0x7f7be40693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:32.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7be9834700 1 -- 172.21.15.53:0/2326077300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be40ffd20 msgr2=0x7f7be40693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:32.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7be9834700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be40ffd20 0x7f7be40693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:32.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7be9834700 1 -- 172.21.15.53:0/2326077300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7be4100f20 msgr2=0x7f7be419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:32.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7be9834700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7be4100f20 0x7f7be419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:32.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7be9834700 1 -- 172.21.15.53:0/2326077300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be0009d00 con 0x7f7be41018c0 2023-11-06T01:33:32.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7be9834700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7be41018c0 0x7f7be419dfe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c970 tx=0x7f7bdc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:32.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7bda7fc700 1 -- 172.21.15.53:0/2326077300 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc007b00 con 0x7f7be41018c0 2023-11-06T01:33:32.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7bda7fc700 1 -- 172.21.15.53:0/2326077300 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bdc00ce60 con 0x7f7be41018c0 2023-11-06T01:33:32.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7beb297700 1 -- 172.21.15.53:0/2326077300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be419ed90 con 0x7f7be41018c0 2023-11-06T01:33:32.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7bda7fc700 1 -- 172.21.15.53:0/2326077300 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc005470 con 0x7f7be41018c0 2023-11-06T01:33:32.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.267+0000 7f7beb297700 1 -- 172.21.15.53:0/2326077300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be419f230 con 0x7f7be41018c0 2023-11-06T01:33:32.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.268+0000 7f7beb297700 1 -- 172.21.15.53:0/2326077300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be4107360 con 0x7f7be41018c0 2023-11-06T01:33:32.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.270+0000 7f7bda7fc700 1 -- 172.21.15.53:0/2326077300 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f7bdc0055d0 con 0x7f7be41018c0 2023-11-06T01:33:32.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.271+0000 7f7bda7fc700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bd00a05b0 0x7f7bd00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:32.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.271+0000 7f7be9033700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bd00a05b0 0x7f7bd00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:32.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.271+0000 7f7bda7fc700 1 -- 172.21.15.53:0/2326077300 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7bdc0cbaf0 con 0x7f7be41018c0 2023-11-06T01:33:32.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.271+0000 7f7be9033700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bd00a05b0 0x7f7bd00a2a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009fd0 tx=0x7f7be000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:32.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.274+0000 7f7bda7fc700 1 -- 172.21.15.53:0/2326077300 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7bdc095510 con 0x7f7be41018c0 2023-11-06T01:33:32.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:32 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.483+0000 7f7beb297700 1 -- 172.21.15.53:0/2326077300 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7be40611d0 con 0x7f7bd00a05b0 2023-11-06T01:33:32.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:32 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.501+0000 7f7bda7fc700 1 -- 172.21.15.53:0/2326077300 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f7be40611d0 con 0x7f7bd00a05b0 2023-11-06T01:33:32.503 INFO:teuthology.orchestra.run.smithi053.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-11-06T01:33:32.504 INFO:teuthology.orchestra.run.smithi053.stdout:mds.all 12/12 1s ago 98s smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2023-11-06T01:33:32.504 INFO:teuthology.orchestra.run.smithi053.stdout:mgr 3/3 1s ago 6m smithi053=x;smithi062=y;smithi163=z;count:3 2023-11-06T01:33:32.504 INFO:teuthology.orchestra.run.smithi053.stdout:mon 3/3 1s ago 6m smithi053:172.21.15.53=a;smithi062:172.21.15.62=b;smithi163:172.21.15.163=c;count:3 2023-11-06T01:33:32.504 INFO:teuthology.orchestra.run.smithi053.stdout:osd 12 1s ago - 2023-11-06T01:33:32.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.503+0000 7f7beb297700 1 -- 172.21.15.53:0/2326077300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bd00a05b0 msgr2=0x7f7bd00a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:32.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.504+0000 7f7beb297700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bd00a05b0 0x7f7bd00a2a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009fd0 tx=0x7f7be000b040 comp rx=0 tx=0).stop 2023-11-06T01:33:32.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.504+0000 7f7beb297700 1 -- 172.21.15.53:0/2326077300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7be41018c0 msgr2=0x7f7be419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:32.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.504+0000 7f7beb297700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7be41018c0 0x7f7be419dfe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c970 tx=0x7f7bdc00cc80 comp rx=0 tx=0).stop 2023-11-06T01:33:32.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.504+0000 7f7beb297700 1 -- 172.21.15.53:0/2326077300 shutdown_connections 2023-11-06T01:33:32.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.504+0000 7f7beb297700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7bd00a05b0 0x7f7bd00a2a70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:32.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.504+0000 7f7beb297700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7be41018c0 0x7f7be419dfe0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:32.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.504+0000 7f7beb297700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be40ffd20 0x7f7be40693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:32.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.504+0000 7f7beb297700 1 --2- 172.21.15.53:0/2326077300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7be4100f20 0x7f7be419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:32.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.504+0000 7f7beb297700 1 -- 172.21.15.53:0/2326077300 >> 172.21.15.53:0/2326077300 conn(0x7f7be40fb2a0 msgr2=0x7f7be41044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:32.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.504+0000 7f7beb297700 1 -- 172.21.15.53:0/2326077300 shutdown_connections 2023-11-06T01:33:32.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:32.504+0000 7f7beb297700 1 -- 172.21.15.53:0/2326077300 wait complete. 2023-11-06T01:33:32.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:32 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:32.870 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-11-06T01:33:33.429 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:33.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:33 smithi163 ceph-mon[104416]: pgmap v287: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 228 B/s, 2 keys/s, 2 objects/s recovering 2023-11-06T01:33:33.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:33 smithi163 ceph-mon[104416]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-11-06T01:33:33.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:33 smithi163 ceph-mon[104416]: Cluster is now healthy 2023-11-06T01:33:33.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:33:33.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:33:33.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:33.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:33 smithi053 ceph-mon[99472]: pgmap v287: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 228 B/s, 2 keys/s, 2 objects/s recovering 2023-11-06T01:33:33.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:33 smithi053 ceph-mon[99472]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-11-06T01:33:33.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:33 smithi053 ceph-mon[99472]: Cluster is now healthy 2023-11-06T01:33:33.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:33:33.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:33:33.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:33.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:33 smithi062 ceph-mon[103924]: pgmap v287: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 228 B/s, 2 keys/s, 2 objects/s recovering 2023-11-06T01:33:33.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:33 smithi062 ceph-mon[103924]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-11-06T01:33:33.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:33 smithi062 ceph-mon[103924]: Cluster is now healthy 2023-11-06T01:33:33.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:33:33.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:33:33.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:33:34.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:34 smithi163 ceph-mon[104416]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:34.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:34 smithi053 ceph-mon[99472]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:34.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:34 smithi062 ceph-mon[103924]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:34.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.658+0000 7f7062689700 1 -- 172.21.15.53:0/2572209285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f705c0ffd20 msgr2=0x7f705c100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:34.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.658+0000 7f7062689700 1 --2- 172.21.15.53:0/2572209285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f705c0ffd20 0x7f705c100140 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f704c0099c0 tx=0x7f704c009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:34.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.658+0000 7f7062689700 1 -- 172.21.15.53:0/2572209285 shutdown_connections 2023-11-06T01:33:34.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.658+0000 7f7062689700 1 --2- 172.21.15.53:0/2572209285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c1018c0 0x7f705c108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:34.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.658+0000 7f7062689700 1 --2- 172.21.15.53:0/2572209285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f705c0ffd20 0x7f705c100140 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:34.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.658+0000 7f7062689700 1 --2- 172.21.15.53:0/2572209285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c100f20 0x7f705c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:34.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.658+0000 7f7062689700 1 -- 172.21.15.53:0/2572209285 >> 172.21.15.53:0/2572209285 conn(0x7f705c0fb2a0 msgr2=0x7f705c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:34.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.658+0000 7f7062689700 1 -- 172.21.15.53:0/2572209285 shutdown_connections 2023-11-06T01:33:34.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.658+0000 7f7062689700 1 -- 172.21.15.53:0/2572209285 wait complete. 2023-11-06T01:33:34.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.659+0000 7f7062689700 1 Processor -- start 2023-11-06T01:33:34.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.659+0000 7f7062689700 1 -- start start 2023-11-06T01:33:34.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.659+0000 7f7062689700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c0ffd20 0x7f705c0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:34.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.659+0000 7f7062689700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f705c100f20 0x7f705c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:34.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.659+0000 7f7062689700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c1018c0 0x7f705c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:34.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.659+0000 7f7062689700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c19e6a0 con 0x7f705c100f20 2023-11-06T01:33:34.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.659+0000 7f7062689700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c19e810 con 0x7f705c0ffd20 2023-11-06T01:33:34.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.659+0000 7f7062689700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c19e9d0 con 0x7f705c1018c0 2023-11-06T01:33:34.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f705b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f705c100f20 0x7f705c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:34.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f705b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f705c100f20 0x7f705c19b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39034/0 (socket says 172.21.15.53:39034) 2023-11-06T01:33:34.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f705b7fe700 1 -- 172.21.15.53:0/3916736328 learned_addr learned my addr 172.21.15.53:0/3916736328 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:34.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f705bfff700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c0ffd20 0x7f705c0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:34.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f7060c26700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c1018c0 0x7f705c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:34.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f705b7fe700 1 -- 172.21.15.53:0/3916736328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c1018c0 msgr2=0x7f705c19df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:34.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f705b7fe700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c1018c0 0x7f705c19df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:34.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f705b7fe700 1 -- 172.21.15.53:0/3916736328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c0ffd20 msgr2=0x7f705c0693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:34.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f705b7fe700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c0ffd20 0x7f705c0693b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:34.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f705b7fe700 1 -- 172.21.15.53:0/3916736328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c0095f0 con 0x7f705c100f20 2023-11-06T01:33:34.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f7060c26700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c1018c0 0x7f705c19df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:34.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f705b7fe700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f705c100f20 0x7f705c19b8c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f705000d950 tx=0x7f705000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:34.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f70597fa700 1 -- 172.21.15.53:0/3916736328 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050009b40 con 0x7f705c100f20 2023-11-06T01:33:34.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f70597fa700 1 -- 172.21.15.53:0/3916736328 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7050009ca0 con 0x7f705c100f20 2023-11-06T01:33:34.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f70597fa700 1 -- 172.21.15.53:0/3916736328 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705000f620 con 0x7f705c100f20 2023-11-06T01:33:34.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f7062689700 1 -- 172.21.15.53:0/3916736328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c19ecb0 con 0x7f705c100f20 2023-11-06T01:33:34.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.660+0000 7f7062689700 1 -- 172.21.15.53:0/3916736328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c19f160 con 0x7f705c100f20 2023-11-06T01:33:34.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.661+0000 7f7062689700 1 -- 172.21.15.53:0/3916736328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c107310 con 0x7f705c100f20 2023-11-06T01:33:34.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.662+0000 7f70597fa700 1 -- 172.21.15.53:0/3916736328 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f7050010460 con 0x7f705c100f20 2023-11-06T01:33:34.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.663+0000 7f70597fa700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70440a05b0 0x7f70440a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:34.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.663+0000 7f705bfff700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70440a05b0 0x7f70440a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:34.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.663+0000 7f70597fa700 1 -- 172.21.15.53:0/3916736328 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f70500cbed0 con 0x7f705c100f20 2023-11-06T01:33:34.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.663+0000 7f705bfff700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70440a05b0 0x7f70440a2a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f704c00b580 tx=0x7f704c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:34.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.667+0000 7f70597fa700 1 -- 172.21.15.53:0/3916736328 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f70500958f0 con 0x7f705c100f20 2023-11-06T01:33:34.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.871+0000 7f7062689700 1 -- 172.21.15.53:0/3916736328 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f705c0611d0 con 0x7f70440a05b0 2023-11-06T01:33:34.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.873+0000 7f70597fa700 1 -- 172.21.15.53:0/3916736328 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f705c0611d0 con 0x7f70440a05b0 2023-11-06T01:33:34.874 INFO:teuthology.orchestra.run.smithi053.stdout:HOST ADDR LABELS STATUS 2023-11-06T01:33:34.875 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053 172.21.15.53 2023-11-06T01:33:34.875 INFO:teuthology.orchestra.run.smithi053.stdout:smithi062 172.21.15.62 2023-11-06T01:33:34.875 INFO:teuthology.orchestra.run.smithi053.stdout:smithi163 172.21.15.163 2023-11-06T01:33:34.875 INFO:teuthology.orchestra.run.smithi053.stdout:3 hosts in cluster 2023-11-06T01:33:34.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 -- 172.21.15.53:0/3916736328 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70440a05b0 msgr2=0x7f70440a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:34.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70440a05b0 0x7f70440a2a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f704c00b580 tx=0x7f704c005e80 comp rx=0 tx=0).stop 2023-11-06T01:33:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 -- 172.21.15.53:0/3916736328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f705c100f20 msgr2=0x7f705c19b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f705c100f20 0x7f705c19b8c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f705000d950 tx=0x7f705000dd10 comp rx=0 tx=0).stop 2023-11-06T01:33:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 -- 172.21.15.53:0/3916736328 shutdown_connections 2023-11-06T01:33:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70440a05b0 0x7f70440a2a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c0ffd20 0x7f705c0693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c1018c0 0x7f705c19df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 --2- 172.21.15.53:0/3916736328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f705c100f20 0x7f705c19b8c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 -- 172.21.15.53:0/3916736328 >> 172.21.15.53:0/3916736328 conn(0x7f705c0fb2a0 msgr2=0x7f705c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 -- 172.21.15.53:0/3916736328 shutdown_connections 2023-11-06T01:33:34.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:34.875+0000 7f7062689700 1 -- 172.21.15.53:0/3916736328 wait complete. 2023-11-06T01:33:35.289 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-11-06T01:33:35.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:35 smithi163 ceph-mon[104416]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 228 B/s, 2 keys/s, 2 objects/s recovering 2023-11-06T01:33:35.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:35 smithi053 ceph-mon[99472]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 228 B/s, 2 keys/s, 2 objects/s recovering 2023-11-06T01:33:35.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:35 smithi062 ceph-mon[103924]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 228 B/s, 2 keys/s, 2 objects/s recovering 2023-11-06T01:33:35.849 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:36.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:36 smithi163 ceph-mon[104416]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:36.522 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:36 smithi053 ceph-mon[99472]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:36.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:36 smithi062 ceph-mon[103924]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.413+0000 7fc9562a7700 1 -- 172.21.15.53:0/222299542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950101930 msgr2=0x7fc950103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.413+0000 7fc9562a7700 1 --2- 172.21.15.53:0/222299542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950101930 0x7fc950103d20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc940009990 tx=0x7fc940009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.413+0000 7fc9562a7700 1 -- 172.21.15.53:0/222299542 shutdown_connections 2023-11-06T01:33:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.413+0000 7fc9562a7700 1 --2- 172.21.15.53:0/222299542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc950106b90 0x7fc950108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.413+0000 7fc9562a7700 1 --2- 172.21.15.53:0/222299542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950101930 0x7fc950103d20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.413+0000 7fc9562a7700 1 --2- 172.21.15.53:0/222299542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc950104260 0x7fc950106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.413+0000 7fc9562a7700 1 -- 172.21.15.53:0/222299542 >> 172.21.15.53:0/222299542 conn(0x7fc9500fb2a0 msgr2=0x7fc9500fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:37.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.413+0000 7fc9562a7700 1 -- 172.21.15.53:0/222299542 shutdown_connections 2023-11-06T01:33:37.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.413+0000 7fc9562a7700 1 -- 172.21.15.53:0/222299542 wait complete. 2023-11-06T01:33:37.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.414+0000 7fc9562a7700 1 Processor -- start 2023-11-06T01:33:37.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.414+0000 7fc9562a7700 1 -- start start 2023-11-06T01:33:37.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc9562a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc950101930 0x7fc950196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:37.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc9562a7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc950104260 0x7fc9501970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:37.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc9562a7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950106b90 0x7fc95019e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:37.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc9562a7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95019e710 con 0x7fc950106b90 2023-11-06T01:33:37.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc9562a7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95019e850 con 0x7fc950104260 2023-11-06T01:33:37.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc9562a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95019e9e0 con 0x7fc950101930 2023-11-06T01:33:37.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc954844700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950106b90 0x7fc95019e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:37.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc954844700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950106b90 0x7fc95019e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39048/0 (socket says 172.21.15.53:39048) 2023-11-06T01:33:37.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc954844700 1 -- 172.21.15.53:0/3318952474 learned_addr learned my addr 172.21.15.53:0/3318952474 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:37.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc94ffff700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc950101930 0x7fc950196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:37.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc94f7fe700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc950104260 0x7fc9501970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:37.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc954844700 1 -- 172.21.15.53:0/3318952474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc950101930 msgr2=0x7fc950196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:37.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc954844700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc950101930 0x7fc950196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:37.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc954844700 1 -- 172.21.15.53:0/3318952474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc950104260 msgr2=0x7fc9501970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:37.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc954844700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc950104260 0x7fc9501970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:37.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc954844700 1 -- 172.21.15.53:0/3318952474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400095f0 con 0x7fc950106b90 2023-11-06T01:33:37.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc954844700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950106b90 0x7fc95019e140 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc94800ba10 tx=0x7fc94800bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:37.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc94d7fa700 1 -- 172.21.15.53:0/3318952474 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800d8e0 con 0x7fc950106b90 2023-11-06T01:33:37.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc94d7fa700 1 -- 172.21.15.53:0/3318952474 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc948009880 con 0x7fc950106b90 2023-11-06T01:33:37.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc9562a7700 1 -- 172.21.15.53:0/3318952474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95019ecc0 con 0x7fc950106b90 2023-11-06T01:33:37.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.415+0000 7fc94d7fa700 1 -- 172.21.15.53:0/3318952474 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9480143d0 con 0x7fc950106b90 2023-11-06T01:33:37.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.416+0000 7fc9562a7700 1 -- 172.21.15.53:0/3318952474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95019f170 con 0x7fc950106b90 2023-11-06T01:33:37.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.416+0000 7fc9562a7700 1 -- 172.21.15.53:0/3318952474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95010c6b0 con 0x7fc950106b90 2023-11-06T01:33:37.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.418+0000 7fc94d7fa700 1 -- 172.21.15.53:0/3318952474 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc948014530 con 0x7fc950106b90 2023-11-06T01:33:37.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.418+0000 7fc94d7fa700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9380a05b0 0x7fc9380a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:37.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.418+0000 7fc94ffff700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9380a05b0 0x7fc9380a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:37.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.418+0000 7fc94d7fa700 1 -- 172.21.15.53:0/3318952474 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc948011070 con 0x7fc950106b90 2023-11-06T01:33:37.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.419+0000 7fc94ffff700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9380a05b0 0x7fc9380a2a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc94000b5c0 tx=0x7fc940005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:37.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.423+0000 7fc94d7fa700 1 -- 172.21.15.53:0/3318952474 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc94804f0e0 con 0x7fc950106b90 2023-11-06T01:33:37.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:37 smithi163 ceph-mon[104416]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 195 B/s, 1 keys/s, 2 objects/s recovering 2023-11-06T01:33:37.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:37 smithi053 ceph-mon[99472]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 195 B/s, 1 keys/s, 2 objects/s recovering 2023-11-06T01:33:37.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.629+0000 7fc9562a7700 1 -- 172.21.15.53:0/3318952474 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc950103490 con 0x7fc9380a05b0 2023-11-06T01:33:37.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.631+0000 7fc94d7fa700 1 -- 172.21.15.53:0/3318952474 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fc950103490 con 0x7fc9380a05b0 2023-11-06T01:33:37.632 INFO:teuthology.orchestra.run.smithi053.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-11-06T01:33:37.633 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-11-06T01:33:37.633 INFO:teuthology.orchestra.run.smithi053.stdout:smithi062 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440094400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-11-06T01:33:37.633 INFO:teuthology.orchestra.run.smithi053.stdout:smithi163 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GU400BGN 372G No 113s ago Has a FileSystem, LVM detected 2023-11-06T01:33:37.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 -- 172.21.15.53:0/3318952474 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9380a05b0 msgr2=0x7fc9380a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:37.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9380a05b0 0x7fc9380a2a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc94000b5c0 tx=0x7fc940005fd0 comp rx=0 tx=0).stop 2023-11-06T01:33:37.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 -- 172.21.15.53:0/3318952474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950106b90 msgr2=0x7fc95019e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:37.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950106b90 0x7fc95019e140 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc94800ba10 tx=0x7fc94800bd20 comp rx=0 tx=0).stop 2023-11-06T01:33:37.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 -- 172.21.15.53:0/3318952474 shutdown_connections 2023-11-06T01:33:37.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc9380a05b0 0x7fc9380a2a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:37.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950106b90 0x7fc95019e140 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:37.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc950101930 0x7fc950196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:37.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 --2- 172.21.15.53:0/3318952474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc950104260 0x7fc9501970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:37.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 -- 172.21.15.53:0/3318952474 >> 172.21.15.53:0/3318952474 conn(0x7fc9500fb2a0 msgr2=0x7fc950102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:37.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 -- 172.21.15.53:0/3318952474 shutdown_connections 2023-11-06T01:33:37.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:37.633+0000 7fc9562a7700 1 -- 172.21.15.53:0/3318952474 wait complete. 2023-11-06T01:33:37.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:37 smithi062 ceph-mon[103924]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 195 B/s, 1 keys/s, 2 objects/s recovering 2023-11-06T01:33:38.209 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-06T01:33:38.221 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi053.front.sepia.ceph.com 2023-11-06T01:33:38.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-11-06T01:33:38.785 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:39.304 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:39 smithi053 ceph-mon[99472]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 176 B/s, 1 keys/s, 1 objects/s recovering 2023-11-06T01:33:39.304 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:39 smithi053 ceph-mon[99472]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:39.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:39 smithi062 ceph-mon[103924]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 176 B/s, 1 keys/s, 1 objects/s recovering 2023-11-06T01:33:39.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:39 smithi062 ceph-mon[103924]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:39.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:39 smithi163 ceph-mon[104416]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 176 B/s, 1 keys/s, 1 objects/s recovering 2023-11-06T01:33:39.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:39 smithi163 ceph-mon[104416]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-06T01:33:40.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.067+0000 7f448b2a7700 1 -- 172.21.15.53:0/474524909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44841018c0 msgr2=0x7f4484108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:40.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.067+0000 7f448b2a7700 1 --2- 172.21.15.53:0/474524909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44841018c0 0x7f4484108170 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f447c00b0d0 tx=0x7f447c00b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:40.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.067+0000 7f448b2a7700 1 -- 172.21.15.53:0/474524909 shutdown_connections 2023-11-06T01:33:40.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.067+0000 7f448b2a7700 1 --2- 172.21.15.53:0/474524909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44840ffd20 0x7f4484100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:40.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.067+0000 7f448b2a7700 1 --2- 172.21.15.53:0/474524909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44841018c0 0x7f4484108170 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:40.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.067+0000 7f448b2a7700 1 --2- 172.21.15.53:0/474524909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4484100f20 0x7f4484101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:40.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.067+0000 7f448b2a7700 1 -- 172.21.15.53:0/474524909 >> 172.21.15.53:0/474524909 conn(0x7f44840fb2a0 msgr2=0x7f44840fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:40.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.067+0000 7f448b2a7700 1 -- 172.21.15.53:0/474524909 shutdown_connections 2023-11-06T01:33:40.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.067+0000 7f448b2a7700 1 -- 172.21.15.53:0/474524909 wait complete. 2023-11-06T01:33:40.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.067+0000 7f448b2a7700 1 Processor -- start 2023-11-06T01:33:40.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.067+0000 7f448b2a7700 1 -- start start 2023-11-06T01:33:40.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f448b2a7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44840ffd20 0x7f4484069c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:40.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f448b2a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4484100f20 0x7f448419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:40.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f448b2a7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44841018c0 0x7f448419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:40.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f448b2a7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419e670 con 0x7f44841018c0 2023-11-06T01:33:40.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f448b2a7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419e7e0 con 0x7f44840ffd20 2023-11-06T01:33:40.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f448b2a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419e9a0 con 0x7f4484100f20 2023-11-06T01:33:40.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f4489844700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44841018c0 0x7f448419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:40.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f4489844700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44841018c0 0x7f448419ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39058/0 (socket says 172.21.15.53:39058) 2023-11-06T01:33:40.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f4489844700 1 -- 172.21.15.53:0/2525525205 learned_addr learned my addr 172.21.15.53:0/2525525205 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:40.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f4489043700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44840ffd20 0x7f4484069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:40.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f4488842700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4484100f20 0x7f448419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:40.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f4489844700 1 -- 172.21.15.53:0/2525525205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4484100f20 msgr2=0x7f448419b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:40.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f4489844700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4484100f20 0x7f448419b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:40.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f4489844700 1 -- 172.21.15.53:0/2525525205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44840ffd20 msgr2=0x7f4484069c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:40.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f4489844700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44840ffd20 0x7f4484069c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:40.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f4489844700 1 -- 172.21.15.53:0/2525525205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c009d00 con 0x7f44841018c0 2023-11-06T01:33:40.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.068+0000 7f4489844700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44841018c0 0x7f448419ded0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f447c007a00 tx=0x7f447c0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:40.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.069+0000 7f447a7fc700 1 -- 172.21.15.53:0/2525525205 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00e040 con 0x7f44841018c0 2023-11-06T01:33:40.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.069+0000 7f447a7fc700 1 -- 172.21.15.53:0/2525525205 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f447c003e10 con 0x7f44841018c0 2023-11-06T01:33:40.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.069+0000 7f448b2a7700 1 -- 172.21.15.53:0/2525525205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448419ec20 con 0x7f44841018c0 2023-11-06T01:33:40.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.069+0000 7f447a7fc700 1 -- 172.21.15.53:0/2525525205 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c012a90 con 0x7f44841018c0 2023-11-06T01:33:40.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.069+0000 7f448b2a7700 1 -- 172.21.15.53:0/2525525205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448419f030 con 0x7f44841018c0 2023-11-06T01:33:40.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.070+0000 7f448b2a7700 1 -- 172.21.15.53:0/2525525205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4484107230 con 0x7f44841018c0 2023-11-06T01:33:40.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.071+0000 7f447a7fc700 1 -- 172.21.15.53:0/2525525205 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f447c0056c0 con 0x7f44841018c0 2023-11-06T01:33:40.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.071+0000 7f447a7fc700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44700a05b0 0x7f44700a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:40.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.071+0000 7f447a7fc700 1 -- 172.21.15.53:0/2525525205 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f447c0cd9e0 con 0x7f44841018c0 2023-11-06T01:33:40.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.071+0000 7f4489043700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44700a05b0 0x7f44700a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:40.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.072+0000 7f4489043700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44700a05b0 0x7f44700a2a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4484100d80 tx=0x7f4480007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:40.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.075+0000 7f447a7fc700 1 -- 172.21.15.53:0/2525525205 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f447c097490 con 0x7f44841018c0 2023-11-06T01:33:40.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.319+0000 7f448b2a7700 1 -- 172.21.15.53:0/2525525205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f448402d090 con 0x7f44841018c0 2023-11-06T01:33:40.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.320+0000 7f447a7fc700 1 -- 172.21.15.53:0/2525525205 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2895 (secure 0 0 0) 0x7f447c09aab0 con 0x7f44841018c0 2023-11-06T01:33:40.322 INFO:teuthology.orchestra.run.smithi053.stdout:e27 2023-11-06T01:33:40.322 INFO:teuthology.orchestra.run.smithi053.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-11-06T01:33:40.322 INFO:teuthology.orchestra.run.smithi053.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-06T01:33:40.323 INFO:teuthology.orchestra.run.smithi053.stdout:legacy client fscid: 1 2023-11-06T01:33:40.323 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:40.323 INFO:teuthology.orchestra.run.smithi053.stdout:Filesystem 'cephfs' (1) 2023-11-06T01:33:40.323 INFO:teuthology.orchestra.run.smithi053.stdout:fs_name cephfs 2023-11-06T01:33:40.323 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 27 2023-11-06T01:33:40.324 INFO:teuthology.orchestra.run.smithi053.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-11-06T01:33:40.324 INFO:teuthology.orchestra.run.smithi053.stdout:created 2023-11-06T01:32:26.093194+0000 2023-11-06T01:33:40.324 INFO:teuthology.orchestra.run.smithi053.stdout:modified 2023-11-06T01:32:33.006369+0000 2023-11-06T01:33:40.324 INFO:teuthology.orchestra.run.smithi053.stdout:tableserver 0 2023-11-06T01:33:40.324 INFO:teuthology.orchestra.run.smithi053.stdout:root 0 2023-11-06T01:33:40.324 INFO:teuthology.orchestra.run.smithi053.stdout:session_timeout 300 2023-11-06T01:33:40.325 INFO:teuthology.orchestra.run.smithi053.stdout:session_autoclose 300 2023-11-06T01:33:40.325 INFO:teuthology.orchestra.run.smithi053.stdout:max_file_size 1099511627776 2023-11-06T01:33:40.325 INFO:teuthology.orchestra.run.smithi053.stdout:required_client_features {} 2023-11-06T01:33:40.325 INFO:teuthology.orchestra.run.smithi053.stdout:last_failure 0 2023-11-06T01:33:40.325 INFO:teuthology.orchestra.run.smithi053.stdout:last_failure_osd_epoch 0 2023-11-06T01:33:40.326 INFO:teuthology.orchestra.run.smithi053.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-06T01:33:40.326 INFO:teuthology.orchestra.run.smithi053.stdout:max_mds 5 2023-11-06T01:33:40.326 INFO:teuthology.orchestra.run.smithi053.stdout:in 0,1,2,3,4 2023-11-06T01:33:40.327 INFO:teuthology.orchestra.run.smithi053.stdout:up {0=24487,1=24472,2=24464,3=24422,4=24395} 2023-11-06T01:33:40.327 INFO:teuthology.orchestra.run.smithi053.stdout:failed 2023-11-06T01:33:40.327 INFO:teuthology.orchestra.run.smithi053.stdout:damaged 2023-11-06T01:33:40.327 INFO:teuthology.orchestra.run.smithi053.stdout:stopped 2023-11-06T01:33:40.328 INFO:teuthology.orchestra.run.smithi053.stdout:data_pools [3] 2023-11-06T01:33:40.328 INFO:teuthology.orchestra.run.smithi053.stdout:metadata_pool 2 2023-11-06T01:33:40.328 INFO:teuthology.orchestra.run.smithi053.stdout:inline_data disabled 2023-11-06T01:33:40.328 INFO:teuthology.orchestra.run.smithi053.stdout:balancer 2023-11-06T01:33:40.329 INFO:teuthology.orchestra.run.smithi053.stdout:bal_rank_mask -1 2023-11-06T01:33:40.329 INFO:teuthology.orchestra.run.smithi053.stdout:standby_count_wanted 0 2023-11-06T01:33:40.329 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.i{0:24487} state up:active seq 4 addr [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.329 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.f{0:24460} state up:standby-replay seq 1 addr [v2:172.21.15.163:6834/392449784,v1:172.21.15.163:6835/392449784] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.329 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c{1:24472} state up:active seq 6 addr [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.330 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.h{1:24440} state up:standby-replay seq 1 addr [v2:172.21.15.62:6836/644941927,v1:172.21.15.62:6837/644941927] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.330 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b{2:24464} state up:active seq 4 addr [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.330 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.l{2:24430} state up:standby-replay seq 1 addr [v2:172.21.15.163:6832/731722016,v1:172.21.15.163:6833/731722016] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.330 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.e{3:24422} state up:active seq 8 addr [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.330 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a{3:14628} state up:standby-replay seq 1 addr [v2:172.21.15.53:6840/1972292821,v1:172.21.15.53:6841/1972292821] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.331 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.k{4:24395} state up:active seq 10 addr [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.331 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.g{4:14616} state up:standby-replay seq 1 addr [v2:172.21.15.53:6838/1353368498,v1:172.21.15.53:6839/1353368498] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.331 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:40.331 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:40.331 INFO:teuthology.orchestra.run.smithi053.stdout:Standby daemons: 2023-11-06T01:33:40.332 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:40.332 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.53:6834/275572743,v1:172.21.15.53:6835/275572743] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.332 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.53:6836/4211608225,v1:172.21.15.53:6837/4211608225] compat {c=[1],r=[1],i=[7ff]}] 2023-11-06T01:33:40.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 -- 172.21.15.53:0/2525525205 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44700a05b0 msgr2=0x7f44700a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:40.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44700a05b0 0x7f44700a2a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4484100d80 tx=0x7f4480007400 comp rx=0 tx=0).stop 2023-11-06T01:33:40.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 -- 172.21.15.53:0/2525525205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44841018c0 msgr2=0x7f448419ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:40.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44841018c0 0x7f448419ded0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f447c007a00 tx=0x7f447c0094c0 comp rx=0 tx=0).stop 2023-11-06T01:33:40.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 -- 172.21.15.53:0/2525525205 shutdown_connections 2023-11-06T01:33:40.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44700a05b0 0x7f44700a2a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:40.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44840ffd20 0x7f4484069c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:40.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44841018c0 0x7f448419ded0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:40.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 --2- 172.21.15.53:0/2525525205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4484100f20 0x7f448419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:40.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 -- 172.21.15.53:0/2525525205 >> 172.21.15.53:0/2525525205 conn(0x7f44840fb2a0 msgr2=0x7f4484104370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:40.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 -- 172.21.15.53:0/2525525205 shutdown_connections 2023-11-06T01:33:40.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:40.322+0000 7f448b2a7700 1 -- 172.21.15.53:0/2525525205 wait complete. 2023-11-06T01:33:40.335 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-11-06T01:33:40.746 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-11-06T01:33:41.305 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:41.551 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:41 smithi053 ceph-mon[99472]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 152 B/s, 1 keys/s, 1 objects/s recovering 2023-11-06T01:33:41.552 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:41 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2525525205' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-06T01:33:41.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:41 smithi062 ceph-mon[103924]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 152 B/s, 1 keys/s, 1 objects/s recovering 2023-11-06T01:33:41.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:41 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2525525205' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-06T01:33:41.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:41 smithi163 ceph-mon[104416]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 152 B/s, 1 keys/s, 1 objects/s recovering 2023-11-06T01:33:41.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:41 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2525525205' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-06T01:33:42.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.533+0000 7f391a49d700 1 -- 172.21.15.53:0/3578202540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3914106bb0 msgr2=0x7f3914108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:42.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.533+0000 7f391a49d700 1 --2- 172.21.15.53:0/3578202540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3914106bb0 0x7f3914108fa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f390c00b0a0 tx=0x7f390c00b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:42.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.533+0000 7f391a49d700 1 -- 172.21.15.53:0/3578202540 shutdown_connections 2023-11-06T01:33:42.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.533+0000 7f391a49d700 1 --2- 172.21.15.53:0/3578202540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3914101950 0x7f3914103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:42.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.533+0000 7f391a49d700 1 --2- 172.21.15.53:0/3578202540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3914106bb0 0x7f3914108fa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:42.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.533+0000 7f391a49d700 1 --2- 172.21.15.53:0/3578202540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3914104280 0x7f3914106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:42.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.533+0000 7f391a49d700 1 -- 172.21.15.53:0/3578202540 >> 172.21.15.53:0/3578202540 conn(0x7f39140fb2a0 msgr2=0x7f39140fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:42.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.533+0000 7f391a49d700 1 -- 172.21.15.53:0/3578202540 shutdown_connections 2023-11-06T01:33:42.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.533+0000 7f391a49d700 1 -- 172.21.15.53:0/3578202540 wait complete. 2023-11-06T01:33:42.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f391a49d700 1 Processor -- start 2023-11-06T01:33:42.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f391a49d700 1 -- start start 2023-11-06T01:33:42.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f391a49d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3914101950 0x7f3914196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:42.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f391a49d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3914104280 0x7f3914197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:42.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f391a49d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3914106bb0 0x7f391419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:42.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f391a49d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391419e760 con 0x7f3914104280 2023-11-06T01:33:42.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f391a49d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391419e8a0 con 0x7f3914106bb0 2023-11-06T01:33:42.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f391a49d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391419ea30 con 0x7f3914101950 2023-11-06T01:33:42.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f39137fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3914104280 0x7f3914197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:42.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f39137fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3914104280 0x7f3914197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39068/0 (socket says 172.21.15.53:39068) 2023-11-06T01:33:42.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f39137fe700 1 -- 172.21.15.53:0/3922484149 learned_addr learned my addr 172.21.15.53:0/3922484149 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:42.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f3918a3a700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3914106bb0 0x7f391419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:42.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f39137fe700 1 -- 172.21.15.53:0/3922484149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3914101950 msgr2=0x7f3914196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:42.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f39137fe700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3914101950 0x7f3914196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:42.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f39137fe700 1 -- 172.21.15.53:0/3922484149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3914106bb0 msgr2=0x7f391419e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:42.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f39137fe700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3914106bb0 0x7f391419e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:42.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f39137fe700 1 -- 172.21.15.53:0/3922484149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c009d00 con 0x7f3914104280 2023-11-06T01:33:42.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.534+0000 7f3918a3a700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3914106bb0 0x7f391419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:42.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.535+0000 7f39137fe700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3914104280 0x7f3914197110 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00d9b0 tx=0x7f38fc00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:42.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.535+0000 7f39117fa700 1 -- 172.21.15.53:0/3922484149 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00bb60 con 0x7f3914104280 2023-11-06T01:33:42.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.535+0000 7f391a49d700 1 -- 172.21.15.53:0/3922484149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39140ff860 con 0x7f3914104280 2023-11-06T01:33:42.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.535+0000 7f39117fa700 1 -- 172.21.15.53:0/3922484149 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38fc007e80 con 0x7f3914104280 2023-11-06T01:33:42.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.535+0000 7f39117fa700 1 -- 172.21.15.53:0/3922484149 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc004dd0 con 0x7f3914104280 2023-11-06T01:33:42.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.535+0000 7f391a49d700 1 -- 172.21.15.53:0/3922484149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39140ffd60 con 0x7f3914104280 2023-11-06T01:33:42.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.536+0000 7f391a49d700 1 -- 172.21.15.53:0/3922484149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391410c610 con 0x7f3914104280 2023-11-06T01:33:42.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.537+0000 7f39117fa700 1 -- 172.21.15.53:0/3922484149 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f38fc004020 con 0x7f3914104280 2023-11-06T01:33:42.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.538+0000 7f39117fa700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39000a05b0 0x7f39000a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:42.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.538+0000 7f39117fa700 1 -- 172.21.15.53:0/3922484149 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f38fc0cbf80 con 0x7f3914104280 2023-11-06T01:33:42.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.538+0000 7f3913fff700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39000a05b0 0x7f39000a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:42.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.538+0000 7f3913fff700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39000a05b0 0x7f39000a2a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f390400a990 tx=0x7f3904005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:42.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.541+0000 7f39117fa700 1 -- 172.21.15.53:0/3922484149 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f38fc0959a0 con 0x7f3914104280 2023-11-06T01:33:42.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.759+0000 7f391a49d700 1 -- 172.21.15.53:0/3922484149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f39141003a0 con 0x7f3914104280 2023-11-06T01:33:42.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.760+0000 7f39117fa700 1 -- 172.21.15.53:0/3922484149 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) v1 ==== 56+0+5319 (secure 0 0 0) 0x7f38fc098fc0 con 0x7f3914104280 2023-11-06T01:33:42.761 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 84 2023-11-06T01:33:42.762 INFO:teuthology.orchestra.run.smithi053.stdout:fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:33:42.762 INFO:teuthology.orchestra.run.smithi053.stdout:created 2023-11-06T01:25:25.499519+0000 2023-11-06T01:33:42.762 INFO:teuthology.orchestra.run.smithi053.stdout:modified 2023-11-06T01:33:27.113678+0000 2023-11-06T01:33:42.762 INFO:teuthology.orchestra.run.smithi053.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-11-06T01:33:42.762 INFO:teuthology.orchestra.run.smithi053.stdout:crush_version 26 2023-11-06T01:33:42.763 INFO:teuthology.orchestra.run.smithi053.stdout:full_ratio 0.95 2023-11-06T01:33:42.763 INFO:teuthology.orchestra.run.smithi053.stdout:backfillfull_ratio 0.9 2023-11-06T01:33:42.763 INFO:teuthology.orchestra.run.smithi053.stdout:nearfull_ratio 0.85 2023-11-06T01:33:42.763 INFO:teuthology.orchestra.run.smithi053.stdout:require_min_compat_client luminous 2023-11-06T01:33:42.763 INFO:teuthology.orchestra.run.smithi053.stdout:min_compat_client luminous 2023-11-06T01:33:42.764 INFO:teuthology.orchestra.run.smithi053.stdout:require_osd_release reef 2023-11-06T01:33:42.764 INFO:teuthology.orchestra.run.smithi053.stdout:stretch_mode_enabled false 2023-11-06T01:33:42.764 INFO:teuthology.orchestra.run.smithi053.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-11-06T01:33:42.764 INFO:teuthology.orchestra.run.smithi053.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-11-06T01:33:42.764 INFO:teuthology.orchestra.run.smithi053.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-11-06T01:33:42.765 INFO:teuthology.orchestra.run.smithi053.stdout:max_osd 12 2023-11-06T01:33:42.765 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6802/2480021073,v1:172.21.15.53:6803/2480021073] [v2:172.21.15.53:6804/2480021073,v1:172.21.15.53:6805/2480021073] exists,up 1b30174e-feb1-42fd-8b43-0ffd686e01ee 2023-11-06T01:33:42.765 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6800/395697393,v1:172.21.15.62:6801/395697393] [v2:172.21.15.62:6802/395697393,v1:172.21.15.62:6803/395697393] exists,up 73a75b02-34df-4def-b5f6-e29f1190c1e8 2023-11-06T01:33:42.765 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2 up in weight 1 up_from 18 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6800/1520683016,v1:172.21.15.163:6801/1520683016] [v2:172.21.15.163:6802/1520683016,v1:172.21.15.163:6803/1520683016] exists,up 6650ac65-e6f8-4a01-b2bd-60b0c7995997 2023-11-06T01:33:42.765 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3 up in weight 1 up_from 26 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6810/3352913345,v1:172.21.15.53:6811/3352913345] [v2:172.21.15.53:6812/3352913345,v1:172.21.15.53:6813/3352913345] exists,up e0faf95d-f034-418c-8604-278ed2d8a98c 2023-11-06T01:33:42.766 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4 up in weight 1 up_from 34 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6808/1956539718,v1:172.21.15.62:6809/1956539718] [v2:172.21.15.62:6810/1956539718,v1:172.21.15.62:6811/1956539718] exists,up f8542e9c-72e2-4551-b1f7-204be77127f0 2023-11-06T01:33:42.766 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5 up in weight 1 up_from 39 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6808/464090439,v1:172.21.15.163:6809/464090439] [v2:172.21.15.163:6810/464090439,v1:172.21.15.163:6811/464090439] exists,up 4adb65ab-f410-41a2-b4dc-7e12bcaba8ba 2023-11-06T01:33:42.766 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6 up in weight 1 up_from 46 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6818/4140594819,v1:172.21.15.53:6819/4140594819] [v2:172.21.15.53:6820/4140594819,v1:172.21.15.53:6821/4140594819] exists,up 2b8502f9-4651-43b4-b212-d37785c4655a 2023-11-06T01:33:42.766 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7 up in weight 1 up_from 51 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6816/1541716587,v1:172.21.15.62:6817/1541716587] [v2:172.21.15.62:6818/1541716587,v1:172.21.15.62:6819/1541716587] exists,up e6ac514c-4385-49cc-b408-f24f49a200ee 2023-11-06T01:33:42.766 INFO:teuthology.orchestra.run.smithi053.stdout:osd.8 up in weight 1 up_from 58 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6816/2361123317,v1:172.21.15.163:6817/2361123317] [v2:172.21.15.163:6818/2361123317,v1:172.21.15.163:6819/2361123317] exists,up 14d1dc49-9a61-48da-9250-70a70c4a7fd8 2023-11-06T01:33:42.767 INFO:teuthology.orchestra.run.smithi053.stdout:osd.9 up in weight 1 up_from 65 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6826/825228850,v1:172.21.15.53:6827/825228850] [v2:172.21.15.53:6828/825228850,v1:172.21.15.53:6829/825228850] exists,up 23af8247-4e33-4d08-8467-452c339e1eee 2023-11-06T01:33:42.767 INFO:teuthology.orchestra.run.smithi053.stdout:osd.10 up in weight 1 up_from 70 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6824/1978188861,v1:172.21.15.62:6825/1978188861] [v2:172.21.15.62:6826/1978188861,v1:172.21.15.62:6827/1978188861] exists,up 4e90a07b-b1b1-4120-a456-6c8106cc1b07 2023-11-06T01:33:42.767 INFO:teuthology.orchestra.run.smithi053.stdout:osd.11 up in weight 1 up_from 77 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6824/1021137705,v1:172.21.15.163:6825/1021137705] [v2:172.21.15.163:6826/1021137705,v1:172.21.15.163:6827/1021137705] exists,up 758da1b3-af80-435f-bc06-3c9a3af6e501 2023-11-06T01:33:42.767 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 2.6 [8,11] 2023-11-06T01:33:42.767 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 2.13 [2,11] 2023-11-06T01:33:42.768 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 2.1b [2,1] 2023-11-06T01:33:42.768 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 2.1f [8,4] 2023-11-06T01:33:42.768 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.5 [3,9] 2023-11-06T01:33:42.768 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.11 [5,9] 2023-11-06T01:33:42.768 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.17 [4,7] 2023-11-06T01:33:42.769 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.20 [5,9] 2023-11-06T01:33:42.769 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.23 [4,8] 2023-11-06T01:33:42.769 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.31 [4,6] 2023-11-06T01:33:42.769 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/2774626263 expires 2023-11-07T01:26:25.398163+0000 2023-11-06T01:33:42.769 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/749598229 expires 2023-11-07T01:25:59.898321+0000 2023-11-06T01:33:42.769 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:6800/3877634851 expires 2023-11-07T01:25:59.898321+0000 2023-11-06T01:33:42.770 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:6801/3877634851 expires 2023-11-07T01:25:59.898321+0000 2023-11-06T01:33:42.770 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:6801/2231847638 expires 2023-11-07T01:26:25.398163+0000 2023-11-06T01:33:42.770 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/2187559879 expires 2023-11-07T01:25:59.898321+0000 2023-11-06T01:33:42.770 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/429543130 expires 2023-11-07T01:25:59.898321+0000 2023-11-06T01:33:42.771 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/3127896038 expires 2023-11-07T01:26:25.398163+0000 2023-11-06T01:33:42.771 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:6800/2231847638 expires 2023-11-07T01:26:25.398163+0000 2023-11-06T01:33:42.771 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/2103213255 expires 2023-11-07T01:26:25.398163+0000 2023-11-06T01:33:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 -- 172.21.15.53:0/3922484149 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39000a05b0 msgr2=0x7f39000a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39000a05b0 0x7f39000a2a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f390400a990 tx=0x7f3904005c80 comp rx=0 tx=0).stop 2023-11-06T01:33:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 -- 172.21.15.53:0/3922484149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3914104280 msgr2=0x7f3914197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3914104280 0x7f3914197110 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00d9b0 tx=0x7f38fc00dd70 comp rx=0 tx=0).stop 2023-11-06T01:33:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 -- 172.21.15.53:0/3922484149 shutdown_connections 2023-11-06T01:33:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39000a05b0 0x7f39000a2a70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:42.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3914106bb0 0x7f391419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:42.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3914101950 0x7f3914196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:42.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 --2- 172.21.15.53:0/3922484149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3914104280 0x7f3914197110 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:42.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 -- 172.21.15.53:0/3922484149 >> 172.21.15.53:0/3922484149 conn(0x7f39140fb2a0 msgr2=0x7f3914102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:42.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 -- 172.21.15.53:0/3922484149 shutdown_connections 2023-11-06T01:33:42.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:42.762+0000 7f391a49d700 1 -- 172.21.15.53:0/3922484149 wait complete. 2023-11-06T01:33:43.167 INFO:teuthology.run_tasks:Running task fs.ready... 2023-11-06T01:33:43.182 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:33:43.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.311+0000 7fc2c9385700 1 -- 172.21.15.53:0/3149617852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c40ffc70 msgr2=0x7fc2c4100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:43.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.311+0000 7fc2c9385700 1 --2- 172.21.15.53:0/3149617852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c40ffc70 0x7fc2c4100090 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40099c0 tx=0x7fc2b4009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.311+0000 7fc2c9385700 1 -- 172.21.15.53:0/3149617852 shutdown_connections 2023-11-06T01:33:43.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.311+0000 7fc2c9385700 1 --2- 172.21.15.53:0/3149617852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2c4101800 0x7fc2c41080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.311+0000 7fc2c9385700 1 --2- 172.21.15.53:0/3149617852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c40ffc70 0x7fc2c4100090 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.311+0000 7fc2c9385700 1 --2- 172.21.15.53:0/3149617852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2c4100e60 0x7fc2c41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.311+0000 7fc2c9385700 1 -- 172.21.15.53:0/3149617852 >> 172.21.15.53:0/3149617852 conn(0x7fc2c40fb1c0 msgr2=0x7fc2c40fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.311+0000 7fc2c9385700 1 -- 172.21.15.53:0/3149617852 shutdown_connections 2023-11-06T01:33:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.311+0000 7fc2c9385700 1 -- 172.21.15.53:0/3149617852 wait complete. 2023-11-06T01:33:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.312+0000 7fc2c9385700 1 Processor -- start 2023-11-06T01:33:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.312+0000 7fc2c9385700 1 -- start start 2023-11-06T01:33:43.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.312+0000 7fc2c9385700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2c40ffc70 0x7fc2c4068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:43.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.312+0000 7fc2c9385700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2c4100e60 0x7fc2c419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:43.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.312+0000 7fc2c9385700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c4101800 0x7fc2c419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:43.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.312+0000 7fc2c9385700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c419e650 con 0x7fc2c4101800 2023-11-06T01:33:43.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.312+0000 7fc2c9385700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c419e7c0 con 0x7fc2c4100e60 2023-11-06T01:33:43.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.312+0000 7fc2c9385700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c419e980 con 0x7fc2c40ffc70 2023-11-06T01:33:43.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.312+0000 7fc2c37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c4101800 0x7fc2c419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:43.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.312+0000 7fc2c37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c4101800 0x7fc2c419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39082/0 (socket says 172.21.15.53:39082) 2023-11-06T01:33:43.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.312+0000 7fc2c37fe700 1 -- 172.21.15.53:0/3403487805 learned_addr learned my addr 172.21.15.53:0/3403487805 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:43.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2c2ffd700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2c40ffc70 0x7fc2c4068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:43.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2c27fc700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2c4100e60 0x7fc2c419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:43.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2c37fe700 1 -- 172.21.15.53:0/3403487805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2c40ffc70 msgr2=0x7fc2c4068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:43.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2c37fe700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2c40ffc70 0x7fc2c4068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2c37fe700 1 -- 172.21.15.53:0/3403487805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2c4100e60 msgr2=0x7fc2c419b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:43.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2c37fe700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2c4100e60 0x7fc2c419b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2c37fe700 1 -- 172.21.15.53:0/3403487805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b40095f0 con 0x7fc2c4101800 2023-11-06T01:33:43.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2c27fc700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2c4100e60 0x7fc2c419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:43.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2c37fe700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c4101800 0x7fc2c419deb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00ada0 tx=0x7fc2bc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:43.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2abfff700 1 -- 172.21.15.53:0/3403487805 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00f070 con 0x7fc2c4101800 2023-11-06T01:33:43.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2c9385700 1 -- 172.21.15.53:0/3403487805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c419ec60 con 0x7fc2c4101800 2023-11-06T01:33:43.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2abfff700 1 -- 172.21.15.53:0/3403487805 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2bc007b30 con 0x7fc2c4101800 2023-11-06T01:33:43.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2abfff700 1 -- 172.21.15.53:0/3403487805 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0053e0 con 0x7fc2c4101800 2023-11-06T01:33:43.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.313+0000 7fc2c9385700 1 -- 172.21.15.53:0/3403487805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c419f170 con 0x7fc2c4101800 2023-11-06T01:33:43.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.314+0000 7fc2c9385700 1 -- 172.21.15.53:0/3403487805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c4107260 con 0x7fc2c4101800 2023-11-06T01:33:43.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.315+0000 7fc2abfff700 1 -- 172.21.15.53:0/3403487805 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc2bc00c980 con 0x7fc2c4101800 2023-11-06T01:33:43.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.316+0000 7fc2abfff700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2ac0a05f0 0x7fc2ac0a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:43.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.316+0000 7fc2c2ffd700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2ac0a05f0 0x7fc2ac0a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:43.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.316+0000 7fc2abfff700 1 -- 172.21.15.53:0/3403487805 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc2bc0cb190 con 0x7fc2c4101800 2023-11-06T01:33:43.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.316+0000 7fc2c2ffd700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2ac0a05f0 0x7fc2ac0a2ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400b580 tx=0x7fc2b4009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:43.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.320+0000 7fc2abfff700 1 -- 172.21.15.53:0/3403487805 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc2bc094be0 con 0x7fc2c4101800 2023-11-06T01:33:43.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.526+0000 7fc2c9385700 1 -- 172.21.15.53:0/3403487805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2c402ce70 con 0x7fc2c4101800 2023-11-06T01:33:43.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.527+0000 7fc2abfff700 1 -- 172.21.15.53:0/3403487805 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22543 (secure 0 0 0) 0x7fc2bc098200 con 0x7fc2c4101800 2023-11-06T01:33:43.529 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:43.530 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:33:27.113678+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:33:43.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 -- 172.21.15.53:0/3403487805 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2ac0a05f0 msgr2=0x7fc2ac0a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:43.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2ac0a05f0 0x7fc2ac0a2ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400b580 tx=0x7fc2b4009f10 comp rx=0 tx=0).stop 2023-11-06T01:33:43.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 -- 172.21.15.53:0/3403487805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c4101800 msgr2=0x7fc2c419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:43.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c4101800 0x7fc2c419deb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00ada0 tx=0x7fc2bc00c330 comp rx=0 tx=0).stop 2023-11-06T01:33:43.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 -- 172.21.15.53:0/3403487805 shutdown_connections 2023-11-06T01:33:43.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2ac0a05f0 0x7fc2ac0a2ab0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c4101800 0x7fc2c419deb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2c40ffc70 0x7fc2c4068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 --2- 172.21.15.53:0/3403487805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2c4100e60 0x7fc2c419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 -- 172.21.15.53:0/3403487805 >> 172.21.15.53:0/3403487805 conn(0x7fc2c40fb1c0 msgr2=0x7fc2c41043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:43.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 -- 172.21.15.53:0/3403487805 shutdown_connections 2023-11-06T01:33:43.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.530+0000 7fc2c9385700 1 -- 172.21.15.53:0/3403487805 wait complete. 2023-11-06T01:33:43.543 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:43 smithi053 ceph-mon[99472]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-11-06T01:33:43.543 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:43 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3922484149' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-06T01:33:43.545 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T01:32:24.604601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T01:32:25.457741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:33:43.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T01:33:43.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:43 smithi062 ceph-mon[103924]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-11-06T01:33:43.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:43 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3922484149' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-06T01:33:43.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.673+0000 7f6541287700 1 -- 172.21.15.53:0/3731148901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f653c102970 msgr2=0x7f653c109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:43.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.673+0000 7f6541287700 1 --2- 172.21.15.53:0/3731148901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f653c102970 0x7f653c109220 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f653400b0d0 tx=0x7f653400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.673+0000 7f6541287700 1 -- 172.21.15.53:0/3731148901 shutdown_connections 2023-11-06T01:33:43.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.673+0000 7f6541287700 1 --2- 172.21.15.53:0/3731148901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f653c102970 0x7f653c109220 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.673+0000 7f6541287700 1 --2- 172.21.15.53:0/3731148901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f653c100de0 0x7f653c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.673+0000 7f6541287700 1 --2- 172.21.15.53:0/3731148901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f653c101fd0 0x7f653c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.673+0000 7f6541287700 1 -- 172.21.15.53:0/3731148901 >> 172.21.15.53:0/3731148901 conn(0x7f653c0fc370 msgr2=0x7f653c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:43.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.673+0000 7f6541287700 1 -- 172.21.15.53:0/3731148901 shutdown_connections 2023-11-06T01:33:43.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.673+0000 7f6541287700 1 -- 172.21.15.53:0/3731148901 wait complete. 2023-11-06T01:33:43.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.674+0000 7f6541287700 1 Processor -- start 2023-11-06T01:33:43.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.674+0000 7f6541287700 1 -- start start 2023-11-06T01:33:43.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.674+0000 7f6541287700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f653c100de0 0x7f653c196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:43.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.674+0000 7f6541287700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f653c101fd0 0x7f653c1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:43.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.674+0000 7f6541287700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f653c102970 0x7f653c19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.674+0000 7f6541287700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c19e720 con 0x7f653c102970 2023-11-06T01:33:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.674+0000 7f6541287700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c19e860 con 0x7f653c101fd0 2023-11-06T01:33:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.674+0000 7f6541287700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c19e9f0 con 0x7f653c100de0 2023-11-06T01:33:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.674+0000 7f653b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f653c102970 0x7f653c19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.674+0000 7f653b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f653c102970 0x7f653c19e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39096/0 (socket says 172.21.15.53:39096) 2023-11-06T01:33:43.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.674+0000 7f653b7fe700 1 -- 172.21.15.53:0/789657046 learned_addr learned my addr 172.21.15.53:0/789657046 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:43.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f653a7fc700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f653c101fd0 0x7f653c1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:43.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f653affd700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f653c100de0 0x7f653c196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:43.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f653b7fe700 1 -- 172.21.15.53:0/789657046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f653c100de0 msgr2=0x7f653c196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:43.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f653b7fe700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f653c100de0 0x7f653c196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f653b7fe700 1 -- 172.21.15.53:0/789657046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f653c101fd0 msgr2=0x7f653c1970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f653b7fe700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f653c101fd0 0x7f653c1970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f653b7fe700 1 -- 172.21.15.53:0/789657046 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6534009d00 con 0x7f653c102970 2023-11-06T01:33:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f653affd700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f653c100de0 0x7f653c196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f653b7fe700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f653c102970 0x7f653c19e150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6534008ec0 tx=0x7f6534008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:43.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f6523fff700 1 -- 172.21.15.53:0/789657046 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653400e040 con 0x7f653c102970 2023-11-06T01:33:43.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f6523fff700 1 -- 172.21.15.53:0/789657046 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f653400be30 con 0x7f653c102970 2023-11-06T01:33:43.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f6523fff700 1 -- 172.21.15.53:0/789657046 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653401b6c0 con 0x7f653c102970 2023-11-06T01:33:43.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f6541287700 1 -- 172.21.15.53:0/789657046 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c19ec70 con 0x7f653c102970 2023-11-06T01:33:43.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.675+0000 7f6541287700 1 -- 172.21.15.53:0/789657046 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c19f040 con 0x7f653c102970 2023-11-06T01:33:43.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.676+0000 7f6541287700 1 -- 172.21.15.53:0/789657046 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653c108320 con 0x7f653c102970 2023-11-06T01:33:43.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.677+0000 7f6523fff700 1 -- 172.21.15.53:0/789657046 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f6534005090 con 0x7f653c102970 2023-11-06T01:33:43.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.678+0000 7f6523fff700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65240a05b0 0x7f65240a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:43.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.678+0000 7f653affd700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65240a05b0 0x7f65240a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:43.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.678+0000 7f6523fff700 1 -- 172.21.15.53:0/789657046 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f65340cd480 con 0x7f653c102970 2023-11-06T01:33:43.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.678+0000 7f653affd700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65240a05b0 0x7f65240a2a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f653c101e30 tx=0x7f652c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:43.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.682+0000 7f6523fff700 1 -- 172.21.15.53:0/789657046 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6534096f30 con 0x7f653c102970 2023-11-06T01:33:43.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:43 smithi163 ceph-mon[104416]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-11-06T01:33:43.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:43 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3922484149' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-06T01:33:43.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.882+0000 7f6541287700 1 -- 172.21.15.53:0/789657046 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f653c10c840 con 0x7f653c102970 2023-11-06T01:33:43.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.882+0000 7f6523fff700 1 -- 172.21.15.53:0/789657046 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f653402ce60 con 0x7f653c102970 2023-11-06T01:33:43.884 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:33:43.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.884+0000 7f6541287700 1 -- 172.21.15.53:0/789657046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65240a05b0 msgr2=0x7f65240a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:43.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.884+0000 7f6541287700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65240a05b0 0x7f65240a2a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f653c101e30 tx=0x7f652c009260 comp rx=0 tx=0).stop 2023-11-06T01:33:43.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.884+0000 7f6541287700 1 -- 172.21.15.53:0/789657046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f653c102970 msgr2=0x7f653c19e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:43.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.884+0000 7f6541287700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f653c102970 0x7f653c19e150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6534008ec0 tx=0x7f6534008ef0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.885+0000 7f6541287700 1 -- 172.21.15.53:0/789657046 shutdown_connections 2023-11-06T01:33:43.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.885+0000 7f6541287700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65240a05b0 0x7f65240a2a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.885+0000 7f6541287700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f653c102970 0x7f653c19e150 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.885+0000 7f6541287700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f653c100de0 0x7f653c196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.885+0000 7f6541287700 1 --2- 172.21.15.53:0/789657046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f653c101fd0 0x7f653c1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:43.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.885+0000 7f6541287700 1 -- 172.21.15.53:0/789657046 >> 172.21.15.53:0/789657046 conn(0x7f653c0fc370 msgr2=0x7f653c105460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:43.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.885+0000 7f6541287700 1 -- 172.21.15.53:0/789657046 shutdown_connections 2023-11-06T01:33:43.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:43.885+0000 7f6541287700 1 -- 172.21.15.53:0/789657046 wait complete. 2023-11-06T01:33:43.899 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T01:33:44.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.025+0000 7feb3b396700 1 -- 172.21.15.53:0/306189643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb340ffc80 msgr2=0x7feb341000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.025+0000 7feb3b396700 1 --2- 172.21.15.53:0/306189643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb340ffc80 0x7feb341000a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7feb300099c0 tx=0x7feb30009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.025+0000 7feb3b396700 1 -- 172.21.15.53:0/306189643 shutdown_connections 2023-11-06T01:33:44.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.025+0000 7feb3b396700 1 --2- 172.21.15.53:0/306189643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb34101810 0x7feb341080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.025+0000 7feb3b396700 1 --2- 172.21.15.53:0/306189643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb340ffc80 0x7feb341000a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.025+0000 7feb3b396700 1 --2- 172.21.15.53:0/306189643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb34100e70 0x7feb341012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.025+0000 7feb3b396700 1 -- 172.21.15.53:0/306189643 >> 172.21.15.53:0/306189643 conn(0x7feb340fb1f0 msgr2=0x7feb340fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:44.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.025+0000 7feb3b396700 1 -- 172.21.15.53:0/306189643 shutdown_connections 2023-11-06T01:33:44.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.025+0000 7feb3b396700 1 -- 172.21.15.53:0/306189643 wait complete. 2023-11-06T01:33:44.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb3b396700 1 Processor -- start 2023-11-06T01:33:44.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb3b396700 1 -- start start 2023-11-06T01:33:44.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb3b396700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb340ffc80 0x7feb34068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb3b396700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb34100e70 0x7feb3419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb3b396700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb34101810 0x7feb3419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb3b396700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3419e590 con 0x7feb34100e70 2023-11-06T01:33:44.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb3b396700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3419e700 con 0x7feb340ffc80 2023-11-06T01:33:44.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb3b396700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3419e8c0 con 0x7feb34101810 2023-11-06T01:33:44.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb38931700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb34100e70 0x7feb3419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:44.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb38931700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb34100e70 0x7feb3419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39108/0 (socket says 172.21.15.53:39108) 2023-11-06T01:33:44.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb38931700 1 -- 172.21.15.53:0/2209606090 learned_addr learned my addr 172.21.15.53:0/2209606090 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:44.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb39132700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb340ffc80 0x7feb34068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:44.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb39933700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb34101810 0x7feb3419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:44.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb38931700 1 -- 172.21.15.53:0/2209606090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb34101810 msgr2=0x7feb3419ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb38931700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb34101810 0x7feb3419ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb38931700 1 -- 172.21.15.53:0/2209606090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb340ffc80 msgr2=0x7feb34068ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb38931700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb340ffc80 0x7feb34068ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb38931700 1 -- 172.21.15.53:0/2209606090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb300095f0 con 0x7feb34100e70 2023-11-06T01:33:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb39933700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb34101810 0x7feb3419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.026+0000 7feb38931700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb34100e70 0x7feb3419b760 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feb2800e9a0 tx=0x7feb2800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.027+0000 7feb267fc700 1 -- 172.21.15.53:0/2209606090 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28009790 con 0x7feb34100e70 2023-11-06T01:33:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.027+0000 7feb267fc700 1 -- 172.21.15.53:0/2209606090 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb2800cbd0 con 0x7feb34100e70 2023-11-06T01:33:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.027+0000 7feb3b396700 1 -- 172.21.15.53:0/2209606090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3419eba0 con 0x7feb34100e70 2023-11-06T01:33:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.027+0000 7feb267fc700 1 -- 172.21.15.53:0/2209606090 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28018680 con 0x7feb34100e70 2023-11-06T01:33:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.027+0000 7feb3b396700 1 -- 172.21.15.53:0/2209606090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3419f0b0 con 0x7feb34100e70 2023-11-06T01:33:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.028+0000 7feb3b396700 1 -- 172.21.15.53:0/2209606090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb34107260 con 0x7feb34100e70 2023-11-06T01:33:44.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.029+0000 7feb267fc700 1 -- 172.21.15.53:0/2209606090 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7feb2800cd40 con 0x7feb34100e70 2023-11-06T01:33:44.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.029+0000 7feb267fc700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb200a05b0 0x7feb200a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.030+0000 7feb39132700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb200a05b0 0x7feb200a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:44.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.030+0000 7feb267fc700 1 -- 172.21.15.53:0/2209606090 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7feb28014070 con 0x7feb34100e70 2023-11-06T01:33:44.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.030+0000 7feb39132700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb200a05b0 0x7feb200a2a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feb30009990 tx=0x7feb30005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:44.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.034+0000 7feb267fc700 1 -- 172.21.15.53:0/2209606090 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7feb280520e0 con 0x7feb34100e70 2023-11-06T01:33:44.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.237+0000 7feb3b396700 1 -- 172.21.15.53:0/2209606090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb3402cec0 con 0x7feb34100e70 2023-11-06T01:33:44.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.237+0000 7feb267fc700 1 -- 172.21.15.53:0/2209606090 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb280957e0 con 0x7feb34100e70 2023-11-06T01:33:44.239 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-06T01:33:44.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 -- 172.21.15.53:0/2209606090 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb200a05b0 msgr2=0x7feb200a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb200a05b0 0x7feb200a2a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feb30009990 tx=0x7feb30005fb0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 -- 172.21.15.53:0/2209606090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb34100e70 msgr2=0x7feb3419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb34100e70 0x7feb3419b760 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feb2800e9a0 tx=0x7feb2800ecb0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 -- 172.21.15.53:0/2209606090 shutdown_connections 2023-11-06T01:33:44.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb200a05b0 0x7feb200a2a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb340ffc80 0x7feb34068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb34101810 0x7feb3419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 --2- 172.21.15.53:0/2209606090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb34100e70 0x7feb3419b760 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 -- 172.21.15.53:0/2209606090 >> 172.21.15.53:0/2209606090 conn(0x7feb340fb1f0 msgr2=0x7feb341043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:44.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 -- 172.21.15.53:0/2209606090 shutdown_connections 2023-11-06T01:33:44.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.239+0000 7feb3b396700 1 -- 172.21.15.53:0/2209606090 wait complete. 2023-11-06T01:33:44.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T01:33:44.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.382+0000 7f1962ca3700 1 -- 172.21.15.53:0/908890484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c100990 msgr2=0x7f195c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.382+0000 7f1962ca3700 1 --2- 172.21.15.53:0/908890484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c100990 0x7f195c100e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f195400b0a0 tx=0x7f195400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.382+0000 7f1962ca3700 1 -- 172.21.15.53:0/908890484 shutdown_connections 2023-11-06T01:33:44.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.382+0000 7f1962ca3700 1 --2- 172.21.15.53:0/908890484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195c0ff670 0x7f195c0ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.382+0000 7f1962ca3700 1 --2- 172.21.15.53:0/908890484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c100990 0x7f195c100e10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.382+0000 7f1962ca3700 1 --2- 172.21.15.53:0/908890484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f195c0fffd0 0x7f195c100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.382+0000 7f1962ca3700 1 -- 172.21.15.53:0/908890484 >> 172.21.15.53:0/908890484 conn(0x7f195c0fb1f0 msgr2=0x7f195c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:44.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.382+0000 7f1962ca3700 1 -- 172.21.15.53:0/908890484 shutdown_connections 2023-11-06T01:33:44.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.382+0000 7f1962ca3700 1 -- 172.21.15.53:0/908890484 wait complete. 2023-11-06T01:33:44.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.382+0000 7f1962ca3700 1 Processor -- start 2023-11-06T01:33:44.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1962ca3700 1 -- start start 2023-11-06T01:33:44.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1962ca3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c0ff670 0x7f195c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1962ca3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195c0fffd0 0x7f195c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1962ca3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f195c100990 0x7f195c1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1962ca3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c1a0850 con 0x7f195c0ff670 2023-11-06T01:33:44.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1962ca3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c1a09c0 con 0x7f195c0fffd0 2023-11-06T01:33:44.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1962ca3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c1a0b80 con 0x7f195c100990 2023-11-06T01:33:44.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1960a3f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c0ff670 0x7f195c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:44.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1960a3f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c0ff670 0x7f195c198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39120/0 (socket says 172.21.15.53:39120) 2023-11-06T01:33:44.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1960a3f700 1 -- 172.21.15.53:0/2533924421 learned_addr learned my addr 172.21.15.53:0/2533924421 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:44.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1961240700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f195c100990 0x7f195c1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:44.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1960a3f700 1 -- 172.21.15.53:0/2533924421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f195c100990 msgr2=0x7f195c1a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1960a3f700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f195c100990 0x7f195c1a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1960a3f700 1 -- 172.21.15.53:0/2533924421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195c0fffd0 msgr2=0x7f195c199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1960a3f700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195c0fffd0 0x7f195c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1960a3f700 1 -- 172.21.15.53:0/2533924421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1954009d00 con 0x7f195c0ff670 2023-11-06T01:33:44.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1961240700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f195c100990 0x7f195c1a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:44.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1960a3f700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c0ff670 0x7f195c198d10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f195000c9b0 tx=0x7f195000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:44.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1959ffb700 1 -- 172.21.15.53:0/2533924421 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950007b00 con 0x7f195c0ff670 2023-11-06T01:33:44.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1959ffb700 1 -- 172.21.15.53:0/2533924421 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1950007c60 con 0x7f195c0ff670 2023-11-06T01:33:44.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.383+0000 7f1962ca3700 1 -- 172.21.15.53:0/2533924421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c1a0e60 con 0x7f195c0ff670 2023-11-06T01:33:44.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.384+0000 7f1959ffb700 1 -- 172.21.15.53:0/2533924421 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950018660 con 0x7f195c0ff670 2023-11-06T01:33:44.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.384+0000 7f1962ca3700 1 -- 172.21.15.53:0/2533924421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c1a1280 con 0x7f195c0ff670 2023-11-06T01:33:44.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.385+0000 7f1962ca3700 1 -- 172.21.15.53:0/2533924421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195c10a450 con 0x7f195c0ff670 2023-11-06T01:33:44.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.385+0000 7f1959ffb700 1 -- 172.21.15.53:0/2533924421 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f1950018880 con 0x7f195c0ff670 2023-11-06T01:33:44.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.386+0000 7f1959ffb700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f194c0a05b0 0x7f194c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.386+0000 7f195bfff700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f194c0a05b0 0x7f194c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:44.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.386+0000 7f1959ffb700 1 -- 172.21.15.53:0/2533924421 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f19500cbc50 con 0x7f195c0ff670 2023-11-06T01:33:44.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.387+0000 7f195bfff700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f194c0a05b0 0x7f194c0a2a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1948005fd0 tx=0x7f1948005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:44.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.391+0000 7f1959ffb700 1 -- 172.21.15.53:0/2533924421 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1950095610 con 0x7f195c0ff670 2023-11-06T01:33:44.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.591+0000 7f1962ca3700 1 -- 172.21.15.53:0/2533924421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f195c02cc70 con 0x7f195c0ff670 2023-11-06T01:33:44.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.592+0000 7f1959ffb700 1 -- 172.21.15.53:0/2533924421 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1950098c30 con 0x7f195c0ff670 2023-11-06T01:33:44.594 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-06T01:33:44.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:44 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3403487805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:44.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:44 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/789657046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:44.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:44 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2209606090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 -- 172.21.15.53:0/2533924421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f194c0a05b0 msgr2=0x7f194c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f194c0a05b0 0x7f194c0a2a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1948005fd0 tx=0x7f1948005f60 comp rx=0 tx=0).stop 2023-11-06T01:33:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 -- 172.21.15.53:0/2533924421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c0ff670 msgr2=0x7f195c198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c0ff670 0x7f195c198d10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f195000c9b0 tx=0x7f195000cd70 comp rx=0 tx=0).stop 2023-11-06T01:33:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 -- 172.21.15.53:0/2533924421 shutdown_connections 2023-11-06T01:33:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f194c0a05b0 0x7f194c0a2a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f195c100990 0x7f195c1a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f195c0ff670 0x7f195c198d10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 --2- 172.21.15.53:0/2533924421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195c0fffd0 0x7f195c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 -- 172.21.15.53:0/2533924421 >> 172.21.15.53:0/2533924421 conn(0x7f195c0fb1f0 msgr2=0x7f195c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 -- 172.21.15.53:0/2533924421 shutdown_connections 2023-11-06T01:33:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.594+0000 7f1962ca3700 1 -- 172.21.15.53:0/2533924421 wait complete. 2023-11-06T01:33:44.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:33:44.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:44 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3403487805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:44.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:44 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/789657046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:44.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:44 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2209606090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:44.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.732+0000 7f686e575700 1 -- 172.21.15.53:0/2679658515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868101800 msgr2=0x7f68681080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.732+0000 7f686e575700 1 --2- 172.21.15.53:0/2679658515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868101800 0x7f68681080b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f686000b0d0 tx=0x7f686000b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.732+0000 7f686e575700 1 -- 172.21.15.53:0/2679658515 shutdown_connections 2023-11-06T01:33:44.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.732+0000 7f686e575700 1 --2- 172.21.15.53:0/2679658515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68680ffc70 0x7f6868100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.732+0000 7f686e575700 1 --2- 172.21.15.53:0/2679658515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868101800 0x7f68681080b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.732+0000 7f686e575700 1 --2- 172.21.15.53:0/2679658515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6868100e60 0x7f68681012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.732+0000 7f686e575700 1 -- 172.21.15.53:0/2679658515 >> 172.21.15.53:0/2679658515 conn(0x7f68680fb1c0 msgr2=0x7f68680fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:44.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.732+0000 7f686e575700 1 -- 172.21.15.53:0/2679658515 shutdown_connections 2023-11-06T01:33:44.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.732+0000 7f686e575700 1 -- 172.21.15.53:0/2679658515 wait complete. 2023-11-06T01:33:44.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f686e575700 1 Processor -- start 2023-11-06T01:33:44.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f686e575700 1 -- start start 2023-11-06T01:33:44.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f686e575700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68680ffc70 0x7f6868103c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f686e575700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868100e60 0x7f6868102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f686e575700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6868101800 0x7f6868102780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f686e575700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6868102f20 con 0x7f6868100e60 2023-11-06T01:33:44.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f686e575700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6868103090 con 0x7f68680ffc70 2023-11-06T01:33:44.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f686e575700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681a0b30 con 0x7f6868101800 2023-11-06T01:33:44.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f68677fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868100e60 0x7f6868102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:44.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f68677fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868100e60 0x7f6868102240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39136/0 (socket says 172.21.15.53:39136) 2023-11-06T01:33:44.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f68677fe700 1 -- 172.21.15.53:0/824861796 learned_addr learned my addr 172.21.15.53:0/824861796 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:44.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f686cb12700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6868101800 0x7f6868102780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:44.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f68677fe700 1 -- 172.21.15.53:0/824861796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6868101800 msgr2=0x7f6868102780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f68677fe700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6868101800 0x7f6868102780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f68677fe700 1 -- 172.21.15.53:0/824861796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68680ffc70 msgr2=0x7f6868103c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f68677fe700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68680ffc70 0x7f6868103c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f68677fe700 1 -- 172.21.15.53:0/824861796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6860009d00 con 0x7f6868100e60 2023-11-06T01:33:44.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.733+0000 7f68677fe700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868100e60 0x7f6868102240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f685c00d950 tx=0x7f685c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:44.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.734+0000 7f68657fa700 1 -- 172.21.15.53:0/824861796 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c009b40 con 0x7f6868100e60 2023-11-06T01:33:44.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.734+0000 7f68657fa700 1 -- 172.21.15.53:0/824861796 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f685c009ca0 con 0x7f6868100e60 2023-11-06T01:33:44.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.734+0000 7f686e575700 1 -- 172.21.15.53:0/824861796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68681a0e10 con 0x7f6868100e60 2023-11-06T01:33:44.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.734+0000 7f68657fa700 1 -- 172.21.15.53:0/824861796 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c00f620 con 0x7f6868100e60 2023-11-06T01:33:44.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.734+0000 7f686e575700 1 -- 172.21.15.53:0/824861796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68681a1290 con 0x7f6868100e60 2023-11-06T01:33:44.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.735+0000 7f686e575700 1 -- 172.21.15.53:0/824861796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686810b490 con 0x7f6868100e60 2023-11-06T01:33:44.746 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:44 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3403487805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:44.747 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:44 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/789657046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:44.747 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:44 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2209606090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:44.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.736+0000 7f68657fa700 1 -- 172.21.15.53:0/824861796 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f685c010460 con 0x7f6868100e60 2023-11-06T01:33:44.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.736+0000 7f68657fa700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68500a05f0 0x7f68500a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:44.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.737+0000 7f68657fa700 1 -- 172.21.15.53:0/824861796 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f685c0cb910 con 0x7f6868100e60 2023-11-06T01:33:44.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.737+0000 7f6867fff700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68500a05f0 0x7f68500a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:44.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.737+0000 7f6867fff700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68500a05f0 0x7f68500a2ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6868100cc0 tx=0x7f6858005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:44.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.741+0000 7f68657fa700 1 -- 172.21.15.53:0/824861796 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f685c095330 con 0x7f6868100e60 2023-11-06T01:33:44.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.989+0000 7f686e575700 1 -- 172.21.15.53:0/824861796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68681a18a0 con 0x7f6868100e60 2023-11-06T01:33:44.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.990+0000 7f68657fa700 1 -- 172.21.15.53:0/824861796 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10815 (secure 0 0 0) 0x7f685c016090 con 0x7f6868100e60 2023-11-06T01:33:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:44.992 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-06T01:32:26.093194+0000","modified":"2023-11-06T01:32:33.006369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24472,"mds_2":24464,"mds_3":24422,"mds_4":24395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24395":{"gid":24395,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T01:33:44.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.992+0000 7f686e575700 1 -- 172.21.15.53:0/824861796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68500a05f0 msgr2=0x7f68500a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.992+0000 7f686e575700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68500a05f0 0x7f68500a2ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6868100cc0 tx=0x7f6858005ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.992+0000 7f686e575700 1 -- 172.21.15.53:0/824861796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868100e60 msgr2=0x7f6868102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:44.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.992+0000 7f686e575700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868100e60 0x7f6868102240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f685c00d950 tx=0x7f685c00dd10 comp rx=0 tx=0).stop 2023-11-06T01:33:44.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.992+0000 7f686e575700 1 -- 172.21.15.53:0/824861796 shutdown_connections 2023-11-06T01:33:44.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.992+0000 7f686e575700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68500a05f0 0x7f68500a2ab0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.992+0000 7f686e575700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68680ffc70 0x7f6868103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.992+0000 7f686e575700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6868101800 0x7f6868102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.992+0000 7f686e575700 1 --2- 172.21.15.53:0/824861796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868100e60 0x7f6868102240 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:44.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.992+0000 7f686e575700 1 -- 172.21.15.53:0/824861796 >> 172.21.15.53:0/824861796 conn(0x7f68680fb1c0 msgr2=0x7f6868067f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:44.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.993+0000 7f686e575700 1 -- 172.21.15.53:0/824861796 shutdown_connections 2023-11-06T01:33:44.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:44.993+0000 7f686e575700 1 -- 172.21.15.53:0/824861796 wait complete. 2023-11-06T01:33:44.997 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-11-06T01:33:45.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:33:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.132+0000 7f4cd299d700 1 -- 172.21.15.53:0/244861535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc0ffb00 msgr2=0x7f4ccc0fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.132+0000 7f4cd299d700 1 --2- 172.21.15.53:0/244861535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc0ffb00 0x7f4ccc0fff20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80099c0 tx=0x7f4cc8009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.132+0000 7f4cd299d700 1 -- 172.21.15.53:0/244861535 shutdown_connections 2023-11-06T01:33:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.132+0000 7f4cd299d700 1 --2- 172.21.15.53:0/244861535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ccc101690 0x7f4ccc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.132+0000 7f4cd299d700 1 --2- 172.21.15.53:0/244861535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc0ffb00 0x7f4ccc0fff20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.132+0000 7f4cd299d700 1 --2- 172.21.15.53:0/244861535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ccc100cf0 0x7f4ccc101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.132+0000 7f4cd299d700 1 -- 172.21.15.53:0/244861535 >> 172.21.15.53:0/244861535 conn(0x7f4ccc0fb050 msgr2=0x7f4ccc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.132+0000 7f4cd299d700 1 -- 172.21.15.53:0/244861535 shutdown_connections 2023-11-06T01:33:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.132+0000 7f4cd299d700 1 -- 172.21.15.53:0/244861535 wait complete. 2023-11-06T01:33:45.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd299d700 1 Processor -- start 2023-11-06T01:33:45.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd299d700 1 -- start start 2023-11-06T01:33:45.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd299d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc0ffb00 0x7f4ccc1956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd299d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ccc100cf0 0x7f4ccc195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd299d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ccc101690 0x7f4ccc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd299d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc19d230 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd299d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc19d370 con 0x7f4ccc100cf0 2023-11-06T01:33:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd299d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc19d500 con 0x7f4ccc101690 2023-11-06T01:33:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd199b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc0ffb00 0x7f4ccc1956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd199b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc0ffb00 0x7f4ccc1956b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39148/0 (socket says 172.21.15.53:39148) 2023-11-06T01:33:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd199b700 1 -- 172.21.15.53:0/1102117550 learned_addr learned my addr 172.21.15.53:0/1102117550 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd119a700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ccc100cf0 0x7f4ccc195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd219c700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ccc101690 0x7f4ccc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd199b700 1 -- 172.21.15.53:0/1102117550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ccc101690 msgr2=0x7f4ccc19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd199b700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ccc101690 0x7f4ccc19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd199b700 1 -- 172.21.15.53:0/1102117550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ccc100cf0 msgr2=0x7f4ccc195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd199b700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ccc100cf0 0x7f4ccc195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd199b700 1 -- 172.21.15.53:0/1102117550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc80095f0 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.133+0000 7f4cd199b700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc0ffb00 0x7f4ccc1956b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009990 tx=0x7f4cc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.134+0000 7f4cbeffd700 1 -- 172.21.15.53:0/1102117550 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc801d070 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.134+0000 7f4cbeffd700 1 -- 172.21.15.53:0/1102117550 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc8004010 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.134+0000 7f4cd299d700 1 -- 172.21.15.53:0/1102117550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ccc19d780 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.134+0000 7f4cbeffd700 1 -- 172.21.15.53:0/1102117550 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8022cd0 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.134+0000 7f4cd299d700 1 -- 172.21.15.53:0/1102117550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ccc19dc80 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.135+0000 7f4cd299d700 1 -- 172.21.15.53:0/1102117550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ccc10b310 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.136+0000 7f4cbeffd700 1 -- 172.21.15.53:0/1102117550 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f4cc8022470 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.137+0000 7f4cbeffd700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4cb80a0730 0x7f4cb80a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.137+0000 7f4cd119a700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4cb80a0730 0x7f4cb80a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.137+0000 7f4cbeffd700 1 -- 172.21.15.53:0/1102117550 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4cc80cceb0 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.137+0000 7f4cd119a700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4cb80a0730 0x7f4cb80a2bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0007c60 tx=0x7f4cc00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.141+0000 7f4cbeffd700 1 -- 172.21.15.53:0/1102117550 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4cc8096960 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:45 smithi053 ceph-mon[99472]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:45.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:45 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2533924421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:45.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:45 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/824861796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:45.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.350+0000 7f4cd299d700 1 -- 172.21.15.53:0/1102117550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ccc10b4d0 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.351+0000 7f4cbeffd700 1 -- 172.21.15.53:0/1102117550 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22543 (secure 0 0 0) 0x7f4cc8099f80 con 0x7f4ccc0ffb00 2023-11-06T01:33:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:45.354 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:33:27.113678+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:33:45.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 -- 172.21.15.53:0/1102117550 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4cb80a0730 msgr2=0x7f4cb80a2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4cb80a0730 0x7f4cb80a2bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0007c60 tx=0x7f4cc00073d0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 -- 172.21.15.53:0/1102117550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc0ffb00 msgr2=0x7f4ccc1956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc0ffb00 0x7f4ccc1956b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009990 tx=0x7f4cc8004730 comp rx=0 tx=0).stop 2023-11-06T01:33:45.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 -- 172.21.15.53:0/1102117550 shutdown_connections 2023-11-06T01:33:45.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4cb80a0730 0x7f4cb80a2bf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ccc101690 0x7f4ccc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc0ffb00 0x7f4ccc1956b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 --2- 172.21.15.53:0/1102117550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ccc100cf0 0x7f4ccc195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 -- 172.21.15.53:0/1102117550 >> 172.21.15.53:0/1102117550 conn(0x7f4ccc0fb050 msgr2=0x7f4ccc0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:45.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 -- 172.21.15.53:0/1102117550 shutdown_connections 2023-11-06T01:33:45.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.354+0000 7f4cd299d700 1 -- 172.21.15.53:0/1102117550 wait complete. 2023-11-06T01:33:45.370 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T01:32:24.604601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T01:32:25.457741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:33:45.371 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T01:33:45.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.509+0000 7f57bc2aa700 1 -- 172.21.15.53:0/749779110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b41005e0 msgr2=0x7f57b4100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.509+0000 7f57bc2aa700 1 --2- 172.21.15.53:0/749779110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b41005e0 0x7f57b4100a40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009990 tx=0x7f57a4009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.509+0000 7f57bc2aa700 1 -- 172.21.15.53:0/749779110 shutdown_connections 2023-11-06T01:33:45.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.509+0000 7f57bc2aa700 1 --2- 172.21.15.53:0/749779110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b4100f80 0x7f57b4107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.509+0000 7f57bc2aa700 1 --2- 172.21.15.53:0/749779110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b40ff3f0 0x7f57b40ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.509+0000 7f57bc2aa700 1 --2- 172.21.15.53:0/749779110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b41005e0 0x7f57b4100a40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.509+0000 7f57bc2aa700 1 -- 172.21.15.53:0/749779110 >> 172.21.15.53:0/749779110 conn(0x7f57b40fa960 msgr2=0x7f57b40fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.509+0000 7f57bc2aa700 1 -- 172.21.15.53:0/749779110 shutdown_connections 2023-11-06T01:33:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.509+0000 7f57bc2aa700 1 -- 172.21.15.53:0/749779110 wait complete. 2023-11-06T01:33:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.509+0000 7f57bc2aa700 1 Processor -- start 2023-11-06T01:33:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57bc2aa700 1 -- start start 2023-11-06T01:33:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57bc2aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b40ff3f0 0x7f57b4196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57bc2aa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b41005e0 0x7f57b4196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57bc2aa700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b4100f80 0x7f57b419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57bc2aa700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419e5c0 con 0x7f57b4100f80 2023-11-06T01:33:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57bc2aa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419e700 con 0x7f57b40ff3f0 2023-11-06T01:33:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57bc2aa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419e890 con 0x7f57b41005e0 2023-11-06T01:33:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57ba847700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b4100f80 0x7f57b419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57ba847700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b4100f80 0x7f57b419dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39166/0 (socket says 172.21.15.53:39166) 2023-11-06T01:33:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57ba847700 1 -- 172.21.15.53:0/3120343839 learned_addr learned my addr 172.21.15.53:0/3120343839 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57b9845700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b41005e0 0x7f57b4196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57ba046700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b40ff3f0 0x7f57b4196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57ba847700 1 -- 172.21.15.53:0/3120343839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b41005e0 msgr2=0x7f57b4196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57ba847700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b41005e0 0x7f57b4196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57ba847700 1 -- 172.21.15.53:0/3120343839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b40ff3f0 msgr2=0x7f57b4196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57ba847700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b40ff3f0 0x7f57b4196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57ba847700 1 -- 172.21.15.53:0/3120343839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a40095f0 con 0x7f57b4100f80 2023-11-06T01:33:45.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57ba046700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b40ff3f0 0x7f57b4196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:45.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.510+0000 7f57ba847700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b4100f80 0x7f57b419dff0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00ef30 tx=0x7f57ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:45.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.511+0000 7f57ab7fe700 1 -- 172.21.15.53:0/3120343839 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac018070 con 0x7f57b4100f80 2023-11-06T01:33:45.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.511+0000 7f57bc2aa700 1 -- 172.21.15.53:0/3120343839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b419eb10 con 0x7f57b4100f80 2023-11-06T01:33:45.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.511+0000 7f57ab7fe700 1 -- 172.21.15.53:0/3120343839 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57ac00f040 con 0x7f57b4100f80 2023-11-06T01:33:45.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.511+0000 7f57ab7fe700 1 -- 172.21.15.53:0/3120343839 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac00cdb0 con 0x7f57b4100f80 2023-11-06T01:33:45.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.511+0000 7f57bc2aa700 1 -- 172.21.15.53:0/3120343839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b419f020 con 0x7f57b4100f80 2023-11-06T01:33:45.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.512+0000 7f57bc2aa700 1 -- 172.21.15.53:0/3120343839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b4106370 con 0x7f57b4100f80 2023-11-06T01:33:45.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.513+0000 7f57ab7fe700 1 -- 172.21.15.53:0/3120343839 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f57ac003a00 con 0x7f57b4100f80 2023-11-06T01:33:45.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.513+0000 7f57ab7fe700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57a00a05b0 0x7f57a00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.513+0000 7f57ba046700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57a00a05b0 0x7f57a00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.513+0000 7f57ab7fe700 1 -- 172.21.15.53:0/3120343839 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f57ac0cb730 con 0x7f57b4100f80 2023-11-06T01:33:45.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.514+0000 7f57ba046700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57a00a05b0 0x7f57a00a2a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f57b4100440 tx=0x7f57b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:45.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.517+0000 7f57ab7fe700 1 -- 172.21.15.53:0/3120343839 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f57ac0951e0 con 0x7f57b4100f80 2023-11-06T01:33:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:45 smithi062 ceph-mon[103924]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:45 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2533924421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:45 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/824861796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:45.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.720+0000 7f57bc2aa700 1 -- 172.21.15.53:0/3120343839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57b402cc70 con 0x7f57b4100f80 2023-11-06T01:33:45.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.720+0000 7f57ab7fe700 1 -- 172.21.15.53:0/3120343839 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57ac098800 con 0x7f57b4100f80 2023-11-06T01:33:45.724 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:33:45.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.722+0000 7f57bc2aa700 1 -- 172.21.15.53:0/3120343839 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57a00a05b0 msgr2=0x7f57a00a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.722+0000 7f57bc2aa700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57a00a05b0 0x7f57a00a2a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f57b4100440 tx=0x7f57b0009260 comp rx=0 tx=0).stop 2023-11-06T01:33:45.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.722+0000 7f57bc2aa700 1 -- 172.21.15.53:0/3120343839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b4100f80 msgr2=0x7f57b419dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.722+0000 7f57bc2aa700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b4100f80 0x7f57b419dff0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00ef30 tx=0x7f57ac00c3a0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.722+0000 7f57bc2aa700 1 -- 172.21.15.53:0/3120343839 shutdown_connections 2023-11-06T01:33:45.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.722+0000 7f57bc2aa700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57a00a05b0 0x7f57a00a2a70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.722+0000 7f57bc2aa700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b40ff3f0 0x7f57b4196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.722+0000 7f57bc2aa700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b4100f80 0x7f57b419dff0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.722+0000 7f57bc2aa700 1 --2- 172.21.15.53:0/3120343839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b41005e0 0x7f57b4196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.722+0000 7f57bc2aa700 1 -- 172.21.15.53:0/3120343839 >> 172.21.15.53:0/3120343839 conn(0x7f57b40fa960 msgr2=0x7f57b40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:45.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.723+0000 7f57bc2aa700 1 -- 172.21.15.53:0/3120343839 shutdown_connections 2023-11-06T01:33:45.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.723+0000 7f57bc2aa700 1 -- 172.21.15.53:0/3120343839 wait complete. 2023-11-06T01:33:45.736 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T01:33:45.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:45 smithi163 ceph-mon[104416]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:45.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:45 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2533924421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:45.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:45 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/824861796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:45.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 -- 172.21.15.53:0/3095836042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4100e70 msgr2=0x7f71a41012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 --2- 172.21.15.53:0/3095836042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4100e70 0x7f71a41012d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f71900099c0 tx=0x7f7190009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 -- 172.21.15.53:0/3095836042 shutdown_connections 2023-11-06T01:33:45.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 --2- 172.21.15.53:0/3095836042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a40ffc80 0x7f71a41000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 --2- 172.21.15.53:0/3095836042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71a4101810 0x7f71a41080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 --2- 172.21.15.53:0/3095836042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4100e70 0x7f71a41012d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 -- 172.21.15.53:0/3095836042 >> 172.21.15.53:0/3095836042 conn(0x7f71a40fb1f0 msgr2=0x7f71a40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:45.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 -- 172.21.15.53:0/3095836042 shutdown_connections 2023-11-06T01:33:45.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 -- 172.21.15.53:0/3095836042 wait complete. 2023-11-06T01:33:45.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 Processor -- start 2023-11-06T01:33:45.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 -- start start 2023-11-06T01:33:45.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a40ffc80 0x7f71a4068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4100e70 0x7f71a419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71a4101810 0x7f71a41a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41a0740 con 0x7f71a4100e70 2023-11-06T01:33:45.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41a08b0 con 0x7f71a40ffc80 2023-11-06T01:33:45.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.864+0000 7f71ab04b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41a0a70 con 0x7f71a4101810 2023-11-06T01:33:45.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4100e70 0x7f71a419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4100e70 0x7f71a419db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39178/0 (socket says 172.21.15.53:39178) 2023-11-06T01:33:45.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a3fff700 1 -- 172.21.15.53:0/3531164463 learned_addr learned my addr 172.21.15.53:0/3531164463 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:45.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a8de7700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a40ffc80 0x7f71a4068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a95e8700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71a4101810 0x7f71a41a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a3fff700 1 -- 172.21.15.53:0/3531164463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71a4101810 msgr2=0x7f71a41a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a3fff700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71a4101810 0x7f71a41a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a3fff700 1 -- 172.21.15.53:0/3531164463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a40ffc80 msgr2=0x7f71a4068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:45.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a3fff700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a40ffc80 0x7f71a4068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:45.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a3fff700 1 -- 172.21.15.53:0/3531164463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71900095f0 con 0x7f71a4100e70 2023-11-06T01:33:45.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a95e8700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71a4101810 0x7f71a41a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:45.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a3fff700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4100e70 0x7f71a419db40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7190009990 tx=0x7f7190003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:45.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a1ffb700 1 -- 172.21.15.53:0/3531164463 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719001d070 con 0x7f71a4100e70 2023-11-06T01:33:45.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71ab04b700 1 -- 172.21.15.53:0/3531164463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a41a0cf0 con 0x7f71a4100e70 2023-11-06T01:33:45.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a1ffb700 1 -- 172.21.15.53:0/3531164463 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7190004640 con 0x7f71a4100e70 2023-11-06T01:33:45.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71a1ffb700 1 -- 172.21.15.53:0/3531164463 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719000f460 con 0x7f71a4100e70 2023-11-06T01:33:45.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.865+0000 7f71ab04b700 1 -- 172.21.15.53:0/3531164463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a41a11a0 con 0x7f71a4100e70 2023-11-06T01:33:45.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.866+0000 7f71ab04b700 1 -- 172.21.15.53:0/3531164463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a4106b80 con 0x7f71a4100e70 2023-11-06T01:33:45.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.867+0000 7f71a1ffb700 1 -- 172.21.15.53:0/3531164463 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f7190004060 con 0x7f71a4100e70 2023-11-06T01:33:45.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.868+0000 7f71a1ffb700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f71940a05b0 0x7f71940a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:45.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.868+0000 7f71a8de7700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f71940a05b0 0x7f71940a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:45.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.868+0000 7f71a1ffb700 1 -- 172.21.15.53:0/3531164463 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f71900cd090 con 0x7f71a4100e70 2023-11-06T01:33:45.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.869+0000 7f71a8de7700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f71940a05b0 0x7f71940a2a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f71a4100cd0 tx=0x7f719800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:45.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:45.872+0000 7f71a1ffb700 1 -- 172.21.15.53:0/3531164463 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7190096ae0 con 0x7f71a4100e70 2023-11-06T01:33:46.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.075+0000 7f71ab04b700 1 -- 172.21.15.53:0/3531164463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71a402ce40 con 0x7f71a4100e70 2023-11-06T01:33:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.075+0000 7f71a1ffb700 1 -- 172.21.15.53:0/3531164463 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f719009a100 con 0x7f71a4100e70 2023-11-06T01:33:46.078 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-06T01:33:46.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 -- 172.21.15.53:0/3531164463 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f71940a05b0 msgr2=0x7f71940a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f71940a05b0 0x7f71940a2a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f71a4100cd0 tx=0x7f719800a380 comp rx=0 tx=0).stop 2023-11-06T01:33:46.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 -- 172.21.15.53:0/3531164463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4100e70 msgr2=0x7f71a419db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4100e70 0x7f71a419db40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7190009990 tx=0x7f7190003af0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 -- 172.21.15.53:0/3531164463 shutdown_connections 2023-11-06T01:33:46.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f71940a05b0 0x7f71940a2a70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71a40ffc80 0x7f71a4068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71a4101810 0x7f71a41a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 --2- 172.21.15.53:0/3531164463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4100e70 0x7f71a419db40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 -- 172.21.15.53:0/3531164463 >> 172.21.15.53:0/3531164463 conn(0x7f71a40fb1f0 msgr2=0x7f71a40fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:46.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 -- 172.21.15.53:0/3531164463 shutdown_connections 2023-11-06T01:33:46.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.078+0000 7f71ab04b700 1 -- 172.21.15.53:0/3531164463 wait complete. 2023-11-06T01:33:46.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T01:33:46.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.223+0000 7f68f02bf700 1 -- 172.21.15.53:0/185338018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e81018b0 msgr2=0x7f68e8103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.223+0000 7f68f02bf700 1 --2- 172.21.15.53:0/185338018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e81018b0 0x7f68e8103ca0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009990 tx=0x7f68d8009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.223+0000 7f68f02bf700 1 -- 172.21.15.53:0/185338018 shutdown_connections 2023-11-06T01:33:46.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.223+0000 7f68f02bf700 1 --2- 172.21.15.53:0/185338018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8106b10 0x7f68e8108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.223+0000 7f68f02bf700 1 --2- 172.21.15.53:0/185338018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e81018b0 0x7f68e8103ca0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.223+0000 7f68f02bf700 1 --2- 172.21.15.53:0/185338018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e81041e0 0x7f68e81065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.223+0000 7f68f02bf700 1 -- 172.21.15.53:0/185338018 >> 172.21.15.53:0/185338018 conn(0x7f68e80fb220 msgr2=0x7f68e80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:46.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.223+0000 7f68f02bf700 1 -- 172.21.15.53:0/185338018 shutdown_connections 2023-11-06T01:33:46.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.223+0000 7f68f02bf700 1 -- 172.21.15.53:0/185338018 wait complete. 2023-11-06T01:33:46.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.223+0000 7f68f02bf700 1 Processor -- start 2023-11-06T01:33:46.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68f02bf700 1 -- start start 2023-11-06T01:33:46.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68f02bf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e81018b0 0x7f68e8196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68f02bf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e81041e0 0x7f68e8196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68f02bf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8106b10 0x7f68e819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68f02bf700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e819e5e0 con 0x7f68e81041e0 2023-11-06T01:33:46.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68f02bf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e819e720 con 0x7f68e8106b10 2023-11-06T01:33:46.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68f02bf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e819e8b0 con 0x7f68e81018b0 2023-11-06T01:33:46.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68ed85a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e81041e0 0x7f68e8196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68ed85a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e81041e0 0x7f68e8196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39192/0 (socket says 172.21.15.53:39192) 2023-11-06T01:33:46.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68ed85a700 1 -- 172.21.15.53:0/632795193 learned_addr learned my addr 172.21.15.53:0/632795193 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:46.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68ee05b700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e81018b0 0x7f68e8196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68ee85c700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8106b10 0x7f68e819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68ed85a700 1 -- 172.21.15.53:0/632795193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e81018b0 msgr2=0x7f68e8196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68ed85a700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e81018b0 0x7f68e8196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68ed85a700 1 -- 172.21.15.53:0/632795193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8106b10 msgr2=0x7f68e819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68ed85a700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8106b10 0x7f68e819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68ed85a700 1 -- 172.21.15.53:0/632795193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d80095f0 con 0x7f68e81041e0 2023-11-06T01:33:46.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68ed85a700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e81041e0 0x7f68e8196fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f68e400d950 tx=0x7f68e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:46.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.224+0000 7f68df7fe700 1 -- 172.21.15.53:0/632795193 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400f890 con 0x7f68e81041e0 2023-11-06T01:33:46.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.225+0000 7f68f02bf700 1 -- 172.21.15.53:0/632795193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e819eb90 con 0x7f68e81041e0 2023-11-06T01:33:46.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.225+0000 7f68df7fe700 1 -- 172.21.15.53:0/632795193 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68e4010460 con 0x7f68e81041e0 2023-11-06T01:33:46.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.225+0000 7f68df7fe700 1 -- 172.21.15.53:0/632795193 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400e610 con 0x7f68e81041e0 2023-11-06T01:33:46.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.225+0000 7f68f02bf700 1 -- 172.21.15.53:0/632795193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e819f0a0 con 0x7f68e81041e0 2023-11-06T01:33:46.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.225+0000 7f68f02bf700 1 -- 172.21.15.53:0/632795193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e810c590 con 0x7f68e81041e0 2023-11-06T01:33:46.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.227+0000 7f68df7fe700 1 -- 172.21.15.53:0/632795193 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f68e40108d0 con 0x7f68e81041e0 2023-11-06T01:33:46.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.227+0000 7f68df7fe700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68d40a05b0 0x7f68d40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.227+0000 7f68ee05b700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68d40a05b0 0x7f68d40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.227+0000 7f68df7fe700 1 -- 172.21.15.53:0/632795193 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f68e40cb900 con 0x7f68e81041e0 2023-11-06T01:33:46.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.228+0000 7f68ee05b700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68d40a05b0 0x7f68d40a2a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f68d800b5c0 tx=0x7f68d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:46.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.231+0000 7f68df7fe700 1 -- 172.21.15.53:0/632795193 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f68e4095320 con 0x7f68e81041e0 2023-11-06T01:33:46.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:46 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1102117550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:46.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:46 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3120343839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:46.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:46 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3531164463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:46.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.432+0000 7f68f02bf700 1 -- 172.21.15.53:0/632795193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68e819f650 con 0x7f68e81041e0 2023-11-06T01:33:46.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.433+0000 7f68df7fe700 1 -- 172.21.15.53:0/632795193 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68e4020020 con 0x7f68e81041e0 2023-11-06T01:33:46.435 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-06T01:33:46.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 -- 172.21.15.53:0/632795193 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68d40a05b0 msgr2=0x7f68d40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68d40a05b0 0x7f68d40a2a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f68d800b5c0 tx=0x7f68d8005fd0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 -- 172.21.15.53:0/632795193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e81041e0 msgr2=0x7f68e8196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e81041e0 0x7f68e8196fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f68e400d950 tx=0x7f68e400dc60 comp rx=0 tx=0).stop 2023-11-06T01:33:46.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 -- 172.21.15.53:0/632795193 shutdown_connections 2023-11-06T01:33:46.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f68d40a05b0 0x7f68d40a2a70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e8106b10 0x7f68e819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e81018b0 0x7f68e8196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 --2- 172.21.15.53:0/632795193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e81041e0 0x7f68e8196fa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 -- 172.21.15.53:0/632795193 >> 172.21.15.53:0/632795193 conn(0x7f68e80fb220 msgr2=0x7f68e8102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:46.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 -- 172.21.15.53:0/632795193 shutdown_connections 2023-11-06T01:33:46.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.435+0000 7f68f02bf700 1 -- 172.21.15.53:0/632795193 wait complete. 2023-11-06T01:33:46.450 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:33:46.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.573+0000 7f04624bd700 1 -- 172.21.15.53:0/1111066095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f045c100e60 msgr2=0x7f045c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.573+0000 7f04624bd700 1 --2- 172.21.15.53:0/1111066095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f045c100e60 0x7f045c1012c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f04500099c0 tx=0x7f0450009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.574+0000 7f04624bd700 1 -- 172.21.15.53:0/1111066095 shutdown_connections 2023-11-06T01:33:46.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.574+0000 7f04624bd700 1 --2- 172.21.15.53:0/1111066095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c0ffc70 0x7f045c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.574+0000 7f04624bd700 1 --2- 172.21.15.53:0/1111066095 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f045c101800 0x7f045c1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.574+0000 7f04624bd700 1 --2- 172.21.15.53:0/1111066095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f045c100e60 0x7f045c1012c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.574+0000 7f04624bd700 1 -- 172.21.15.53:0/1111066095 >> 172.21.15.53:0/1111066095 conn(0x7f045c0fb1c0 msgr2=0x7f045c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:46.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.574+0000 7f04624bd700 1 -- 172.21.15.53:0/1111066095 shutdown_connections 2023-11-06T01:33:46.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.574+0000 7f04624bd700 1 -- 172.21.15.53:0/1111066095 wait complete. 2023-11-06T01:33:46.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.574+0000 7f04624bd700 1 Processor -- start 2023-11-06T01:33:46.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.574+0000 7f04624bd700 1 -- start start 2023-11-06T01:33:46.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f04624bd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f045c0ffc70 0x7f045c068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f04624bd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f045c100e60 0x7f045c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f04624bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c101800 0x7f045c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f04624bd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c19e5f0 con 0x7f045c100e60 2023-11-06T01:33:46.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f04624bd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c19e760 con 0x7f045c101800 2023-11-06T01:33:46.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f04624bd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c19e920 con 0x7f045c0ffc70 2023-11-06T01:33:46.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f045b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f045c100e60 0x7f045c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f045b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f045c100e60 0x7f045c19b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39206/0 (socket says 172.21.15.53:39206) 2023-11-06T01:33:46.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f045b7fe700 1 -- 172.21.15.53:0/389213708 learned_addr learned my addr 172.21.15.53:0/389213708 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:46.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f0460a5a700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c101800 0x7f045c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f045bfff700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f045c0ffc70 0x7f045c068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f045b7fe700 1 -- 172.21.15.53:0/389213708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f045c0ffc70 msgr2=0x7f045c068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f045b7fe700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f045c0ffc70 0x7f045c068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f045b7fe700 1 -- 172.21.15.53:0/389213708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c101800 msgr2=0x7f045c19de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f045b7fe700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c101800 0x7f045c19de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f045b7fe700 1 -- 172.21.15.53:0/389213708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04500095f0 con 0x7f045c100e60 2023-11-06T01:33:46.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f045bfff700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f045c0ffc70 0x7f045c068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:46.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f045b7fe700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f045c100e60 0x7f045c19b810 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f045000c010 tx=0x7f0450009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:46.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f04597fa700 1 -- 172.21.15.53:0/389213708 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045001d070 con 0x7f045c100e60 2023-11-06T01:33:46.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f04597fa700 1 -- 172.21.15.53:0/389213708 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04500040d0 con 0x7f045c100e60 2023-11-06T01:33:46.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f04624bd700 1 -- 172.21.15.53:0/389213708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045c19eba0 con 0x7f045c100e60 2023-11-06T01:33:46.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f04597fa700 1 -- 172.21.15.53:0/389213708 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450005290 con 0x7f045c100e60 2023-11-06T01:33:46.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.575+0000 7f04624bd700 1 -- 172.21.15.53:0/389213708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045c19efc0 con 0x7f045c100e60 2023-11-06T01:33:46.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.576+0000 7f04624bd700 1 -- 172.21.15.53:0/389213708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045c107200 con 0x7f045c100e60 2023-11-06T01:33:46.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.577+0000 7f04597fa700 1 -- 172.21.15.53:0/389213708 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0450004810 con 0x7f045c100e60 2023-11-06T01:33:46.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.578+0000 7f04597fa700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f04440a05f0 0x7f04440a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.578+0000 7f045bfff700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f04440a05f0 0x7f04440a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.578+0000 7f04597fa700 1 -- 172.21.15.53:0/389213708 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f04500ccb20 con 0x7f045c100e60 2023-11-06T01:33:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.578+0000 7f045bfff700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f04440a05f0 0x7f04440a2ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f044c005950 tx=0x7f044c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.582+0000 7f04597fa700 1 -- 172.21.15.53:0/389213708 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0450096690 con 0x7f045c100e60 2023-11-06T01:33:46.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:46 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1102117550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:46.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:46 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3120343839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:46.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:46 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3531164463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:46.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:46 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1102117550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:46.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:46 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3120343839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:46.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:46 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3531164463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:46.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.829+0000 7f04624bd700 1 -- 172.21.15.53:0/389213708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f045c02d090 con 0x7f045c100e60 2023-11-06T01:33:46.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.829+0000 7f04597fa700 1 -- 172.21.15.53:0/389213708 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10815 (secure 0 0 0) 0x7f0450099cb0 con 0x7f045c100e60 2023-11-06T01:33:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:46.831 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-06T01:32:26.093194+0000","modified":"2023-11-06T01:32:33.006369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24472,"mds_2":24464,"mds_3":24422,"mds_4":24395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24395":{"gid":24395,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T01:33:46.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 -- 172.21.15.53:0/389213708 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f04440a05f0 msgr2=0x7f04440a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f04440a05f0 0x7f04440a2ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f044c005950 tx=0x7f044c0058e0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 -- 172.21.15.53:0/389213708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f045c100e60 msgr2=0x7f045c19b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f045c100e60 0x7f045c19b810 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f045000c010 tx=0x7f0450009580 comp rx=0 tx=0).stop 2023-11-06T01:33:46.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 -- 172.21.15.53:0/389213708 shutdown_connections 2023-11-06T01:33:46.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f04440a05f0 0x7f04440a2ab0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c101800 0x7f045c19de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f045c0ffc70 0x7f045c068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 --2- 172.21.15.53:0/389213708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f045c100e60 0x7f045c19b810 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 -- 172.21.15.53:0/389213708 >> 172.21.15.53:0/389213708 conn(0x7f045c0fb1c0 msgr2=0x7f045c104340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:46.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 -- 172.21.15.53:0/389213708 shutdown_connections 2023-11-06T01:33:46.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.832+0000 7f04624bd700 1 -- 172.21.15.53:0/389213708 wait complete. 2023-11-06T01:33:46.836 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-11-06T01:33:46.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:33:46.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.970+0000 7f475ca48700 1 -- 172.21.15.53:0/610719655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4758100e60 msgr2=0x7f47581012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.970+0000 7f475ca48700 1 --2- 172.21.15.53:0/610719655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4758100e60 0x7f47581012c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f474c0099c0 tx=0x7f474c009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.970+0000 7f475ca48700 1 -- 172.21.15.53:0/610719655 shutdown_connections 2023-11-06T01:33:46.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.970+0000 7f475ca48700 1 --2- 172.21.15.53:0/610719655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4758101800 0x7f47581080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.970+0000 7f475ca48700 1 --2- 172.21.15.53:0/610719655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47580ffc70 0x7f4758100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.970+0000 7f475ca48700 1 --2- 172.21.15.53:0/610719655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4758100e60 0x7f47581012c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.970+0000 7f475ca48700 1 -- 172.21.15.53:0/610719655 >> 172.21.15.53:0/610719655 conn(0x7f47580fb1c0 msgr2=0x7f47580fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:46.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.970+0000 7f475ca48700 1 -- 172.21.15.53:0/610719655 shutdown_connections 2023-11-06T01:33:46.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.970+0000 7f475ca48700 1 -- 172.21.15.53:0/610719655 wait complete. 2023-11-06T01:33:46.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475ca48700 1 Processor -- start 2023-11-06T01:33:46.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475ca48700 1 -- start start 2023-11-06T01:33:46.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475ca48700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47580ffc70 0x7f4758068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475ca48700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4758100e60 0x7f475819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475ca48700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758101800 0x7f475819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475ca48700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475819e550 con 0x7f47580ffc70 2023-11-06T01:33:46.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475ca48700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475819e6c0 con 0x7f4758100e60 2023-11-06T01:33:46.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475ca48700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475819e880 con 0x7f4758101800 2023-11-06T01:33:46.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47580ffc70 0x7f4758068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47580ffc70 0x7f4758068a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39218/0 (socket says 172.21.15.53:39218) 2023-11-06T01:33:46.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475659c700 1 -- 172.21.15.53:0/4147884456 learned_addr learned my addr 172.21.15.53:0/4147884456 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:46.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f4756d9d700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758101800 0x7f475819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475659c700 1 -- 172.21.15.53:0/4147884456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758101800 msgr2=0x7f475819ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475659c700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758101800 0x7f475819ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475659c700 1 -- 172.21.15.53:0/4147884456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4758100e60 msgr2=0x7f475819b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:46.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f4755d9b700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4758100e60 0x7f475819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475659c700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4758100e60 0x7f475819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:46.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f475659c700 1 -- 172.21.15.53:0/4147884456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c0095f0 con 0x7f47580ffc70 2023-11-06T01:33:46.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.971+0000 7f4756d9d700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758101800 0x7f475819ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:46.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.972+0000 7f475659c700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47580ffc70 0x7f4758068a60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f474800d980 tx=0x7f474800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:46.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.972+0000 7f47477fe700 1 -- 172.21.15.53:0/4147884456 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474800de60 con 0x7f47580ffc70 2023-11-06T01:33:46.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.972+0000 7f47477fe700 1 -- 172.21.15.53:0/4147884456 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47480044e0 con 0x7f47580ffc70 2023-11-06T01:33:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.972+0000 7f475ca48700 1 -- 172.21.15.53:0/4147884456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475819eb60 con 0x7f47580ffc70 2023-11-06T01:33:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.972+0000 7f47477fe700 1 -- 172.21.15.53:0/4147884456 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748003be0 con 0x7f47580ffc70 2023-11-06T01:33:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.972+0000 7f475ca48700 1 -- 172.21.15.53:0/4147884456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475819f070 con 0x7f47580ffc70 2023-11-06T01:33:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.973+0000 7f475ca48700 1 -- 172.21.15.53:0/4147884456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475810b480 con 0x7f47580ffc70 2023-11-06T01:33:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.974+0000 7f47477fe700 1 -- 172.21.15.53:0/4147884456 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f4748009bb0 con 0x7f47580ffc70 2023-11-06T01:33:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.974+0000 7f47477fe700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47400a05f0 0x7f47400a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.974+0000 7f4755d9b700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47400a05f0 0x7f47400a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.974+0000 7f47477fe700 1 -- 172.21.15.53:0/4147884456 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f47480cbc50 con 0x7f47580ffc70 2023-11-06T01:33:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.975+0000 7f4755d9b700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47400a05f0 0x7f47400a2ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f474c0098e0 tx=0x7f474c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:46.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:46.978+0000 7f47477fe700 1 -- 172.21.15.53:0/4147884456 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4748095670 con 0x7f47580ffc70 2023-11-06T01:33:47.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.181+0000 7f475ca48700 1 -- 172.21.15.53:0/4147884456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f475810b640 con 0x7f47580ffc70 2023-11-06T01:33:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.183+0000 7f47477fe700 1 -- 172.21.15.53:0/4147884456 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22543 (secure 0 0 0) 0x7f4748009940 con 0x7f47580ffc70 2023-11-06T01:33:47.185 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:47.185 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:33:27.113678+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:33:47.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 -- 172.21.15.53:0/4147884456 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47400a05f0 msgr2=0x7f47400a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47400a05f0 0x7f47400a2ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f474c0098e0 tx=0x7f474c00b540 comp rx=0 tx=0).stop 2023-11-06T01:33:47.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 -- 172.21.15.53:0/4147884456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47580ffc70 msgr2=0x7f4758068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47580ffc70 0x7f4758068a60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f474800d980 tx=0x7f474800dd40 comp rx=0 tx=0).stop 2023-11-06T01:33:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 -- 172.21.15.53:0/4147884456 shutdown_connections 2023-11-06T01:33:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47400a05f0 0x7f47400a2ab0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758101800 0x7f475819ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47580ffc70 0x7f4758068a60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 --2- 172.21.15.53:0/4147884456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4758100e60 0x7f475819b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 -- 172.21.15.53:0/4147884456 >> 172.21.15.53:0/4147884456 conn(0x7f47580fb1c0 msgr2=0x7f47580fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 -- 172.21.15.53:0/4147884456 shutdown_connections 2023-11-06T01:33:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.185+0000 7f475ca48700 1 -- 172.21.15.53:0/4147884456 wait complete. 2023-11-06T01:33:47.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-06T01:32:26.093194+0000', 'modified': '2023-11-06T01:32:33.006369+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24487, 'mds_1': 24472, 'mds_2': 24464, 'mds_3': 24422, 'mds_4': 24395}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6839/1353368498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 1353368498}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 1353368498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6841/1972292821', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6840', 'nonce': 1972292821}, {'type': 'v1', 'addr': '172.21.15.53:6841', 'nonce': 1972292821}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24395': {'gid': 24395, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.62:6833/585581012', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 585581012}, {'type': 'v1', 'addr': '172.21.15.62:6833', 'nonce': 585581012}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.62:6835/3901057483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 3901057483}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 3901057483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6833/731722016', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6832', 'nonce': 731722016}, {'type': 'v1', 'addr': '172.21.15.163:6833', 'nonce': 731722016}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/644941927', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 644941927}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 644941927}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6835/392449784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 392449784}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 392449784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.62:6839/1997633670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 1997633670}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 1997633670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.163:6837/1759271265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 1759271265}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 1759271265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.163:6839/3554773881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6838', 'nonce': 3554773881}, {'type': 'v1', 'addr': '172.21.15.163:6839', 'nonce': 3554773881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T01:33:47.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-11-06T01:33:47.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-11-06T01:33:47.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.326+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/2484148634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4a8104190 msgr2=0x7fd4a8106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.326+0000 7fd4ad0ce700 1 --2- 172.21.15.53:0/2484148634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4a8104190 0x7fd4a8106580 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009990 tx=0x7fd49c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.326+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/2484148634 shutdown_connections 2023-11-06T01:33:47.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.326+0000 7fd4ad0ce700 1 --2- 172.21.15.53:0/2484148634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a8106ac0 0x7fd4a8108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.326+0000 7fd4ad0ce700 1 --2- 172.21.15.53:0/2484148634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4a8101860 0x7fd4a8103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.326+0000 7fd4ad0ce700 1 --2- 172.21.15.53:0/2484148634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4a8104190 0x7fd4a8106580 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.326+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/2484148634 >> 172.21.15.53:0/2484148634 conn(0x7fd4a80fb1f0 msgr2=0x7fd4a80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.326+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/2484148634 shutdown_connections 2023-11-06T01:33:47.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.326+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/2484148634 wait complete. 2023-11-06T01:33:47.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.327+0000 7fd4ad0ce700 1 Processor -- start 2023-11-06T01:33:47.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4ad0ce700 1 -- start start 2023-11-06T01:33:47.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4ad0ce700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a8101860 0x7fd4a8196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4ad0ce700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4a8104190 0x7fd4a8197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4ad0ce700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4a8106ac0 0x7fd4a819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4ad0ce700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a819e650 con 0x7fd4a8104190 2023-11-06T01:33:47.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4ad0ce700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a819e790 con 0x7fd4a8101860 2023-11-06T01:33:47.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4ad0ce700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a819e920 con 0x7fd4a8106ac0 2023-11-06T01:33:47.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4a8104190 0x7fd4a8197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4a8104190 0x7fd4a8197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39234/0 (socket says 172.21.15.53:39234) 2023-11-06T01:33:47.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a659c700 1 -- 172.21.15.53:0/3141148018 learned_addr learned my addr 172.21.15.53:0/3141148018 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a8101860 0x7fd4a8196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a759e700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4a8106ac0 0x7fd4a819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a659c700 1 -- 172.21.15.53:0/3141148018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4a8106ac0 msgr2=0x7fd4a819e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a659c700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4a8106ac0 0x7fd4a819e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a659c700 1 -- 172.21.15.53:0/3141148018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a8101860 msgr2=0x7fd4a8196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a659c700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a8101860 0x7fd4a8196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a659c700 1 -- 172.21.15.53:0/3141148018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c0095f0 con 0x7fd4a8104190 2023-11-06T01:33:47.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a8101860 0x7fd4a8196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:47.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a759e700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4a8106ac0 0x7fd4a819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:47.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.328+0000 7fd4a659c700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4a8104190 0x7fd4a8197010 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd49c005910 tx=0x7fd49c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.329+0000 7fd48ffff700 1 -- 172.21.15.53:0/3141148018 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c01d070 con 0x7fd4a8104190 2023-11-06T01:33:47.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.329+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/3141148018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a819eba0 con 0x7fd4a8104190 2023-11-06T01:33:47.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.329+0000 7fd48ffff700 1 -- 172.21.15.53:0/3141148018 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd49c00fc30 con 0x7fd4a8104190 2023-11-06T01:33:47.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.329+0000 7fd48ffff700 1 -- 172.21.15.53:0/3141148018 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c017650 con 0x7fd4a8104190 2023-11-06T01:33:47.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.329+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/3141148018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a819efc0 con 0x7fd4a8104190 2023-11-06T01:33:47.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.331+0000 7fd48ffff700 1 -- 172.21.15.53:0/3141148018 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fd49c022470 con 0x7fd4a8104190 2023-11-06T01:33:47.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.331+0000 7fd48ffff700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd4900a05b0 0x7fd4900a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.331+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd4900a05b0 0x7fd4900a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.331+0000 7fd48ffff700 1 -- 172.21.15.53:0/3141148018 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd49c0ccf70 con 0x7fd4a8104190 2023-11-06T01:33:47.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.332+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd4900a05b0 0x7fd4900a2a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd498005950 tx=0x7fd4980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.332+0000 7fd4b925eb80 1 Processor -- start 2023-11-06T01:33:47.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.332+0000 7fd4b925eb80 1 -- start start 2023-11-06T01:33:47.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.332+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590d0a0 0x564c6590d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.332+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590da20 0x564c6590be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.332+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590c3a0 0x564c6590c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.332+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c658639c0 con 0x564c6590c3a0 2023-11-06T01:33:47.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.332+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c65861380 con 0x564c6590d0a0 2023-11-06T01:33:47.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.332+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6585cc70 con 0x564c6590da20 2023-11-06T01:33:47.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd4a759e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590c3a0 0x564c6590c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd4a759e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590c3a0 0x564c6590c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39242/0 (socket says 172.21.15.53:39242) 2023-11-06T01:33:47.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd4a759e700 1 -- 172.21.15.53:0/1453313341 learned_addr learned my addr 172.21.15.53:0/1453313341 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/1453313341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590d0a0 0x564c6590d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd4a659c700 1 --2- 172.21.15.53:0/1453313341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590da20 0x564c6590be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd4a759e700 1 -- 172.21.15.53:0/1453313341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590da20 msgr2=0x564c6590be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd4a759e700 1 --2- 172.21.15.53:0/1453313341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590da20 0x564c6590be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd4a759e700 1 -- 172.21.15.53:0/1453313341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590d0a0 msgr2=0x564c6590d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd4a759e700 1 --2- 172.21.15.53:0/1453313341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590d0a0 0x564c6590d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd4a759e700 1 -- 172.21.15.53:0/1453313341 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c6568ad90 con 0x564c6590c3a0 2023-11-06T01:33:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd4a759e700 1 --2- 172.21.15.53:0/1453313341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590c3a0 0x564c6590c820 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00084f0 tx=0x7fd4a0008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c826859f94807e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd48d7fa700 1 -- 172.21.15.53:0/1453313341 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0011710 con 0x564c6590c3a0 2023-11-06T01:33:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd48d7fa700 1 -- 172.21.15.53:0/1453313341 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4a0011d30 con 0x564c6590c3a0 2023-11-06T01:33:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.333+0000 7fd48d7fa700 1 -- 172.21.15.53:0/1453313341 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0013d10 con 0x564c6590c3a0 2023-11-06T01:33:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.334+0000 7fd4b925eb80 1 -- 172.21.15.53:0/1453313341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590c3a0 msgr2=0x564c6590c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.334+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/1453313341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590c3a0 0x564c6590c820 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00084f0 tx=0x7fd4a0008800 comp rx=0 tx=0).stop 2023-11-06T01:33:47.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.334+0000 7fd4b925eb80 1 -- 172.21.15.53:0/1453313341 shutdown_connections 2023-11-06T01:33:47.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.334+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/1453313341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590d0a0 0x564c6590d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.334+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/1453313341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590c3a0 0x564c6590c820 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.334+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/1453313341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590da20 0x564c6590be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.334+0000 7fd4b925eb80 1 -- 172.21.15.53:0/1453313341 >> 172.21.15.53:0/1453313341 conn(0x564c6586b080 msgr2=0x564c659068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.334+0000 7fd4b925eb80 1 -- 172.21.15.53:0/1453313341 shutdown_connections 2023-11-06T01:33:47.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.334+0000 7fd4b925eb80 1 -- 172.21.15.53:0/1453313341 wait complete. 2023-11-06T01:33:47.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4b925eb80 1 -- start start 2023-11-06T01:33:47.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4b925eb80 1 Processor -- start 2023-11-06T01:33:47.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590c3a0 0x564c65915960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c65915ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590da20 0x564c6591cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c657f94c0 con 0x564c6590d0a0 2023-11-06T01:33:47.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c657f9160 con 0x564c6590da20 2023-11-06T01:33:47.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c656e08d0 con 0x564c6590c3a0 2023-11-06T01:33:47.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4b925eb80 10 client.0 _ll_register_callbacks cb 0x564c65746a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:47.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4a659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c65915ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4a659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c65915ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39246/0 (socket says 172.21.15.53:39246) 2023-11-06T01:33:47.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4a659c700 1 -- 172.21.15.53:0/3791241826 learned_addr learned my addr 172.21.15.53:0/3791241826 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590c3a0 0x564c65915960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4a759e700 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590da20 0x564c6591cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4a659c700 1 -- 172.21.15.53:0/3791241826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590c3a0 msgr2=0x564c65915960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4a659c700 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590c3a0 0x564c65915960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4a659c700 1 -- 172.21.15.53:0/3791241826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590da20 msgr2=0x564c6591cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4a659c700 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590da20 0x564c6591cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.335+0000 7fd4a659c700 1 -- 172.21.15.53:0/3791241826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a0007d30 con 0x564c6590d0a0 2023-11-06T01:33:47.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd4a659c700 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c65915ea0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd49c096c70 tx=0x7fd49c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6a95f2778cda8f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd48aff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:47.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd48aff5700 1 -- 172.21.15.53:0/3791241826 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c09e5a0 con 0x564c6590d0a0 2023-11-06T01:33:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd48aff5700 1 -- 172.21.15.53:0/3791241826 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd49c0a2750 con 0x564c6590d0a0 2023-11-06T01:33:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3791241826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c65887cc0 con 0x564c6590d0a0 2023-11-06T01:33:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd48aff5700 1 -- 172.21.15.53:0/3791241826 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd49c099c60 con 0x564c6590d0a0 2023-11-06T01:33:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd4b925eb80 10 client.15234 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3791241826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564c65673c10 con 0x564c6590d0a0 2023-11-06T01:33:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd48aff5700 1 -- 172.21.15.53:0/3791241826 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c65887cc0 con 0x564c6590d0a0 2023-11-06T01:33:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd48aff5700 1 -- 172.21.15.53:0/3791241826 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fd49c022720 con 0x564c6590d0a0 2023-11-06T01:33:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd4b925eb80 10 client.15234 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd4b925eb80 10 client.15234 resolve_mds: resolved name 'k' to daemon mds.k 2023-11-06T01:33:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd4b925eb80 20 client.15234 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x564c65920840 0x564c6591fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd4b925eb80 4 client.15234 mds_command: new command op to 24395 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.336+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3791241826 --> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c658dadc0 con 0x564c65920840 2023-11-06T01:33:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.337+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x564c65920840 0x564c6591fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.337+0000 7fd48aff5700 10 client.15234 ms_handle_connect on v2:172.21.15.62:6832/585581012 2023-11-06T01:33:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.337+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x564c65920840 0x564c6591fe10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2577ff7fbc6ec316 server_cookie=e9080166f50bddf1 in_seq=0 out_seq=0 2023-11-06T01:33:47.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd48aff5700 1 -- 172.21.15.53:0/3791241826 <== mds.4 v2:172.21.15.62:6832/585581012 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564c658dadc0 con 0x564c65920840 2023-11-06T01:33:47.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd48aff5700 10 client.15234 handle_command_reply: tid=0 2023-11-06T01:33:47.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 1 client.15234 shutdown 2023-11-06T01:33:47.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3791241826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 msgr2=0x564c65915ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c65915ea0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd49c096c70 tx=0x7fd49c009310 comp rx=0 tx=0).stop 2023-11-06T01:33:47.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3791241826 shutdown_connections 2023-11-06T01:33:47.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x564c65920840 0x564c6591fe10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590da20 0x564c6591cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590c3a0 0x564c65915960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd48aff5700 0 client.15234 ms_handle_reset on v2:172.21.15.62:6832/585581012 2023-11-06T01:33:47.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3791241826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c65915ea0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3791241826 >> 172.21.15.53:0/3791241826 conn(0x564c6586b080 msgr2=0x564c65906060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3791241826 shutdown_connections 2023-11-06T01:33:47.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3791241826 wait complete. 2023-11-06T01:33:47.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.338+0000 7fd4b925eb80 20 client.15234 trim_cache size 0 max 16384 2023-11-06T01:33:47.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.355+0000 7fd4b925eb80 1 Processor -- start 2023-11-06T01:33:47.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4b925eb80 1 -- start start 2023-11-06T01:33:47.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590c3a0 0x564c6590be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c659295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590da20 0x564c65910280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c658dadc0 con 0x564c6590d0a0 2023-11-06T01:33:47.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c65887cc0 con 0x564c6590c3a0 2023-11-06T01:33:47.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c65806830 con 0x564c6590da20 2023-11-06T01:33:47.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c659295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c659295d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39254/0 (socket says 172.21.15.53:39254) 2023-11-06T01:33:47.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a659c700 1 -- 172.21.15.53:0/2549492877 learned_addr learned my addr 172.21.15.53:0/2549492877 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a759e700 1 --2- 172.21.15.53:0/2549492877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590da20 0x564c65910280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/2549492877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590c3a0 0x564c6590be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a659c700 1 -- 172.21.15.53:0/2549492877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590da20 msgr2=0x564c65910280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a659c700 1 --2- 172.21.15.53:0/2549492877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590da20 0x564c65910280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a659c700 1 -- 172.21.15.53:0/2549492877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590c3a0 msgr2=0x564c6590be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a659c700 1 --2- 172.21.15.53:0/2549492877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590c3a0 0x564c6590be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a659c700 1 -- 172.21.15.53:0/2549492877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c656b1200 con 0x564c6590d0a0 2023-11-06T01:33:47.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a759e700 1 --2- 172.21.15.53:0/2549492877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590da20 0x564c65910280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:47.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd4a659c700 1 --2- 172.21.15.53:0/2549492877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c659295d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0a2220 tx=0x7fd49c098d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a643302d7115ed63 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd48dffb700 1 -- 172.21.15.53:0/2549492877 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c060bf0 con 0x564c6590d0a0 2023-11-06T01:33:47.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd48dffb700 1 -- 172.21.15.53:0/2549492877 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd49c096ab0 con 0x564c6590d0a0 2023-11-06T01:33:47.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.356+0000 7fd48dffb700 1 -- 172.21.15.53:0/2549492877 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0cac70 con 0x564c6590d0a0 2023-11-06T01:33:47.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.357+0000 7fd4b925eb80 1 -- 172.21.15.53:0/2549492877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 msgr2=0x564c659295d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.357+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/2549492877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c659295d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0a2220 tx=0x7fd49c098d20 comp rx=0 tx=0).stop 2023-11-06T01:33:47.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.357+0000 7fd4b925eb80 1 -- 172.21.15.53:0/2549492877 shutdown_connections 2023-11-06T01:33:47.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.357+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/2549492877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590c3a0 0x564c6590be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.357+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/2549492877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590da20 0x564c65910280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.357+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/2549492877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590d0a0 0x564c659295d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.357+0000 7fd4b925eb80 1 -- 172.21.15.53:0/2549492877 >> 172.21.15.53:0/2549492877 conn(0x564c6586b080 msgr2=0x564c65913770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.357+0000 7fd4b925eb80 1 -- 172.21.15.53:0/2549492877 shutdown_connections 2023-11-06T01:33:47.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.357+0000 7fd4b925eb80 1 -- 172.21.15.53:0/2549492877 wait complete. 2023-11-06T01:33:47.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.357+0000 7fd4b925eb80 1 -- start start 2023-11-06T01:33:47.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.357+0000 7fd4b925eb80 1 Processor -- start 2023-11-06T01:33:47.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590c3a0 0x564c65924840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590d0a0 0x564c65924d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4b925eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590da20 0x564c65927bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c65928100 con 0x564c6590da20 2023-11-06T01:33:47.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c65928270 con 0x564c6590c3a0 2023-11-06T01:33:47.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4b925eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c65928430 con 0x564c6590d0a0 2023-11-06T01:33:47.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4b925eb80 10 client.0 _ll_register_callbacks cb 0x7fd49c09a2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:47.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4a759e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590da20 0x564c65927bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4a759e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590da20 0x564c65927bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39262/0 (socket says 172.21.15.53:39262) 2023-11-06T01:33:47.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4a759e700 1 -- 172.21.15.53:0/3730406586 learned_addr learned my addr 172.21.15.53:0/3730406586 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590c3a0 0x564c65924840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4a659c700 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590d0a0 0x564c65924d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4a759e700 1 -- 172.21.15.53:0/3730406586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590d0a0 msgr2=0x564c65924d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4a759e700 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590d0a0 0x564c65924d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4a759e700 1 -- 172.21.15.53:0/3730406586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590c3a0 msgr2=0x564c65924840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4a759e700 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590c3a0 0x564c65924840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4a759e700 1 -- 172.21.15.53:0/3730406586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c65673c10 con 0x564c6590da20 2023-11-06T01:33:47.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4a759e700 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590da20 0x564c65927bc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x564c65901400 tx=0x7fd4a0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d195b27a2fb9a21 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd48aff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:47.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd48aff5700 1 -- 172.21.15.53:0/3730406586 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0028660 con 0x564c6590da20 2023-11-06T01:33:47.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3730406586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4a0011d30 con 0x564c6590da20 2023-11-06T01:33:47.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.358+0000 7fd48aff5700 1 -- 172.21.15.53:0/3730406586 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd4a0011ea0 con 0x564c6590da20 2023-11-06T01:33:47.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd48aff5700 1 -- 172.21.15.53:0/3730406586 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4a00287c0 con 0x564c6590da20 2023-11-06T01:33:47.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd4b925eb80 10 client.15240 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:47.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3730406586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564c65689260 con 0x564c6590da20 2023-11-06T01:33:47.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd48aff5700 1 -- 172.21.15.53:0/3730406586 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4a0011d30 con 0x564c6590da20 2023-11-06T01:33:47.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd48aff5700 1 -- 172.21.15.53:0/3730406586 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fd4a0031050 con 0x564c6590da20 2023-11-06T01:33:47.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd4b925eb80 10 client.15240 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:47.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd4b925eb80 10 client.15240 resolve_mds: resolved name 'k' to daemon mds.k 2023-11-06T01:33:47.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd4b925eb80 20 client.15240 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:47.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x564c659778f0 0x564c65974360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd4b925eb80 4 client.15240 mds_command: new command op to 24395 tid=0[{"prefix": "status"}] 2023-11-06T01:33:47.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3730406586 --> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564c65855cf0 con 0x564c659778f0 2023-11-06T01:33:47.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.359+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x564c659778f0 0x564c65974360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd48aff5700 10 client.15240 ms_handle_connect on v2:172.21.15.62:6832/585581012 2023-11-06T01:33:47.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4a6d9d700 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x564c659778f0 0x564c65974360 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c88a4aecbc540c0e server_cookie=d62002a846a1b2a in_seq=0 out_seq=0 2023-11-06T01:33:47.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd48aff5700 1 -- 172.21.15.53:0/3730406586 <== mds.4 v2:172.21.15.62:6832/585581012 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x564c65855cf0 con 0x564c659778f0 2023-11-06T01:33:47.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd48aff5700 10 client.15240 handle_command_reply: tid=0 2023-11-06T01:33:47.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 1 client.15240 shutdown 2023-11-06T01:33:47.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3730406586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590da20 msgr2=0x564c65927bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590da20 0x564c65927bc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x564c65901400 tx=0x7fd4a0011b70 comp rx=0 tx=0).stop 2023-11-06T01:33:47.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3730406586 shutdown_connections 2023-11-06T01:33:47.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x564c659778f0 0x564c65974360 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c6590c3a0 0x564c65924840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd48aff5700 0 client.15240 ms_handle_reset on v2:172.21.15.62:6832/585581012 2023-11-06T01:33:47.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564c6590da20 0x564c65927bc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 1 --2- 172.21.15.53:0/3730406586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c6590d0a0 0x564c65924d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3730406586 >> 172.21.15.53:0/3730406586 conn(0x564c6586b080 msgr2=0x564c65910280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3730406586 shutdown_connections 2023-11-06T01:33:47.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 1 -- 172.21.15.53:0/3730406586 wait complete. 2023-11-06T01:33:47.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.360+0000 7fd4b925eb80 20 client.15240 trim_cache size 0 max 16384 2023-11-06T01:33:47.386 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:47.386 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:33:47.386 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 4, 2023-11-06T01:33:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24395, 2023-11-06T01:33:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-06T01:33:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-06T01:33:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-06T01:33:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 75.358064326000004, 2023-11-06T01:33:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 27, 2023-11-06T01:33:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 84, 2023-11-06T01:33:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 82, 2023-11-06T01:33:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 108.49418421199999 2023-11-06T01:33:47.388 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:47.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/3141148018 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd4900a05b0 msgr2=0x7fd4900a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd4900a05b0 0x7fd4900a2a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd498005950 tx=0x7fd4980058e0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/3141148018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4a8104190 msgr2=0x7fd4a8197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4a8104190 0x7fd4a8197010 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd49c005910 tx=0x7fd49c00f740 comp rx=0 tx=0).stop 2023-11-06T01:33:47.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/3141148018 shutdown_connections 2023-11-06T01:33:47.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd4900a05b0 0x7fd4900a2a70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a8101860 0x7fd4a8196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4a8106ac0 0x7fd4a819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 --2- 172.21.15.53:0/3141148018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4a8104190 0x7fd4a8197010 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/3141148018 >> 172.21.15.53:0/3141148018 conn(0x7fd4a80fb1f0 msgr2=0x7fd4a81024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/3141148018 shutdown_connections 2023-11-06T01:33:47.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.361+0000 7fd4ad0ce700 1 -- 172.21.15.53:0/3141148018 wait complete. 2023-11-06T01:33:47.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-11-06T01:33:47.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.518+0000 7f6617f19700 1 -- 172.21.15.53:0/3831194529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6610100e70 msgr2=0x7f66101012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.518+0000 7f6617f19700 1 --2- 172.21.15.53:0/3831194529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6610100e70 0x7f66101012d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f66000099c0 tx=0x7f6600009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.518+0000 7f6617f19700 1 -- 172.21.15.53:0/3831194529 shutdown_connections 2023-11-06T01:33:47.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.518+0000 7f6617f19700 1 --2- 172.21.15.53:0/3831194529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66100ffc80 0x7f66101000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.518+0000 7f6617f19700 1 --2- 172.21.15.53:0/3831194529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6610101810 0x7f66101080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.518+0000 7f6617f19700 1 --2- 172.21.15.53:0/3831194529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6610100e70 0x7f66101012d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.518+0000 7f6617f19700 1 -- 172.21.15.53:0/3831194529 >> 172.21.15.53:0/3831194529 conn(0x7f66100fb1f0 msgr2=0x7f66100fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.518+0000 7f6617f19700 1 -- 172.21.15.53:0/3831194529 shutdown_connections 2023-11-06T01:33:47.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.518+0000 7f6617f19700 1 -- 172.21.15.53:0/3831194529 wait complete. 2023-11-06T01:33:47.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f6617f19700 1 Processor -- start 2023-11-06T01:33:47.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f6617f19700 1 -- start start 2023-11-06T01:33:47.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f6617f19700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66100ffc80 0x7f6610103fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f6617f19700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610100e70 0x7f6610102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f6617f19700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6610101810 0x7f6610102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f6617f19700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66101032e0 con 0x7f6610101810 2023-11-06T01:33:47.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f6617f19700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6610103450 con 0x7f6610100e70 2023-11-06T01:33:47.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f6617f19700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66101a0ae0 con 0x7f66100ffc80 2023-11-06T01:33:47.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f66164b6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6610101810 0x7f6610102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f66164b6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6610101810 0x7f6610102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39276/0 (socket says 172.21.15.53:39276) 2023-11-06T01:33:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f66164b6700 1 -- 172.21.15.53:0/2073788300 learned_addr learned my addr 172.21.15.53:0/2073788300 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f66154b4700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610100e70 0x7f6610102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.519+0000 7f6615cb5700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66100ffc80 0x7f6610103fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f66164b6700 1 -- 172.21.15.53:0/2073788300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66100ffc80 msgr2=0x7f6610103fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f66164b6700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66100ffc80 0x7f6610103fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f66164b6700 1 -- 172.21.15.53:0/2073788300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610100e70 msgr2=0x7f6610102600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f66164b6700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610100e70 0x7f6610102600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f66164b6700 1 -- 172.21.15.53:0/2073788300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66000095f0 con 0x7f6610101810 2023-11-06T01:33:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f6615cb5700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66100ffc80 0x7f6610103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f66164b6700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6610101810 0x7f6610102b40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f660800ef30 tx=0x7f660800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f6606ffd700 1 -- 172.21.15.53:0/2073788300 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608018070 con 0x7f6610101810 2023-11-06T01:33:47.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f6606ffd700 1 -- 172.21.15.53:0/2073788300 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660800f040 con 0x7f6610101810 2023-11-06T01:33:47.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f6617f19700 1 -- 172.21.15.53:0/2073788300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66101a0d60 con 0x7f6610101810 2023-11-06T01:33:47.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f6606ffd700 1 -- 172.21.15.53:0/2073788300 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800cdb0 con 0x7f6610101810 2023-11-06T01:33:47.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.520+0000 7f6617f19700 1 -- 172.21.15.53:0/2073788300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66101a1210 con 0x7f6610101810 2023-11-06T01:33:47.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.522+0000 7f6606ffd700 1 -- 172.21.15.53:0/2073788300 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f6608003a00 con 0x7f6610101810 2023-11-06T01:33:47.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.523+0000 7f6606ffd700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65fc0a0630 0x7f65fc0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.523+0000 7f6615cb5700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65fc0a0630 0x7f65fc0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.523+0000 7f6606ffd700 1 -- 172.21.15.53:0/2073788300 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f66080cb6b0 con 0x7f6610101810 2023-11-06T01:33:47.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.523+0000 7f6615cb5700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65fc0a0630 0x7f65fc0a2af0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6610100cd0 tx=0x7f660c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.523+0000 7f66240a9b80 1 Processor -- start 2023-11-06T01:33:47.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.523+0000 7f66240a9b80 1 -- start start 2023-11-06T01:33:47.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e64a20 0x55b256e62e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e633a0 0x55b256e63820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66240a9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256dba9c0 con 0x55b256e633a0 2023-11-06T01:33:47.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66240a9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256db8380 con 0x55b256e64a20 2023-11-06T01:33:47.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66240a9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256db3c70 con 0x55b256e640a0 2023-11-06T01:33:47.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66164b6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e633a0 0x55b256e63820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66164b6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e633a0 0x55b256e63820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39284/0 (socket says 172.21.15.53:39284) 2023-11-06T01:33:47.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66164b6700 1 -- 172.21.15.53:0/1388909430 learned_addr learned my addr 172.21.15.53:0/1388909430 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f6615cb5700 1 --2- 172.21.15.53:0/1388909430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66154b4700 1 --2- 172.21.15.53:0/1388909430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e64a20 0x55b256e62e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66164b6700 1 -- 172.21.15.53:0/1388909430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 msgr2=0x55b256e644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66164b6700 1 --2- 172.21.15.53:0/1388909430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66164b6700 1 -- 172.21.15.53:0/1388909430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e64a20 msgr2=0x55b256e62e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66164b6700 1 --2- 172.21.15.53:0/1388909430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e64a20 0x55b256e62e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66164b6700 1 -- 172.21.15.53:0/1388909430 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b256be1d90 con 0x55b256e633a0 2023-11-06T01:33:47.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f6615cb5700 1 --2- 172.21.15.53:0/1388909430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:47.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f66164b6700 1 --2- 172.21.15.53:0/1388909430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e633a0 0x55b256e63820 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f66080090a0 tx=0x7f6608095290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6931a3bd4f24bfc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f65fbfff700 1 -- 172.21.15.53:0/1388909430 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660801b320 con 0x55b256e633a0 2023-11-06T01:33:47.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f65fbfff700 1 -- 172.21.15.53:0/1388909430 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6608099520 con 0x55b256e633a0 2023-11-06T01:33:47.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.524+0000 7f65fbfff700 1 -- 172.21.15.53:0/1388909430 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608099080 con 0x55b256e633a0 2023-11-06T01:33:47.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.525+0000 7f66240a9b80 1 -- 172.21.15.53:0/1388909430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e633a0 msgr2=0x55b256e63820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.525+0000 7f66240a9b80 1 --2- 172.21.15.53:0/1388909430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e633a0 0x55b256e63820 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f66080090a0 tx=0x7f6608095290 comp rx=0 tx=0).stop 2023-11-06T01:33:47.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.525+0000 7f66240a9b80 1 -- 172.21.15.53:0/1388909430 shutdown_connections 2023-11-06T01:33:47.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.525+0000 7f66240a9b80 1 --2- 172.21.15.53:0/1388909430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e633a0 0x55b256e63820 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.525+0000 7f66240a9b80 1 --2- 172.21.15.53:0/1388909430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.525+0000 7f66240a9b80 1 --2- 172.21.15.53:0/1388909430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e64a20 0x55b256e62e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.525+0000 7f66240a9b80 1 -- 172.21.15.53:0/1388909430 >> 172.21.15.53:0/1388909430 conn(0x55b256dc2080 msgr2=0x55b256e5d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.525+0000 7f66240a9b80 1 -- 172.21.15.53:0/1388909430 shutdown_connections 2023-11-06T01:33:47.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.525+0000 7f66240a9b80 1 -- 172.21.15.53:0/1388909430 wait complete. 2023-11-06T01:33:47.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66240a9b80 1 -- start start 2023-11-06T01:33:47.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66240a9b80 1 Processor -- start 2023-11-06T01:33:47.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e6c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e640a0 0x55b256e6cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e64a20 0x55b256e73f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66240a9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256d504c0 con 0x55b256e640a0 2023-11-06T01:33:47.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66240a9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256d50160 con 0x55b256e633a0 2023-11-06T01:33:47.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66240a9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256c378d0 con 0x55b256e64a20 2023-11-06T01:33:47.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66240a9b80 10 client.0 _ll_register_callbacks cb 0x55b256c9da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:47.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66154b4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e640a0 0x55b256e6cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66154b4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e640a0 0x55b256e6cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39288/0 (socket says 172.21.15.53:39288) 2023-11-06T01:33:47.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f66154b4700 1 -- 172.21.15.53:0/1715445864 learned_addr learned my addr 172.21.15.53:0/1715445864 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.526+0000 7f6615cb5700 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e6c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f66164b6700 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e64a20 0x55b256e73f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f66154b4700 1 -- 172.21.15.53:0/1715445864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e64a20 msgr2=0x55b256e73f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f66154b4700 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e64a20 0x55b256e73f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f66154b4700 1 -- 172.21.15.53:0/1715445864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 msgr2=0x55b256e6c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f66154b4700 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e6c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f66154b4700 1 -- 172.21.15.53:0/1715445864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66101a1210 con 0x55b256e640a0 2023-11-06T01:33:47.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f66154b4700 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e640a0 0x55b256e6cf20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f66000058e0 tx=0x7f660000bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa059c78a4b4f893 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f65f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:47.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f65f97fa700 1 -- 172.21.15.53:0/1715445864 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660001d050 con 0x55b256e640a0 2023-11-06T01:33:47.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f65f97fa700 1 -- 172.21.15.53:0/1715445864 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f66000054e0 con 0x55b256e640a0 2023-11-06T01:33:47.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f66240a9b80 1 -- 172.21.15.53:0/1715445864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b256ddecc0 con 0x55b256e640a0 2023-11-06T01:33:47.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f65f97fa700 1 -- 172.21.15.53:0/1715445864 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66000056a0 con 0x55b256e640a0 2023-11-06T01:33:47.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f66240a9b80 10 client.15252 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:47.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f66240a9b80 1 -- 172.21.15.53:0/1715445864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b256bcac10 con 0x55b256e640a0 2023-11-06T01:33:47.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f65f97fa700 1 -- 172.21.15.53:0/1715445864 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b256ddecc0 con 0x55b256e640a0 2023-11-06T01:33:47.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.527+0000 7f65f97fa700 1 -- 172.21.15.53:0/1715445864 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f660000f990 con 0x55b256e640a0 2023-11-06T01:33:47.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.528+0000 7f66240a9b80 10 client.15252 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:47.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.528+0000 7f66240a9b80 10 client.15252 resolve_mds: resolved name 'e' to daemon mds.e 2023-11-06T01:33:47.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.528+0000 7f66240a9b80 20 client.15252 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:47.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.528+0000 7f66240a9b80 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x55b256e77cd0 0x55b256e770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.528+0000 7f66240a9b80 4 client.15252 mds_command: new command op to 24422 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:47.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.528+0000 7f66240a9b80 1 -- 172.21.15.53:0/1715445864 --> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b256e31dc0 con 0x55b256e77cd0 2023-11-06T01:33:47.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.528+0000 7f6615cb5700 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x55b256e77cd0 0x55b256e770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.528+0000 7f6615cb5700 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x55b256e77cd0 0x55b256e770b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ed8958e2f233a068 server_cookie=fa228350f7303b20 in_seq=0 out_seq=0 2023-11-06T01:33:47.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.528+0000 7f65f97fa700 10 client.15252 ms_handle_connect on v2:172.21.15.62:6834/3901057483 2023-11-06T01:33:47.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f65f97fa700 1 -- 172.21.15.53:0/1715445864 <== mds.3 v2:172.21.15.62:6834/3901057483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b256e31dc0 con 0x55b256e77cd0 2023-11-06T01:33:47.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f65f97fa700 10 client.15252 handle_command_reply: tid=0 2023-11-06T01:33:47.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 1 client.15252 shutdown 2023-11-06T01:33:47.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 1 -- 172.21.15.53:0/1715445864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e640a0 msgr2=0x55b256e6cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e640a0 0x55b256e6cf20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f66000058e0 tx=0x7f660000bf00 comp rx=0 tx=0).stop 2023-11-06T01:33:47.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 1 -- 172.21.15.53:0/1715445864 shutdown_connections 2023-11-06T01:33:47.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x55b256e77cd0 0x55b256e770b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e6c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e64a20 0x55b256e73f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f65f97fa700 0 client.15252 ms_handle_reset on v2:172.21.15.62:6834/3901057483 2023-11-06T01:33:47.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 1 --2- 172.21.15.53:0/1715445864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256e640a0 0x55b256e6cf20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 1 -- 172.21.15.53:0/1715445864 >> 172.21.15.53:0/1715445864 conn(0x55b256dc2080 msgr2=0x55b256e5d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 1 -- 172.21.15.53:0/1715445864 shutdown_connections 2023-11-06T01:33:47.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 1 -- 172.21.15.53:0/1715445864 wait complete. 2023-11-06T01:33:47.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.529+0000 7f66240a9b80 20 client.15252 trim_cache size 0 max 16384 2023-11-06T01:33:47.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f66240a9b80 1 Processor -- start 2023-11-06T01:33:47.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f66240a9b80 1 -- start start 2023-11-06T01:33:47.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e67320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e80a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e67860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f66240a9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256e5f480 con 0x55b256dc2080 2023-11-06T01:33:47.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f66240a9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256e31dc0 con 0x55b256e633a0 2023-11-06T01:33:47.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f66240a9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256e6dbd0 con 0x55b256e640a0 2023-11-06T01:33:47.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f6615cb5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e67320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f6615cb5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e67320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39294/0 (socket says 172.21.15.53:39294) 2023-11-06T01:33:47.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f6615cb5700 1 -- 172.21.15.53:0/3192491919 learned_addr learned my addr 172.21.15.53:0/3192491919 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f66154b4700 1 --2- 172.21.15.53:0/3192491919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e80a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f6615cb5700 1 -- 172.21.15.53:0/3192491919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 msgr2=0x55b256e67860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f6615cb5700 1 --2- 172.21.15.53:0/3192491919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e67860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f6615cb5700 1 -- 172.21.15.53:0/3192491919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 msgr2=0x55b256e80a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.547+0000 7f6615cb5700 1 --2- 172.21.15.53:0/3192491919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e80a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f6615cb5700 1 -- 172.21.15.53:0/3192491919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b256c08200 con 0x55b256dc2080 2023-11-06T01:33:47.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f66154b4700 1 --2- 172.21.15.53:0/3192491919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e80a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:47.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f6615cb5700 1 --2- 172.21.15.53:0/3192491919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e67320 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55b256d5d960 tx=0x7f660c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a2352dec1b3859 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f6604ff9700 1 -- 172.21.15.53:0/3192491919 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c024040 con 0x55b256dc2080 2023-11-06T01:33:47.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f6604ff9700 1 -- 172.21.15.53:0/3192491919 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660c004cf0 con 0x55b256dc2080 2023-11-06T01:33:47.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f6604ff9700 1 -- 172.21.15.53:0/3192491919 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c01e430 con 0x55b256dc2080 2023-11-06T01:33:47.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f66240a9b80 1 -- 172.21.15.53:0/3192491919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 msgr2=0x55b256e67320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f66240a9b80 1 --2- 172.21.15.53:0/3192491919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e67320 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55b256d5d960 tx=0x7f660c014000 comp rx=0 tx=0).stop 2023-11-06T01:33:47.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f66240a9b80 1 -- 172.21.15.53:0/3192491919 shutdown_connections 2023-11-06T01:33:47.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f66240a9b80 1 --2- 172.21.15.53:0/3192491919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e67860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f66240a9b80 1 --2- 172.21.15.53:0/3192491919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e67320 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f66240a9b80 1 --2- 172.21.15.53:0/3192491919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e80a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f66240a9b80 1 -- 172.21.15.53:0/3192491919 >> 172.21.15.53:0/3192491919 conn(0x55b256e5dc20 msgr2=0x55b256e770b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.548+0000 7f66240a9b80 1 -- 172.21.15.53:0/3192491919 shutdown_connections 2023-11-06T01:33:47.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66240a9b80 1 -- 172.21.15.53:0/3192491919 wait complete. 2023-11-06T01:33:47.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66240a9b80 1 -- start start 2023-11-06T01:33:47.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66240a9b80 1 Processor -- start 2023-11-06T01:33:47.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e7be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e7c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66240a9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e7f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66240a9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256e7f8a0 con 0x55b256dc2080 2023-11-06T01:33:47.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66240a9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256e7fa10 con 0x55b256e633a0 2023-11-06T01:33:47.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66240a9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256e7fbd0 con 0x55b256e640a0 2023-11-06T01:33:47.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66240a9b80 10 client.0 _ll_register_callbacks cb 0x7f660000baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:47.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f6615cb5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e7be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f6615cb5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e7be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39300/0 (socket says 172.21.15.53:39300) 2023-11-06T01:33:47.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f6615cb5700 1 -- 172.21.15.53:0/32718557 learned_addr learned my addr 172.21.15.53:0/32718557 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66154b4700 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e7c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f66164b6700 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e7f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f6615cb5700 1 -- 172.21.15.53:0/32718557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 msgr2=0x55b256e7f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f6615cb5700 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e7f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.549+0000 7f6615cb5700 1 -- 172.21.15.53:0/32718557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 msgr2=0x55b256e7c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f6615cb5700 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e7c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f6615cb5700 1 -- 172.21.15.53:0/32718557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c00edd0 con 0x55b256dc2080 2023-11-06T01:33:47.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f66154b4700 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e7c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:47.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f66164b6700 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e7f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:47.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f6615cb5700 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e7be00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f660c00f010 tx=0x7f660c005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c640f1388a50d90 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f65f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:47.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f65f97fa700 1 -- 172.21.15.53:0/32718557 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c024070 con 0x55b256dc2080 2023-11-06T01:33:47.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f65f97fa700 1 -- 172.21.15.53:0/32718557 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f660c004950 con 0x55b256dc2080 2023-11-06T01:33:47.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f66240a9b80 1 -- 172.21.15.53:0/32718557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6608099520 con 0x55b256dc2080 2023-11-06T01:33:47.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f65f97fa700 1 -- 172.21.15.53:0/32718557 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f660c004b10 con 0x55b256dc2080 2023-11-06T01:33:47.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f66240a9b80 10 client.15258 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:47.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f66240a9b80 1 -- 172.21.15.53:0/32718557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b256be0260 con 0x55b256dc2080 2023-11-06T01:33:47.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f65f97fa700 1 -- 172.21.15.53:0/32718557 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6608099520 con 0x55b256dc2080 2023-11-06T01:33:47.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f65f97fa700 1 -- 172.21.15.53:0/32718557 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f660c02d020 con 0x55b256dc2080 2023-11-06T01:33:47.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f66240a9b80 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:47.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f66240a9b80 10 client.15258 resolve_mds: resolved name 'e' to daemon mds.e 2023-11-06T01:33:47.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f66240a9b80 20 client.15258 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:47.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f66240a9b80 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x55b256e64a20 0x55b256ecb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f66240a9b80 4 client.15258 mds_command: new command op to 24422 tid=0[{"prefix": "status"}] 2023-11-06T01:33:47.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.550+0000 7f66240a9b80 1 -- 172.21.15.53:0/32718557 --> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b256daccf0 con 0x55b256e64a20 2023-11-06T01:33:47.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66154b4700 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x55b256e64a20 0x55b256ecb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66154b4700 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x55b256e64a20 0x55b256ecb940 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d227d78722d3d75 server_cookie=8755c3ee821004ce in_seq=0 out_seq=0 2023-11-06T01:33:47.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f65f97fa700 10 client.15258 ms_handle_connect on v2:172.21.15.62:6834/3901057483 2023-11-06T01:33:47.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f65f97fa700 1 -- 172.21.15.53:0/32718557 <== mds.3 v2:172.21.15.62:6834/3901057483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b256daccf0 con 0x55b256e64a20 2023-11-06T01:33:47.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f65f97fa700 10 client.15258 handle_command_reply: tid=0 2023-11-06T01:33:47.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66240a9b80 1 client.15258 shutdown 2023-11-06T01:33:47.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66240a9b80 1 -- 172.21.15.53:0/32718557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 msgr2=0x55b256e7be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66240a9b80 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e7be00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f660c00f010 tx=0x7f660c005630 comp rx=0 tx=0).stop 2023-11-06T01:33:47.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66240a9b80 1 -- 172.21.15.53:0/32718557 shutdown_connections 2023-11-06T01:33:47.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66240a9b80 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x55b256e64a20 0x55b256ecb940 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66240a9b80 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b256e640a0 0x55b256e7f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f65f97fa700 0 client.15258 ms_handle_reset on v2:172.21.15.62:6834/3901057483 2023-11-06T01:33:47.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66240a9b80 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b256dc2080 0x55b256e7be00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66240a9b80 1 --2- 172.21.15.53:0/32718557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b256e633a0 0x55b256e7c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66240a9b80 1 -- 172.21.15.53:0/32718557 >> 172.21.15.53:0/32718557 conn(0x55b256e5dc20 msgr2=0x55b256e770b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66240a9b80 1 -- 172.21.15.53:0/32718557 shutdown_connections 2023-11-06T01:33:47.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.551+0000 7f66240a9b80 1 -- 172.21.15.53:0/32718557 wait complete. 2023-11-06T01:33:47.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f66240a9b80 20 client.15258 trim_cache size 0 max 16384 2023-11-06T01:33:47.579 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:33:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 3, 2023-11-06T01:33:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24422, 2023-11-06T01:33:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-06T01:33:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-06T01:33:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-06T01:33:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 76.559201740999995, 2023-11-06T01:33:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 27, 2023-11-06T01:33:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 84, 2023-11-06T01:33:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 82, 2023-11-06T01:33:47.582 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 101.912740193 2023-11-06T01:33:47.582 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:47.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 -- 172.21.15.53:0/2073788300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65fc0a0630 msgr2=0x7f65fc0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65fc0a0630 0x7f65fc0a2af0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6610100cd0 tx=0x7f660c005f90 comp rx=0 tx=0).stop 2023-11-06T01:33:47.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 -- 172.21.15.53:0/2073788300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6610101810 msgr2=0x7f6610102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6610101810 0x7f6610102b40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f660800ef30 tx=0x7f660800c3a0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 -- 172.21.15.53:0/2073788300 shutdown_connections 2023-11-06T01:33:47.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f65fc0a0630 0x7f65fc0a2af0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6610101810 0x7f6610102b40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66100ffc80 0x7f6610103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 --2- 172.21.15.53:0/2073788300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610100e70 0x7f6610102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 -- 172.21.15.53:0/2073788300 >> 172.21.15.53:0/2073788300 conn(0x7f66100fb1f0 msgr2=0x7f66100fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 -- 172.21.15.53:0/2073788300 shutdown_connections 2023-11-06T01:33:47.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.552+0000 7f6617f19700 1 -- 172.21.15.53:0/2073788300 wait complete. 2023-11-06T01:33:47.585 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:47 smithi053 ceph-mon[99472]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-11-06T01:33:47.586 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:47 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/632795193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:47.586 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:47 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/389213708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:47.586 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:47 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/4147884456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:47.587 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-06T01:33:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:47 smithi062 ceph-mon[103924]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-11-06T01:33:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:47 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/632795193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:47 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/389213708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:47 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/4147884456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:47.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.712+0000 7eff297e9700 1 -- 172.21.15.53:0/2306187350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101380 msgr2=0x7eff241017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.712+0000 7eff297e9700 1 --2- 172.21.15.53:0/2306187350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101380 0x7eff241017a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7eff18009990 tx=0x7eff18009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.712+0000 7eff297e9700 1 -- 172.21.15.53:0/2306187350 shutdown_connections 2023-11-06T01:33:47.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.712+0000 7eff297e9700 1 --2- 172.21.15.53:0/2306187350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff24100900 0x7eff241081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.712+0000 7eff297e9700 1 --2- 172.21.15.53:0/2306187350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101380 0x7eff241017a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.712+0000 7eff297e9700 1 --2- 172.21.15.53:0/2306187350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff240fff40 0x7eff241003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.712+0000 7eff297e9700 1 -- 172.21.15.53:0/2306187350 >> 172.21.15.53:0/2306187350 conn(0x7eff240fa9f0 msgr2=0x7eff240fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.712+0000 7eff297e9700 1 -- 172.21.15.53:0/2306187350 shutdown_connections 2023-11-06T01:33:47.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.712+0000 7eff297e9700 1 -- 172.21.15.53:0/2306187350 wait complete. 2023-11-06T01:33:47.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.713+0000 7eff297e9700 1 Processor -- start 2023-11-06T01:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.713+0000 7eff297e9700 1 -- start start 2023-11-06T01:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.713+0000 7eff297e9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff240fff40 0x7eff240ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.713+0000 7eff297e9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff24100900 0x7eff2419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.713+0000 7eff297e9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101380 0x7eff2419de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.713+0000 7eff297e9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2419e5f0 con 0x7eff24101380 2023-11-06T01:33:47.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.713+0000 7eff297e9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2419e760 con 0x7eff240fff40 2023-11-06T01:33:47.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.713+0000 7eff297e9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2419e920 con 0x7eff24100900 2023-11-06T01:33:47.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.713+0000 7eff237fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101380 0x7eff2419de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.713+0000 7eff237fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101380 0x7eff2419de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39318/0 (socket says 172.21.15.53:39318) 2023-11-06T01:33:47.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.713+0000 7eff237fe700 1 -- 172.21.15.53:0/2277160207 learned_addr learned my addr 172.21.15.53:0/2277160207 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff22ffd700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff240fff40 0x7eff240ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff227fc700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff24100900 0x7eff2419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff237fe700 1 -- 172.21.15.53:0/2277160207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff24100900 msgr2=0x7eff2419b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff237fe700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff24100900 0x7eff2419b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff237fe700 1 -- 172.21.15.53:0/2277160207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff240fff40 msgr2=0x7eff240ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff237fe700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff240fff40 0x7eff240ff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff237fe700 1 -- 172.21.15.53:0/2277160207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff180095f0 con 0x7eff24101380 2023-11-06T01:33:47.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff227fc700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff24100900 0x7eff2419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:47.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff237fe700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101380 0x7eff2419de20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00ada0 tx=0x7eff1c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff0bfff700 1 -- 172.21.15.53:0/2277160207 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c00f070 con 0x7eff24101380 2023-11-06T01:33:47.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff0bfff700 1 -- 172.21.15.53:0/2277160207 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff1c007b30 con 0x7eff24101380 2023-11-06T01:33:47.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff297e9700 1 -- 172.21.15.53:0/2277160207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2419ec00 con 0x7eff24101380 2023-11-06T01:33:47.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff0bfff700 1 -- 172.21.15.53:0/2277160207 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c005440 con 0x7eff24101380 2023-11-06T01:33:47.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.714+0000 7eff297e9700 1 -- 172.21.15.53:0/2277160207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2419f0c0 con 0x7eff24101380 2023-11-06T01:33:47.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.716+0000 7eff0bfff700 1 -- 172.21.15.53:0/2277160207 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7eff1c004ad0 con 0x7eff24101380 2023-11-06T01:33:47.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.717+0000 7eff0bfff700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff0c0a05b0 0x7eff0c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.717+0000 7eff22ffd700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff0c0a05b0 0x7eff0c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.717+0000 7eff0bfff700 1 -- 172.21.15.53:0/2277160207 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7eff1c0cb230 con 0x7eff24101380 2023-11-06T01:33:47.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.717+0000 7eff22ffd700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff0c0a05b0 0x7eff0c0a2a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff18006010 tx=0x7eff1800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.718+0000 7eff35979b80 1 Processor -- start 2023-11-06T01:33:47.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.718+0000 7eff35979b80 1 -- start start 2023-11-06T01:33:47.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.718+0000 7eff35979b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838be4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.718+0000 7eff35979b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838bea20 0x5642838bce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.718+0000 7eff35979b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 0x5642838bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.718+0000 7eff35979b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642838149c0 con 0x5642838be0a0 2023-11-06T01:33:47.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.718+0000 7eff35979b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564283812380 con 0x5642838bd3a0 2023-11-06T01:33:47.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff35979b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56428380dc70 con 0x5642838bea20 2023-11-06T01:33:47.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff22ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838be4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff22ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838be4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39322/0 (socket says 172.21.15.53:39322) 2023-11-06T01:33:47.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff22ffd700 1 -- 172.21.15.53:0/1477612120 learned_addr learned my addr 172.21.15.53:0/1477612120 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff22ffd700 1 -- 172.21.15.53:0/1477612120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838bea20 msgr2=0x5642838bce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff22ffd700 1 --2- 172.21.15.53:0/1477612120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838bea20 0x5642838bce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff22ffd700 1 -- 172.21.15.53:0/1477612120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 msgr2=0x5642838bd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff22ffd700 1 --2- 172.21.15.53:0/1477612120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 0x5642838bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff22ffd700 1 -- 172.21.15.53:0/1477612120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56428363bd90 con 0x5642838be0a0 2023-11-06T01:33:47.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff22ffd700 1 --2- 172.21.15.53:0/1477612120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838be4e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7eff1800ba30 tx=0x7eff1801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57a68ca19046fbb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff097fa700 1 -- 172.21.15.53:0/1477612120 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1802b070 con 0x5642838be0a0 2023-11-06T01:33:47.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff097fa700 1 -- 172.21.15.53:0/1477612120 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff1801e040 con 0x5642838be0a0 2023-11-06T01:33:47.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.719+0000 7eff097fa700 1 -- 172.21.15.53:0/1477612120 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18024a40 con 0x5642838be0a0 2023-11-06T01:33:47.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 -- 172.21.15.53:0/1477612120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 msgr2=0x5642838be4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 --2- 172.21.15.53:0/1477612120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838be4e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7eff1800ba30 tx=0x7eff1801f000 comp rx=0 tx=0).stop 2023-11-06T01:33:47.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 -- 172.21.15.53:0/1477612120 shutdown_connections 2023-11-06T01:33:47.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 --2- 172.21.15.53:0/1477612120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 0x5642838bd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 --2- 172.21.15.53:0/1477612120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838be4e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 --2- 172.21.15.53:0/1477612120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838bea20 0x5642838bce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 -- 172.21.15.53:0/1477612120 >> 172.21.15.53:0/1477612120 conn(0x56428381c080 msgr2=0x5642838b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 -- 172.21.15.53:0/1477612120 shutdown_connections 2023-11-06T01:33:47.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 -- 172.21.15.53:0/1477612120 wait complete. 2023-11-06T01:33:47.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 -- start start 2023-11-06T01:33:47.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 Processor -- start 2023-11-06T01:33:47.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838bd3a0 0x5642838c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.720+0000 7eff35979b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff35979b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bea20 0x5642838cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff35979b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642837aa4c0 con 0x5642838be0a0 2023-11-06T01:33:47.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff35979b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642837aa160 con 0x5642838bea20 2023-11-06T01:33:47.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff35979b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642836918d0 con 0x5642838bd3a0 2023-11-06T01:33:47.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff35979b80 10 client.0 _ll_register_callbacks cb 0x5642836f7a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:47.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff227fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff227fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39330/0 (socket says 172.21.15.53:39330) 2023-11-06T01:33:47.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff227fc700 1 -- 172.21.15.53:0/3747509579 learned_addr learned my addr 172.21.15.53:0/3747509579 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff22ffd700 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838bd3a0 0x5642838c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff227fc700 1 -- 172.21.15.53:0/3747509579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838bd3a0 msgr2=0x5642838c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff227fc700 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838bd3a0 0x5642838c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff227fc700 1 -- 172.21.15.53:0/3747509579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bea20 msgr2=0x5642838cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff237fe700 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bea20 0x5642838cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff227fc700 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bea20 0x5642838cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff227fc700 1 -- 172.21.15.53:0/3747509579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff24108b50 con 0x5642838be0a0 2023-11-06T01:33:47.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff227fc700 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838c6e00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7eff1400e680 tx=0x7eff1400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626f0e2048562276 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:47.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff06ff5700 1 -- 172.21.15.53:0/3747509579 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff14004220 con 0x5642838be0a0 2023-11-06T01:33:47.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff06ff5700 1 -- 172.21.15.53:0/3747509579 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7eff14004380 con 0x5642838be0a0 2023-11-06T01:33:47.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff35979b80 1 -- 172.21.15.53:0/3747509579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564283838cc0 con 0x5642838be0a0 2023-11-06T01:33:47.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff06ff5700 1 -- 172.21.15.53:0/3747509579 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff14004540 con 0x5642838be0a0 2023-11-06T01:33:47.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff35979b80 10 client.15270 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:47.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.721+0000 7eff35979b80 1 -- 172.21.15.53:0/3747509579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564283624c10 con 0x5642838be0a0 2023-11-06T01:33:47.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.722+0000 7eff06ff5700 1 -- 172.21.15.53:0/3747509579 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564283838cc0 con 0x5642838be0a0 2023-11-06T01:33:47.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.722+0000 7eff06ff5700 1 -- 172.21.15.53:0/3747509579 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7eff14003be0 con 0x5642838be0a0 2023-11-06T01:33:47.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.722+0000 7eff35979b80 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:47.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.722+0000 7eff35979b80 10 client.15270 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-06T01:33:47.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.722+0000 7eff35979b80 20 client.15270 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:47.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.722+0000 7eff35979b80 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x5642838d1c70 0x5642838d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.722+0000 7eff35979b80 4 client.15270 mds_command: new command op to 24464 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:47.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.722+0000 7eff35979b80 1 -- 172.21.15.53:0/3747509579 --> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56428388bdc0 con 0x5642838d1c70 2023-11-06T01:33:47.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.722+0000 7eff22ffd700 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x5642838d1c70 0x5642838d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.723+0000 7eff06ff5700 10 client.15270 ms_handle_connect on v2:172.21.15.62:6838/1997633670 2023-11-06T01:33:47.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.723+0000 7eff22ffd700 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x5642838d1c70 0x5642838d0ff0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c78f774519842622 server_cookie=3334543e143ccbbe in_seq=0 out_seq=0 2023-11-06T01:33:47.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.723+0000 7eff06ff5700 1 -- 172.21.15.53:0/3747509579 <== mds.2 v2:172.21.15.62:6838/1997633670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56428388bdc0 con 0x5642838d1c70 2023-11-06T01:33:47.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.723+0000 7eff06ff5700 10 client.15270 handle_command_reply: tid=0 2023-11-06T01:33:47.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.723+0000 7eff35979b80 1 client.15270 shutdown 2023-11-06T01:33:47.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff35979b80 1 -- 172.21.15.53:0/3747509579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 msgr2=0x5642838c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff35979b80 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838c6e00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7eff1400e680 tx=0x7eff1400e990 comp rx=0 tx=0).stop 2023-11-06T01:33:47.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff35979b80 1 -- 172.21.15.53:0/3747509579 shutdown_connections 2023-11-06T01:33:47.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff35979b80 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x5642838d1c70 0x5642838d0ff0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff35979b80 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bea20 0x5642838cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff06ff5700 0 client.15270 ms_handle_reset on v2:172.21.15.62:6838/1997633670 2023-11-06T01:33:47.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff35979b80 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838bd3a0 0x5642838c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff35979b80 1 --2- 172.21.15.53:0/3747509579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838c6e00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff35979b80 1 -- 172.21.15.53:0/3747509579 >> 172.21.15.53:0/3747509579 conn(0x56428381c080 msgr2=0x5642838b7060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff35979b80 1 -- 172.21.15.53:0/3747509579 shutdown_connections 2023-11-06T01:33:47.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff35979b80 1 -- 172.21.15.53:0/3747509579 wait complete. 2023-11-06T01:33:47.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.724+0000 7eff35979b80 20 client.15270 trim_cache size 0 max 16384 2023-11-06T01:33:47.746 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:47 smithi163 ceph-mon[104416]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-11-06T01:33:47.746 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:47 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/632795193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:47.747 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:47 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/389213708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:47.747 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:47 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/4147884456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:47.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.741+0000 7eff35979b80 1 Processor -- start 2023-11-06T01:33:47.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.741+0000 7eff35979b80 1 -- start start 2023-11-06T01:33:47.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.741+0000 7eff35979b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56428381c080 0x5642838bce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.741+0000 7eff35979b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 0x5642838da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.741+0000 7eff35979b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838be0a0 0x5642838d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.741+0000 7eff35979b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642838b8520 con 0x56428381c080 2023-11-06T01:33:47.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.741+0000 7eff35979b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642837d80b0 con 0x5642838bd3a0 2023-11-06T01:33:47.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.741+0000 7eff35979b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564283812cf0 con 0x5642838be0a0 2023-11-06T01:33:47.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.741+0000 7eff22ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56428381c080 0x5642838bce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.741+0000 7eff22ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56428381c080 0x5642838bce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39336/0 (socket says 172.21.15.53:39336) 2023-11-06T01:33:47.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff22ffd700 1 -- 172.21.15.53:0/962936783 learned_addr learned my addr 172.21.15.53:0/962936783 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff227fc700 1 --2- 172.21.15.53:0/962936783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 0x5642838da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff237fe700 1 --2- 172.21.15.53:0/962936783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838be0a0 0x5642838d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff22ffd700 1 -- 172.21.15.53:0/962936783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838be0a0 msgr2=0x5642838d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff22ffd700 1 --2- 172.21.15.53:0/962936783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838be0a0 0x5642838d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff22ffd700 1 -- 172.21.15.53:0/962936783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 msgr2=0x5642838da9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff22ffd700 1 --2- 172.21.15.53:0/962936783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 0x5642838da9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff22ffd700 1 -- 172.21.15.53:0/962936783 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564283662200 con 0x56428381c080 2023-11-06T01:33:47.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff22ffd700 1 --2- 172.21.15.53:0/962936783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56428381c080 0x5642838bce10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7eff18031040 tx=0x7eff18009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231847b911b90dc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff09ffb700 1 -- 172.21.15.53:0/962936783 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1802b040 con 0x56428381c080 2023-11-06T01:33:47.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff09ffb700 1 -- 172.21.15.53:0/962936783 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff1802c040 con 0x56428381c080 2023-11-06T01:33:47.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.742+0000 7eff09ffb700 1 -- 172.21.15.53:0/962936783 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18024670 con 0x56428381c080 2023-11-06T01:33:47.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 -- 172.21.15.53:0/962936783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56428381c080 msgr2=0x5642838bce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 --2- 172.21.15.53:0/962936783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56428381c080 0x5642838bce10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7eff18031040 tx=0x7eff18009550 comp rx=0 tx=0).stop 2023-11-06T01:33:47.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 -- 172.21.15.53:0/962936783 shutdown_connections 2023-11-06T01:33:47.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 --2- 172.21.15.53:0/962936783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5642838be0a0 0x5642838d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 --2- 172.21.15.53:0/962936783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56428381c080 0x5642838bce10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 --2- 172.21.15.53:0/962936783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 0x5642838da9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 -- 172.21.15.53:0/962936783 >> 172.21.15.53:0/962936783 conn(0x5642838c5960 msgr2=0x5642838c12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 -- 172.21.15.53:0/962936783 shutdown_connections 2023-11-06T01:33:47.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 -- 172.21.15.53:0/962936783 wait complete. 2023-11-06T01:33:47.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 -- start start 2023-11-06T01:33:47.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 Processor -- start 2023-11-06T01:33:47.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56428381c080 0x5642838d5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 0x5642838da880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642838d5f30 con 0x5642838be0a0 2023-11-06T01:33:47.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564283923760 con 0x5642838bd3a0 2023-11-06T01:33:47.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642839238a0 con 0x56428381c080 2023-11-06T01:33:47.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff35979b80 10 client.0 _ll_register_callbacks cb 0x7eff14008ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:47.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff237fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff237fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838d5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39346/0 (socket says 172.21.15.53:39346) 2023-11-06T01:33:47.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff237fe700 1 -- 172.21.15.53:0/1274938034 learned_addr learned my addr 172.21.15.53:0/1274938034 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.743+0000 7eff22ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56428381c080 0x5642838d5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff227fc700 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 0x5642838da880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff237fe700 1 -- 172.21.15.53:0/1274938034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56428381c080 msgr2=0x5642838d5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff237fe700 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56428381c080 0x5642838d5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff237fe700 1 -- 172.21.15.53:0/1274938034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 msgr2=0x5642838da880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff237fe700 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 0x5642838da880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff237fe700 1 -- 172.21.15.53:0/1274938034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56428363bd90 con 0x5642838be0a0 2023-11-06T01:33:47.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff237fe700 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838d5850 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0986a0 tx=0x7eff1c098380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402437a892f49899 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:47.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff06ff5700 1 -- 172.21.15.53:0/1274938034 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0989a0 con 0x5642838be0a0 2023-11-06T01:33:47.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff06ff5700 1 -- 172.21.15.53:0/1274938034 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7eff1c09c7d0 con 0x5642838be0a0 2023-11-06T01:33:47.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff35979b80 1 -- 172.21.15.53:0/1274938034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff1801e040 con 0x5642838be0a0 2023-11-06T01:33:47.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff06ff5700 1 -- 172.21.15.53:0/1274938034 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff1c0a0980 con 0x5642838be0a0 2023-11-06T01:33:47.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff35979b80 10 client.15276 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:47.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff35979b80 1 -- 172.21.15.53:0/1274938034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56428363a260 con 0x5642838be0a0 2023-11-06T01:33:47.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.744+0000 7eff06ff5700 1 -- 172.21.15.53:0/1274938034 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff1c07b080 con 0x5642838be0a0 2023-11-06T01:33:47.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.745+0000 7eff06ff5700 1 -- 172.21.15.53:0/1274938034 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7eff1c094790 con 0x5642838be0a0 2023-11-06T01:33:47.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.745+0000 7eff35979b80 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:47.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.745+0000 7eff35979b80 10 client.15276 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-06T01:33:47.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.745+0000 7eff35979b80 20 client.15276 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:47.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.745+0000 7eff35979b80 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x5642838bea20 0x564283925000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.745+0000 7eff35979b80 4 client.15276 mds_command: new command op to 24464 tid=0[{"prefix": "status"}] 2023-11-06T01:33:47.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.745+0000 7eff35979b80 1 -- 172.21.15.53:0/1274938034 --> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564283806cf0 con 0x5642838bea20 2023-11-06T01:33:47.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.745+0000 7eff22ffd700 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x5642838bea20 0x564283925000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.745+0000 7eff06ff5700 10 client.15276 ms_handle_connect on v2:172.21.15.62:6838/1997633670 2023-11-06T01:33:47.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.745+0000 7eff22ffd700 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x5642838bea20 0x564283925000 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6179e1b18be3c5d server_cookie=cfaf76855261f89d in_seq=0 out_seq=0 2023-11-06T01:33:47.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff06ff5700 1 -- 172.21.15.53:0/1274938034 <== mds.2 v2:172.21.15.62:6838/1997633670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x564283806cf0 con 0x5642838bea20 2023-11-06T01:33:47.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff06ff5700 10 client.15276 handle_command_reply: tid=0 2023-11-06T01:33:47.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 1 client.15276 shutdown 2023-11-06T01:33:47.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 1 -- 172.21.15.53:0/1274938034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 msgr2=0x5642838d5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838d5850 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0986a0 tx=0x7eff1c098380 comp rx=0 tx=0).stop 2023-11-06T01:33:47.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 1 -- 172.21.15.53:0/1274938034 shutdown_connections 2023-11-06T01:33:47.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x5642838bea20 0x564283925000 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5642838be0a0 0x5642838d5850 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56428381c080 0x5642838d5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff06ff5700 0 client.15276 ms_handle_reset on v2:172.21.15.62:6838/1997633670 2023-11-06T01:33:47.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 1 --2- 172.21.15.53:0/1274938034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642838bd3a0 0x5642838da880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 1 -- 172.21.15.53:0/1274938034 >> 172.21.15.53:0/1274938034 conn(0x5642838c5960 msgr2=0x5642838d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 1 -- 172.21.15.53:0/1274938034 shutdown_connections 2023-11-06T01:33:47.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 1 -- 172.21.15.53:0/1274938034 wait complete. 2023-11-06T01:33:47.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff35979b80 20 client.15276 trim_cache size 0 max 16384 2023-11-06T01:33:47.769 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:33:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 2, 2023-11-06T01:33:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24464, 2023-11-06T01:33:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-06T01:33:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-06T01:33:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-06T01:33:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 77.300099978000006, 2023-11-06T01:33:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 27, 2023-11-06T01:33:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 84, 2023-11-06T01:33:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 82, 2023-11-06T01:33:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 86.476643148999997 2023-11-06T01:33:47.771 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:47.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff297e9700 1 -- 172.21.15.53:0/2277160207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff0c0a05b0 msgr2=0x7eff0c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff297e9700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff0c0a05b0 0x7eff0c0a2a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff18006010 tx=0x7eff1800b540 comp rx=0 tx=0).stop 2023-11-06T01:33:47.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff297e9700 1 -- 172.21.15.53:0/2277160207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101380 msgr2=0x7eff2419de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff297e9700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101380 0x7eff2419de20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00ada0 tx=0x7eff1c00c330 comp rx=0 tx=0).stop 2023-11-06T01:33:47.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff297e9700 1 -- 172.21.15.53:0/2277160207 shutdown_connections 2023-11-06T01:33:47.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff297e9700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7eff0c0a05b0 0x7eff0c0a2a70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff297e9700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff240fff40 0x7eff240ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff297e9700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101380 0x7eff2419de20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff297e9700 1 --2- 172.21.15.53:0/2277160207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff24100900 0x7eff2419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.746+0000 7eff297e9700 1 -- 172.21.15.53:0/2277160207 >> 172.21.15.53:0/2277160207 conn(0x7eff240fa9f0 msgr2=0x7eff240fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.747+0000 7eff297e9700 1 -- 172.21.15.53:0/2277160207 shutdown_connections 2023-11-06T01:33:47.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.747+0000 7eff297e9700 1 -- 172.21.15.53:0/2277160207 wait complete. 2023-11-06T01:33:47.775 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T01:33:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.901+0000 7f6707762700 1 -- 172.21.15.53:0/2074006753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6700101250 msgr2=0x7f6700101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.901+0000 7f6707762700 1 --2- 172.21.15.53:0/2074006753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6700101250 0x7f6700101670 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009990 tx=0x7f66fc009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.901+0000 7f6707762700 1 -- 172.21.15.53:0/2074006753 shutdown_connections 2023-11-06T01:33:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.901+0000 7f6707762700 1 --2- 172.21.15.53:0/2074006753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6700100890 0x7f6700108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.901+0000 7f6707762700 1 --2- 172.21.15.53:0/2074006753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6700101250 0x7f6700101670 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.901+0000 7f6707762700 1 --2- 172.21.15.53:0/2074006753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67000ffef0 0x7f6700100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.901+0000 7f6707762700 1 -- 172.21.15.53:0/2074006753 >> 172.21.15.53:0/2074006753 conn(0x7f67000fa960 msgr2=0x7f67000fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.901+0000 7f6707762700 1 -- 172.21.15.53:0/2074006753 shutdown_connections 2023-11-06T01:33:47.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.901+0000 7f6707762700 1 -- 172.21.15.53:0/2074006753 wait complete. 2023-11-06T01:33:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6707762700 1 Processor -- start 2023-11-06T01:33:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6707762700 1 -- start start 2023-11-06T01:33:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6707762700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67000ffef0 0x7f67000ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6707762700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6700100890 0x7f670019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6707762700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6700101250 0x7f670019dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6707762700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670019e640 con 0x7f6700100890 2023-11-06T01:33:47.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6707762700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670019e7b0 con 0x7f6700101250 2023-11-06T01:33:47.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6707762700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670019e970 con 0x7f67000ffef0 2023-11-06T01:33:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6704cfd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6700100890 0x7f670019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6704cfd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6700100890 0x7f670019b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39358/0 (socket says 172.21.15.53:39358) 2023-11-06T01:33:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6704cfd700 1 -- 172.21.15.53:0/3271865612 learned_addr learned my addr 172.21.15.53:0/3271865612 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f67054fe700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67000ffef0 0x7f67000ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.902+0000 7f6705cff700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6700101250 0x7f670019dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.903+0000 7f6704cfd700 1 -- 172.21.15.53:0/3271865612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67000ffef0 msgr2=0x7f67000ff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.903+0000 7f6704cfd700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67000ffef0 0x7f67000ff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.903+0000 7f6704cfd700 1 -- 172.21.15.53:0/3271865612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6700101250 msgr2=0x7f670019dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.903+0000 7f6704cfd700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6700101250 0x7f670019dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.903+0000 7f6704cfd700 1 -- 172.21.15.53:0/3271865612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc0095f0 con 0x7f6700100890 2023-11-06T01:33:47.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.903+0000 7f6704cfd700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6700100890 0x7f670019b810 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f66f400c970 tx=0x7f66f400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.903+0000 7f66f27fc700 1 -- 172.21.15.53:0/3271865612 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f40078d0 con 0x7f6700100890 2023-11-06T01:33:47.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.903+0000 7f6707762700 1 -- 172.21.15.53:0/3271865612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670019ec50 con 0x7f6700100890 2023-11-06T01:33:47.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.903+0000 7f66f27fc700 1 -- 172.21.15.53:0/3271865612 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66f4018480 con 0x7f6700100890 2023-11-06T01:33:47.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.903+0000 7f66f27fc700 1 -- 172.21.15.53:0/3271865612 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f400f690 con 0x7f6700100890 2023-11-06T01:33:47.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.903+0000 7f6707762700 1 -- 172.21.15.53:0/3271865612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670019f110 con 0x7f6700100890 2023-11-06T01:33:47.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.905+0000 7f66f27fc700 1 -- 172.21.15.53:0/3271865612 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f66f40185f0 con 0x7f6700100890 2023-11-06T01:33:47.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.906+0000 7f66f27fc700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f66ec0a05b0 0x7f66ec0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.906+0000 7f67054fe700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f66ec0a05b0 0x7f66ec0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.906+0000 7f66f27fc700 1 -- 172.21.15.53:0/3271865612 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f66f4013070 con 0x7f6700100890 2023-11-06T01:33:47.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.906+0000 7f67054fe700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f66ec0a05b0 0x7f66ec0a2a70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00b5c0 tx=0x7f66fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f67138f2b80 1 Processor -- start 2023-11-06T01:33:47.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f67138f2b80 1 -- start start 2023-11-06T01:33:47.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e634e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2e61e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e623a0 0x5603d2e62820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f67138f2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2db99c0 con 0x5603d2e63a20 2023-11-06T01:33:47.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f67138f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2db7380 con 0x5603d2e630a0 2023-11-06T01:33:47.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f67138f2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2db2c70 con 0x5603d2e623a0 2023-11-06T01:33:47.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f6704cfd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2e61e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f6704cfd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2e61e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39366/0 (socket says 172.21.15.53:39366) 2023-11-06T01:33:47.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f6704cfd700 1 -- 172.21.15.53:0/1859278068 learned_addr learned my addr 172.21.15.53:0/1859278068 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f6705cff700 1 --2- 172.21.15.53:0/1859278068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e623a0 0x5603d2e62820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f67054fe700 1 --2- 172.21.15.53:0/1859278068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f6704cfd700 1 -- 172.21.15.53:0/1859278068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e623a0 msgr2=0x5603d2e62820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f6704cfd700 1 --2- 172.21.15.53:0/1859278068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e623a0 0x5603d2e62820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f6704cfd700 1 -- 172.21.15.53:0/1859278068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 msgr2=0x5603d2e634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f6704cfd700 1 --2- 172.21.15.53:0/1859278068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e634e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f6704cfd700 1 -- 172.21.15.53:0/1859278068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603d2be0d90 con 0x5603d2e63a20 2023-11-06T01:33:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.907+0000 7f6704cfd700 1 --2- 172.21.15.53:0/1859278068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2e61e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f66f4095e00 tx=0x7f66f4098d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52743e715191cb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f66eb7fe700 1 -- 172.21.15.53:0/1859278068 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4084070 con 0x5603d2e63a20 2023-11-06T01:33:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f66eb7fe700 1 -- 172.21.15.53:0/1859278068 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66f4099a90 con 0x5603d2e63a20 2023-11-06T01:33:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f66eb7fe700 1 -- 172.21.15.53:0/1859278068 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f40188d0 con 0x5603d2e63a20 2023-11-06T01:33:47.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f67138f2b80 1 -- 172.21.15.53:0/1859278068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 msgr2=0x5603d2e61e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f67138f2b80 1 --2- 172.21.15.53:0/1859278068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2e61e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f66f4095e00 tx=0x7f66f4098d70 comp rx=0 tx=0).stop 2023-11-06T01:33:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f67138f2b80 1 -- 172.21.15.53:0/1859278068 shutdown_connections 2023-11-06T01:33:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f67138f2b80 1 --2- 172.21.15.53:0/1859278068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f67138f2b80 1 --2- 172.21.15.53:0/1859278068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e623a0 0x5603d2e62820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f67138f2b80 1 --2- 172.21.15.53:0/1859278068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2e61e10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f67138f2b80 1 -- 172.21.15.53:0/1859278068 >> 172.21.15.53:0/1859278068 conn(0x5603d2dc1080 msgr2=0x5603d2e5c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f67138f2b80 1 -- 172.21.15.53:0/1859278068 shutdown_connections 2023-11-06T01:33:47.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.908+0000 7f67138f2b80 1 -- 172.21.15.53:0/1859278068 wait complete. 2023-11-06T01:33:47.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.909+0000 7f67138f2b80 1 -- start start 2023-11-06T01:33:47.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.909+0000 7f67138f2b80 1 Processor -- start 2023-11-06T01:33:47.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.909+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e6b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.909+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e6bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.909+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e63a20 0x5603d2e72ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.909+0000 7f67138f2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2d4f4c0 con 0x5603d2e623a0 2023-11-06T01:33:47.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.909+0000 7f67138f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2d4f160 con 0x5603d2e630a0 2023-11-06T01:33:47.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.909+0000 7f67138f2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2c368d0 con 0x5603d2e63a20 2023-11-06T01:33:47.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67054fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e6b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67054fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e6b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39370/0 (socket says 172.21.15.53:39370) 2023-11-06T01:33:47.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67054fe700 1 -- 172.21.15.53:0/2287597675 learned_addr learned my addr 172.21.15.53:0/2287597675 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67138f2b80 10 client.0 _ll_register_callbacks cb 0x5603d2c9ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:47.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f6704cfd700 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e6bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f6705cff700 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e63a20 0x5603d2e72ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67054fe700 1 -- 172.21.15.53:0/2287597675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e63a20 msgr2=0x5603d2e72ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67054fe700 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e63a20 0x5603d2e72ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67054fe700 1 -- 172.21.15.53:0/2287597675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 msgr2=0x5603d2e6bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67054fe700 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e6bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67054fe700 1 -- 172.21.15.53:0/2287597675 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670019f110 con 0x5603d2e623a0 2023-11-06T01:33:47.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67054fe700 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e6b960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f66fc015000 tx=0x7f66fc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c559dd0288158525 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f66e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:47.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f66e8ff9700 1 -- 172.21.15.53:0/2287597675 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc00f660 con 0x5603d2e623a0 2023-11-06T01:33:47.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f66e8ff9700 1 -- 172.21.15.53:0/2287597675 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f66fc00f7c0 con 0x5603d2e623a0 2023-11-06T01:33:47.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67138f2b80 1 -- 172.21.15.53:0/2287597675 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603d2dddcc0 con 0x5603d2e623a0 2023-11-06T01:33:47.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f66e8ff9700 1 -- 172.21.15.53:0/2287597675 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66fc00f980 con 0x5603d2e623a0 2023-11-06T01:33:47.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67138f2b80 10 client.15288 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:47.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.910+0000 7f67138f2b80 1 -- 172.21.15.53:0/2287597675 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5603d2bc9c10 con 0x5603d2e623a0 2023-11-06T01:33:47.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.911+0000 7f66e8ff9700 1 -- 172.21.15.53:0/2287597675 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66fc03e080 con 0x5603d2e623a0 2023-11-06T01:33:47.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.911+0000 7f66e8ff9700 1 -- 172.21.15.53:0/2287597675 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f66fc038030 con 0x5603d2e623a0 2023-11-06T01:33:47.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.911+0000 7f67138f2b80 10 client.15288 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:47.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.911+0000 7f67138f2b80 10 client.15288 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T01:33:47.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.911+0000 7f67138f2b80 20 client.15288 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:47.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.911+0000 7f67138f2b80 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5603d2e76c10 0x5603d2e75fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.911+0000 7f67138f2b80 4 client.15288 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:47.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.911+0000 7f67138f2b80 1 -- 172.21.15.53:0/2287597675 --> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603d2e30dc0 con 0x5603d2e76c10 2023-11-06T01:33:47.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.911+0000 7f6704cfd700 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5603d2e76c10 0x5603d2e75fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.911+0000 7f66e8ff9700 10 client.15288 ms_handle_connect on v2:172.21.15.163:6836/1759271265 2023-11-06T01:33:47.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.911+0000 7f6704cfd700 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5603d2e76c10 0x5603d2e75fe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84ed9d7284436e7e server_cookie=37be84801ae0897a in_seq=0 out_seq=0 2023-11-06T01:33:47.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f66e8ff9700 1 -- 172.21.15.53:0/2287597675 <== mds.1 v2:172.21.15.163:6836/1759271265 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5603d2e30dc0 con 0x5603d2e76c10 2023-11-06T01:33:47.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f66e8ff9700 10 client.15288 handle_command_reply: tid=0 2023-11-06T01:33:47.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f67138f2b80 1 client.15288 shutdown 2023-11-06T01:33:47.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f67138f2b80 1 -- 172.21.15.53:0/2287597675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 msgr2=0x5603d2e6b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f67138f2b80 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e6b960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f66fc015000 tx=0x7f66fc009310 comp rx=0 tx=0).stop 2023-11-06T01:33:47.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f67138f2b80 1 -- 172.21.15.53:0/2287597675 shutdown_connections 2023-11-06T01:33:47.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f67138f2b80 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5603d2e76c10 0x5603d2e75fe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f67138f2b80 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e63a20 0x5603d2e72ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f67138f2b80 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e6b960 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f67138f2b80 1 --2- 172.21.15.53:0/2287597675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e6bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f66e8ff9700 0 client.15288 ms_handle_reset on v2:172.21.15.163:6836/1759271265 2023-11-06T01:33:47.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f67138f2b80 1 -- 172.21.15.53:0/2287597675 >> 172.21.15.53:0/2287597675 conn(0x5603d2dc1080 msgr2=0x5603d2e5c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f67138f2b80 1 -- 172.21.15.53:0/2287597675 shutdown_connections 2023-11-06T01:33:47.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.912+0000 7f67138f2b80 1 -- 172.21.15.53:0/2287597675 wait complete. 2023-11-06T01:33:47.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.913+0000 7f67138f2b80 20 client.15288 trim_cache size 0 max 16384 2023-11-06T01:33:47.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67138f2b80 1 Processor -- start 2023-11-06T01:33:47.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67138f2b80 1 -- start start 2023-11-06T01:33:47.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e61e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e7f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e63a20 0x5603d2e66320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67138f2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2e6a450 con 0x5603d2e623a0 2023-11-06T01:33:47.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67138f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2e77f30 con 0x5603d2e630a0 2023-11-06T01:33:47.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67138f2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2e6cb50 con 0x5603d2e63a20 2023-11-06T01:33:47.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67054fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e61e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67054fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e61e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39378/0 (socket says 172.21.15.53:39378) 2023-11-06T01:33:47.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67054fe700 1 -- 172.21.15.53:0/1522103293 learned_addr learned my addr 172.21.15.53:0/1522103293 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f6704cfd700 1 --2- 172.21.15.53:0/1522103293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e7f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f6705cff700 1 --2- 172.21.15.53:0/1522103293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e63a20 0x5603d2e66320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67054fe700 1 -- 172.21.15.53:0/1522103293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e63a20 msgr2=0x5603d2e66320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.930+0000 7f67054fe700 1 --2- 172.21.15.53:0/1522103293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e63a20 0x5603d2e66320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67054fe700 1 -- 172.21.15.53:0/1522103293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 msgr2=0x5603d2e7f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67054fe700 1 --2- 172.21.15.53:0/1522103293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e7f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67054fe700 1 -- 172.21.15.53:0/1522103293 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603d2c07200 con 0x5603d2e623a0 2023-11-06T01:33:47.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f6704cfd700 1 --2- 172.21.15.53:0/1522103293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e7f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:47.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67054fe700 1 --2- 172.21.15.53:0/1522103293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e61e10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0098e0 tx=0x7f66fc02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9d939b1417c86a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f66ebfff700 1 -- 172.21.15.53:0/1522103293 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc02f6c0 con 0x5603d2e623a0 2023-11-06T01:33:47.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f66ebfff700 1 -- 172.21.15.53:0/1522103293 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5603d2dddcc0 con 0x5603d2e623a0 2023-11-06T01:33:47.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f66ebfff700 1 -- 172.21.15.53:0/1522103293 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc023460 con 0x5603d2e623a0 2023-11-06T01:33:47.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67138f2b80 1 -- 172.21.15.53:0/1522103293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 msgr2=0x5603d2e61e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67138f2b80 1 --2- 172.21.15.53:0/1522103293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e61e10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0098e0 tx=0x7f66fc02f5b0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67138f2b80 1 -- 172.21.15.53:0/1522103293 shutdown_connections 2023-11-06T01:33:47.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67138f2b80 1 --2- 172.21.15.53:0/1522103293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e63a20 0x5603d2e66320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67138f2b80 1 --2- 172.21.15.53:0/1522103293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e623a0 0x5603d2e61e10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67138f2b80 1 --2- 172.21.15.53:0/1522103293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e7f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67138f2b80 1 -- 172.21.15.53:0/1522103293 >> 172.21.15.53:0/1522103293 conn(0x5603d2dc1080 msgr2=0x5603d2e75fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67138f2b80 1 -- 172.21.15.53:0/1522103293 shutdown_connections 2023-11-06T01:33:47.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.931+0000 7f67138f2b80 1 -- 172.21.15.53:0/1522103293 wait complete. 2023-11-06T01:33:47.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f67138f2b80 1 -- start start 2023-11-06T01:33:47.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f67138f2b80 1 Processor -- start 2023-11-06T01:33:47.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e623a0 0x5603d2e7a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e7af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f67138f2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2ec8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f67138f2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2ec8c60 con 0x5603d2e63a20 2023-11-06T01:33:47.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f67138f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2ec8da0 con 0x5603d2e630a0 2023-11-06T01:33:47.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f67138f2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d2ec8f30 con 0x5603d2e623a0 2023-11-06T01:33:47.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f67138f2b80 10 client.0 _ll_register_callbacks cb 0x7f66fc003ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:47.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f6705cff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2ec8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f6705cff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2ec8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39388/0 (socket says 172.21.15.53:39388) 2023-11-06T01:33:47.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f6705cff700 1 -- 172.21.15.53:0/3934509620 learned_addr learned my addr 172.21.15.53:0/3934509620 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:47.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f6704cfd700 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e7af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.932+0000 7f67054fe700 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e623a0 0x5603d2e7a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f6705cff700 1 -- 172.21.15.53:0/3934509620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e623a0 msgr2=0x5603d2e7a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f6705cff700 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e623a0 0x5603d2e7a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f6705cff700 1 -- 172.21.15.53:0/3934509620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 msgr2=0x5603d2e7af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:47.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f6705cff700 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e7af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f6705cff700 1 -- 172.21.15.53:0/3934509620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603d2bc9c10 con 0x5603d2e63a20 2023-11-06T01:33:47.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f6705cff700 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2ec8720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f66f8008ec0 tx=0x7f66f8008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183453cc46023918 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:47.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f66e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:47.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f66e8ff9700 1 -- 172.21.15.53:0/3934509620 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800f040 con 0x5603d2e63a20 2023-11-06T01:33:47.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f66e8ff9700 1 -- 172.21.15.53:0/3934509620 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f66f8016c70 con 0x5603d2e63a20 2023-11-06T01:33:47.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f67138f2b80 1 -- 172.21.15.53:0/3934509620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66f4099a90 con 0x5603d2e63a20 2023-11-06T01:33:47.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f66e8ff9700 1 -- 172.21.15.53:0/3934509620 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66f8016e30 con 0x5603d2e63a20 2023-11-06T01:33:47.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f67138f2b80 10 client.15294 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:47.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f67138f2b80 1 -- 172.21.15.53:0/3934509620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5603d2bdf260 con 0x5603d2e63a20 2023-11-06T01:33:47.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f66e8ff9700 1 -- 172.21.15.53:0/3934509620 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66f4099a90 con 0x5603d2e63a20 2023-11-06T01:33:47.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.933+0000 7f66e8ff9700 1 -- 172.21.15.53:0/3934509620 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f66f80185b0 con 0x5603d2e63a20 2023-11-06T01:33:47.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f67138f2b80 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:47.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f67138f2b80 10 client.15294 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T01:33:47.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f67138f2b80 20 client.15294 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:47.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f67138f2b80 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5603d2ecdd10 0x5603d2eca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:47.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f67138f2b80 4 client.15294 mds_command: new command op to 24472 tid=0[{"prefix": "status"}] 2023-11-06T01:33:47.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f67138f2b80 1 -- 172.21.15.53:0/3934509620 --> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5603d2dabcf0 con 0x5603d2ecdd10 2023-11-06T01:33:47.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f67054fe700 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5603d2ecdd10 0x5603d2eca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:47.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f66e8ff9700 10 client.15294 ms_handle_connect on v2:172.21.15.163:6836/1759271265 2023-11-06T01:33:47.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f67054fe700 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5603d2ecdd10 0x5603d2eca6e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d6730dc0dd5b786 server_cookie=7ddb5dbca26298fa in_seq=0 out_seq=0 2023-11-06T01:33:47.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f66e8ff9700 1 -- 172.21.15.53:0/3934509620 <== mds.1 v2:172.21.15.163:6836/1759271265 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5603d2dabcf0 con 0x5603d2ecdd10 2023-11-06T01:33:47.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f66e8ff9700 10 client.15294 handle_command_reply: tid=0 2023-11-06T01:33:47.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.934+0000 7f67138f2b80 1 client.15294 shutdown 2023-11-06T01:33:47.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f67138f2b80 1 -- 172.21.15.53:0/3934509620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 msgr2=0x5603d2ec8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f67138f2b80 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2ec8720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f66f8008ec0 tx=0x7f66f8008ef0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f67138f2b80 1 -- 172.21.15.53:0/3934509620 shutdown_connections 2023-11-06T01:33:47.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f67138f2b80 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x5603d2ecdd10 0x5603d2eca6e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f67138f2b80 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603d2e63a20 0x5603d2ec8720 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f67138f2b80 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603d2e623a0 0x5603d2e7a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f67138f2b80 1 --2- 172.21.15.53:0/3934509620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603d2e630a0 0x5603d2e7af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f66e8ff9700 0 client.15294 ms_handle_reset on v2:172.21.15.163:6836/1759271265 2023-11-06T01:33:47.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f67138f2b80 1 -- 172.21.15.53:0/3934509620 >> 172.21.15.53:0/3934509620 conn(0x5603d2dc1080 msgr2=0x5603d2e6a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f67138f2b80 1 -- 172.21.15.53:0/3934509620 shutdown_connections 2023-11-06T01:33:47.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f67138f2b80 1 -- 172.21.15.53:0/3934509620 wait complete. 2023-11-06T01:33:47.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f67138f2b80 20 client.15294 trim_cache size 0 max 16384 2023-11-06T01:33:47.959 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:47.959 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:33:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 1, 2023-11-06T01:33:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24472, 2023-11-06T01:33:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-06T01:33:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-06T01:33:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-06T01:33:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 78.498686266000007, 2023-11-06T01:33:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 27, 2023-11-06T01:33:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 84, 2023-11-06T01:33:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 82, 2023-11-06T01:33:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 98.047199824000003 2023-11-06T01:33:47.961 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:47.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f6707762700 1 -- 172.21.15.53:0/3271865612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f66ec0a05b0 msgr2=0x7f66ec0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f6707762700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f66ec0a05b0 0x7f66ec0a2a70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00b5c0 tx=0x7f66fc005fb0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f6707762700 1 -- 172.21.15.53:0/3271865612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6700100890 msgr2=0x7f670019b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:47.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f6707762700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6700100890 0x7f670019b810 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f66f400c970 tx=0x7f66f400cc80 comp rx=0 tx=0).stop 2023-11-06T01:33:47.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f6707762700 1 -- 172.21.15.53:0/3271865612 shutdown_connections 2023-11-06T01:33:47.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f6707762700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f66ec0a05b0 0x7f66ec0a2a70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f6707762700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6700101250 0x7f670019dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f6707762700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67000ffef0 0x7f67000ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f6707762700 1 --2- 172.21.15.53:0/3271865612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6700100890 0x7f670019b810 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:47.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.935+0000 7f6707762700 1 -- 172.21.15.53:0/3271865612 >> 172.21.15.53:0/3271865612 conn(0x7f67000fa960 msgr2=0x7f67000fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:47.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.936+0000 7f6707762700 1 -- 172.21.15.53:0/3271865612 shutdown_connections 2023-11-06T01:33:47.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:47.936+0000 7f6707762700 1 -- 172.21.15.53:0/3271865612 wait complete. 2023-11-06T01:33:47.966 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-11-06T01:33:48.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.091+0000 7f35f294a700 1 -- 172.21.15.53:0/2119210474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ec104930 msgr2=0x7f35ec104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:48.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.091+0000 7f35f294a700 1 --2- 172.21.15.53:0/2119210474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ec104930 0x7f35ec104db0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f35e400b0a0 tx=0x7f35e400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.091+0000 7f35f294a700 1 -- 172.21.15.53:0/2119210474 shutdown_connections 2023-11-06T01:33:48.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.091+0000 7f35f294a700 1 --2- 172.21.15.53:0/2119210474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec0ff5f0 0x7f35ec101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.091+0000 7f35f294a700 1 --2- 172.21.15.53:0/2119210474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ec104930 0x7f35ec104db0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.091+0000 7f35f294a700 1 --2- 172.21.15.53:0/2119210474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec101f60 0x7f35ec1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.091+0000 7f35f294a700 1 -- 172.21.15.53:0/2119210474 >> 172.21.15.53:0/2119210474 conn(0x7f35ec0fb1f0 msgr2=0x7f35ec0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:48.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.091+0000 7f35f294a700 1 -- 172.21.15.53:0/2119210474 shutdown_connections 2023-11-06T01:33:48.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.091+0000 7f35f294a700 1 -- 172.21.15.53:0/2119210474 wait complete. 2023-11-06T01:33:48.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35f294a700 1 Processor -- start 2023-11-06T01:33:48.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35f294a700 1 -- start start 2023-11-06T01:33:48.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35f294a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec0ff5f0 0x7f35ec19d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35f294a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ec101f60 0x7f35ec19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35f294a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec104930 0x7f35ec1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35f294a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec1a4d60 con 0x7f35ec101f60 2023-11-06T01:33:48.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35f294a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec1a4ea0 con 0x7f35ec0ff5f0 2023-11-06T01:33:48.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35f294a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec1a5030 con 0x7f35ec104930 2023-11-06T01:33:48.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ec101f60 0x7f35ec19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ec101f60 0x7f35ec19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39400/0 (socket says 172.21.15.53:39400) 2023-11-06T01:33:48.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35eb7fe700 1 -- 172.21.15.53:0/1014100098 learned_addr learned my addr 172.21.15.53:0/1014100098 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:48.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.092+0000 7f35ebfff700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec0ff5f0 0x7f35ec19d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35f0ee7700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec104930 0x7f35ec1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35eb7fe700 1 -- 172.21.15.53:0/1014100098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec104930 msgr2=0x7f35ec1a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec104930 0x7f35ec1a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35eb7fe700 1 -- 172.21.15.53:0/1014100098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec0ff5f0 msgr2=0x7f35ec19d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec0ff5f0 0x7f35ec19d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35eb7fe700 1 -- 172.21.15.53:0/1014100098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4009d00 con 0x7f35ec101f60 2023-11-06T01:33:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35f0ee7700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec104930 0x7f35ec1a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ec101f60 0x7f35ec19d720 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e9a0 tx=0x7f35dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35e97fa700 1 -- 172.21.15.53:0/1014100098 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc009790 con 0x7f35ec101f60 2023-11-06T01:33:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35e97fa700 1 -- 172.21.15.53:0/1014100098 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35dc00cbd0 con 0x7f35ec101f60 2023-11-06T01:33:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35f294a700 1 -- 172.21.15.53:0/1014100098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ec1a5310 con 0x7f35ec101f60 2023-11-06T01:33:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35e97fa700 1 -- 172.21.15.53:0/1014100098 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc018680 con 0x7f35ec101f60 2023-11-06T01:33:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.093+0000 7f35f294a700 1 -- 172.21.15.53:0/1014100098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec1a5820 con 0x7f35ec101f60 2023-11-06T01:33:48.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.096+0000 7f35e97fa700 1 -- 172.21.15.53:0/1014100098 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f35dc00cd40 con 0x7f35ec101f60 2023-11-06T01:33:48.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.096+0000 7f35e97fa700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f35d80a05b0 0x7f35d80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35ebfff700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f35d80a05b0 0x7f35d80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35e97fa700 1 -- 172.21.15.53:0/1014100098 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f35dc014070 con 0x7f35ec101f60 2023-11-06T01:33:48.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35ebfff700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f35d80a05b0 0x7f35d80a2a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009c60 tx=0x7f35d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:48.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35feadab80 1 Processor -- start 2023-11-06T01:33:48.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35feadab80 1 -- start start 2023-11-06T01:33:48.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35feadab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4e0a0 0x558331e4e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35feadab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4ea20 0x558331e4ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35feadab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e4d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35feadab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331da49c0 con 0x558331e4d3a0 2023-11-06T01:33:48.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35feadab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331da2380 con 0x558331e4ea20 2023-11-06T01:33:48.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35feadab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331d9dc70 con 0x558331e4e0a0 2023-11-06T01:33:48.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35f0ee7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e4d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35f0ee7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e4d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39410/0 (socket says 172.21.15.53:39410) 2023-11-06T01:33:48.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35f0ee7700 1 -- 172.21.15.53:0/3898216192 learned_addr learned my addr 172.21.15.53:0/3898216192 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:48.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/3898216192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4ea20 0x558331e4ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.097+0000 7f35ebfff700 1 --2- 172.21.15.53:0/3898216192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4e0a0 0x558331e4e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.098+0000 7f35f0ee7700 1 -- 172.21.15.53:0/3898216192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4e0a0 msgr2=0x558331e4e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:48.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.098+0000 7f35f0ee7700 1 --2- 172.21.15.53:0/3898216192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4e0a0 0x558331e4e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.098+0000 7f35f0ee7700 1 -- 172.21.15.53:0/3898216192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4ea20 msgr2=0x558331e4ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:48.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.098+0000 7f35f0ee7700 1 --2- 172.21.15.53:0/3898216192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4ea20 0x558331e4ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.098+0000 7f35f0ee7700 1 -- 172.21.15.53:0/3898216192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558331bcbd90 con 0x558331e4d3a0 2023-11-06T01:33:48.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.098+0000 7f35f0ee7700 1 --2- 172.21.15.53:0/3898216192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e4d820 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009c40 tx=0x7f35e4009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acaac10b4a32f85 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:48.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.098+0000 7f35e27fc700 1 -- 172.21.15.53:0/3898216192 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400e040 con 0x558331e4d3a0 2023-11-06T01:33:48.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.098+0000 7f35e27fc700 1 -- 172.21.15.53:0/3898216192 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e4009490 con 0x558331e4d3a0 2023-11-06T01:33:48.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.098+0000 7f35e27fc700 1 -- 172.21.15.53:0/3898216192 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e401d6d0 con 0x558331e4d3a0 2023-11-06T01:33:48.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.099+0000 7f35feadab80 1 -- 172.21.15.53:0/3898216192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 msgr2=0x558331e4d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:48.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.099+0000 7f35feadab80 1 --2- 172.21.15.53:0/3898216192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e4d820 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009c40 tx=0x7f35e4009c70 comp rx=0 tx=0).stop 2023-11-06T01:33:48.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.099+0000 7f35feadab80 1 -- 172.21.15.53:0/3898216192 shutdown_connections 2023-11-06T01:33:48.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.099+0000 7f35feadab80 1 --2- 172.21.15.53:0/3898216192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e4d820 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.099+0000 7f35feadab80 1 --2- 172.21.15.53:0/3898216192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4e0a0 0x558331e4e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.099+0000 7f35feadab80 1 --2- 172.21.15.53:0/3898216192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4ea20 0x558331e4ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.099+0000 7f35feadab80 1 -- 172.21.15.53:0/3898216192 >> 172.21.15.53:0/3898216192 conn(0x558331dac080 msgr2=0x558331e478f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:48.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.099+0000 7f35feadab80 1 -- 172.21.15.53:0/3898216192 shutdown_connections 2023-11-06T01:33:48.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.099+0000 7f35feadab80 1 -- 172.21.15.53:0/3898216192 wait complete. 2023-11-06T01:33:48.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.099+0000 7f35feadab80 1 -- start start 2023-11-06T01:33:48.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.099+0000 7f35feadab80 1 Processor -- start 2023-11-06T01:33:48.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35feadab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35feadab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4e0a0 0x558331e56de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35feadab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4ea20 0x558331e5de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35feadab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331d3a4c0 con 0x558331e4d3a0 2023-11-06T01:33:48.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35feadab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331d3a160 con 0x558331e4e0a0 2023-11-06T01:33:48.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35feadab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331c218d0 con 0x558331e4ea20 2023-11-06T01:33:48.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35feadab80 10 client.0 _ll_register_callbacks cb 0x558331c87a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:48.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35ebfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35ebfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39412/0 (socket says 172.21.15.53:39412) 2023-11-06T01:33:48.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35ebfff700 1 -- 172.21.15.53:0/3689066764 learned_addr learned my addr 172.21.15.53:0/3689066764 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:48.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35f0ee7700 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4ea20 0x558331e5de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4e0a0 0x558331e56de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35ebfff700 1 -- 172.21.15.53:0/3689066764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4ea20 msgr2=0x558331e5de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:48.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35ebfff700 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4ea20 0x558331e5de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35ebfff700 1 -- 172.21.15.53:0/3689066764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4e0a0 msgr2=0x558331e56de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:48.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35ebfff700 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4e0a0 0x558331e56de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35ebfff700 1 -- 172.21.15.53:0/3689066764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec1058e0 con 0x558331e4d3a0 2023-11-06T01:33:48.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35f0ee7700 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4ea20 0x558331e5de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:33:48.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35ebfff700 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e568a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f35d4003900 tx=0x7f35d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626ba73609b0a4cf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:48.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:48.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35cb7fe700 1 -- 172.21.15.53:0/3689066764 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4022070 con 0x558331e4d3a0 2023-11-06T01:33:48.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35cb7fe700 1 -- 172.21.15.53:0/3689066764 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f35d401d620 con 0x558331e4d3a0 2023-11-06T01:33:48.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.100+0000 7f35feadab80 1 -- 172.21.15.53:0/3689066764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558331dc8cc0 con 0x558331e4d3a0 2023-11-06T01:33:48.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35cb7fe700 1 -- 172.21.15.53:0/3689066764 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35d401d7e0 con 0x558331e4d3a0 2023-11-06T01:33:48.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35feadab80 10 client.15306 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:48.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35feadab80 1 -- 172.21.15.53:0/3689066764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558331bb4c10 con 0x558331e4d3a0 2023-11-06T01:33:48.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35cb7fe700 1 -- 172.21.15.53:0/3689066764 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558331dc8cc0 con 0x558331e4d3a0 2023-11-06T01:33:48.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35cb7fe700 1 -- 172.21.15.53:0/3689066764 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f35d402f020 con 0x558331e4d3a0 2023-11-06T01:33:48.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35feadab80 10 client.15306 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:48.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35feadab80 10 client.15306 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-06T01:33:48.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35feadab80 20 client.15306 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:48.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35feadab80 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558331e61bf0 0x558331e60f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35feadab80 4 client.15306 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:48.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35feadab80 1 -- 172.21.15.53:0/3689066764 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558331e1bdc0 con 0x558331e61bf0 2023-11-06T01:33:48.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.101+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558331e61bf0 0x558331e60f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.102+0000 7f35cb7fe700 10 client.15306 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:48.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.102+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558331e61bf0 0x558331e60f80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4675f082382b5a73 server_cookie=cdcaaf30343f9774 in_seq=0 out_seq=0 2023-11-06T01:33:48.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.102+0000 7f35cb7fe700 1 -- 172.21.15.53:0/3689066764 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558331e1bdc0 con 0x558331e61bf0 2023-11-06T01:33:48.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.102+0000 7f35cb7fe700 10 client.15306 handle_command_reply: tid=0 2023-11-06T01:33:48.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.102+0000 7f35feadab80 1 client.15306 shutdown 2023-11-06T01:33:48.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.102+0000 7f35feadab80 1 -- 172.21.15.53:0/3689066764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 msgr2=0x558331e568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:48.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.102+0000 7f35feadab80 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e568a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f35d4003900 tx=0x7f35d40047c0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.102+0000 7f35feadab80 1 -- 172.21.15.53:0/3689066764 shutdown_connections 2023-11-06T01:33:48.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.102+0000 7f35feadab80 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558331e61bf0 0x558331e60f80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.102+0000 7f35feadab80 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4ea20 0x558331e5de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.103+0000 7f35feadab80 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4d3a0 0x558331e568a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.103+0000 7f35cb7fe700 0 client.15306 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:48.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.103+0000 7f35feadab80 1 --2- 172.21.15.53:0/3689066764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4e0a0 0x558331e56de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.103+0000 7f35feadab80 1 -- 172.21.15.53:0/3689066764 >> 172.21.15.53:0/3689066764 conn(0x558331dac080 msgr2=0x558331e47060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:48.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.103+0000 7f35feadab80 1 -- 172.21.15.53:0/3689066764 shutdown_connections 2023-11-06T01:33:48.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.103+0000 7f35feadab80 1 -- 172.21.15.53:0/3689066764 wait complete. 2023-11-06T01:33:48.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.103+0000 7f35feadab80 20 client.15306 trim_cache size 0 max 16384 2023-11-06T01:33:48.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.120+0000 7f35feadab80 1 Processor -- start 2023-11-06T01:33:48.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.120+0000 7f35feadab80 1 -- start start 2023-11-06T01:33:48.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.120+0000 7f35feadab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 0x558331e4ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.120+0000 7f35feadab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4e0a0 0x558331e6a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.120+0000 7f35feadab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4ea20 0x558331e60f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.120+0000 7f35feadab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331dc8cc0 con 0x558331e4ea20 2023-11-06T01:33:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.120+0000 7f35feadab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331d680b0 con 0x558331e4d3a0 2023-11-06T01:33:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.120+0000 7f35feadab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331da2cf0 con 0x558331e4e0a0 2023-11-06T01:33:48.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.120+0000 7f35f0ee7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4ea20 0x558331e60f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35f0ee7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4ea20 0x558331e60f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39424/0 (socket says 172.21.15.53:39424) 2023-11-06T01:33:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35ebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 0x558331e4ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35f0ee7700 1 -- 172.21.15.53:0/1549459823 learned_addr learned my addr 172.21.15.53:0/1549459823 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:48.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/1549459823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4e0a0 0x558331e6a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35f0ee7700 1 -- 172.21.15.53:0/1549459823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4e0a0 msgr2=0x558331e6a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:48.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35f0ee7700 1 --2- 172.21.15.53:0/1549459823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4e0a0 0x558331e6a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35f0ee7700 1 -- 172.21.15.53:0/1549459823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 msgr2=0x558331e4ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:48.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35f0ee7700 1 --2- 172.21.15.53:0/1549459823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 0x558331e4ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35f0ee7700 1 -- 172.21.15.53:0/1549459823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558331bf2200 con 0x558331e4ea20 2023-11-06T01:33:48.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/1549459823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4e0a0 0x558331e6a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:48.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35ebfff700 1 --2- 172.21.15.53:0/1549459823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 0x558331e4ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:33:48.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35f0ee7700 1 --2- 172.21.15.53:0/1549459823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4ea20 0x558331e60f80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f35e4021000 tx=0x7f35e400bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a834c74da2c2f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:48.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35e2ffd700 1 -- 172.21.15.53:0/1549459823 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400e040 con 0x558331e4ea20 2023-11-06T01:33:48.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35e2ffd700 1 -- 172.21.15.53:0/1549459823 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e400b780 con 0x558331e4ea20 2023-11-06T01:33:48.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.121+0000 7f35e2ffd700 1 -- 172.21.15.53:0/1549459823 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e401d730 con 0x558331e4ea20 2023-11-06T01:33:48.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 -- 172.21.15.53:0/1549459823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4ea20 msgr2=0x558331e60f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:48.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 --2- 172.21.15.53:0/1549459823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4ea20 0x558331e60f80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f35e4021000 tx=0x7f35e400bdc0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 -- 172.21.15.53:0/1549459823 shutdown_connections 2023-11-06T01:33:48.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 --2- 172.21.15.53:0/1549459823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 0x558331e4ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 --2- 172.21.15.53:0/1549459823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4ea20 0x558331e60f80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 --2- 172.21.15.53:0/1549459823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4e0a0 0x558331e6a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 -- 172.21.15.53:0/1549459823 >> 172.21.15.53:0/1549459823 conn(0x558331dac080 msgr2=0x558331e54710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 -- 172.21.15.53:0/1549459823 shutdown_connections 2023-11-06T01:33:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 -- 172.21.15.53:0/1549459823 wait complete. 2023-11-06T01:33:48.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 -- start start 2023-11-06T01:33:48.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 Processor -- start 2023-11-06T01:33:48.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 0x558331e67c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4e0a0 0x558331e6a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4ea20 0x558331e6a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331eb3700 con 0x558331e4e0a0 2023-11-06T01:33:48.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331eb3840 con 0x558331e4d3a0 2023-11-06T01:33:48.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.122+0000 7f35feadab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558331eb3980 con 0x558331e4ea20 2023-11-06T01:33:48.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35feadab80 10 client.0 _ll_register_callbacks cb 0x7f35d40043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:48.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4e0a0 0x558331e6a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4e0a0 0x558331e6a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39428/0 (socket says 172.21.15.53:39428) 2023-11-06T01:33:48.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35eb7fe700 1 -- 172.21.15.53:0/970865178 learned_addr learned my addr 172.21.15.53:0/970865178 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:48.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35f0ee7700 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4ea20 0x558331e6a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35ebfff700 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 0x558331e67c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35eb7fe700 1 -- 172.21.15.53:0/970865178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4ea20 msgr2=0x558331e6a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:48.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4ea20 0x558331e6a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35eb7fe700 1 -- 172.21.15.53:0/970865178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 msgr2=0x558331e67c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:48.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 0x558331e67c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35eb7fe700 1 -- 172.21.15.53:0/970865178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558331bb4c10 con 0x558331e4e0a0 2023-11-06T01:33:48.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35ebfff700 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 0x558331e67c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:48.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35eb7fe700 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4e0a0 0x558331e6a150 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e970 tx=0x7f35dc0cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d79e1b063b9f500 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:48.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:48.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35cb7fe700 1 -- 172.21.15.53:0/970865178 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0a1550 con 0x558331e4e0a0 2023-11-06T01:33:48.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35cb7fe700 1 -- 172.21.15.53:0/970865178 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f35dc099740 con 0x558331e4e0a0 2023-11-06T01:33:48.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35feadab80 1 -- 172.21.15.53:0/970865178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35e4009490 con 0x558331e4e0a0 2023-11-06T01:33:48.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35cb7fe700 1 -- 172.21.15.53:0/970865178 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35dc099900 con 0x558331e4e0a0 2023-11-06T01:33:48.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35feadab80 10 client.15312 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:48.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.123+0000 7f35feadab80 1 -- 172.21.15.53:0/970865178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558331bcbd90 con 0x558331e4e0a0 2023-11-06T01:33:48.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.124+0000 7f35cb7fe700 1 -- 172.21.15.53:0/970865178 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35dc065030 con 0x558331e4e0a0 2023-11-06T01:33:48.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.124+0000 7f35cb7fe700 1 -- 172.21.15.53:0/970865178 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f35dc098a60 con 0x558331e4e0a0 2023-11-06T01:33:48.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.124+0000 7f35feadab80 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:48.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.124+0000 7f35feadab80 10 client.15312 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-06T01:33:48.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.124+0000 7f35feadab80 20 client.15312 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:48.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.124+0000 7f35feadab80 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558331eb8700 0x558331eb5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:48.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.124+0000 7f35feadab80 4 client.15312 mds_command: new command op to 24487 tid=0[{"prefix": "status"}] 2023-11-06T01:33:48.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.124+0000 7f35feadab80 1 -- 172.21.15.53:0/970865178 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558331d96cf0 con 0x558331eb8700 2023-11-06T01:33:48.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.124+0000 7f35ebfff700 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558331eb8700 0x558331eb5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:48.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.124+0000 7f35cb7fe700 10 client.15312 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:48.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.124+0000 7f35ebfff700 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558331eb8700 0x558331eb5010 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff6092f5bd223533 server_cookie=18e87100a504887e in_seq=0 out_seq=0 2023-11-06T01:33:48.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35cb7fe700 1 -- 172.21.15.53:0/970865178 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558331d96cf0 con 0x558331eb8700 2023-11-06T01:33:48.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35cb7fe700 10 client.15312 handle_command_reply: tid=0 2023-11-06T01:33:48.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 1 client.15312 shutdown 2023-11-06T01:33:48.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 1 -- 172.21.15.53:0/970865178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4e0a0 msgr2=0x558331e6a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:48.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4e0a0 0x558331e6a150 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e970 tx=0x7f35dc0cb040 comp rx=0 tx=0).stop 2023-11-06T01:33:48.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 1 -- 172.21.15.53:0/970865178 shutdown_connections 2023-11-06T01:33:48.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558331e4d3a0 0x558331e67c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558331eb8700 0x558331eb5010 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558331e4ea20 0x558331e6a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35cb7fe700 0 client.15312 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 1 --2- 172.21.15.53:0/970865178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558331e4e0a0 0x558331e6a150 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 1 -- 172.21.15.53:0/970865178 >> 172.21.15.53:0/970865178 conn(0x558331dac080 msgr2=0x558331e60f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:48.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 1 -- 172.21.15.53:0/970865178 shutdown_connections 2023-11-06T01:33:48.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 1 -- 172.21.15.53:0/970865178 wait complete. 2023-11-06T01:33:48.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35feadab80 20 client.15312 trim_cache size 0 max 16384 2023-11-06T01:33:48.149 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "3f13dbd6-7c43-11ee-8db9-212e2dc638e7", 2023-11-06T01:33:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-11-06T01:33:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24487, 2023-11-06T01:33:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-11-06T01:33:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-11-06T01:33:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-11-06T01:33:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 82.030086322000002, 2023-11-06T01:33:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 27, 2023-11-06T01:33:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 84, 2023-11-06T01:33:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 82, 2023-11-06T01:33:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 93.675822561000004 2023-11-06T01:33:48.152 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:48.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35f294a700 1 -- 172.21.15.53:0/1014100098 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f35d80a05b0 msgr2=0x7f35d80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:48.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35f294a700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f35d80a05b0 0x7f35d80a2a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009c60 tx=0x7f35d4009260 comp rx=0 tx=0).stop 2023-11-06T01:33:48.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35f294a700 1 -- 172.21.15.53:0/1014100098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ec101f60 msgr2=0x7f35ec19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:48.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.125+0000 7f35f294a700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ec101f60 0x7f35ec19d720 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e9a0 tx=0x7f35dc00ecb0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.126+0000 7f35f294a700 1 -- 172.21.15.53:0/1014100098 shutdown_connections 2023-11-06T01:33:48.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.126+0000 7f35f294a700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f35d80a05b0 0x7f35d80a2a70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.126+0000 7f35f294a700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec0ff5f0 0x7f35ec19d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.126+0000 7f35f294a700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec104930 0x7f35ec1a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.126+0000 7f35f294a700 1 --2- 172.21.15.53:0/1014100098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ec101f60 0x7f35ec19d720 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:48.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.126+0000 7f35f294a700 1 -- 172.21.15.53:0/1014100098 >> 172.21.15.53:0/1014100098 conn(0x7f35ec0fb1f0 msgr2=0x7f35ec1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:48.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.126+0000 7f35f294a700 1 -- 172.21.15.53:0/1014100098 shutdown_connections 2023-11-06T01:33:48.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:48.126+0000 7f35f294a700 1 -- 172.21.15.53:0/1014100098 wait complete. 2023-11-06T01:33:48.156 INFO:teuthology.run_tasks:Running task kclient... 2023-11-06T01:33:48.178 INFO:tasks.kclient:Mounting kernel clients... 2023-11-06T01:33:48.178 INFO:tasks.kclient:config is {'client.0': {}} 2023-11-06T01:33:48.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:48.179 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-06T01:33:48.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:48.197 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2023-11-06T01:33:48.266 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2023-11-06T01:33:48.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T01:33:48.268 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2023-11-06T01:33:48.268 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-06T01:33:48.269 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-11-06T01:33:48.269 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-11-06T01:33:48.269 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph fs ls 2023-11-06T01:33:48.869 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:33:49.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:49 smithi053 ceph-mon[99472]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:49.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:49 smithi062 ceph-mon[103924]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:49.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:49 smithi163 ceph-mon[104416]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-11-06T01:33:50.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.106+0000 7f319e4c1700 1 -- 172.21.15.53:0/903809010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3198105d80 msgr2=0x7f3198106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:50.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.106+0000 7f319e4c1700 1 --2- 172.21.15.53:0/903809010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3198105d80 0x7f3198106160 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3188009990 tx=0x7f3188009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:50.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.107+0000 7f319e4c1700 1 -- 172.21.15.53:0/903809010 shutdown_connections 2023-11-06T01:33:50.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.107+0000 7f319e4c1700 1 --2- 172.21.15.53:0/903809010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3198100690 0x7f319810cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:50.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.107+0000 7f319e4c1700 1 --2- 172.21.15.53:0/903809010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3198105d80 0x7f3198106160 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:50.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.107+0000 7f319e4c1700 1 --2- 172.21.15.53:0/903809010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31980ffcd0 0x7f3198100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:50.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.107+0000 7f319e4c1700 1 -- 172.21.15.53:0/903809010 >> 172.21.15.53:0/903809010 conn(0x7f3198076d90 msgr2=0x7f31980771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:50.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.107+0000 7f319e4c1700 1 -- 172.21.15.53:0/903809010 shutdown_connections 2023-11-06T01:33:50.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.107+0000 7f319e4c1700 1 -- 172.21.15.53:0/903809010 wait complete. 2023-11-06T01:33:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.107+0000 7f319e4c1700 1 Processor -- start 2023-11-06T01:33:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.107+0000 7f319e4c1700 1 -- start start 2023-11-06T01:33:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319e4c1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31980ffcd0 0x7f3198196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319e4c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3198100690 0x7f3198197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319e4c1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3198105d80 0x7f319819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319e4c1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319819bdb0 con 0x7f3198105d80 2023-11-06T01:33:50.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319e4c1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319819bf20 con 0x7f31980ffcd0 2023-11-06T01:33:50.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319e4c1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319819c0e0 con 0x7f3198100690 2023-11-06T01:33:50.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319ca5e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3198105d80 0x7f319819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:50.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319ca5e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3198105d80 0x7f319819b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39444/0 (socket says 172.21.15.53:39444) 2023-11-06T01:33:50.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319ca5e700 1 -- 172.21.15.53:0/3490411481 learned_addr learned my addr 172.21.15.53:0/3490411481 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:50.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f3197fff700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31980ffcd0 0x7f3198196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:50.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f31977fe700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3198100690 0x7f3198197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:50.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319ca5e700 1 -- 172.21.15.53:0/3490411481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3198100690 msgr2=0x7f3198197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:50.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319ca5e700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3198100690 0x7f3198197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:50.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319ca5e700 1 -- 172.21.15.53:0/3490411481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31980ffcd0 msgr2=0x7f3198196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:50.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319ca5e700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31980ffcd0 0x7f3198196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:50.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f319ca5e700 1 -- 172.21.15.53:0/3490411481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31880095f0 con 0x7f3198105d80 2023-11-06T01:33:50.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f31977fe700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3198100690 0x7f3198197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:50.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.108+0000 7f3197fff700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31980ffcd0 0x7f3198196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:50.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.109+0000 7f319ca5e700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3198105d80 0x7f319819b7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f319000cca0 tx=0x7f319000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:50.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.109+0000 7f31957fa700 1 -- 172.21.15.53:0/3490411481 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000cea0 con 0x7f3198105d80 2023-11-06T01:33:50.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.109+0000 7f31957fa700 1 -- 172.21.15.53:0/3490411481 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f319000ad60 con 0x7f3198105d80 2023-11-06T01:33:50.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.109+0000 7f319e4c1700 1 -- 172.21.15.53:0/3490411481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319819c3c0 con 0x7f3198105d80 2023-11-06T01:33:50.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.109+0000 7f31957fa700 1 -- 172.21.15.53:0/3490411481 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31900116f0 con 0x7f3198105d80 2023-11-06T01:33:50.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.109+0000 7f319e4c1700 1 -- 172.21.15.53:0/3490411481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31981a3c20 con 0x7f3198105d80 2023-11-06T01:33:50.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.110+0000 7f319e4c1700 1 -- 172.21.15.53:0/3490411481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319804ea90 con 0x7f3198105d80 2023-11-06T01:33:50.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.111+0000 7f31957fa700 1 -- 172.21.15.53:0/3490411481 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f3190012420 con 0x7f3198105d80 2023-11-06T01:33:50.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.112+0000 7f31957fa700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f31800a0630 0x7f31800a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:50.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.112+0000 7f3197fff700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f31800a0630 0x7f31800a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:50.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.112+0000 7f31957fa700 1 -- 172.21.15.53:0/3490411481 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f31900cc240 con 0x7f3198105d80 2023-11-06T01:33:50.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.112+0000 7f3197fff700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f31800a0630 0x7f31800a2af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f318800b5c0 tx=0x7f3188005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:50.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.115+0000 7f31957fa700 1 -- 172.21.15.53:0/3490411481 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3190095d70 con 0x7f3198105d80 2023-11-06T01:33:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.358+0000 7f319e4c1700 1 -- 172.21.15.53:0/3490411481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3198198150 con 0x7f3198105d80 2023-11-06T01:33:50.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.359+0000 7f31957fa700 1 -- 172.21.15.53:0/3490411481 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3190099390 con 0x7f3198105d80 2023-11-06T01:33:50.361 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T01:33:50.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 -- 172.21.15.53:0/3490411481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f31800a0630 msgr2=0x7f31800a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:50.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f31800a0630 0x7f31800a2af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f318800b5c0 tx=0x7f3188005fd0 comp rx=0 tx=0).stop 2023-11-06T01:33:50.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 -- 172.21.15.53:0/3490411481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3198105d80 msgr2=0x7f319819b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:50.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3198105d80 0x7f319819b7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f319000cca0 tx=0x7f319000a340 comp rx=0 tx=0).stop 2023-11-06T01:33:50.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 -- 172.21.15.53:0/3490411481 shutdown_connections 2023-11-06T01:33:50.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f31800a0630 0x7f31800a2af0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:50.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31980ffcd0 0x7f3198196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3198105d80 0x7f319819b7a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 --2- 172.21.15.53:0/3490411481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3198100690 0x7f3198197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 -- 172.21.15.53:0/3490411481 >> 172.21.15.53:0/3490411481 conn(0x7f3198076d90 msgr2=0x7f31980fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 -- 172.21.15.53:0/3490411481 shutdown_connections 2023-11-06T01:33:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:50.361+0000 7f319e4c1700 1 -- 172.21.15.53:0/3490411481 wait complete. 2023-11-06T01:33:50.828 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T01:33:50.829 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T01:33:50.829 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-11-06T01:33:50.829 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T01:33:50.829 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-11-06T01:33:50.830 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T01:33:50.830 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T01:33:50.830 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T01:33:50.830 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-11-06T01:33:50.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:50.831 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-11-06T01:33:50.850 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T01:33:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T01:33:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T01:33:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-06T01:33:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-11-06T01:33:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-06T01:33:50.851 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T01:33:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-11-06T01:33:50.851 INFO:teuthology.orchestra.run.smithi053.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T01:33:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-11-06T01:33:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T01:33:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 41755sec preferred_lft 41755sec 2023-11-06T01:33:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link 2023-11-06T01:33:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-11-06T01:33:50.853 INFO:teuthology.orchestra.run.smithi053.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T01:33:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-11-06T01:33:50.853 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T01:33:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-11-06T01:33:50.854 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-06T01:33:50.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:50.855 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-06T01:33:50.855 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-06T01:33:50.855 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add name ceph-brx type bridge 2023-11-06T01:33:50.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr flush dev ceph-brx 2023-11-06T01:33:50.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set ceph-brx up 2023-11-06T01:33:50.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-06T01:33:50.857 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-06T01:33:51.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:51.034 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T01:33:51.064 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-11-06T01:33:51.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:51.066 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-11-06T01:33:51.134 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-11-06T01:33:51.134 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T01:33:51.135 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T01:33:51.135 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T01:33:51.135 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T01:33:51.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:51.138 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-06T01:33:51.138 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-06T01:33:51.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T01:33:51.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T01:33:51.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T01:33:51.140 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-06T01:33:51.300 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:51 smithi053 ceph-mon[99472]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-11-06T01:33:51.301 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:51 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3490411481' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-06T01:33:51.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:51.390 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-06T01:33:51.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:51.408 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2023-11-06T01:33:51.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:51.465 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-06T01:33:51.465 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-06T01:33:51.466 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--tmp-cephfs 2023-11-06T01:33:51.466 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-11-06T01:33:51.466 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-06T01:33:51.561 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-11-06T01:33:51.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:51.562 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-06T01:33:51.563 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-06T01:33:51.563 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-11-06T01:33:51.563 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-06T01:33:51.564 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-11-06T01:33:51.564 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-11-06T01:33:51.564 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-11-06T01:33:51.564 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-06T01:33:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:51 smithi062 ceph-mon[103924]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-11-06T01:33:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:51 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3490411481' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-06T01:33:51.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:51 smithi163 ceph-mon[104416]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-11-06T01:33:51.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:51 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3490411481' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-06T01:33:51.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:51.772 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-11-06T01:33:51.773 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-11-06T01:33:51.773 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-11-06T01:33:51.773 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.0 master ceph-brx 2023-11-06T01:33:51.773 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-11-06T01:33:51.874 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T01:33:51.874 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /tmp/cephfs 2023-11-06T01:33:51.891 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/tmp/cephfs' 2023-11-06T01:33:51.892 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T01:33:51.892 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /tmp/cephfs 2023-11-06T01:33:51.948 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-11-06T01:33:51.948 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-11-06T01:33:51.949 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/9b213191-048f-4ff8-846e-a578c140694d 2023-11-06T01:33:51.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:51.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/9b213191-048f-4ff8-846e-a578c140694d /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2023-11-06T01:33:52.040 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-11-06T01:33:52.040 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-11-06T01:33:52.165 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-11-06T01:33:52.166 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-11-06T01:33:52.166 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.62:3300,172.21.15.163:3300" 2023-11-06T01:33:52.166 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@3f13dbd6-7c43-11ee-8db9-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/9b213191-048f-4ff8-846e-a578c140694d 2023-11-06T01:33:52.166 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.53:3300/172.21.15.62:3300/172.21.15.163:3300" will pass to kernel 2023-11-06T01:33:52.167 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: v2 mount support check returned 2 2023-11-06T01:33:52.167 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-06T01:33:52.167 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:3300,172.21.15.62:3300,172.21.15.163:3300:/volumes/_nogroup/sv_1/9b213191-048f-4ff8-846e-a578c140694d 2023-11-06T01:33:52.167 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mds_namespace=cephfs,fsid=3f13dbd6-7c43-11ee-8db9-212e2dc638e7" will pass to kernel 2023-11-06T01:33:52.172 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T01:33:52.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:52.173 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /tmp/cephfs 2023-11-06T01:33:52.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:52.235 DEBUG:teuthology.orchestra.run.smithi053:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-06T01:33:52.293 INFO:teuthology.orchestra.run.smithi053.stdout:client24911 2023-11-06T01:33:52.293 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:33:52.294 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T01:33:52.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:52.313 DEBUG:teuthology.orchestra.run.smithi053:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-06T01:33:52.411 INFO:teuthology.orchestra.run.smithi053.stdout:client24911 2023-11-06T01:33:52.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:52.412 DEBUG:teuthology.orchestra.run.smithi053:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-06T01:33:52.463 INFO:teuthology.orchestra.run.smithi053.stdout:client24911 2023-11-06T01:33:52.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:33:52.464 DEBUG:teuthology.orchestra.run.smithi053:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7.client24911/status') 2023-11-06T01:33:52.521 INFO:teuthology.orchestra.run.smithi053.stdout:instance: client.24911 (3)172.21.15.53:0/1089936501 2023-11-06T01:33:52.522 INFO:teuthology.orchestra.run.smithi053.stdout:blocklisted: false 2023-11-06T01:33:52.523 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2023-11-06T01:33:52.523 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2023-11-06T01:33:52.523 INFO:teuthology.orchestra.run.smithi053.stderr:71 bytes copied, 2.4254e-05 s, 2.9 MB/s 2023-11-06T01:33:52.524 INFO:teuthology.run_tasks:Running task check-counter... 2023-11-06T01:33:52.535 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-11-06T01:33:52.575 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:33:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.700+0000 7fe1e9911700 1 -- 172.21.15.53:0/2999329463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1e41017f0 msgr2=0x7fe1e4103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.700+0000 7fe1e9911700 1 --2- 172.21.15.53:0/2999329463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1e41017f0 0x7fe1e4103be0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009990 tx=0x7fe1d4009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.700+0000 7fe1e9911700 1 -- 172.21.15.53:0/2999329463 shutdown_connections 2023-11-06T01:33:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.700+0000 7fe1e9911700 1 --2- 172.21.15.53:0/2999329463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e4106a50 0x7fe1e4108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.700+0000 7fe1e9911700 1 --2- 172.21.15.53:0/2999329463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1e41017f0 0x7fe1e4103be0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.700+0000 7fe1e9911700 1 --2- 172.21.15.53:0/2999329463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e4104120 0x7fe1e4106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.700+0000 7fe1e9911700 1 -- 172.21.15.53:0/2999329463 >> 172.21.15.53:0/2999329463 conn(0x7fe1e40fb1c0 msgr2=0x7fe1e40fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:52.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.700+0000 7fe1e9911700 1 -- 172.21.15.53:0/2999329463 shutdown_connections 2023-11-06T01:33:52.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.700+0000 7fe1e9911700 1 -- 172.21.15.53:0/2999329463 wait complete. 2023-11-06T01:33:52.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.701+0000 7fe1e9911700 1 Processor -- start 2023-11-06T01:33:52.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.701+0000 7fe1e9911700 1 -- start start 2023-11-06T01:33:52.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.701+0000 7fe1e9911700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e41017f0 0x7fe1e4101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:52.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.701+0000 7fe1e9911700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e4104120 0x7fe1e40ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.701+0000 7fe1e9911700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1e4106a50 0x7fe1e40ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.701+0000 7fe1e9911700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e4100480 con 0x7fe1e4106a50 2023-11-06T01:33:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.701+0000 7fe1e9911700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e41005f0 con 0x7fe1e41017f0 2023-11-06T01:33:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.701+0000 7fe1e9911700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e41a0a70 con 0x7fe1e4104120 2023-11-06T01:33:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.701+0000 7fe1e37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1e4106a50 0x7fe1e40ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.701+0000 7fe1e37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1e4106a50 0x7fe1e40ffce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39464/0 (socket says 172.21.15.53:39464) 2023-11-06T01:33:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.701+0000 7fe1e37fe700 1 -- 172.21.15.53:0/1606232621 learned_addr learned my addr 172.21.15.53:0/1606232621 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e27fc700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e4104120 0x7fe1e40ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e2ffd700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e41017f0 0x7fe1e4101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e37fe700 1 -- 172.21.15.53:0/1606232621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e4104120 msgr2=0x7fe1e40ff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e37fe700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e4104120 0x7fe1e40ff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e37fe700 1 -- 172.21.15.53:0/1606232621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e41017f0 msgr2=0x7fe1e4101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e37fe700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e41017f0 0x7fe1e4101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e37fe700 1 -- 172.21.15.53:0/1606232621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d40095f0 con 0x7fe1e4106a50 2023-11-06T01:33:52.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e37fe700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1e4106a50 0x7fe1e40ffce0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc007ee0 tx=0x7fe1dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:52.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e890f700 1 -- 172.21.15.53:0/1606232621 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc00dd20 con 0x7fe1e4106a50 2023-11-06T01:33:52.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e9911700 1 -- 172.21.15.53:0/1606232621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e41a0d50 con 0x7fe1e4106a50 2023-11-06T01:33:52.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e890f700 1 -- 172.21.15.53:0/1606232621 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1dc00f040 con 0x7fe1e4106a50 2023-11-06T01:33:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e890f700 1 -- 172.21.15.53:0/1606232621 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0048a0 con 0x7fe1e4106a50 2023-11-06T01:33:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.702+0000 7fe1e9911700 1 -- 172.21.15.53:0/1606232621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e41a1200 con 0x7fe1e4106a50 2023-11-06T01:33:52.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.703+0000 7fe1e9911700 1 -- 172.21.15.53:0/1606232621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e40fcd40 con 0x7fe1e4106a50 2023-11-06T01:33:52.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.704+0000 7fe1e890f700 1 -- 172.21.15.53:0/1606232621 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fe1dc003a00 con 0x7fe1e4106a50 2023-11-06T01:33:52.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.705+0000 7fe1e890f700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe1cc0a05f0 0x7fe1cc0a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:52.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.705+0000 7fe1e890f700 1 -- 172.21.15.53:0/1606232621 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe1dc0cbc30 con 0x7fe1e4106a50 2023-11-06T01:33:52.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.705+0000 7fe1e2ffd700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe1cc0a05f0 0x7fe1cc0a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:52.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.705+0000 7fe1e2ffd700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe1cc0a05f0 0x7fe1cc0a2ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400b5c0 tx=0x7fe1d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:52.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.708+0000 7fe1e890f700 1 -- 172.21.15.53:0/1606232621 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe1dc0955f0 con 0x7fe1e4106a50 2023-11-06T01:33:52.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.912+0000 7fe1e9911700 1 -- 172.21.15.53:0/1606232621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1e402ce40 con 0x7fe1e4106a50 2023-11-06T01:33:52.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.914+0000 7fe1e890f700 1 -- 172.21.15.53:0/1606232621 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22543 (secure 0 0 0) 0x7fe1dc098c10 con 0x7fe1e4106a50 2023-11-06T01:33:52.916 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:52.916 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:33:27.113678+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:33:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 -- 172.21.15.53:0/1606232621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe1cc0a05f0 msgr2=0x7fe1cc0a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe1cc0a05f0 0x7fe1cc0a2ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400b5c0 tx=0x7fe1d4005fd0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 -- 172.21.15.53:0/1606232621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1e4106a50 msgr2=0x7fe1e40ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1e4106a50 0x7fe1e40ffce0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc007ee0 tx=0x7fe1dc00d3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 -- 172.21.15.53:0/1606232621 shutdown_connections 2023-11-06T01:33:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe1cc0a05f0 0x7fe1cc0a2ab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e41017f0 0x7fe1e4101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1e4106a50 0x7fe1e40ffce0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 --2- 172.21.15.53:0/1606232621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e4104120 0x7fe1e40ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 -- 172.21.15.53:0/1606232621 >> 172.21.15.53:0/1606232621 conn(0x7fe1e40fb1c0 msgr2=0x7fe1e41024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 -- 172.21.15.53:0/1606232621 shutdown_connections 2023-11-06T01:33:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:52.916+0000 7fe1e9911700 1 -- 172.21.15.53:0/1606232621 wait complete. 2023-11-06T01:33:52.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T01:32:24.604601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T01:32:25.457741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:33:52.932 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T01:33:53.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.059+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2720792366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00ffca0 msgr2=0x7f7af01000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.059+0000 7f7af4ab6700 1 --2- 172.21.15.53:0/2720792366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00ffca0 0x7f7af01000c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00099c0 tx=0x7f7ae0009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.059+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2720792366 shutdown_connections 2023-11-06T01:33:53.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.059+0000 7f7af4ab6700 1 --2- 172.21.15.53:0/2720792366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af0101830 0x7f7af01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.059+0000 7f7af4ab6700 1 --2- 172.21.15.53:0/2720792366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00ffca0 0x7f7af01000c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.059+0000 7f7af4ab6700 1 --2- 172.21.15.53:0/2720792366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7af0100e90 0x7f7af01012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.059+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2720792366 >> 172.21.15.53:0/2720792366 conn(0x7f7af00fb250 msgr2=0x7f7af00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:53.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.059+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2720792366 shutdown_connections 2023-11-06T01:33:53.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.059+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2720792366 wait complete. 2023-11-06T01:33:53.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.060+0000 7f7af4ab6700 1 Processor -- start 2023-11-06T01:33:53.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.060+0000 7f7af4ab6700 1 -- start start 2023-11-06T01:33:53.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.060+0000 7f7af4ab6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00ffca0 0x7f7af0069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.060+0000 7f7af4ab6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af0100e90 0x7f7af019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.060+0000 7f7af4ab6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7af0101830 0x7f7af019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.060+0000 7f7af4ab6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af019e600 con 0x7f7af00ffca0 2023-11-06T01:33:53.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.060+0000 7f7af4ab6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af019e770 con 0x7f7af0100e90 2023-11-06T01:33:53.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.060+0000 7f7af4ab6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af019e930 con 0x7f7af0101830 2023-11-06T01:33:53.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.060+0000 7f7aee59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00ffca0 0x7f7af0069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:53.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aee59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00ffca0 0x7f7af0069cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39474/0 (socket says 172.21.15.53:39474) 2023-11-06T01:33:53.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aee59c700 1 -- 172.21.15.53:0/2250518868 learned_addr learned my addr 172.21.15.53:0/2250518868 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:53.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aeed9d700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7af0101830 0x7f7af019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:53.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aedd9b700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af0100e90 0x7f7af019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:53.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aee59c700 1 -- 172.21.15.53:0/2250518868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7af0101830 msgr2=0x7f7af019de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aee59c700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7af0101830 0x7f7af019de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aee59c700 1 -- 172.21.15.53:0/2250518868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af0100e90 msgr2=0x7f7af019b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aee59c700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af0100e90 0x7f7af019b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aee59c700 1 -- 172.21.15.53:0/2250518868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae00095f0 con 0x7f7af00ffca0 2023-11-06T01:33:53.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aeed9d700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7af0101830 0x7f7af019de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:53.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aedd9b700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af0100e90 0x7f7af019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:53.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7aee59c700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00ffca0 0x7f7af0069cb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009f50 tx=0x7f7ae0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:53.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7ae77fe700 1 -- 172.21.15.53:0/2250518868 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae001d070 con 0x7f7af00ffca0 2023-11-06T01:33:53.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7ae77fe700 1 -- 172.21.15.53:0/2250518868 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae0004180 con 0x7f7af00ffca0 2023-11-06T01:33:53.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2250518868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af019ebb0 con 0x7f7af00ffca0 2023-11-06T01:33:53.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7ae77fe700 1 -- 172.21.15.53:0/2250518868 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000f460 con 0x7f7af00ffca0 2023-11-06T01:33:53.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.061+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2250518868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af019f030 con 0x7f7af00ffca0 2023-11-06T01:33:53.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.062+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2250518868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af0107260 con 0x7f7af00ffca0 2023-11-06T01:33:53.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.063+0000 7f7ae77fe700 1 -- 172.21.15.53:0/2250518868 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f7ae0022650 con 0x7f7af00ffca0 2023-11-06T01:33:53.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.064+0000 7f7ae77fe700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7adc0a0630 0x7f7adc0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.064+0000 7f7aedd9b700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7adc0a0630 0x7f7adc0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:53.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.064+0000 7f7ae77fe700 1 -- 172.21.15.53:0/2250518868 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7ae00cd5d0 con 0x7f7af00ffca0 2023-11-06T01:33:53.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.065+0000 7f7aedd9b700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7adc0a0630 0x7f7adc0a2af0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8005fd0 tx=0x7f7ad8005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:53.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.068+0000 7f7ae77fe700 1 -- 172.21.15.53:0/2250518868 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7ae0097020 con 0x7f7af00ffca0 2023-11-06T01:33:53.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.268+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2250518868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7af002ce00 con 0x7f7af00ffca0 2023-11-06T01:33:53.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.270+0000 7f7ae77fe700 1 -- 172.21.15.53:0/2250518868 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ae009a640 con 0x7f7af00ffca0 2023-11-06T01:33:53.272 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:33:53.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2250518868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7adc0a0630 msgr2=0x7f7adc0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7adc0a0630 0x7f7adc0a2af0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8005fd0 tx=0x7f7ad8005e20 comp rx=0 tx=0).stop 2023-11-06T01:33:53.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2250518868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00ffca0 msgr2=0x7f7af0069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00ffca0 0x7f7af0069cb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009f50 tx=0x7f7ae0005ef0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2250518868 shutdown_connections 2023-11-06T01:33:53.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7adc0a0630 0x7f7adc0a2af0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7af0101830 0x7f7af019de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00ffca0 0x7f7af0069cb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 --2- 172.21.15.53:0/2250518868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af0100e90 0x7f7af019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2250518868 >> 172.21.15.53:0/2250518868 conn(0x7f7af00fb250 msgr2=0x7f7af01043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:53.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2250518868 shutdown_connections 2023-11-06T01:33:53.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.272+0000 7f7af4ab6700 1 -- 172.21.15.53:0/2250518868 wait complete. 2023-11-06T01:33:53.287 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T01:33:53.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.415+0000 7f52dd35c700 1 -- 172.21.15.53:0/1793917730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52d80ffca0 msgr2=0x7f52d81000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.415+0000 7f52dd35c700 1 --2- 172.21.15.53:0/1793917730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52d80ffca0 0x7f52d81000c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f52c00099c0 tx=0x7f52c0009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.415+0000 7f52dd35c700 1 -- 172.21.15.53:0/1793917730 shutdown_connections 2023-11-06T01:33:53.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.415+0000 7f52dd35c700 1 --2- 172.21.15.53:0/1793917730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52d8101830 0x7f52d81080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.415+0000 7f52dd35c700 1 --2- 172.21.15.53:0/1793917730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52d80ffca0 0x7f52d81000c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.415+0000 7f52dd35c700 1 --2- 172.21.15.53:0/1793917730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52d8100e90 0x7f52d81012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.415+0000 7f52dd35c700 1 -- 172.21.15.53:0/1793917730 >> 172.21.15.53:0/1793917730 conn(0x7f52d80fb250 msgr2=0x7f52d80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:53.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.415+0000 7f52dd35c700 1 -- 172.21.15.53:0/1793917730 shutdown_connections 2023-11-06T01:33:53.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.415+0000 7f52dd35c700 1 -- 172.21.15.53:0/1793917730 wait complete. 2023-11-06T01:33:53.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52dd35c700 1 Processor -- start 2023-11-06T01:33:53.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52dd35c700 1 -- start start 2023-11-06T01:33:53.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52dd35c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52d80ffca0 0x7f52d8196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52dd35c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52d8100e90 0x7f52d8196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52dd35c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52d8101830 0x7f52d819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52dd35c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d819e620 con 0x7f52d80ffca0 2023-11-06T01:33:53.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52dd35c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d819e760 con 0x7f52d8101830 2023-11-06T01:33:53.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52dd35c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d819e8f0 con 0x7f52d8100e90 2023-11-06T01:33:53.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52d6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52d80ffca0 0x7f52d8196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:53.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52d6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52d80ffca0 0x7f52d8196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39488/0 (socket says 172.21.15.53:39488) 2023-11-06T01:33:53.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52d6ffd700 1 -- 172.21.15.53:0/217361122 learned_addr learned my addr 172.21.15.53:0/217361122 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:53.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52d77fe700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52d8101830 0x7f52d819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:53.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52d6ffd700 1 -- 172.21.15.53:0/217361122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52d8100e90 msgr2=0x7f52d8196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52d6ffd700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52d8100e90 0x7f52d8196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52d6ffd700 1 -- 172.21.15.53:0/217361122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52d8101830 msgr2=0x7f52d819e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52d6ffd700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52d8101830 0x7f52d819e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52d6ffd700 1 -- 172.21.15.53:0/217361122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00095f0 con 0x7f52d80ffca0 2023-11-06T01:33:53.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.416+0000 7f52d77fe700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52d8101830 0x7f52d819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:53.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.417+0000 7f52d6ffd700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52d80ffca0 0x7f52d8196aa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009fd0 tx=0x7f52c000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:53.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.417+0000 7f52cffff700 1 -- 172.21.15.53:0/217361122 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c001d070 con 0x7f52d80ffca0 2023-11-06T01:33:53.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.417+0000 7f52cffff700 1 -- 172.21.15.53:0/217361122 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52c00054f0 con 0x7f52d80ffca0 2023-11-06T01:33:53.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.417+0000 7f52cffff700 1 -- 172.21.15.53:0/217361122 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c0004c40 con 0x7f52d80ffca0 2023-11-06T01:33:53.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.417+0000 7f52dd35c700 1 -- 172.21.15.53:0/217361122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d819eb70 con 0x7f52d80ffca0 2023-11-06T01:33:53.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.417+0000 7f52dd35c700 1 -- 172.21.15.53:0/217361122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d819ef40 con 0x7f52d80ffca0 2023-11-06T01:33:53.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.418+0000 7f52dd35c700 1 -- 172.21.15.53:0/217361122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d810b4b0 con 0x7f52d80ffca0 2023-11-06T01:33:53.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.419+0000 7f52cffff700 1 -- 172.21.15.53:0/217361122 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f52c000fc30 con 0x7f52d80ffca0 2023-11-06T01:33:53.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.419+0000 7f52cffff700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52c40a05b0 0x7f52c40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.420+0000 7f52cffff700 1 -- 172.21.15.53:0/217361122 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f52c00cce80 con 0x7f52d80ffca0 2023-11-06T01:33:53.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.420+0000 7f52d67fc700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52c40a05b0 0x7f52c40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:53.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.420+0000 7f52d67fc700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52c40a05b0 0x7f52c40a2a70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c8005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:53.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.423+0000 7f52cffff700 1 -- 172.21.15.53:0/217361122 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f52c0096930 con 0x7f52d80ffca0 2023-11-06T01:33:53.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:53 smithi053 ceph-mon[99472]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-11-06T01:33:53.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:53 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1606232621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:53.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.626+0000 7f52dd35c700 1 -- 172.21.15.53:0/217361122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52d819f550 con 0x7f52d80ffca0 2023-11-06T01:33:53.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.626+0000 7f52cffff700 1 -- 172.21.15.53:0/217361122 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52c0099f50 con 0x7f52d80ffca0 2023-11-06T01:33:53.629 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-06T01:33:53.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.628+0000 7f52dd35c700 1 -- 172.21.15.53:0/217361122 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52c40a05b0 msgr2=0x7f52c40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.628+0000 7f52dd35c700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52c40a05b0 0x7f52c40a2a70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c8005d00 comp rx=0 tx=0).stop 2023-11-06T01:33:53.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.628+0000 7f52dd35c700 1 -- 172.21.15.53:0/217361122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52d80ffca0 msgr2=0x7f52d8196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.628+0000 7f52dd35c700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52d80ffca0 0x7f52d8196aa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009fd0 tx=0x7f52c000f740 comp rx=0 tx=0).stop 2023-11-06T01:33:53.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.628+0000 7f52dd35c700 1 -- 172.21.15.53:0/217361122 shutdown_connections 2023-11-06T01:33:53.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.628+0000 7f52dd35c700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52c40a05b0 0x7f52c40a2a70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.628+0000 7f52dd35c700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52d8101830 0x7f52d819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.628+0000 7f52dd35c700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52d80ffca0 0x7f52d8196aa0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.628+0000 7f52dd35c700 1 --2- 172.21.15.53:0/217361122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52d8100e90 0x7f52d8196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.629+0000 7f52dd35c700 1 -- 172.21.15.53:0/217361122 >> 172.21.15.53:0/217361122 conn(0x7f52d80fb250 msgr2=0x7f52d8069100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:53.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.629+0000 7f52dd35c700 1 -- 172.21.15.53:0/217361122 shutdown_connections 2023-11-06T01:33:53.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.629+0000 7f52dd35c700 1 -- 172.21.15.53:0/217361122 wait complete. 2023-11-06T01:33:53.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T01:33:53.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:53 smithi062 ceph-mon[103924]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-11-06T01:33:53.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:53 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1606232621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:53.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:53 smithi163 ceph-mon[104416]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-11-06T01:33:53.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:53 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1606232621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:53.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.773+0000 7fe78ab59700 1 -- 172.21.15.53:0/3536514959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7840fede0 msgr2=0x7fe7840ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.773+0000 7fe78ab59700 1 --2- 172.21.15.53:0/3536514959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7840fede0 0x7fe7840ff260 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe770009990 tx=0x7fe770009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.773+0000 7fe78ab59700 1 -- 172.21.15.53:0/3536514959 shutdown_connections 2023-11-06T01:33:53.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.773+0000 7fe78ab59700 1 --2- 172.21.15.53:0/3536514959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe784102370 0x7fe784102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.773+0000 7fe78ab59700 1 --2- 172.21.15.53:0/3536514959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7840ff7a0 0x7fe784109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.773+0000 7fe78ab59700 1 --2- 172.21.15.53:0/3536514959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7840fede0 0x7fe7840ff260 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.773+0000 7fe78ab59700 1 -- 172.21.15.53:0/3536514959 >> 172.21.15.53:0/3536514959 conn(0x7fe7840fa960 msgr2=0x7fe7840fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:53.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.773+0000 7fe78ab59700 1 -- 172.21.15.53:0/3536514959 shutdown_connections 2023-11-06T01:33:53.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.773+0000 7fe78ab59700 1 -- 172.21.15.53:0/3536514959 wait complete. 2023-11-06T01:33:53.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe78ab59700 1 Processor -- start 2023-11-06T01:33:53.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe78ab59700 1 -- start start 2023-11-06T01:33:53.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe78ab59700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7840fede0 0x7fe784196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe78ab59700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7840ff7a0 0x7fe784196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe78ab59700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe784102370 0x7fe78419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe78ab59700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78419e620 con 0x7fe7840fede0 2023-11-06T01:33:53.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe78ab59700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78419e760 con 0x7fe7840ff7a0 2023-11-06T01:33:53.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe78ab59700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78419e8f0 con 0x7fe784102370 2023-11-06T01:33:53.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe7888f5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7840fede0 0x7fe784196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:53.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe7888f5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7840fede0 0x7fe784196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39508/0 (socket says 172.21.15.53:39508) 2023-11-06T01:33:53.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe7888f5700 1 -- 172.21.15.53:0/1618155718 learned_addr learned my addr 172.21.15.53:0/1618155718 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:53.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe7890f6700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe784102370 0x7fe78419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:53.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe783fff700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7840ff7a0 0x7fe784196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:53.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe7888f5700 1 -- 172.21.15.53:0/1618155718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe784102370 msgr2=0x7fe78419e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe7888f5700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe784102370 0x7fe78419e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe7888f5700 1 -- 172.21.15.53:0/1618155718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7840ff7a0 msgr2=0x7fe784196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe7888f5700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7840ff7a0 0x7fe784196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe7888f5700 1 -- 172.21.15.53:0/1618155718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7700095f0 con 0x7fe7840fede0 2023-11-06T01:33:53.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe783fff700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7840ff7a0 0x7fe784196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:53.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.774+0000 7fe7888f5700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7840fede0 0x7fe784196aa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe77800d980 tx=0x7fe77800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:53.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.775+0000 7fe781ffb700 1 -- 172.21.15.53:0/1618155718 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77800b950 con 0x7fe7840fede0 2023-11-06T01:33:53.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.775+0000 7fe781ffb700 1 -- 172.21.15.53:0/1618155718 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe778007620 con 0x7fe7840fede0 2023-11-06T01:33:53.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.775+0000 7fe78ab59700 1 -- 172.21.15.53:0/1618155718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78419ebd0 con 0x7fe7840fede0 2023-11-06T01:33:53.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.775+0000 7fe781ffb700 1 -- 172.21.15.53:0/1618155718 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77800f3c0 con 0x7fe7840fede0 2023-11-06T01:33:53.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.775+0000 7fe78ab59700 1 -- 172.21.15.53:0/1618155718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78419f0e0 con 0x7fe7840fede0 2023-11-06T01:33:53.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.776+0000 7fe78ab59700 1 -- 172.21.15.53:0/1618155718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe784107ef0 con 0x7fe7840fede0 2023-11-06T01:33:53.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.777+0000 7fe781ffb700 1 -- 172.21.15.53:0/1618155718 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fe778010640 con 0x7fe7840fede0 2023-11-06T01:33:53.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.777+0000 7fe781ffb700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7740a05b0 0x7fe7740a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:53.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.778+0000 7fe783fff700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7740a05b0 0x7fe7740a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:53.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.778+0000 7fe781ffb700 1 -- 172.21.15.53:0/1618155718 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe7780cb970 con 0x7fe7840fede0 2023-11-06T01:33:53.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.778+0000 7fe783fff700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7740a05b0 0x7fe7740a2a70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe770006010 tx=0x7fe77000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:53.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.782+0000 7fe781ffb700 1 -- 172.21.15.53:0/1618155718 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe778095390 con 0x7fe7840fede0 2023-11-06T01:33:53.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.985+0000 7fe78ab59700 1 -- 172.21.15.53:0/1618155718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe78402cf20 con 0x7fe7840fede0 2023-11-06T01:33:53.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.985+0000 7fe781ffb700 1 -- 172.21.15.53:0/1618155718 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7780160a0 con 0x7fe7840fede0 2023-11-06T01:33:53.987 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-06T01:33:53.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 -- 172.21.15.53:0/1618155718 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7740a05b0 msgr2=0x7fe7740a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7740a05b0 0x7fe7740a2a70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe770006010 tx=0x7fe77000b540 comp rx=0 tx=0).stop 2023-11-06T01:33:53.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 -- 172.21.15.53:0/1618155718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7840fede0 msgr2=0x7fe784196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:53.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7840fede0 0x7fe784196aa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe77800d980 tx=0x7fe77800dc90 comp rx=0 tx=0).stop 2023-11-06T01:33:53.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 -- 172.21.15.53:0/1618155718 shutdown_connections 2023-11-06T01:33:53.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7740a05b0 0x7fe7740a2a70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe784102370 0x7fe78419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7840fede0 0x7fe784196aa0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 --2- 172.21.15.53:0/1618155718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7840ff7a0 0x7fe784196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:53.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 -- 172.21.15.53:0/1618155718 >> 172.21.15.53:0/1618155718 conn(0x7fe7840fa960 msgr2=0x7fe7840fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:53.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 -- 172.21.15.53:0/1618155718 shutdown_connections 2023-11-06T01:33:53.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:53.988+0000 7fe78ab59700 1 -- 172.21.15.53:0/1618155718 wait complete. 2023-11-06T01:33:54.002 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:33:54.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.129+0000 7effbf515700 1 -- 172.21.15.53:0/1555273611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb80ffc70 msgr2=0x7effb8100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.129+0000 7effbf515700 1 --2- 172.21.15.53:0/1555273611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb80ffc70 0x7effb8100090 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7effb40099c0 tx=0x7effb4009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.129+0000 7effbf515700 1 -- 172.21.15.53:0/1555273611 shutdown_connections 2023-11-06T01:33:54.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.129+0000 7effbf515700 1 --2- 172.21.15.53:0/1555273611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb8101800 0x7effb81080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.129+0000 7effbf515700 1 --2- 172.21.15.53:0/1555273611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb80ffc70 0x7effb8100090 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.129+0000 7effbf515700 1 --2- 172.21.15.53:0/1555273611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effb8100e60 0x7effb81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.129+0000 7effbf515700 1 -- 172.21.15.53:0/1555273611 >> 172.21.15.53:0/1555273611 conn(0x7effb80fb1c0 msgr2=0x7effb80fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:54.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.129+0000 7effbf515700 1 -- 172.21.15.53:0/1555273611 shutdown_connections 2023-11-06T01:33:54.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.129+0000 7effbf515700 1 -- 172.21.15.53:0/1555273611 wait complete. 2023-11-06T01:33:54.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.129+0000 7effbf515700 1 Processor -- start 2023-11-06T01:33:54.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.129+0000 7effbf515700 1 -- start start 2023-11-06T01:33:54.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbf515700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effb80ffc70 0x7effb8068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbf515700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb8100e60 0x7effb819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbf515700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb8101800 0x7effb819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbf515700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb819e540 con 0x7effb8100e60 2023-11-06T01:33:54.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbf515700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb819e6b0 con 0x7effb8101800 2023-11-06T01:33:54.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbf515700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb819e870 con 0x7effb80ffc70 2023-11-06T01:33:54.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbcab0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb8100e60 0x7effb819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbcab0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb8100e60 0x7effb819b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39532/0 (socket says 172.21.15.53:39532) 2023-11-06T01:33:54.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbcab0700 1 -- 172.21.15.53:0/3499172627 learned_addr learned my addr 172.21.15.53:0/3499172627 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:54.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbd2b1700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effb80ffc70 0x7effb8068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbdab2700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb8101800 0x7effb819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbcab0700 1 -- 172.21.15.53:0/3499172627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effb80ffc70 msgr2=0x7effb8068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbcab0700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effb80ffc70 0x7effb8068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbcab0700 1 -- 172.21.15.53:0/3499172627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb8101800 msgr2=0x7effb819dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbcab0700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb8101800 0x7effb819dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbcab0700 1 -- 172.21.15.53:0/3499172627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb40095f0 con 0x7effb8100e60 2023-11-06T01:33:54.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbcab0700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb8100e60 0x7effb819b760 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7effa800e9a0 tx=0x7effa800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:54.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effae7fc700 1 -- 172.21.15.53:0/3499172627 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8009790 con 0x7effb8100e60 2023-11-06T01:33:54.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbf515700 1 -- 172.21.15.53:0/3499172627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb819eb50 con 0x7effb8100e60 2023-11-06T01:33:54.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effae7fc700 1 -- 172.21.15.53:0/3499172627 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effa8010430 con 0x7effb8100e60 2023-11-06T01:33:54.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effae7fc700 1 -- 172.21.15.53:0/3499172627 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8018680 con 0x7effb8100e60 2023-11-06T01:33:54.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.130+0000 7effbf515700 1 -- 172.21.15.53:0/3499172627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb819f060 con 0x7effb8100e60 2023-11-06T01:33:54.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.131+0000 7effbf515700 1 -- 172.21.15.53:0/3499172627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb810b480 con 0x7effb8100e60 2023-11-06T01:33:54.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.132+0000 7effae7fc700 1 -- 172.21.15.53:0/3499172627 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7effa80105a0 con 0x7effb8100e60 2023-11-06T01:33:54.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.133+0000 7effae7fc700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effa40a05f0 0x7effa40a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.133+0000 7effbd2b1700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effa40a05f0 0x7effa40a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.133+0000 7effae7fc700 1 -- 172.21.15.53:0/3499172627 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7effa8014070 con 0x7effb8100e60 2023-11-06T01:33:54.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.134+0000 7effbd2b1700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effa40a05f0 0x7effa40a2ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7effb4009990 tx=0x7effb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:54.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.137+0000 7effae7fc700 1 -- 172.21.15.53:0/3499172627 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7effa80958f0 con 0x7effb8100e60 2023-11-06T01:33:54.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.340+0000 7effbf515700 1 -- 172.21.15.53:0/3499172627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effb8106b90 con 0x7effb8100e60 2023-11-06T01:33:54.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.342+0000 7effae7fc700 1 -- 172.21.15.53:0/3499172627 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22543 (secure 0 0 0) 0x7effa8026090 con 0x7effb8100e60 2023-11-06T01:33:54.344 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:54.344 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:33:27.113678+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:33:54.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 -- 172.21.15.53:0/3499172627 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effa40a05f0 msgr2=0x7effa40a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effa40a05f0 0x7effa40a2ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7effb4009990 tx=0x7effb4005fb0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 -- 172.21.15.53:0/3499172627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb8100e60 msgr2=0x7effb819b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb8100e60 0x7effb819b760 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7effa800e9a0 tx=0x7effa800ecb0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 -- 172.21.15.53:0/3499172627 shutdown_connections 2023-11-06T01:33:54.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7effa40a05f0 0x7effa40a2ab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb8101800 0x7effb819dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effb80ffc70 0x7effb8068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 --2- 172.21.15.53:0/3499172627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb8100e60 0x7effb819b760 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 -- 172.21.15.53:0/3499172627 >> 172.21.15.53:0/3499172627 conn(0x7effb80fb1c0 msgr2=0x7effb80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:54.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 -- 172.21.15.53:0/3499172627 shutdown_connections 2023-11-06T01:33:54.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.344+0000 7effbf515700 1 -- 172.21.15.53:0/3499172627 wait complete. 2023-11-06T01:33:54.359 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T01:32:24.604601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T01:32:25.457741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:33:54.359 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T01:33:54.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.487+0000 7fe7d17c0700 1 -- 172.21.15.53:0/3709894210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7cc100e70 msgr2=0x7fe7cc1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.487+0000 7fe7d17c0700 1 --2- 172.21.15.53:0/3709894210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7cc100e70 0x7fe7cc1012d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0099c0 tx=0x7fe7bc009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.487+0000 7fe7d17c0700 1 -- 172.21.15.53:0/3709894210 shutdown_connections 2023-11-06T01:33:54.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.487+0000 7fe7d17c0700 1 --2- 172.21.15.53:0/3709894210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7cc101810 0x7fe7cc1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.487+0000 7fe7d17c0700 1 --2- 172.21.15.53:0/3709894210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7cc0ffc80 0x7fe7cc1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.487+0000 7fe7d17c0700 1 --2- 172.21.15.53:0/3709894210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7cc100e70 0x7fe7cc1012d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.487+0000 7fe7d17c0700 1 -- 172.21.15.53:0/3709894210 >> 172.21.15.53:0/3709894210 conn(0x7fe7cc0fb1f0 msgr2=0x7fe7cc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.487+0000 7fe7d17c0700 1 -- 172.21.15.53:0/3709894210 shutdown_connections 2023-11-06T01:33:54.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.487+0000 7fe7d17c0700 1 -- 172.21.15.53:0/3709894210 wait complete. 2023-11-06T01:33:54.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7d17c0700 1 Processor -- start 2023-11-06T01:33:54.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7d17c0700 1 -- start start 2023-11-06T01:33:54.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7d17c0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7cc0ffc80 0x7fe7cc194870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7d17c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7cc100e70 0x7fe7cc194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7d17c0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7cc101810 0x7fe7cc19be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7d17c0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc19c3f0 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7d17c0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc19c530 con 0x7fe7cc101810 2023-11-06T01:33:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7d17c0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc19c6c0 con 0x7fe7cc100e70 2023-11-06T01:33:54.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7caffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7cc0ffc80 0x7fe7cc194870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7caffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7cc0ffc80 0x7fe7cc194870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39544/0 (socket says 172.21.15.53:39544) 2023-11-06T01:33:54.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7caffd700 1 -- 172.21.15.53:0/1266418992 learned_addr learned my addr 172.21.15.53:0/1266418992 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:54.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7cb7fe700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7cc101810 0x7fe7cc19be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7ca7fc700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7cc100e70 0x7fe7cc194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7caffd700 1 -- 172.21.15.53:0/1266418992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7cc100e70 msgr2=0x7fe7cc194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7caffd700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7cc100e70 0x7fe7cc194db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7caffd700 1 -- 172.21.15.53:0/1266418992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7cc101810 msgr2=0x7fe7cc19be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7caffd700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7cc101810 0x7fe7cc19be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.488+0000 7fe7caffd700 1 -- 172.21.15.53:0/1266418992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc0095f0 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.489+0000 7fe7caffd700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7cc0ffc80 0x7fe7cc194870 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400a9b0 tx=0x7fe7b400acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:54.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.489+0000 7fe7c3fff700 1 -- 172.21.15.53:0/1266418992 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4010070 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.489+0000 7fe7c3fff700 1 -- 172.21.15.53:0/1266418992 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7b4007d20 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.489+0000 7fe7d17c0700 1 -- 172.21.15.53:0/1266418992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc19c9a0 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.489+0000 7fe7c3fff700 1 -- 172.21.15.53:0/1266418992 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4014d70 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.489+0000 7fe7d17c0700 1 -- 172.21.15.53:0/1266418992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc19ceb0 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.490+0000 7fe7d17c0700 1 -- 172.21.15.53:0/1266418992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc106c00 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.491+0000 7fe7c3fff700 1 -- 172.21.15.53:0/1266418992 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fe7b4014410 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.491+0000 7fe7c3fff700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7b80a0630 0x7fe7b80a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.491+0000 7fe7c3fff700 1 -- 172.21.15.53:0/1266418992 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe7b40cafc0 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.492+0000 7fe7ca7fc700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7b80a0630 0x7fe7b80a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.492+0000 7fe7ca7fc700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7b80a0630 0x7fe7b80a2af0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00b580 tx=0x7fe7bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.496+0000 7fe7c3fff700 1 -- 172.21.15.53:0/1266418992 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe7b4094a80 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:54 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2250518868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:54.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:54 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/217361122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:54.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:54 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1618155718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:54.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:54 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2250518868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:54.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:54 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/217361122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:54.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:54 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1618155718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:54.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.696+0000 7fe7d17c0700 1 -- 172.21.15.53:0/1266418992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7cc02ceb0 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.697+0000 7fe7c3fff700 1 -- 172.21.15.53:0/1266418992 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7b40980a0 con 0x7fe7cc0ffc80 2023-11-06T01:33:54.699 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:33:54.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 -- 172.21.15.53:0/1266418992 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7b80a0630 msgr2=0x7fe7b80a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7b80a0630 0x7fe7b80a2af0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00b580 tx=0x7fe7bc01a040 comp rx=0 tx=0).stop 2023-11-06T01:33:54.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 -- 172.21.15.53:0/1266418992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7cc0ffc80 msgr2=0x7fe7cc194870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7cc0ffc80 0x7fe7cc194870 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400a9b0 tx=0x7fe7b400acc0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 -- 172.21.15.53:0/1266418992 shutdown_connections 2023-11-06T01:33:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7b80a0630 0x7fe7b80a2af0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7cc101810 0x7fe7cc19be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7cc0ffc80 0x7fe7cc194870 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 --2- 172.21.15.53:0/1266418992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7cc100e70 0x7fe7cc194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 -- 172.21.15.53:0/1266418992 >> 172.21.15.53:0/1266418992 conn(0x7fe7cc0fb1f0 msgr2=0x7fe7cc0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 -- 172.21.15.53:0/1266418992 shutdown_connections 2023-11-06T01:33:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.699+0000 7fe7d17c0700 1 -- 172.21.15.53:0/1266418992 wait complete. 2023-11-06T01:33:54.713 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T01:33:54.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:54 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2250518868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:54.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:54 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/217361122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:54.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:54 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1618155718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:54.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.841+0000 7fc18993f700 1 -- 172.21.15.53:0/1476783993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc184101870 msgr2=0x7fc184108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.841+0000 7fc18993f700 1 --2- 172.21.15.53:0/1476783993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc184101870 0x7fc184108120 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00b0d0 tx=0x7fc17c00b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.841+0000 7fc18993f700 1 -- 172.21.15.53:0/1476783993 shutdown_connections 2023-11-06T01:33:54.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.841+0000 7fc18993f700 1 --2- 172.21.15.53:0/1476783993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1840ffce0 0x7fc184100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.841+0000 7fc18993f700 1 --2- 172.21.15.53:0/1476783993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc184101870 0x7fc184108120 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.841+0000 7fc18993f700 1 --2- 172.21.15.53:0/1476783993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc184100ed0 0x7fc184101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.841+0000 7fc18993f700 1 -- 172.21.15.53:0/1476783993 >> 172.21.15.53:0/1476783993 conn(0x7fc1840fb250 msgr2=0x7fc1840fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:54.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.841+0000 7fc18993f700 1 -- 172.21.15.53:0/1476783993 shutdown_connections 2023-11-06T01:33:54.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.841+0000 7fc18993f700 1 -- 172.21.15.53:0/1476783993 wait complete. 2023-11-06T01:33:54.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.841+0000 7fc18993f700 1 Processor -- start 2023-11-06T01:33:54.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc18993f700 1 -- start start 2023-11-06T01:33:54.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc18993f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1840ffce0 0x7fc184069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc18993f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc184100ed0 0x7fc18419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc18993f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc184101870 0x7fc18419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc18993f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18419e670 con 0x7fc184100ed0 2023-11-06T01:33:54.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc18993f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18419e7e0 con 0x7fc184101870 2023-11-06T01:33:54.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc18993f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18419e9a0 con 0x7fc1840ffce0 2023-11-06T01:33:54.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc1827fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc184100ed0 0x7fc18419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc1827fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc184100ed0 0x7fc18419b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39560/0 (socket says 172.21.15.53:39560) 2023-11-06T01:33:54.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc1827fc700 1 -- 172.21.15.53:0/2716047519 learned_addr learned my addr 172.21.15.53:0/2716047519 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:54.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc182ffd700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1840ffce0 0x7fc184069cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc1837fe700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc184101870 0x7fc18419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc1827fc700 1 -- 172.21.15.53:0/2716047519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1840ffce0 msgr2=0x7fc184069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc1827fc700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1840ffce0 0x7fc184069cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc1827fc700 1 -- 172.21.15.53:0/2716047519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc184101870 msgr2=0x7fc18419ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:54.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc1827fc700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc184101870 0x7fc18419ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:54.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc1827fc700 1 -- 172.21.15.53:0/2716047519 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c009d00 con 0x7fc184100ed0 2023-11-06T01:33:54.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.842+0000 7fc1827fc700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc184100ed0 0x7fc18419b890 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc17800e9a0 tx=0x7fc17800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:54.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.843+0000 7fc18893d700 1 -- 172.21.15.53:0/2716047519 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178009790 con 0x7fc184100ed0 2023-11-06T01:33:54.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.843+0000 7fc18893d700 1 -- 172.21.15.53:0/2716047519 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc178010430 con 0x7fc184100ed0 2023-11-06T01:33:54.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.843+0000 7fc18993f700 1 -- 172.21.15.53:0/2716047519 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18419ec80 con 0x7fc184100ed0 2023-11-06T01:33:54.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.843+0000 7fc18893d700 1 -- 172.21.15.53:0/2716047519 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178018680 con 0x7fc184100ed0 2023-11-06T01:33:54.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.843+0000 7fc18993f700 1 -- 172.21.15.53:0/2716047519 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18419f190 con 0x7fc184100ed0 2023-11-06T01:33:54.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.844+0000 7fc18993f700 1 -- 172.21.15.53:0/2716047519 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc184107250 con 0x7fc184100ed0 2023-11-06T01:33:54.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.845+0000 7fc18893d700 1 -- 172.21.15.53:0/2716047519 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc1780105a0 con 0x7fc184100ed0 2023-11-06T01:33:54.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.846+0000 7fc18893d700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc16c0a05b0 0x7fc16c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:54.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.846+0000 7fc182ffd700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc16c0a05b0 0x7fc16c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:54.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.846+0000 7fc18893d700 1 -- 172.21.15.53:0/2716047519 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc178014070 con 0x7fc184100ed0 2023-11-06T01:33:54.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.846+0000 7fc182ffd700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc16c0a05b0 0x7fc16c0a2a70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc184100d30 tx=0x7fc174009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:54.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:54.849+0000 7fc18893d700 1 -- 172.21.15.53:0/2716047519 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc1780958f0 con 0x7fc184100ed0 2023-11-06T01:33:55.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.052+0000 7fc18993f700 1 -- 172.21.15.53:0/2716047519 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc18402ce80 con 0x7fc184100ed0 2023-11-06T01:33:55.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.052+0000 7fc18893d700 1 -- 172.21.15.53:0/2716047519 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc178098f10 con 0x7fc184100ed0 2023-11-06T01:33:55.054 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-06T01:33:55.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.054+0000 7fc18993f700 1 -- 172.21.15.53:0/2716047519 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc16c0a05b0 msgr2=0x7fc16c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.054+0000 7fc18993f700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc16c0a05b0 0x7fc16c0a2a70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc184100d30 tx=0x7fc174009260 comp rx=0 tx=0).stop 2023-11-06T01:33:55.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.054+0000 7fc18993f700 1 -- 172.21.15.53:0/2716047519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc184100ed0 msgr2=0x7fc18419b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.054+0000 7fc18993f700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc184100ed0 0x7fc18419b890 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc17800e9a0 tx=0x7fc17800ecb0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.054+0000 7fc18993f700 1 -- 172.21.15.53:0/2716047519 shutdown_connections 2023-11-06T01:33:55.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.054+0000 7fc18993f700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc16c0a05b0 0x7fc16c0a2a70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.054+0000 7fc18993f700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc184101870 0x7fc18419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.054+0000 7fc18993f700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1840ffce0 0x7fc184069cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.054+0000 7fc18993f700 1 --2- 172.21.15.53:0/2716047519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc184100ed0 0x7fc18419b890 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.054+0000 7fc18993f700 1 -- 172.21.15.53:0/2716047519 >> 172.21.15.53:0/2716047519 conn(0x7fc1840fb250 msgr2=0x7fc184104390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:55.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.054+0000 7fc18993f700 1 -- 172.21.15.53:0/2716047519 shutdown_connections 2023-11-06T01:33:55.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.055+0000 7fc18993f700 1 -- 172.21.15.53:0/2716047519 wait complete. 2023-11-06T01:33:55.068 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T01:33:55.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.197+0000 7fb64878a700 1 -- 172.21.15.53:0/1594418286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb640100e20 msgr2=0x7fb640101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.197+0000 7fb64878a700 1 --2- 172.21.15.53:0/1594418286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb640100e20 0x7fb640101280 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb6300099c0 tx=0x7fb630009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.197+0000 7fb64878a700 1 -- 172.21.15.53:0/1594418286 shutdown_connections 2023-11-06T01:33:55.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.197+0000 7fb64878a700 1 --2- 172.21.15.53:0/1594418286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6400ffc30 0x7fb640100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.197+0000 7fb64878a700 1 --2- 172.21.15.53:0/1594418286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6401017c0 0x7fb640108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.197+0000 7fb64878a700 1 --2- 172.21.15.53:0/1594418286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb640100e20 0x7fb640101280 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.197+0000 7fb64878a700 1 -- 172.21.15.53:0/1594418286 >> 172.21.15.53:0/1594418286 conn(0x7fb6400fb1a0 msgr2=0x7fb6400fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.197+0000 7fb64878a700 1 -- 172.21.15.53:0/1594418286 shutdown_connections 2023-11-06T01:33:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.197+0000 7fb64878a700 1 -- 172.21.15.53:0/1594418286 wait complete. 2023-11-06T01:33:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.197+0000 7fb64878a700 1 Processor -- start 2023-11-06T01:33:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.197+0000 7fb64878a700 1 -- start start 2023-11-06T01:33:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb64878a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6400ffc30 0x7fb640103e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb64878a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb640100e20 0x7fb6401024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb64878a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6401017c0 0x7fb640102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb64878a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6401031a0 con 0x7fb640100e20 2023-11-06T01:33:55.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb64878a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb640103310 con 0x7fb6401017c0 2023-11-06T01:33:55.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb64878a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6401a0a00 con 0x7fb6400ffc30 2023-11-06T01:33:55.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb645d25700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb640100e20 0x7fb6401024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb645d25700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb640100e20 0x7fb6401024c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39574/0 (socket says 172.21.15.53:39574) 2023-11-06T01:33:55.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb645d25700 1 -- 172.21.15.53:0/3092090133 learned_addr learned my addr 172.21.15.53:0/3092090133 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:55.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb646d27700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6401017c0 0x7fb640102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb646526700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6400ffc30 0x7fb640103e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb645d25700 1 -- 172.21.15.53:0/3092090133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6400ffc30 msgr2=0x7fb640103e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb645d25700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6400ffc30 0x7fb640103e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb645d25700 1 -- 172.21.15.53:0/3092090133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6401017c0 msgr2=0x7fb640102a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb645d25700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6401017c0 0x7fb640102a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb645d25700 1 -- 172.21.15.53:0/3092090133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6300095f0 con 0x7fb640100e20 2023-11-06T01:33:55.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb646526700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6400ffc30 0x7fb640103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:55.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb645d25700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb640100e20 0x7fb6401024c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c010 tx=0x7fb63000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:55.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb6377fe700 1 -- 172.21.15.53:0/3092090133 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63001d070 con 0x7fb640100e20 2023-11-06T01:33:55.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb6377fe700 1 -- 172.21.15.53:0/3092090133 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb63000fb80 con 0x7fb640100e20 2023-11-06T01:33:55.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb64878a700 1 -- 172.21.15.53:0/3092090133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6401a0c80 con 0x7fb640100e20 2023-11-06T01:33:55.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.198+0000 7fb6377fe700 1 -- 172.21.15.53:0/3092090133 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6300173f0 con 0x7fb640100e20 2023-11-06T01:33:55.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.199+0000 7fb64878a700 1 -- 172.21.15.53:0/3092090133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6401a10a0 con 0x7fb640100e20 2023-11-06T01:33:55.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.200+0000 7fb64878a700 1 -- 172.21.15.53:0/3092090133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb640107cc0 con 0x7fb640100e20 2023-11-06T01:33:55.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.201+0000 7fb6377fe700 1 -- 172.21.15.53:0/3092090133 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fb6300050c0 con 0x7fb640100e20 2023-11-06T01:33:55.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.201+0000 7fb6377fe700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb62c0a05b0 0x7fb62c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.201+0000 7fb6377fe700 1 -- 172.21.15.53:0/3092090133 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb6300cce50 con 0x7fb640100e20 2023-11-06T01:33:55.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.201+0000 7fb646526700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb62c0a05b0 0x7fb62c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.202+0000 7fb646526700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb62c0a05b0 0x7fb62c0a2a70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb640100c80 tx=0x7fb63c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:55.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.205+0000 7fb6377fe700 1 -- 172.21.15.53:0/3092090133 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb63005d0e0 con 0x7fb640100e20 2023-11-06T01:33:55.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:55 smithi053 ceph-mon[99472]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:55.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:55 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3499172627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:55.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:55 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1266418992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:55.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:55 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2716047519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:55.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.411+0000 7fb64878a700 1 -- 172.21.15.53:0/3092090133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb64010b4b0 con 0x7fb640100e20 2023-11-06T01:33:55.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.412+0000 7fb6377fe700 1 -- 172.21.15.53:0/3092090133 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb630096900 con 0x7fb640100e20 2023-11-06T01:33:55.414 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-06T01:33:55.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 -- 172.21.15.53:0/3092090133 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb62c0a05b0 msgr2=0x7fb62c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb62c0a05b0 0x7fb62c0a2a70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb640100c80 tx=0x7fb63c009260 comp rx=0 tx=0).stop 2023-11-06T01:33:55.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 -- 172.21.15.53:0/3092090133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb640100e20 msgr2=0x7fb6401024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb640100e20 0x7fb6401024c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c010 tx=0x7fb63000f690 comp rx=0 tx=0).stop 2023-11-06T01:33:55.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 -- 172.21.15.53:0/3092090133 shutdown_connections 2023-11-06T01:33:55.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb62c0a05b0 0x7fb62c0a2a70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6401017c0 0x7fb640102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6400ffc30 0x7fb640103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 --2- 172.21.15.53:0/3092090133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb640100e20 0x7fb6401024c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 -- 172.21.15.53:0/3092090133 >> 172.21.15.53:0/3092090133 conn(0x7fb6400fb1a0 msgr2=0x7fb6400fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:55.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 -- 172.21.15.53:0/3092090133 shutdown_connections 2023-11-06T01:33:55.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.414+0000 7fb64878a700 1 -- 172.21.15.53:0/3092090133 wait complete. 2023-11-06T01:33:55.428 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-11-06T01:33:55.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:33:55.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.574+0000 7fcbc16c9700 1 -- 172.21.15.53:0/3719926551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbbc106b40 msgr2=0x7fcbbc108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.574+0000 7fcbc16c9700 1 --2- 172.21.15.53:0/3719926551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbbc106b40 0x7fcbbc108f30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400b0a0 tx=0x7fcbb400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.574+0000 7fcbc16c9700 1 -- 172.21.15.53:0/3719926551 shutdown_connections 2023-11-06T01:33:55.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.574+0000 7fcbc16c9700 1 --2- 172.21.15.53:0/3719926551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbbc106b40 0x7fcbbc108f30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.574+0000 7fcbc16c9700 1 --2- 172.21.15.53:0/3719926551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbbc1018e0 0x7fcbbc103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.574+0000 7fcbc16c9700 1 --2- 172.21.15.53:0/3719926551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc104210 0x7fcbbc106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.574+0000 7fcbc16c9700 1 -- 172.21.15.53:0/3719926551 >> 172.21.15.53:0/3719926551 conn(0x7fcbbc0fb250 msgr2=0x7fcbbc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:55.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.574+0000 7fcbc16c9700 1 -- 172.21.15.53:0/3719926551 shutdown_connections 2023-11-06T01:33:55.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.574+0000 7fcbc16c9700 1 -- 172.21.15.53:0/3719926551 wait complete. 2023-11-06T01:33:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.574+0000 7fcbc16c9700 1 Processor -- start 2023-11-06T01:33:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbc16c9700 1 -- start start 2023-11-06T01:33:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbc16c9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc1018e0 0x7fcbbc196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbc16c9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbbc104210 0x7fcbbc196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbc16c9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbbc106b40 0x7fcbbc19e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbc16c9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc19e610 con 0x7fcbbc104210 2023-11-06T01:33:55.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbc16c9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc19e750 con 0x7fcbbc1018e0 2023-11-06T01:33:55.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbc16c9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc19e8e0 con 0x7fcbbc106b40 2023-11-06T01:33:55.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbba7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbbc104210 0x7fcbbc196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbba7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbbc104210 0x7fcbbc196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39588/0 (socket says 172.21.15.53:39588) 2023-11-06T01:33:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbba7fc700 1 -- 172.21.15.53:0/140383426 learned_addr learned my addr 172.21.15.53:0/140383426 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbbaffd700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc1018e0 0x7fcbbc196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbbb7fe700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbbc106b40 0x7fcbbc19e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbba7fc700 1 -- 172.21.15.53:0/140383426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbbc106b40 msgr2=0x7fcbbc19e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbba7fc700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbbc106b40 0x7fcbbc19e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbba7fc700 1 -- 172.21.15.53:0/140383426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc1018e0 msgr2=0x7fcbbc196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbba7fc700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc1018e0 0x7fcbbc196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbba7fc700 1 -- 172.21.15.53:0/140383426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb4009d00 con 0x7fcbbc104210 2023-11-06T01:33:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbbb7fe700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbbc106b40 0x7fcbbc19e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.575+0000 7fcbba7fc700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbbc104210 0x7fcbbc196fd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00d980 tx=0x7fcbac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.576+0000 7fcba3fff700 1 -- 172.21.15.53:0/140383426 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac004220 con 0x7fcbbc104210 2023-11-06T01:33:55.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.576+0000 7fcbc16c9700 1 -- 172.21.15.53:0/140383426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbbc19ebc0 con 0x7fcbbc104210 2023-11-06T01:33:55.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.576+0000 7fcba3fff700 1 -- 172.21.15.53:0/140383426 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbac007640 con 0x7fcbbc104210 2023-11-06T01:33:55.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.576+0000 7fcba3fff700 1 -- 172.21.15.53:0/140383426 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac005610 con 0x7fcbbc104210 2023-11-06T01:33:55.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.576+0000 7fcbc16c9700 1 -- 172.21.15.53:0/140383426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc19f0d0 con 0x7fcbbc104210 2023-11-06T01:33:55.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.577+0000 7fcbc16c9700 1 -- 172.21.15.53:0/140383426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbbc0fcdd0 con 0x7fcbbc104210 2023-11-06T01:33:55.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.578+0000 7fcba3fff700 1 -- 172.21.15.53:0/140383426 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fcbac004750 con 0x7fcbbc104210 2023-11-06T01:33:55.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.578+0000 7fcba3fff700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcba40a05f0 0x7fcba40a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.578+0000 7fcba3fff700 1 -- 172.21.15.53:0/140383426 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcbac0cbe30 con 0x7fcbbc104210 2023-11-06T01:33:55.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.578+0000 7fcbbaffd700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcba40a05f0 0x7fcba40a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.579+0000 7fcbbaffd700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcba40a05f0 0x7fcba40a2ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0005950 tx=0x7fcbb000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:55.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.582+0000 7fcba3fff700 1 -- 172.21.15.53:0/140383426 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcbac095850 con 0x7fcbbc104210 2023-11-06T01:33:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:55 smithi062 ceph-mon[103924]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:55 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3499172627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:55 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1266418992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:55 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2716047519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:55.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:55 smithi163 ceph-mon[104416]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-11-06T01:33:55.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:55 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3499172627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:55.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:55 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1266418992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:55.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:55 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2716047519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:55.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.835+0000 7fcbc16c9700 1 -- 172.21.15.53:0/140383426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbbc02cec0 con 0x7fcbbc104210 2023-11-06T01:33:55.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.835+0000 7fcba3fff700 1 -- 172.21.15.53:0/140383426 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10815 (secure 0 0 0) 0x7fcbac098e70 con 0x7fcbbc104210 2023-11-06T01:33:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:55.838 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-06T01:32:26.093194+0000","modified":"2023-11-06T01:32:33.006369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24472,"mds_2":24464,"mds_3":24422,"mds_4":24395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24395":{"gid":24395,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T01:33:55.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.837+0000 7fcbc16c9700 1 -- 172.21.15.53:0/140383426 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcba40a05f0 msgr2=0x7fcba40a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.837+0000 7fcbc16c9700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcba40a05f0 0x7fcba40a2ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0005950 tx=0x7fcbb000b410 comp rx=0 tx=0).stop 2023-11-06T01:33:55.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.837+0000 7fcbc16c9700 1 -- 172.21.15.53:0/140383426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbbc104210 msgr2=0x7fcbbc196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.837+0000 7fcbc16c9700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbbc104210 0x7fcbbc196fd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00d980 tx=0x7fcbac00dc90 comp rx=0 tx=0).stop 2023-11-06T01:33:55.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.837+0000 7fcbc16c9700 1 -- 172.21.15.53:0/140383426 shutdown_connections 2023-11-06T01:33:55.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.837+0000 7fcbc16c9700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcba40a05f0 0x7fcba40a2ab0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.837+0000 7fcbc16c9700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc1018e0 0x7fcbbc196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.837+0000 7fcbc16c9700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbbc106b40 0x7fcbbc19e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.837+0000 7fcbc16c9700 1 --2- 172.21.15.53:0/140383426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbbc104210 0x7fcbbc196fd0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.837+0000 7fcbc16c9700 1 -- 172.21.15.53:0/140383426 >> 172.21.15.53:0/140383426 conn(0x7fcbbc0fb250 msgr2=0x7fcbbc102550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:55.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.838+0000 7fcbc16c9700 1 -- 172.21.15.53:0/140383426 shutdown_connections 2023-11-06T01:33:55.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.838+0000 7fcbc16c9700 1 -- 172.21.15.53:0/140383426 wait complete. 2023-11-06T01:33:55.848 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-11-06T01:33:55.852 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-11-06T01:33:55.853 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-11-06T01:33:55.853 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T01:33:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.982+0000 7f431ce18700 1 -- 172.21.15.53:0/310920527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4318101800 msgr2=0x7f43181080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.982+0000 7f431ce18700 1 --2- 172.21.15.53:0/310920527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4318101800 0x7f43181080b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f431000b0d0 tx=0x7f431000b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.983+0000 7f431ce18700 1 -- 172.21.15.53:0/310920527 shutdown_connections 2023-11-06T01:33:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.983+0000 7f431ce18700 1 --2- 172.21.15.53:0/310920527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43180ffc70 0x7f4318100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.983+0000 7f431ce18700 1 --2- 172.21.15.53:0/310920527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4318101800 0x7f43181080b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.983+0000 7f431ce18700 1 --2- 172.21.15.53:0/310920527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4318100e60 0x7f43181012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.983+0000 7f431ce18700 1 -- 172.21.15.53:0/310920527 >> 172.21.15.53:0/310920527 conn(0x7f43180fb1c0 msgr2=0x7f43180fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.983+0000 7f431ce18700 1 -- 172.21.15.53:0/310920527 shutdown_connections 2023-11-06T01:33:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.983+0000 7f431ce18700 1 -- 172.21.15.53:0/310920527 wait complete. 2023-11-06T01:33:55.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.983+0000 7f431ce18700 1 Processor -- start 2023-11-06T01:33:55.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.983+0000 7f431ce18700 1 -- start start 2023-11-06T01:33:55.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f431ce18700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43180ffc70 0x7f43181959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f431ce18700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4318100e60 0x7f4318195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f431ce18700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4318101800 0x7f431819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f431ce18700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431819d550 con 0x7f4318100e60 2023-11-06T01:33:55.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f431ce18700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431819d690 con 0x7f4318101800 2023-11-06T01:33:55.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f431ce18700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431819d820 con 0x7f43180ffc70 2023-11-06T01:33:55.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f4315d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4318100e60 0x7f4318195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f4315d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4318100e60 0x7f4318195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39600/0 (socket says 172.21.15.53:39600) 2023-11-06T01:33:55.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f4315d9b700 1 -- 172.21.15.53:0/3584635513 learned_addr learned my addr 172.21.15.53:0/3584635513 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:55.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f4316d9d700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4318101800 0x7f431819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f431659c700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43180ffc70 0x7f43181959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f4315d9b700 1 -- 172.21.15.53:0/3584635513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43180ffc70 msgr2=0x7f43181959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f4315d9b700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43180ffc70 0x7f43181959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f4315d9b700 1 -- 172.21.15.53:0/3584635513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4318101800 msgr2=0x7f431819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:55.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f4315d9b700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4318101800 0x7f431819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f4315d9b700 1 -- 172.21.15.53:0/3584635513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310009d00 con 0x7f4318100e60 2023-11-06T01:33:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f4315d9b700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4318100e60 0x7f4318195f10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c9b0 tx=0x7f430c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f43077fe700 1 -- 172.21.15.53:0/3584635513 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c007b00 con 0x7f4318100e60 2023-11-06T01:33:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f43077fe700 1 -- 172.21.15.53:0/3584635513 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f430c007c60 con 0x7f4318100e60 2023-11-06T01:33:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f431ce18700 1 -- 172.21.15.53:0/3584635513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431819db00 con 0x7f4318100e60 2023-11-06T01:33:55.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f43077fe700 1 -- 172.21.15.53:0/3584635513 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c018660 con 0x7f4318100e60 2023-11-06T01:33:55.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.984+0000 7f431ce18700 1 -- 172.21.15.53:0/3584635513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431819e010 con 0x7f4318100e60 2023-11-06T01:33:55.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.985+0000 7f431ce18700 1 -- 172.21.15.53:0/3584635513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431810b4e0 con 0x7f4318100e60 2023-11-06T01:33:55.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.987+0000 7f43077fe700 1 -- 172.21.15.53:0/3584635513 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f430c00f450 con 0x7f4318100e60 2023-11-06T01:33:55.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.987+0000 7f43077fe700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43000a05f0 0x7f43000a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:55.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.987+0000 7f431659c700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43000a05f0 0x7f43000a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:55.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.987+0000 7f43077fe700 1 -- 172.21.15.53:0/3584635513 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f430c0cbae0 con 0x7f4318100e60 2023-11-06T01:33:55.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.988+0000 7f431659c700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43000a05f0 0x7f43000a2ab0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4308005950 tx=0x7f43080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:55.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:55.992+0000 7f43077fe700 1 -- 172.21.15.53:0/3584635513 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f430c095500 con 0x7f4318100e60 2023-11-06T01:33:56.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.198+0000 7f431ce18700 1 -- 172.21.15.53:0/3584635513 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43180611d0 con 0x7f43000a05f0 2023-11-06T01:33:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.210+0000 7f43077fe700 1 -- 172.21.15.53:0/3584635513 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343607 (secure 0 0 0) 0x7f43180611d0 con 0x7f43000a05f0 2023-11-06T01:33:56.212 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:56.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 -- 172.21.15.53:0/3584635513 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43000a05f0 msgr2=0x7f43000a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:56.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43000a05f0 0x7f43000a2ab0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4308005950 tx=0x7f43080058e0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 -- 172.21.15.53:0/3584635513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4318100e60 msgr2=0x7f4318195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:56.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4318100e60 0x7f4318195f10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c9b0 tx=0x7f430c00ccc0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 -- 172.21.15.53:0/3584635513 shutdown_connections 2023-11-06T01:33:56.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43000a05f0 0x7f43000a2ab0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4318101800 0x7f431819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43180ffc70 0x7f43181959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 --2- 172.21.15.53:0/3584635513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4318100e60 0x7f4318195f10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 -- 172.21.15.53:0/3584635513 >> 172.21.15.53:0/3584635513 conn(0x7f43180fb1c0 msgr2=0x7f43180fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:56.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 -- 172.21.15.53:0/3584635513 shutdown_connections 2023-11-06T01:33:56.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.214+0000 7f431ce18700 1 -- 172.21.15.53:0/3584635513 wait complete. 2023-11-06T01:33:56.225 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-06T01:33:56.229 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-11-06T01:33:55.498729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656232,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":981,"num_read_kb":605,"num_write":415,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1182566,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334636,"kb_used_data":11212,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607188,"statfs":{"total":1151940427776,"available":1151597760512,"internally_reserved":0,"allocated":11481088,"data_stored":7182927,"data_compressed":177927,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4956,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":72,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003045"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121147+0000","last_change":"2023-11-06T01:32:26.110793+0000","last_active":"2023-11-06T01:33:26.121147+0000","last_peered":"2023-11-06T01:33:26.121147+0000","last_clean":"2023-11-06T01:33:26.121147+0000","last_became_active":"2023-11-06T01:32:26.110489+0000","last_became_peered":"2023-11-06T01:32:26.110489+0000","last_unstale":"2023-11-06T01:33:26.121147+0000","last_undegraded":"2023-11-06T01:33:26.121147+0000","last_fullsized":"2023-11-06T01:33:26.121147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:01:08.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122964+0000","last_change":"2023-11-06T01:32:26.107273+0000","last_active":"2023-11-06T01:33:26.122964+0000","last_peered":"2023-11-06T01:33:26.122964+0000","last_clean":"2023-11-06T01:33:26.122964+0000","last_became_active":"2023-11-06T01:32:26.106961+0000","last_became_peered":"2023-11-06T01:32:26.106961+0000","last_unstale":"2023-11-06T01:33:26.122964+0000","last_undegraded":"2023-11-06T01:33:26.122964+0000","last_fullsized":"2023-11-06T01:33:26.122964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:40:39.073926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122761+0000","last_change":"2023-11-06T01:32:26.121316+0000","last_active":"2023-11-06T01:33:26.122761+0000","last_peered":"2023-11-06T01:33:26.122761+0000","last_clean":"2023-11-06T01:33:26.122761+0000","last_became_active":"2023-11-06T01:32:26.120756+0000","last_became_peered":"2023-11-06T01:32:26.120756+0000","last_unstale":"2023-11-06T01:33:26.122761+0000","last_undegraded":"2023-11-06T01:33:26.122761+0000","last_fullsized":"2023-11-06T01:33:26.122761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:02:45.814577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120866+0000","last_change":"2023-11-06T01:32:26.121951+0000","last_active":"2023-11-06T01:33:26.120866+0000","last_peered":"2023-11-06T01:33:26.120866+0000","last_clean":"2023-11-06T01:33:26.120866+0000","last_became_active":"2023-11-06T01:32:26.121636+0000","last_became_peered":"2023-11-06T01:32:26.121636+0000","last_unstale":"2023-11-06T01:33:26.120866+0000","last_undegraded":"2023-11-06T01:33:26.120866+0000","last_fullsized":"2023-11-06T01:33:26.120866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:02.381355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120004+0000","last_change":"2023-11-06T01:32:26.117788+0000","last_active":"2023-11-06T01:33:26.120004+0000","last_peered":"2023-11-06T01:33:26.120004+0000","last_clean":"2023-11-06T01:33:26.120004+0000","last_became_active":"2023-11-06T01:32:26.117161+0000","last_became_peered":"2023-11-06T01:32:26.117161+0000","last_unstale":"2023-11-06T01:33:26.120004+0000","last_undegraded":"2023-11-06T01:33:26.120004+0000","last_fullsized":"2023-11-06T01:33:26.120004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:31.665690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123916+0000","last_change":"2023-11-06T01:32:26.109532+0000","last_active":"2023-11-06T01:33:26.123916+0000","last_peered":"2023-11-06T01:33:26.123916+0000","last_clean":"2023-11-06T01:33:26.123916+0000","last_became_active":"2023-11-06T01:32:26.106144+0000","last_became_peered":"2023-11-06T01:32:26.106144+0000","last_unstale":"2023-11-06T01:33:26.123916+0000","last_undegraded":"2023-11-06T01:33:26.123916+0000","last_fullsized":"2023-11-06T01:33:26.123916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:54.220515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119514+0000","last_change":"2023-11-06T01:32:26.116280+0000","last_active":"2023-11-06T01:33:26.119514+0000","last_peered":"2023-11-06T01:33:26.119514+0000","last_clean":"2023-11-06T01:33:26.119514+0000","last_became_active":"2023-11-06T01:32:26.115957+0000","last_became_peered":"2023-11-06T01:32:26.115957+0000","last_unstale":"2023-11-06T01:33:26.119514+0000","last_undegraded":"2023-11-06T01:33:26.119514+0000","last_fullsized":"2023-11-06T01:33:26.119514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:25.956902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118127+0000","last_change":"2023-11-06T01:32:26.124818+0000","last_active":"2023-11-06T01:33:26.118127+0000","last_peered":"2023-11-06T01:33:26.118127+0000","last_clean":"2023-11-06T01:33:26.118127+0000","last_became_active":"2023-11-06T01:32:26.124511+0000","last_became_peered":"2023-11-06T01:32:26.124511+0000","last_unstale":"2023-11-06T01:33:26.118127+0000","last_undegraded":"2023-11-06T01:33:26.118127+0000","last_fullsized":"2023-11-06T01:33:26.118127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:35.853898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122869+0000","last_change":"2023-11-06T01:32:26.114243+0000","last_active":"2023-11-06T01:33:26.122869+0000","last_peered":"2023-11-06T01:33:26.122869+0000","last_clean":"2023-11-06T01:33:26.122869+0000","last_became_active":"2023-11-06T01:32:26.113923+0000","last_became_peered":"2023-11-06T01:32:26.113923+0000","last_unstale":"2023-11-06T01:33:26.122869+0000","last_undegraded":"2023-11-06T01:33:26.122869+0000","last_fullsized":"2023-11-06T01:33:26.122869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:48.197329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124318+0000","last_change":"2023-11-06T01:32:26.113819+0000","last_active":"2023-11-06T01:33:26.124318+0000","last_peered":"2023-11-06T01:33:26.124318+0000","last_clean":"2023-11-06T01:33:26.124318+0000","last_became_active":"2023-11-06T01:32:26.113506+0000","last_became_peered":"2023-11-06T01:32:26.113506+0000","last_unstale":"2023-11-06T01:33:26.124318+0000","last_undegraded":"2023-11-06T01:33:26.124318+0000","last_fullsized":"2023-11-06T01:33:26.124318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:33.122100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120607+0000","last_change":"2023-11-06T01:32:26.116504+0000","last_active":"2023-11-06T01:33:26.120607+0000","last_peered":"2023-11-06T01:33:26.120607+0000","last_clean":"2023-11-06T01:33:26.120607+0000","last_became_active":"2023-11-06T01:32:26.115220+0000","last_became_peered":"2023-11-06T01:32:26.115220+0000","last_unstale":"2023-11-06T01:33:26.120607+0000","last_undegraded":"2023-11-06T01:33:26.120607+0000","last_fullsized":"2023-11-06T01:33:26.120607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:34.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120346+0000","last_change":"2023-11-06T01:32:26.111596+0000","last_active":"2023-11-06T01:33:26.120346+0000","last_peered":"2023-11-06T01:33:26.120346+0000","last_clean":"2023-11-06T01:33:26.120346+0000","last_became_active":"2023-11-06T01:32:26.111309+0000","last_became_peered":"2023-11-06T01:32:26.111309+0000","last_unstale":"2023-11-06T01:33:26.120346+0000","last_undegraded":"2023-11-06T01:33:26.120346+0000","last_fullsized":"2023-11-06T01:33:26.120346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:34.917068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129435+0000","last_change":"2023-11-06T01:32:26.114536+0000","last_active":"2023-11-06T01:33:26.129435+0000","last_peered":"2023-11-06T01:33:26.129435+0000","last_clean":"2023-11-06T01:33:26.129435+0000","last_became_active":"2023-11-06T01:32:26.114124+0000","last_became_peered":"2023-11-06T01:32:26.114124+0000","last_unstale":"2023-11-06T01:33:26.129435+0000","last_undegraded":"2023-11-06T01:33:26.129435+0000","last_fullsized":"2023-11-06T01:33:26.129435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:05:10.855240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122520+0000","last_change":"2023-11-06T01:32:26.111139+0000","last_active":"2023-11-06T01:33:26.122520+0000","last_peered":"2023-11-06T01:33:26.122520+0000","last_clean":"2023-11-06T01:33:26.122520+0000","last_became_active":"2023-11-06T01:32:26.110764+0000","last_became_peered":"2023-11-06T01:32:26.110764+0000","last_unstale":"2023-11-06T01:33:26.122520+0000","last_undegraded":"2023-11-06T01:33:26.122520+0000","last_fullsized":"2023-11-06T01:33:26.122520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:46.422701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.127151+0000","last_change":"2023-11-06T01:33:27.127151+0000","last_active":"2023-11-06T01:33:27.127151+0000","last_peered":"2023-11-06T01:33:27.127151+0000","last_clean":"2023-11-06T01:33:27.127151+0000","last_became_active":"2023-11-06T01:33:27.126884+0000","last_became_peered":"2023-11-06T01:33:27.126884+0000","last_unstale":"2023-11-06T01:33:27.127151+0000","last_undegraded":"2023-11-06T01:33:27.127151+0000","last_fullsized":"2023-11-06T01:33:27.127151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:03:18.021947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122447+0000","last_change":"2023-11-06T01:32:26.118169+0000","last_active":"2023-11-06T01:33:26.122447+0000","last_peered":"2023-11-06T01:33:26.122447+0000","last_clean":"2023-11-06T01:33:26.122447+0000","last_became_active":"2023-11-06T01:32:26.117837+0000","last_became_peered":"2023-11-06T01:32:26.117837+0000","last_unstale":"2023-11-06T01:33:26.122447+0000","last_undegraded":"2023-11-06T01:33:26.122447+0000","last_fullsized":"2023-11-06T01:33:26.122447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:46.838927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121234+0000","last_change":"2023-11-06T01:32:26.110020+0000","last_active":"2023-11-06T01:33:26.121234+0000","last_peered":"2023-11-06T01:33:26.121234+0000","last_clean":"2023-11-06T01:33:26.121234+0000","last_became_active":"2023-11-06T01:32:26.109707+0000","last_became_peered":"2023-11-06T01:32:26.109707+0000","last_unstale":"2023-11-06T01:33:26.121234+0000","last_undegraded":"2023-11-06T01:33:26.121234+0000","last_fullsized":"2023-11-06T01:33:26.121234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:00.016258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.115738+0000","last_change":"2023-11-06T01:32:26.119949+0000","last_active":"2023-11-06T01:33:26.115738+0000","last_peered":"2023-11-06T01:33:26.115738+0000","last_clean":"2023-11-06T01:33:26.115738+0000","last_became_active":"2023-11-06T01:32:26.117594+0000","last_became_peered":"2023-11-06T01:32:26.117594+0000","last_unstale":"2023-11-06T01:33:26.115738+0000","last_undegraded":"2023-11-06T01:33:26.115738+0000","last_fullsized":"2023-11-06T01:33:26.115738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:17:22.757846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121991+0000","last_change":"2023-11-06T01:32:26.123176+0000","last_active":"2023-11-06T01:33:26.121991+0000","last_peered":"2023-11-06T01:33:26.121991+0000","last_clean":"2023-11-06T01:33:26.121991+0000","last_became_active":"2023-11-06T01:32:26.116687+0000","last_became_peered":"2023-11-06T01:32:26.116687+0000","last_unstale":"2023-11-06T01:33:26.121991+0000","last_undegraded":"2023-11-06T01:33:26.121991+0000","last_fullsized":"2023-11-06T01:33:26.121991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:30.422426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":131,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:51.423488+0000","last_change":"2023-11-06T01:33:27.401065+0000","last_active":"2023-11-06T01:33:51.423488+0000","last_peered":"2023-11-06T01:33:51.423488+0000","last_clean":"2023-11-06T01:33:51.423488+0000","last_became_active":"2023-11-06T01:33:27.391423+0000","last_became_peered":"2023-11-06T01:33:27.391423+0000","last_unstale":"2023-11-06T01:33:51.423488+0000","last_undegraded":"2023-11-06T01:33:51.423488+0000","last_fullsized":"2023-11-06T01:33:51.423488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:00:05.573076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118711+0000","last_change":"2023-11-06T01:32:26.126350+0000","last_active":"2023-11-06T01:33:26.118711+0000","last_peered":"2023-11-06T01:33:26.118711+0000","last_clean":"2023-11-06T01:33:26.118711+0000","last_became_active":"2023-11-06T01:32:26.124703+0000","last_became_peered":"2023-11-06T01:32:26.124703+0000","last_unstale":"2023-11-06T01:33:26.118711+0000","last_undegraded":"2023-11-06T01:33:26.118711+0000","last_fullsized":"2023-11-06T01:33:26.118711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:08:25.866045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118659+0000","last_change":"2023-11-06T01:32:25.680184+0000","last_active":"2023-11-06T01:33:26.118659+0000","last_peered":"2023-11-06T01:33:26.118659+0000","last_clean":"2023-11-06T01:33:26.118659+0000","last_became_active":"2023-11-06T01:32:25.679859+0000","last_became_peered":"2023-11-06T01:32:25.679859+0000","last_unstale":"2023-11-06T01:33:26.118659+0000","last_undegraded":"2023-11-06T01:33:26.118659+0000","last_fullsized":"2023-11-06T01:33:26.118659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:04:34.153562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124048+0000","last_change":"2023-11-06T01:32:26.114511+0000","last_active":"2023-11-06T01:33:26.124048+0000","last_peered":"2023-11-06T01:33:26.124048+0000","last_clean":"2023-11-06T01:33:26.124048+0000","last_became_active":"2023-11-06T01:32:26.114182+0000","last_became_peered":"2023-11-06T01:32:26.114182+0000","last_unstale":"2023-11-06T01:33:26.124048+0000","last_undegraded":"2023-11-06T01:33:26.124048+0000","last_fullsized":"2023-11-06T01:33:26.124048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:49:10.694358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120210+0000","last_change":"2023-11-06T01:32:25.681453+0000","last_active":"2023-11-06T01:33:26.120210+0000","last_peered":"2023-11-06T01:33:26.120210+0000","last_clean":"2023-11-06T01:33:26.120210+0000","last_became_active":"2023-11-06T01:32:25.681180+0000","last_became_peered":"2023-11-06T01:32:25.681180+0000","last_unstale":"2023-11-06T01:33:26.120210+0000","last_undegraded":"2023-11-06T01:33:26.120210+0000","last_fullsized":"2023-11-06T01:33:26.120210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:39:19.738479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122292+0000","last_change":"2023-11-06T01:32:26.118748+0000","last_active":"2023-11-06T01:33:26.122292+0000","last_peered":"2023-11-06T01:33:26.122292+0000","last_clean":"2023-11-06T01:33:26.122292+0000","last_became_active":"2023-11-06T01:32:26.118449+0000","last_became_peered":"2023-11-06T01:32:26.118449+0000","last_unstale":"2023-11-06T01:33:26.122292+0000","last_undegraded":"2023-11-06T01:33:26.122292+0000","last_fullsized":"2023-11-06T01:33:26.122292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:28.134553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119846+0000","last_change":"2023-11-06T01:32:25.680967+0000","last_active":"2023-11-06T01:33:26.119846+0000","last_peered":"2023-11-06T01:33:26.119846+0000","last_clean":"2023-11-06T01:33:26.119846+0000","last_became_active":"2023-11-06T01:32:25.678829+0000","last_became_peered":"2023-11-06T01:32:25.678829+0000","last_unstale":"2023-11-06T01:33:26.119846+0000","last_undegraded":"2023-11-06T01:33:26.119846+0000","last_fullsized":"2023-11-06T01:33:26.119846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:40:07.691852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123063+0000","last_change":"2023-11-06T01:32:26.122692+0000","last_active":"2023-11-06T01:33:26.123063+0000","last_peered":"2023-11-06T01:33:26.123063+0000","last_clean":"2023-11-06T01:33:26.123063+0000","last_became_active":"2023-11-06T01:32:26.121960+0000","last_became_peered":"2023-11-06T01:32:26.121960+0000","last_unstale":"2023-11-06T01:33:26.123063+0000","last_undegraded":"2023-11-06T01:33:26.123063+0000","last_fullsized":"2023-11-06T01:33:26.123063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:48.010467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:52.426673+0000","last_change":"2023-11-06T01:33:27.403594+0000","last_active":"2023-11-06T01:33:52.426673+0000","last_peered":"2023-11-06T01:33:52.426673+0000","last_clean":"2023-11-06T01:33:52.426673+0000","last_became_active":"2023-11-06T01:33:27.391131+0000","last_became_peered":"2023-11-06T01:33:27.391131+0000","last_unstale":"2023-11-06T01:33:52.426673+0000","last_undegraded":"2023-11-06T01:33:52.426673+0000","last_fullsized":"2023-11-06T01:33:52.426673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:55:10.577522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119781+0000","last_change":"2023-11-06T01:32:26.116839+0000","last_active":"2023-11-06T01:33:26.119781+0000","last_peered":"2023-11-06T01:33:26.119781+0000","last_clean":"2023-11-06T01:33:26.119781+0000","last_became_active":"2023-11-06T01:32:26.116535+0000","last_became_peered":"2023-11-06T01:32:26.116535+0000","last_unstale":"2023-11-06T01:33:26.119781+0000","last_undegraded":"2023-11-06T01:33:26.119781+0000","last_fullsized":"2023-11-06T01:33:26.119781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:11.219626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123841+0000","last_change":"2023-11-06T01:32:25.678426+0000","last_active":"2023-11-06T01:33:26.123841+0000","last_peered":"2023-11-06T01:33:26.123841+0000","last_clean":"2023-11-06T01:33:26.123841+0000","last_became_active":"2023-11-06T01:32:25.678072+0000","last_became_peered":"2023-11-06T01:32:25.678072+0000","last_unstale":"2023-11-06T01:33:26.123841+0000","last_undegraded":"2023-11-06T01:33:26.123841+0000","last_fullsized":"2023-11-06T01:33:26.123841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:09:41.630098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124752+0000","last_change":"2023-11-06T01:32:26.119184+0000","last_active":"2023-11-06T01:33:26.124752+0000","last_peered":"2023-11-06T01:33:26.124752+0000","last_clean":"2023-11-06T01:33:26.124752+0000","last_became_active":"2023-11-06T01:32:26.118896+0000","last_became_peered":"2023-11-06T01:32:26.118896+0000","last_unstale":"2023-11-06T01:33:26.124752+0000","last_undegraded":"2023-11-06T01:33:26.124752+0000","last_fullsized":"2023-11-06T01:33:26.124752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:09:17.351315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129133+0000","last_change":"2023-11-06T01:32:25.685621+0000","last_active":"2023-11-06T01:33:26.129133+0000","last_peered":"2023-11-06T01:33:26.129133+0000","last_clean":"2023-11-06T01:33:26.129133+0000","last_became_active":"2023-11-06T01:32:25.684965+0000","last_became_peered":"2023-11-06T01:32:25.684965+0000","last_unstale":"2023-11-06T01:33:26.129133+0000","last_undegraded":"2023-11-06T01:33:26.129133+0000","last_fullsized":"2023-11-06T01:33:26.129133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:28:00.922680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118211+0000","last_change":"2023-11-06T01:32:26.122750+0000","last_active":"2023-11-06T01:33:26.118211+0000","last_peered":"2023-11-06T01:33:26.118211+0000","last_clean":"2023-11-06T01:33:26.118211+0000","last_became_active":"2023-11-06T01:32:26.122241+0000","last_became_peered":"2023-11-06T01:32:26.122241+0000","last_unstale":"2023-11-06T01:33:26.118211+0000","last_undegraded":"2023-11-06T01:33:26.118211+0000","last_fullsized":"2023-11-06T01:33:26.118211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:04.402197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124078+0000","last_change":"2023-11-06T01:32:25.681594+0000","last_active":"2023-11-06T01:33:26.124078+0000","last_peered":"2023-11-06T01:33:26.124078+0000","last_clean":"2023-11-06T01:33:26.124078+0000","last_became_active":"2023-11-06T01:32:25.681303+0000","last_became_peered":"2023-11-06T01:32:25.681303+0000","last_unstale":"2023-11-06T01:33:26.124078+0000","last_undegraded":"2023-11-06T01:33:26.124078+0000","last_fullsized":"2023-11-06T01:33:26.124078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:33.141440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.115522+0000","last_change":"2023-11-06T01:32:26.117234+0000","last_active":"2023-11-06T01:33:26.115522+0000","last_peered":"2023-11-06T01:33:26.115522+0000","last_clean":"2023-11-06T01:33:26.115522+0000","last_became_active":"2023-11-06T01:32:26.116915+0000","last_became_peered":"2023-11-06T01:32:26.116915+0000","last_unstale":"2023-11-06T01:33:26.115522+0000","last_undegraded":"2023-11-06T01:33:26.115522+0000","last_fullsized":"2023-11-06T01:33:26.115522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:05:27.521552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":185,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:54.145360+0000","last_change":"2023-11-06T01:32:25.682827+0000","last_active":"2023-11-06T01:33:54.145360+0000","last_peered":"2023-11-06T01:33:54.145360+0000","last_clean":"2023-11-06T01:33:54.145360+0000","last_became_active":"2023-11-06T01:32:25.682470+0000","last_became_peered":"2023-11-06T01:32:25.682470+0000","last_unstale":"2023-11-06T01:33:54.145360+0000","last_undegraded":"2023-11-06T01:33:54.145360+0000","last_fullsized":"2023-11-06T01:33:54.145360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:43.632070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123248+0000","last_change":"2023-11-06T01:32:26.123918+0000","last_active":"2023-11-06T01:33:26.123248+0000","last_peered":"2023-11-06T01:33:26.123248+0000","last_clean":"2023-11-06T01:33:26.123248+0000","last_became_active":"2023-11-06T01:32:26.123450+0000","last_became_peered":"2023-11-06T01:32:26.123450+0000","last_unstale":"2023-11-06T01:33:26.123248+0000","last_undegraded":"2023-11-06T01:33:26.123248+0000","last_fullsized":"2023-11-06T01:33:26.123248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:38.227311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121215+0000","last_change":"2023-11-06T01:32:25.680051+0000","last_active":"2023-11-06T01:33:26.121215+0000","last_peered":"2023-11-06T01:33:26.121215+0000","last_clean":"2023-11-06T01:33:26.121215+0000","last_became_active":"2023-11-06T01:32:25.679705+0000","last_became_peered":"2023-11-06T01:32:25.679705+0000","last_unstale":"2023-11-06T01:33:26.121215+0000","last_undegraded":"2023-11-06T01:33:26.121215+0000","last_fullsized":"2023-11-06T01:33:26.121215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:18:22.620036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.126354+0000","last_change":"2023-11-06T01:33:27.126354+0000","last_active":"2023-11-06T01:33:27.126354+0000","last_peered":"2023-11-06T01:33:27.126354+0000","last_clean":"2023-11-06T01:33:27.126354+0000","last_became_active":"2023-11-06T01:33:27.125998+0000","last_became_peered":"2023-11-06T01:33:27.125998+0000","last_unstale":"2023-11-06T01:33:27.126354+0000","last_undegraded":"2023-11-06T01:33:27.126354+0000","last_fullsized":"2023-11-06T01:33:27.126354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:00:47.689894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123147+0000","last_change":"2023-11-06T01:32:25.678689+0000","last_active":"2023-11-06T01:33:26.123147+0000","last_peered":"2023-11-06T01:33:26.123147+0000","last_clean":"2023-11-06T01:33:26.123147+0000","last_became_active":"2023-11-06T01:32:25.678355+0000","last_became_peered":"2023-11-06T01:32:25.678355+0000","last_unstale":"2023-11-06T01:33:26.123147+0000","last_undegraded":"2023-11-06T01:33:26.123147+0000","last_fullsized":"2023-11-06T01:33:26.123147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:23.007596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122348+0000","last_change":"2023-11-06T01:32:26.118120+0000","last_active":"2023-11-06T01:33:26.122348+0000","last_peered":"2023-11-06T01:33:26.122348+0000","last_clean":"2023-11-06T01:33:26.122348+0000","last_became_active":"2023-11-06T01:32:26.117762+0000","last_became_peered":"2023-11-06T01:32:26.117762+0000","last_unstale":"2023-11-06T01:33:26.122348+0000","last_undegraded":"2023-11-06T01:33:26.122348+0000","last_fullsized":"2023-11-06T01:33:26.122348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:28:42.417463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'30","reported_seq":219,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:54.430124+0000","last_change":"2023-11-06T01:32:25.676036+0000","last_active":"2023-11-06T01:33:54.430124+0000","last_peered":"2023-11-06T01:33:54.430124+0000","last_clean":"2023-11-06T01:33:54.430124+0000","last_became_active":"2023-11-06T01:32:25.675672+0000","last_became_peered":"2023-11-06T01:32:25.675672+0000","last_unstale":"2023-11-06T01:33:54.430124+0000","last_undegraded":"2023-11-06T01:33:54.430124+0000","last_fullsized":"2023-11-06T01:33:54.430124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:30:54.060926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58677,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":175,"num_read_kb":140,"num_write":32,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120047+0000","last_change":"2023-11-06T01:32:26.118832+0000","last_active":"2023-11-06T01:33:26.120047+0000","last_peered":"2023-11-06T01:33:26.120047+0000","last_clean":"2023-11-06T01:33:26.120047+0000","last_became_active":"2023-11-06T01:32:26.118551+0000","last_became_peered":"2023-11-06T01:32:26.118551+0000","last_unstale":"2023-11-06T01:33:26.120047+0000","last_undegraded":"2023-11-06T01:33:26.120047+0000","last_fullsized":"2023-11-06T01:33:26.120047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:23.031203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.136439+0000","last_change":"2023-11-06T01:33:27.136439+0000","last_active":"2023-11-06T01:33:27.136439+0000","last_peered":"2023-11-06T01:33:27.136439+0000","last_clean":"2023-11-06T01:33:27.136439+0000","last_became_active":"2023-11-06T01:33:27.123153+0000","last_became_peered":"2023-11-06T01:33:27.123153+0000","last_unstale":"2023-11-06T01:33:27.136439+0000","last_undegraded":"2023-11-06T01:33:27.136439+0000","last_fullsized":"2023-11-06T01:33:27.136439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:52:29.385129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129312+0000","last_change":"2023-11-06T01:32:26.113421+0000","last_active":"2023-11-06T01:33:26.129312+0000","last_peered":"2023-11-06T01:33:26.129312+0000","last_clean":"2023-11-06T01:33:26.129312+0000","last_became_active":"2023-11-06T01:32:26.113085+0000","last_became_peered":"2023-11-06T01:32:26.113085+0000","last_unstale":"2023-11-06T01:33:26.129312+0000","last_undegraded":"2023-11-06T01:33:26.129312+0000","last_fullsized":"2023-11-06T01:33:26.129312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:15:04.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123170+0000","last_change":"2023-11-06T01:32:25.680293+0000","last_active":"2023-11-06T01:33:26.123170+0000","last_peered":"2023-11-06T01:33:26.123170+0000","last_clean":"2023-11-06T01:33:26.123170+0000","last_became_active":"2023-11-06T01:32:25.679803+0000","last_became_peered":"2023-11-06T01:32:25.679803+0000","last_unstale":"2023-11-06T01:33:26.123170+0000","last_undegraded":"2023-11-06T01:33:26.123170+0000","last_fullsized":"2023-11-06T01:33:26.123170+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:28.562797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120305+0000","last_change":"2023-11-06T01:32:26.115996+0000","last_active":"2023-11-06T01:33:26.120305+0000","last_peered":"2023-11-06T01:33:26.120305+0000","last_clean":"2023-11-06T01:33:26.120305+0000","last_became_active":"2023-11-06T01:32:26.115407+0000","last_became_peered":"2023-11-06T01:32:26.115407+0000","last_unstale":"2023-11-06T01:33:26.120305+0000","last_undegraded":"2023-11-06T01:33:26.120305+0000","last_fullsized":"2023-11-06T01:33:26.120305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:08.069698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":103,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:51.124706+0000","last_change":"2023-11-06T01:32:25.680958+0000","last_active":"2023-11-06T01:33:51.124706+0000","last_peered":"2023-11-06T01:33:51.124706+0000","last_clean":"2023-11-06T01:33:51.124706+0000","last_became_active":"2023-11-06T01:32:25.680616+0000","last_became_peered":"2023-11-06T01:32:25.680616+0000","last_unstale":"2023-11-06T01:33:51.124706+0000","last_undegraded":"2023-11-06T01:33:51.124706+0000","last_fullsized":"2023-11-06T01:33:51.124706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:02.631755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120959+0000","last_change":"2023-11-06T01:32:26.115464+0000","last_active":"2023-11-06T01:33:26.120959+0000","last_peered":"2023-11-06T01:33:26.120959+0000","last_clean":"2023-11-06T01:33:26.120959+0000","last_became_active":"2023-11-06T01:32:26.115090+0000","last_became_peered":"2023-11-06T01:32:26.115090+0000","last_unstale":"2023-11-06T01:33:26.120959+0000","last_undegraded":"2023-11-06T01:33:26.120959+0000","last_fullsized":"2023-11-06T01:33:26.120959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:08.018044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119958+0000","last_change":"2023-11-06T01:32:25.677787+0000","last_active":"2023-11-06T01:33:26.119958+0000","last_peered":"2023-11-06T01:33:26.119958+0000","last_clean":"2023-11-06T01:33:26.119958+0000","last_became_active":"2023-11-06T01:32:25.677527+0000","last_became_peered":"2023-11-06T01:32:25.677527+0000","last_unstale":"2023-11-06T01:33:26.119958+0000","last_undegraded":"2023-11-06T01:33:26.119958+0000","last_fullsized":"2023-11-06T01:33:26.119958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:34:09.452965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.391503+0000","last_change":"2023-11-06T01:33:27.391176+0000","last_active":"2023-11-06T01:33:27.391503+0000","last_peered":"2023-11-06T01:33:27.391503+0000","last_clean":"2023-11-06T01:33:27.391503+0000","last_became_active":"2023-11-06T01:33:27.390893+0000","last_became_peered":"2023-11-06T01:33:27.390893+0000","last_unstale":"2023-11-06T01:33:27.391503+0000","last_undegraded":"2023-11-06T01:33:27.391503+0000","last_fullsized":"2023-11-06T01:33:27.391503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:20:47.382152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120659+0000","last_change":"2023-11-06T01:32:25.677257+0000","last_active":"2023-11-06T01:33:26.120659+0000","last_peered":"2023-11-06T01:33:26.120659+0000","last_clean":"2023-11-06T01:33:26.120659+0000","last_became_active":"2023-11-06T01:32:25.676536+0000","last_became_peered":"2023-11-06T01:32:25.676536+0000","last_unstale":"2023-11-06T01:33:26.120659+0000","last_undegraded":"2023-11-06T01:33:26.120659+0000","last_fullsized":"2023-11-06T01:33:26.120659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:53:33.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119820+0000","last_change":"2023-11-06T01:32:26.111447+0000","last_active":"2023-11-06T01:33:26.119820+0000","last_peered":"2023-11-06T01:33:26.119820+0000","last_clean":"2023-11-06T01:33:26.119820+0000","last_became_active":"2023-11-06T01:32:26.111132+0000","last_became_peered":"2023-11-06T01:32:26.111132+0000","last_unstale":"2023-11-06T01:33:26.119820+0000","last_undegraded":"2023-11-06T01:33:26.119820+0000","last_fullsized":"2023-11-06T01:33:26.119820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:55.385003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.115705+0000","last_change":"2023-11-06T01:32:25.682562+0000","last_active":"2023-11-06T01:33:26.115705+0000","last_peered":"2023-11-06T01:33:26.115705+0000","last_clean":"2023-11-06T01:33:26.115705+0000","last_became_active":"2023-11-06T01:32:25.682227+0000","last_became_peered":"2023-11-06T01:32:25.682227+0000","last_unstale":"2023-11-06T01:33:26.115705+0000","last_undegraded":"2023-11-06T01:33:26.115705+0000","last_fullsized":"2023-11-06T01:33:26.115705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:40.113429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.129164+0000","last_change":"2023-11-06T01:33:27.129164+0000","last_active":"2023-11-06T01:33:27.129164+0000","last_peered":"2023-11-06T01:33:27.129164+0000","last_clean":"2023-11-06T01:33:27.129164+0000","last_became_active":"2023-11-06T01:33:27.123473+0000","last_became_peered":"2023-11-06T01:33:27.123473+0000","last_unstale":"2023-11-06T01:33:27.129164+0000","last_undegraded":"2023-11-06T01:33:27.129164+0000","last_fullsized":"2023-11-06T01:33:27.129164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:03:52.561653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:53.427221+0000","last_change":"2023-11-06T01:32:25.688176+0000","last_active":"2023-11-06T01:33:53.427221+0000","last_peered":"2023-11-06T01:33:53.427221+0000","last_clean":"2023-11-06T01:33:53.427221+0000","last_became_active":"2023-11-06T01:32:25.687745+0000","last_became_peered":"2023-11-06T01:32:25.687745+0000","last_unstale":"2023-11-06T01:33:53.427221+0000","last_undegraded":"2023-11-06T01:33:53.427221+0000","last_fullsized":"2023-11-06T01:33:53.427221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:21:23.566389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124027+0000","last_change":"2023-11-06T01:32:26.111700+0000","last_active":"2023-11-06T01:33:26.124027+0000","last_peered":"2023-11-06T01:33:26.124027+0000","last_clean":"2023-11-06T01:33:26.124027+0000","last_became_active":"2023-11-06T01:32:26.111380+0000","last_became_peered":"2023-11-06T01:32:26.111380+0000","last_unstale":"2023-11-06T01:33:26.124027+0000","last_undegraded":"2023-11-06T01:33:26.124027+0000","last_fullsized":"2023-11-06T01:33:26.124027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:56:47.874663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124206+0000","last_change":"2023-11-06T01:32:26.121906+0000","last_active":"2023-11-06T01:33:26.124206+0000","last_peered":"2023-11-06T01:33:26.124206+0000","last_clean":"2023-11-06T01:33:26.124206+0000","last_became_active":"2023-11-06T01:32:26.121292+0000","last_became_peered":"2023-11-06T01:32:26.121292+0000","last_unstale":"2023-11-06T01:33:26.124206+0000","last_undegraded":"2023-11-06T01:33:26.124206+0000","last_fullsized":"2023-11-06T01:33:26.124206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.532098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":103,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:52.125485+0000","last_change":"2023-11-06T01:32:25.685814+0000","last_active":"2023-11-06T01:33:52.125485+0000","last_peered":"2023-11-06T01:33:52.125485+0000","last_clean":"2023-11-06T01:33:52.125485+0000","last_became_active":"2023-11-06T01:32:25.684890+0000","last_became_peered":"2023-11-06T01:32:25.684890+0000","last_unstale":"2023-11-06T01:33:52.125485+0000","last_undegraded":"2023-11-06T01:33:52.125485+0000","last_fullsized":"2023-11-06T01:33:52.125485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:21.950945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118629+0000","last_change":"2023-11-06T01:32:26.121752+0000","last_active":"2023-11-06T01:33:26.118629+0000","last_peered":"2023-11-06T01:33:26.118629+0000","last_clean":"2023-11-06T01:33:26.118629+0000","last_became_active":"2023-11-06T01:32:26.121447+0000","last_became_peered":"2023-11-06T01:32:26.121447+0000","last_unstale":"2023-11-06T01:33:26.118629+0000","last_undegraded":"2023-11-06T01:33:26.118629+0000","last_fullsized":"2023-11-06T01:33:26.118629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:51:19.727335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121153+0000","last_change":"2023-11-06T01:32:25.682727+0000","last_active":"2023-11-06T01:33:26.121153+0000","last_peered":"2023-11-06T01:33:26.121153+0000","last_clean":"2023-11-06T01:33:26.121153+0000","last_became_active":"2023-11-06T01:32:25.682377+0000","last_became_peered":"2023-11-06T01:32:25.682377+0000","last_unstale":"2023-11-06T01:33:26.121153+0000","last_undegraded":"2023-11-06T01:33:26.121153+0000","last_fullsized":"2023-11-06T01:33:26.121153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:17:38.614350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119733+0000","last_change":"2023-11-06T01:31:27.931076+0000","last_active":"2023-11-06T01:33:26.119733+0000","last_peered":"2023-11-06T01:33:26.119733+0000","last_clean":"2023-11-06T01:33:26.119733+0000","last_became_active":"2023-11-06T01:31:27.930669+0000","last_became_peered":"2023-11-06T01:31:27.930669+0000","last_unstale":"2023-11-06T01:33:26.119733+0000","last_undegraded":"2023-11-06T01:33:26.119733+0000","last_fullsized":"2023-11-06T01:33:26.119733+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_clean_scrub_stamp":"2023-11-06T01:28:36.252549+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:47:48.729325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.128723+0000","last_change":"2023-11-06T01:32:26.110055+0000","last_active":"2023-11-06T01:33:26.128723+0000","last_peered":"2023-11-06T01:33:26.128723+0000","last_clean":"2023-11-06T01:33:26.128723+0000","last_became_active":"2023-11-06T01:32:26.109640+0000","last_became_peered":"2023-11-06T01:32:26.109640+0000","last_unstale":"2023-11-06T01:33:26.128723+0000","last_undegraded":"2023-11-06T01:33:26.128723+0000","last_fullsized":"2023-11-06T01:33:26.128723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:45:43.076910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119178+0000","last_change":"2023-11-06T01:32:25.679885+0000","last_active":"2023-11-06T01:33:26.119178+0000","last_peered":"2023-11-06T01:33:26.119178+0000","last_clean":"2023-11-06T01:33:26.119178+0000","last_became_active":"2023-11-06T01:32:25.679518+0000","last_became_peered":"2023-11-06T01:32:25.679518+0000","last_unstale":"2023-11-06T01:33:26.119178+0000","last_undegraded":"2023-11-06T01:33:26.119178+0000","last_fullsized":"2023-11-06T01:33:26.119178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:44.558534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120105+0000","last_change":"2023-11-06T01:32:25.682016+0000","last_active":"2023-11-06T01:33:26.120105+0000","last_peered":"2023-11-06T01:33:26.120105+0000","last_clean":"2023-11-06T01:33:26.120105+0000","last_became_active":"2023-11-06T01:32:25.681753+0000","last_became_peered":"2023-11-06T01:32:25.681753+0000","last_unstale":"2023-11-06T01:33:26.120105+0000","last_undegraded":"2023-11-06T01:33:26.120105+0000","last_fullsized":"2023-11-06T01:33:26.120105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:08.575728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129585+0000","last_change":"2023-11-06T01:32:26.117109+0000","last_active":"2023-11-06T01:33:26.129585+0000","last_peered":"2023-11-06T01:33:26.129585+0000","last_clean":"2023-11-06T01:33:26.129585+0000","last_became_active":"2023-11-06T01:32:26.116769+0000","last_became_peered":"2023-11-06T01:32:26.116769+0000","last_unstale":"2023-11-06T01:33:26.129585+0000","last_undegraded":"2023-11-06T01:33:26.129585+0000","last_fullsized":"2023-11-06T01:33:26.129585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:48:04.188806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.142547+0000","last_change":"2023-11-06T01:33:27.142547+0000","last_active":"2023-11-06T01:33:27.142547+0000","last_peered":"2023-11-06T01:33:27.142547+0000","last_clean":"2023-11-06T01:33:27.142547+0000","last_became_active":"2023-11-06T01:33:27.127408+0000","last_became_peered":"2023-11-06T01:33:27.127408+0000","last_unstale":"2023-11-06T01:33:27.142547+0000","last_undegraded":"2023-11-06T01:33:27.142547+0000","last_fullsized":"2023-11-06T01:33:27.142547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:08:10.084437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118671+0000","last_change":"2023-11-06T01:32:26.116351+0000","last_active":"2023-11-06T01:33:26.118671+0000","last_peered":"2023-11-06T01:33:26.118671+0000","last_clean":"2023-11-06T01:33:26.118671+0000","last_became_active":"2023-11-06T01:32:26.116044+0000","last_became_peered":"2023-11-06T01:32:26.116044+0000","last_unstale":"2023-11-06T01:33:26.118671+0000","last_undegraded":"2023-11-06T01:33:26.118671+0000","last_fullsized":"2023-11-06T01:33:26.118671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:20.816412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121520+0000","last_change":"2023-11-06T01:32:25.676764+0000","last_active":"2023-11-06T01:33:26.121520+0000","last_peered":"2023-11-06T01:33:26.121520+0000","last_clean":"2023-11-06T01:33:26.121520+0000","last_became_active":"2023-11-06T01:32:25.676418+0000","last_became_peered":"2023-11-06T01:32:25.676418+0000","last_unstale":"2023-11-06T01:33:26.121520+0000","last_undegraded":"2023-11-06T01:33:26.121520+0000","last_fullsized":"2023-11-06T01:33:26.121520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:49.320532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123031+0000","last_change":"2023-11-06T01:32:26.121107+0000","last_active":"2023-11-06T01:33:26.123031+0000","last_peered":"2023-11-06T01:33:26.123031+0000","last_clean":"2023-11-06T01:33:26.123031+0000","last_became_active":"2023-11-06T01:32:26.120790+0000","last_became_peered":"2023-11-06T01:32:26.120790+0000","last_unstale":"2023-11-06T01:33:26.123031+0000","last_undegraded":"2023-11-06T01:33:26.123031+0000","last_fullsized":"2023-11-06T01:33:26.123031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:34:46.660301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122657+0000","last_change":"2023-11-06T01:32:25.678959+0000","last_active":"2023-11-06T01:33:26.122657+0000","last_peered":"2023-11-06T01:33:26.122657+0000","last_clean":"2023-11-06T01:33:26.122657+0000","last_became_active":"2023-11-06T01:32:25.678564+0000","last_became_peered":"2023-11-06T01:32:25.678564+0000","last_unstale":"2023-11-06T01:33:26.122657+0000","last_undegraded":"2023-11-06T01:33:26.122657+0000","last_fullsized":"2023-11-06T01:33:26.122657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:30:06.304889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119508+0000","last_change":"2023-11-06T01:32:26.113222+0000","last_active":"2023-11-06T01:33:26.119508+0000","last_peered":"2023-11-06T01:33:26.119508+0000","last_clean":"2023-11-06T01:33:26.119508+0000","last_became_active":"2023-11-06T01:32:26.112926+0000","last_became_peered":"2023-11-06T01:32:26.112926+0000","last_unstale":"2023-11-06T01:33:26.119508+0000","last_undegraded":"2023-11-06T01:33:26.119508+0000","last_fullsized":"2023-11-06T01:33:26.119508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:09:51.042469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118301+0000","last_change":"2023-11-06T01:32:25.684985+0000","last_active":"2023-11-06T01:33:26.118301+0000","last_peered":"2023-11-06T01:33:26.118301+0000","last_clean":"2023-11-06T01:33:26.118301+0000","last_became_active":"2023-11-06T01:32:25.684656+0000","last_became_peered":"2023-11-06T01:32:25.684656+0000","last_unstale":"2023-11-06T01:33:26.118301+0000","last_undegraded":"2023-11-06T01:33:26.118301+0000","last_fullsized":"2023-11-06T01:33:26.118301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:42:29.406803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120058+0000","last_change":"2023-11-06T01:32:26.107484+0000","last_active":"2023-11-06T01:33:26.120058+0000","last_peered":"2023-11-06T01:33:26.120058+0000","last_clean":"2023-11-06T01:33:26.120058+0000","last_became_active":"2023-11-06T01:32:26.106092+0000","last_became_peered":"2023-11-06T01:32:26.106092+0000","last_unstale":"2023-11-06T01:33:26.120058+0000","last_undegraded":"2023-11-06T01:33:26.120058+0000","last_fullsized":"2023-11-06T01:33:26.120058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:43:39.481290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123047+0000","last_change":"2023-11-06T01:32:25.679531+0000","last_active":"2023-11-06T01:33:26.123047+0000","last_peered":"2023-11-06T01:33:26.123047+0000","last_clean":"2023-11-06T01:33:26.123047+0000","last_became_active":"2023-11-06T01:32:25.679076+0000","last_became_peered":"2023-11-06T01:32:25.679076+0000","last_unstale":"2023-11-06T01:33:26.123047+0000","last_undegraded":"2023-11-06T01:33:26.123047+0000","last_fullsized":"2023-11-06T01:33:26.123047+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:37.381547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118418+0000","last_change":"2023-11-06T01:32:26.118584+0000","last_active":"2023-11-06T01:33:26.118418+0000","last_peered":"2023-11-06T01:33:26.118418+0000","last_clean":"2023-11-06T01:33:26.118418+0000","last_became_active":"2023-11-06T01:32:26.118285+0000","last_became_peered":"2023-11-06T01:32:26.118285+0000","last_unstale":"2023-11-06T01:33:26.118418+0000","last_undegraded":"2023-11-06T01:33:26.118418+0000","last_fullsized":"2023-11-06T01:33:26.118418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:52:33.057377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119913+0000","last_change":"2023-11-06T01:32:25.681214+0000","last_active":"2023-11-06T01:33:26.119913+0000","last_peered":"2023-11-06T01:33:26.119913+0000","last_clean":"2023-11-06T01:33:26.119913+0000","last_became_active":"2023-11-06T01:32:25.680907+0000","last_became_peered":"2023-11-06T01:32:25.680907+0000","last_unstale":"2023-11-06T01:33:26.119913+0000","last_undegraded":"2023-11-06T01:33:26.119913+0000","last_fullsized":"2023-11-06T01:33:26.119913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:04:58.248787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119830+0000","last_change":"2023-11-06T01:32:26.120862+0000","last_active":"2023-11-06T01:33:26.119830+0000","last_peered":"2023-11-06T01:33:26.119830+0000","last_clean":"2023-11-06T01:33:26.119830+0000","last_became_active":"2023-11-06T01:32:26.120307+0000","last_became_peered":"2023-11-06T01:32:26.120307+0000","last_unstale":"2023-11-06T01:33:26.119830+0000","last_undegraded":"2023-11-06T01:33:26.119830+0000","last_fullsized":"2023-11-06T01:33:26.119830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:45:35.611541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":105,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:53.144644+0000","last_change":"2023-11-06T01:32:25.675607+0000","last_active":"2023-11-06T01:33:53.144644+0000","last_peered":"2023-11-06T01:33:53.144644+0000","last_clean":"2023-11-06T01:33:53.144644+0000","last_became_active":"2023-11-06T01:32:25.675293+0000","last_became_peered":"2023-11-06T01:32:25.675293+0000","last_unstale":"2023-11-06T01:33:53.144644+0000","last_undegraded":"2023-11-06T01:33:53.144644+0000","last_fullsized":"2023-11-06T01:33:53.144644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:05.190844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1440,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120568+0000","last_change":"2023-11-06T01:32:26.116987+0000","last_active":"2023-11-06T01:33:26.120568+0000","last_peered":"2023-11-06T01:33:26.120568+0000","last_clean":"2023-11-06T01:33:26.120568+0000","last_became_active":"2023-11-06T01:32:26.116756+0000","last_became_peered":"2023-11-06T01:32:26.116756+0000","last_unstale":"2023-11-06T01:33:26.120568+0000","last_undegraded":"2023-11-06T01:33:26.120568+0000","last_fullsized":"2023-11-06T01:33:26.120568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:44.039520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122980+0000","last_change":"2023-11-06T01:32:25.675483+0000","last_active":"2023-11-06T01:33:26.122980+0000","last_peered":"2023-11-06T01:33:26.122980+0000","last_clean":"2023-11-06T01:33:26.122980+0000","last_became_active":"2023-11-06T01:32:25.675137+0000","last_became_peered":"2023-11-06T01:32:25.675137+0000","last_unstale":"2023-11-06T01:33:26.122980+0000","last_undegraded":"2023-11-06T01:33:26.122980+0000","last_fullsized":"2023-11-06T01:33:26.122980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:42.415165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120363+0000","last_change":"2023-11-06T01:32:26.115575+0000","last_active":"2023-11-06T01:33:26.120363+0000","last_peered":"2023-11-06T01:33:26.120363+0000","last_clean":"2023-11-06T01:33:26.120363+0000","last_became_active":"2023-11-06T01:32:26.115142+0000","last_became_peered":"2023-11-06T01:32:26.115142+0000","last_unstale":"2023-11-06T01:33:26.120363+0000","last_undegraded":"2023-11-06T01:33:26.120363+0000","last_fullsized":"2023-11-06T01:33:26.120363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:52.859340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129692+0000","last_change":"2023-11-06T01:32:25.676972+0000","last_active":"2023-11-06T01:33:26.129692+0000","last_peered":"2023-11-06T01:33:26.129692+0000","last_clean":"2023-11-06T01:33:26.129692+0000","last_became_active":"2023-11-06T01:32:25.676685+0000","last_became_peered":"2023-11-06T01:32:25.676685+0000","last_unstale":"2023-11-06T01:33:26.129692+0000","last_undegraded":"2023-11-06T01:33:26.129692+0000","last_fullsized":"2023-11-06T01:33:26.129692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:30:21.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121970+0000","last_change":"2023-11-06T01:32:26.115164+0000","last_active":"2023-11-06T01:33:26.121970+0000","last_peered":"2023-11-06T01:33:26.121970+0000","last_clean":"2023-11-06T01:33:26.121970+0000","last_became_active":"2023-11-06T01:32:26.114868+0000","last_became_peered":"2023-11-06T01:32:26.114868+0000","last_unstale":"2023-11-06T01:33:26.121970+0000","last_undegraded":"2023-11-06T01:33:26.121970+0000","last_fullsized":"2023-11-06T01:33:26.121970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:01:34.149871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.127653+0000","last_change":"2023-11-06T01:33:27.127653+0000","last_active":"2023-11-06T01:33:27.127653+0000","last_peered":"2023-11-06T01:33:27.127653+0000","last_clean":"2023-11-06T01:33:27.127653+0000","last_became_active":"2023-11-06T01:33:27.127151+0000","last_became_peered":"2023-11-06T01:33:27.127151+0000","last_unstale":"2023-11-06T01:33:27.127653+0000","last_undegraded":"2023-11-06T01:33:27.127653+0000","last_fullsized":"2023-11-06T01:33:27.127653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:54.825681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122463+0000","last_change":"2023-11-06T01:32:26.115336+0000","last_active":"2023-11-06T01:33:26.122463+0000","last_peered":"2023-11-06T01:33:26.122463+0000","last_clean":"2023-11-06T01:33:26.122463+0000","last_became_active":"2023-11-06T01:32:26.114946+0000","last_became_peered":"2023-11-06T01:32:26.114946+0000","last_unstale":"2023-11-06T01:33:26.122463+0000","last_undegraded":"2023-11-06T01:33:26.122463+0000","last_fullsized":"2023-11-06T01:33:26.122463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:19:13.651494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122948+0000","last_change":"2023-11-06T01:32:26.117794+0000","last_active":"2023-11-06T01:33:26.122948+0000","last_peered":"2023-11-06T01:33:26.122948+0000","last_clean":"2023-11-06T01:33:26.122948+0000","last_became_active":"2023-11-06T01:32:26.117425+0000","last_became_peered":"2023-11-06T01:32:26.117425+0000","last_unstale":"2023-11-06T01:33:26.122948+0000","last_undegraded":"2023-11-06T01:33:26.122948+0000","last_fullsized":"2023-11-06T01:33:26.122948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:00:24.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.125881+0000","last_change":"2023-11-06T01:33:27.125881+0000","last_active":"2023-11-06T01:33:27.125881+0000","last_peered":"2023-11-06T01:33:27.125881+0000","last_clean":"2023-11-06T01:33:27.125881+0000","last_became_active":"2023-11-06T01:33:27.125445+0000","last_became_peered":"2023-11-06T01:33:27.125445+0000","last_unstale":"2023-11-06T01:33:27.125881+0000","last_undegraded":"2023-11-06T01:33:27.125881+0000","last_fullsized":"2023-11-06T01:33:27.125881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:31.654256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129629+0000","last_change":"2023-11-06T01:32:26.109910+0000","last_active":"2023-11-06T01:33:26.129629+0000","last_peered":"2023-11-06T01:33:26.129629+0000","last_clean":"2023-11-06T01:33:26.129629+0000","last_became_active":"2023-11-06T01:32:26.109568+0000","last_became_peered":"2023-11-06T01:32:26.109568+0000","last_unstale":"2023-11-06T01:33:26.129629+0000","last_undegraded":"2023-11-06T01:33:26.129629+0000","last_fullsized":"2023-11-06T01:33:26.129629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:11.049749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120801+0000","last_change":"2023-11-06T01:32:26.114824+0000","last_active":"2023-11-06T01:33:26.120801+0000","last_peered":"2023-11-06T01:33:26.120801+0000","last_clean":"2023-11-06T01:33:26.120801+0000","last_became_active":"2023-11-06T01:32:26.114539+0000","last_became_peered":"2023-11-06T01:32:26.114539+0000","last_unstale":"2023-11-06T01:33:26.120801+0000","last_undegraded":"2023-11-06T01:33:26.120801+0000","last_fullsized":"2023-11-06T01:33:26.120801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:03:02.018769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122658+0000","last_change":"2023-11-06T01:32:26.114743+0000","last_active":"2023-11-06T01:33:26.122658+0000","last_peered":"2023-11-06T01:33:26.122658+0000","last_clean":"2023-11-06T01:33:26.122658+0000","last_became_active":"2023-11-06T01:32:26.114432+0000","last_became_peered":"2023-11-06T01:32:26.114432+0000","last_unstale":"2023-11-06T01:33:26.122658+0000","last_undegraded":"2023-11-06T01:33:26.122658+0000","last_fullsized":"2023-11-06T01:33:26.122658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:27:18.209810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118548+0000","last_change":"2023-11-06T01:32:26.125724+0000","last_active":"2023-11-06T01:33:26.118548+0000","last_peered":"2023-11-06T01:33:26.118548+0000","last_clean":"2023-11-06T01:33:26.118548+0000","last_became_active":"2023-11-06T01:32:26.125455+0000","last_became_peered":"2023-11-06T01:32:26.125455+0000","last_unstale":"2023-11-06T01:33:26.118548+0000","last_undegraded":"2023-11-06T01:33:26.118548+0000","last_fullsized":"2023-11-06T01:33:26.118548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:49.099308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124151+0000","last_change":"2023-11-06T01:32:26.119328+0000","last_active":"2023-11-06T01:33:26.124151+0000","last_peered":"2023-11-06T01:33:26.124151+0000","last_clean":"2023-11-06T01:33:26.124151+0000","last_became_active":"2023-11-06T01:32:26.119042+0000","last_became_peered":"2023-11-06T01:32:26.119042+0000","last_unstale":"2023-11-06T01:33:26.124151+0000","last_undegraded":"2023-11-06T01:33:26.124151+0000","last_fullsized":"2023-11-06T01:33:26.124151+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:36.143788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118310+0000","last_change":"2023-11-06T01:32:26.126538+0000","last_active":"2023-11-06T01:33:26.118310+0000","last_peered":"2023-11-06T01:33:26.118310+0000","last_clean":"2023-11-06T01:33:26.118310+0000","last_became_active":"2023-11-06T01:32:26.124898+0000","last_became_peered":"2023-11-06T01:32:26.124898+0000","last_unstale":"2023-11-06T01:33:26.118310+0000","last_undegraded":"2023-11-06T01:33:26.118310+0000","last_fullsized":"2023-11-06T01:33:26.118310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:33.019326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122059+0000","last_change":"2023-11-06T01:32:26.118751+0000","last_active":"2023-11-06T01:33:26.122059+0000","last_peered":"2023-11-06T01:33:26.122059+0000","last_clean":"2023-11-06T01:33:26.122059+0000","last_became_active":"2023-11-06T01:32:26.118359+0000","last_became_peered":"2023-11-06T01:32:26.118359+0000","last_unstale":"2023-11-06T01:33:26.122059+0000","last_undegraded":"2023-11-06T01:33:26.122059+0000","last_fullsized":"2023-11-06T01:33:26.122059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:55.960400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129434+0000","last_change":"2023-11-06T01:32:26.110133+0000","last_active":"2023-11-06T01:33:26.129434+0000","last_peered":"2023-11-06T01:33:26.129434+0000","last_clean":"2023-11-06T01:33:26.129434+0000","last_became_active":"2023-11-06T01:32:26.109793+0000","last_became_peered":"2023-11-06T01:32:26.109793+0000","last_unstale":"2023-11-06T01:33:26.129434+0000","last_undegraded":"2023-11-06T01:33:26.129434+0000","last_fullsized":"2023-11-06T01:33:26.129434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:00:03.956947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120467+0000","last_change":"2023-11-06T01:32:26.109828+0000","last_active":"2023-11-06T01:33:26.120467+0000","last_peered":"2023-11-06T01:33:26.120467+0000","last_clean":"2023-11-06T01:33:26.120467+0000","last_became_active":"2023-11-06T01:32:26.109526+0000","last_became_peered":"2023-11-06T01:32:26.109526+0000","last_unstale":"2023-11-06T01:33:26.120467+0000","last_undegraded":"2023-11-06T01:33:26.120467+0000","last_fullsized":"2023-11-06T01:33:26.120467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:36.561908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250075,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65618,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":855,"num_read_kb":496,"num_write":232,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1707,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":192216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481821,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":436587,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95799999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":1.002,"15min":1.002},"last":0.59599999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51200000000000001,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.68100000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.75700000000000001,"15min":0.77300000000000002},"min":{"1min":0.48299999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.86499999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80100000000000005,"5min":2.137,"15min":2.137},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71399999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78000000000000003,"5min":1.056,"15min":1.056},"last":0.54100000000000004}]},{"osd":6,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.877}]},{"osd":7,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.51900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92100000000000004,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.81899999999999995,"15min":0.83499999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89900000000000002,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.70599999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.66300000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1970000000000001,"5min":1.343,"15min":1.343},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72099999999999997,"15min":0.71499999999999997},"min":{"1min":0.58899999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65000000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":0.877},"min":{"1min":0.626,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.085,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78200000000000003,"15min":0.78500000000000003},"min":{"1min":0.59499999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.94999999999999996,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.78900000000000003}]}]},{"osd":10,"up_from":70,"seq":300647710754,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":1081580,"data_compressed":41972,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.54000000000000004}]},{"osd":1,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55800000000000005,"15min":0.55700000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.443}]},{"osd":2,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.55000000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0629999999999999,"5min":1.071,"15min":1.071},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0900000000000001,"5min":1.169,"15min":1.169},"last":0.63600000000000001}]},{"osd":3,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76300000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.48399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84799999999999998,"5min":1.052,"15min":1.052},"last":0.60399999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61899999999999999,"15min":0.60999999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.88900000000000001,"5min":1.133,"15min":1.133},"last":0.58299999999999996}]},{"osd":5,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.50800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.105,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82899999999999996,"15min":0.82699999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.044,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77900000000000003,"15min":0.78000000000000003},"min":{"1min":0.51800000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95099999999999996,"5min":1.147,"15min":1.147},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78700000000000003,"15min":0.78900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.079,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.875,"15min":0.875},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.133,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91600000000000004},"min":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.179,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.82099999999999995}]},{"osd":9,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.57099999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.026,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.85199999999999998,"15min":0.86099999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0549999999999999,"5min":1.246,"15min":1.246},"last":0.754}]},{"osd":11,"last update":"Mon Nov 6 01:33:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.145,"5min":1.226,"15min":1.226},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.82599999999999996,"15min":0.82099999999999995},"min":{"1min":0.63,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.76200000000000001}]}]},{"osd":9,"up_from":65,"seq":279172874278,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":436620,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.49199999999999999,"15min":0.50700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.9780000000000002,"5min":3.9780000000000002,"15min":3.9780000000000002},"last":0.28000000000000003}]},{"osd":1,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.66900000000000004,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77800000000000002,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":6.6210000000000004,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":0.46600000000000003}]},{"osd":2,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.67700000000000005,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.476,"5min":6.476,"15min":6.476},"last":0.39800000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59799999999999998,"15min":0.58499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":4.0270000000000001,"5min":4.0270000000000001,"15min":4.0270000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82199999999999995,"5min":0.87,"15min":0.87},"last":0.34799999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.70899999999999996,"15min":0.68000000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.47699999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77200000000000002,"15min":0.77100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":4.1589999999999998,"5min":4.1589999999999998,"15min":4.1589999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.79300000000000004,"15min":0.81499999999999995},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.44700000000000001}]},{"osd":6,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.80100000000000005,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":6.056,"5min":6.056,"15min":6.056},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.3700000000000001,"5min":6.3700000000000001,"15min":6.3700000000000001},"last":0.45900000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85999999999999999,"15min":0.86199999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.97799999999999998,"5min":1.603,"15min":1.603},"last":0.57599999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.83899999999999997,"15min":0.81599999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.524,"5min":6.524,"15min":6.524},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.85999999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":4.1900000000000004,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":0.41999999999999998}]},{"osd":10,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.84199999999999997,"15min":0.81799999999999995},"min":{"1min":0.51900000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.85899999999999999,"15min":0.84499999999999997},"min":{"1min":0.56799999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":4.1280000000000001,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":0.55000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":8,"up_from":58,"seq":249108103209,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":435872,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80400000000000005,"5min":1.177,"15min":1.177},"last":0.497},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":1.196,"15min":1.196},"last":0.70499999999999996}]},{"osd":1,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.50800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84399999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71699999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.47199999999999998,"15min":0.46999999999999997},"min":{"1min":0.35299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.48699999999999999,"15min":0.48199999999999998},"min":{"1min":0.40500000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60399999999999998}]},{"osd":3,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78300000000000003,"5min":1.083,"15min":1.083},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.497,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.498,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":1.24,"15min":1.24},"last":0.627},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.60199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94299999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72599999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58099999999999996,"15min":0.56799999999999995},"min":{"1min":0.52900000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.95599999999999996,"5min":1.036,"15min":1.036},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.61199999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.754}]},{"osd":7,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.57399999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.0600000000000001,"5min":1.353,"15min":1.353},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.72299999999999998,"15min":0.70499999999999996},"min":{"1min":0.58699999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.0149999999999999,"5min":1.276,"15min":1.276},"last":0.78500000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81399999999999995}]},{"osd":10,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.77200000000000002,"15min":0.75900000000000001},"min":{"1min":0.61899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0840000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.78400000000000003,"15min":0.76400000000000001},"min":{"1min":0.60399999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.88700000000000001}]},{"osd":11,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86699999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":1029257,"data_compressed":41972,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.873,"5min":1.157,"15min":1.157},"last":0.376}]},{"osd":1,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.433,"15min":0.40899999999999997},"min":{"1min":0.35899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79500000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.307,"15min":0.307},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.41099999999999998}]},{"osd":2,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.60899999999999999,"15min":0.58299999999999996},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.60599999999999998,"15min":0.57699999999999996},"min":{"1min":0.44700000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.69599999999999995,"15min":0.67600000000000005},"min":{"1min":0.45900000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.8090000000000002,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.63200000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56699999999999995,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.53700000000000003,"15min":0.50600000000000001},"min":{"1min":0.372,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":0.379}]},{"osd":5,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.66800000000000004,"15min":0.64200000000000002},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.66000000000000003,"15min":0.626},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71899999999999997,"15min":0.69599999999999995},"min":{"1min":0.57299999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73099999999999998,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.48299999999999998}]},{"osd":8,"last update":"Mon Nov 6 01:33:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.75900000000000001,"15min":0.72799999999999998},"min":{"1min":0.624,"5min":0.442,"15min":0.442},"max":{"1min":1.0209999999999999,"5min":2.395,"15min":2.395},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78300000000000003,"15min":0.753},"min":{"1min":0.63,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.089,"5min":2.484,"15min":2.484},"last":0.73699999999999999}]},{"osd":9,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.79400000000000004,"15min":0.78200000000000003},"min":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.78400000000000003}]},{"osd":10,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.51900000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.84899999999999998}]},{"osd":11,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.77300000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":436172,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.52100000000000002,"15min":0.52600000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.39800000000000002}]},{"osd":1,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.499},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.47199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.68899999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.53500000000000003}]},{"osd":2,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.60699999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70899999999999996,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52300000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.57399999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.47799999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.61199999999999999,"15min":0.62},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78800000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.44900000000000001}]},{"osd":5,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86799999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46899999999999997}]},{"osd":7,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.75,"15min":0.76800000000000002},"min":{"1min":0.496,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.78200000000000003,"5min":1.042,"15min":1.042},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.68899999999999995,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.879,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.622}]},{"osd":8,"last update":"Mon Nov 6 01:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.78900000000000003,"15min":0.82999999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.74199999999999999,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.54900000000000004}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.42999999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88900000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.82399999999999995,"15min":0.84799999999999998},"min":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64600000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.76900000000000002,"15min":0.78700000000000003},"min":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.93500000000000005,"5min":1.238,"15min":1.238},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.82999999999999996,"15min":0.84899999999999998},"min":{"1min":0.58699999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.86299999999999999,"5min":1.141,"15min":1.141},"last":0.68000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":492403,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82699999999999996,"5min":1.093,"15min":1.093},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53100000000000003,"15min":0.498},"min":{"1min":0.31,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.85999999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.751}]},{"osd":2,"last update":"Mon Nov 6 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.60899999999999999,"15min":0.53700000000000003},"min":{"1min":0.51500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.95699999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.58599999999999997,"15min":0.52100000000000002},"min":{"1min":0.48099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.874},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.61699999999999999,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1459999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.52000000000000002,"15min":0.47499999999999998},"min":{"1min":0.308,"5min":0.252,"15min":0.252},"max":{"1min":1.0580000000000001,"5min":1.617,"15min":1.617},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.57799999999999996,"15min":0.52400000000000002},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.93400000000000005,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.56599999999999995}]},{"osd":5,"last update":"Mon Nov 6 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.65900000000000003,"15min":0.60399999999999998},"min":{"1min":0.59099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0189999999999999,"5min":1.056,"15min":1.056},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.66400000000000003,"15min":0.61799999999999999},"min":{"1min":0.54800000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82399999999999995}]},{"osd":6,"last update":"Mon Nov 6 01:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.64700000000000002,"15min":0.61499999999999999},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96699999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.69899999999999995,"15min":0.66600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.64500000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.66400000000000003,"15min":0.58799999999999997},"min":{"1min":0.49099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.137,"5min":2.27,"15min":2.27},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.68400000000000005,"15min":0.59499999999999997},"min":{"1min":0.53300000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1739999999999999,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.65900000000000003}]},{"osd":8,"last update":"Mon Nov 6 01:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.75800000000000001,"15min":0.70599999999999996},"min":{"1min":0.68999999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.79600000000000004,"15min":0.751},"min":{"1min":0.68799999999999994,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.194,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.78900000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.45300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.82799999999999996,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":0.96599999999999997}]},{"osd":10,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.88300000000000001,"15min":0.875},"min":{"1min":0.56200000000000006,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.23,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.876,"15min":0.86699999999999999},"min":{"1min":0.57999999999999996,"5min":0.501,"15min":0.501},"max":{"1min":1.1579999999999999,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.996},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.94399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":437505,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 01:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.53500000000000003},"min":{"1min":0.434,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74099999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.32200000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.42499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72099999999999997,"5min":1.052,"15min":1.052},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55600000000000005}]},{"osd":5,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70999999999999996,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":1.198,"15min":1.198},"last":0.58499999999999996}]},{"osd":6,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52800000000000002,"15min":0.55300000000000005},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66300000000000003,"5min":1.075,"15min":1.075},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.33500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.68000000000000005,"5min":1.222,"15min":1.222},"last":0.39200000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.52400000000000002}]},{"osd":8,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.71299999999999997,"15min":0.75700000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80000000000000004,"5min":1.292,"15min":1.292},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.71299999999999997,"15min":0.753},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76400000000000001,"5min":1.341,"15min":1.341},"last":0.66300000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.66000000000000003,"15min":0.67500000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":1.123,"15min":1.123},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76000000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.57399999999999995}]},{"osd":10,"last update":"Mon Nov 6 01:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.71299999999999997,"15min":0.72499999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86899999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86799999999999999,"5min":1.165,"15min":1.165},"last":0.61599999999999999}]},{"osd":11,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.56299999999999994,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.51000000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79600000000000004,"5min":0.878,"15min":0.878},"last":0.624}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":495604,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.497},"min":{"1min":0.46100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53800000000000003,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Nov 6 01:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81200000000000006,"5min":1.044,"15min":1.044},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.44500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81499999999999995}]},{"osd":3,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.75},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60399999999999998,"15min":0.57599999999999996},"min":{"1min":0.443,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67000000000000004}]},{"osd":5,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.49199999999999999,"15min":0.45200000000000001},"min":{"1min":0.34499999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56100000000000005,"15min":0.52700000000000002},"min":{"1min":0.499,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76100000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.73699999999999999}]},{"osd":6,"last update":"Mon Nov 6 01:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.46200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.86199999999999999}]},{"osd":7,"last update":"Mon Nov 6 01:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.54700000000000004}]},{"osd":8,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.46700000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76900000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.48599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":1,"15min":1},"last":0.79400000000000004}]},{"osd":9,"last update":"Mon Nov 6 01:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76500000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.89600000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.60699999999999998,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.022,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.96199999999999997,"5min":1.264,"15min":1.264},"last":0.78100000000000003}]},{"osd":11,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.83399999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149759,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":435928,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.36399999999999999},"min":{"1min":0.27300000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51500000000000001,"5min":0.627,"15min":0.627},"last":0.443},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.31,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":1.119,"15min":1.119},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54500000000000004,"15min":0.496},"min":{"1min":0.42099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60399999999999998,"15min":0.55800000000000005},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83099999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61299999999999999,"15min":0.59299999999999997},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.72499999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.39000000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.46300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.66800000000000004,"5min":1.159,"15min":1.159},"last":0.73199999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.68100000000000005,"5min":1.042,"15min":1.042},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59799999999999998,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76100000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.318}]},{"osd":6,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89800000000000002,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.498}]},{"osd":7,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65900000000000003,"15min":0.64700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83999999999999997,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67200000000000004,"15min":0.66000000000000003},"min":{"1min":0.497,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93400000000000005,"5min":1.149,"15min":1.149},"last":0.64800000000000002}]},{"osd":8,"last update":"Mon Nov 6 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.622},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.0430000000000001,"5min":6.0430000000000001,"15min":6.0430000000000001},"last":0.76500000000000001}]},{"osd":9,"last update":"Mon Nov 6 01:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.74199999999999999,"15min":0.76900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85899999999999999,"5min":1.319,"15min":1.319},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.46400000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82199999999999995,"5min":1.214,"15min":1.214},"last":0.54500000000000004}]},{"osd":10,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.79100000000000004,"15min":0.81200000000000006},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.83699999999999997,"5min":1.173,"15min":1.173},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88700000000000001,"5min":1.286,"15min":1.286},"last":0.747}]},{"osd":11,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58999999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888122,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":436699,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.45300000000000001},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.49399999999999999,"15min":0.47799999999999998},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.749,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42999999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.36499999999999999},"min":{"1min":0.34499999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58499999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.41099999999999998,"15min":0.375},"min":{"1min":0.38900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.38600000000000001}]},{"osd":2,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55000000000000004,"15min":0.51600000000000001},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76400000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.504},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58899999999999997,"15min":0.53300000000000003},"min":{"1min":0.46600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81100000000000005,"5min":1.028,"15min":1.028},"last":0.55100000000000005}]},{"osd":3,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":0.63400000000000001,"15min":0.53100000000000003},"min":{"1min":0.53000000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":6.9050000000000002,"5min":6.9050000000000002,"15min":6.9050000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.4770000000000001,"5min":0.69999999999999996,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":12.869,"5min":12.869,"15min":12.869},"last":0.72799999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61499999999999999,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92900000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.624,"15min":0.57799999999999996},"min":{"1min":0.53700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.623}]},{"osd":6,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.68999999999999995,"15min":0.64800000000000002},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":5.3949999999999996,"5min":5.3949999999999996,"15min":5.3949999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.127,"5min":0.73399999999999999,"15min":0.66800000000000004},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.64100000000000001}]},{"osd":7,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.65100000000000002,"15min":0.623},"min":{"1min":0.51500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999}]},{"osd":8,"last update":"Mon Nov 6 01:33:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66400000000000003,"15min":0.63900000000000001},"min":{"1min":0.51100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.92700000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76100000000000001,"15min":0.73499999999999999},"min":{"1min":0.65000000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77900000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":0.82999999999999996,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.0659999999999998,"5min":7.0659999999999998,"15min":7.0659999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.73699999999999999,"15min":0.71599999999999997},"min":{"1min":0.63900000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.68999999999999995}]},{"osd":10,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.70199999999999996,"15min":0.68600000000000005},"min":{"1min":0.55000000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.65300000000000002,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.75800000000000001}]},{"osd":11,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.84399999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":1028700,"data_compressed":41972,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.53700000000000003},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76400000000000001,"5min":1.01,"15min":1.01},"last":0.48499999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56999999999999995,"15min":0.53800000000000003},"min":{"1min":0.436,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81699999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57899999999999996,"15min":0.55000000000000004},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66300000000000003}]},{"osd":2,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.41499999999999998,"15min":0.39100000000000001},"min":{"1min":0.32700000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.41899999999999998,"15min":0.38300000000000001},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63500000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.31900000000000001}]},{"osd":3,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79600000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.439,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.74199999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.49199999999999999}]},{"osd":4,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59999999999999998,"15min":0.57199999999999995},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.61399999999999999,"15min":0.57199999999999995},"min":{"1min":0.47599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86099999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.80400000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55400000000000005}]},{"osd":7,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.67100000000000004,"15min":0.66000000000000003},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92100000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75}]},{"osd":8,"last update":"Mon Nov 6 01:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.59699999999999998},"min":{"1min":0.45800000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93100000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.42099999999999999}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.85899999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59399999999999997}]},{"osd":10,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.55400000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94499999999999995,"5min":1.024,"15min":1.024},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96799999999999997,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T01:33:56.236 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T01:33:56.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.403+0000 7ff7a4026700 1 -- 172.21.15.53:0/2762469137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c0ff660 msgr2=0x7ff79c0ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:56.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.403+0000 7ff7a4026700 1 --2- 172.21.15.53:0/2762469137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c0ff660 0x7ff79c0ffa80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff798009990 tx=0x7ff798009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.403+0000 7ff7a4026700 1 -- 172.21.15.53:0/2762469137 shutdown_connections 2023-11-06T01:33:56.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.403+0000 7ff7a4026700 1 --2- 172.21.15.53:0/2762469137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff79c100980 0x7ff79c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.403+0000 7ff7a4026700 1 --2- 172.21.15.53:0/2762469137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c0ff660 0x7ff79c0ffa80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.403+0000 7ff7a4026700 1 --2- 172.21.15.53:0/2762469137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff79c0fffc0 0x7ff79c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.403+0000 7ff7a4026700 1 -- 172.21.15.53:0/2762469137 >> 172.21.15.53:0/2762469137 conn(0x7ff79c0fb1c0 msgr2=0x7ff79c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.403+0000 7ff7a4026700 1 -- 172.21.15.53:0/2762469137 shutdown_connections 2023-11-06T01:33:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.403+0000 7ff7a4026700 1 -- 172.21.15.53:0/2762469137 wait complete. 2023-11-06T01:33:56.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.403+0000 7ff7a4026700 1 Processor -- start 2023-11-06T01:33:56.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a4026700 1 -- start start 2023-11-06T01:33:56.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a4026700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff79c0ff660 0x7ff79c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:56.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a4026700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c0fffc0 0x7ff79c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:56.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a4026700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff79c100980 0x7ff79c1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:56.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a4026700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c1a0740 con 0x7ff79c0fffc0 2023-11-06T01:33:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a4026700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c1a0880 con 0x7ff79c0ff660 2023-11-06T01:33:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a4026700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c1a0a10 con 0x7ff79c100980 2023-11-06T01:33:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a15c1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c0fffc0 0x7ff79c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a15c1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c0fffc0 0x7ff79c199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39614/0 (socket says 172.21.15.53:39614) 2023-11-06T01:33:56.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a15c1700 1 -- 172.21.15.53:0/2637532645 learned_addr learned my addr 172.21.15.53:0/2637532645 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:56.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a1dc2700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff79c0ff660 0x7ff79c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:56.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a25c3700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff79c100980 0x7ff79c1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:56.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a15c1700 1 -- 172.21.15.53:0/2637532645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff79c100980 msgr2=0x7ff79c1a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:56.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a15c1700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff79c100980 0x7ff79c1a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a15c1700 1 -- 172.21.15.53:0/2637532645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff79c0ff660 msgr2=0x7ff79c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:56.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a15c1700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff79c0ff660 0x7ff79c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a15c1700 1 -- 172.21.15.53:0/2637532645 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980095f0 con 0x7ff79c0fffc0 2023-11-06T01:33:56.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a25c3700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff79c100980 0x7ff79c1a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:56.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a15c1700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c0fffc0 0x7ff79c199100 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c920 tx=0x7ff79000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:56.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff78effd700 1 -- 172.21.15.53:0/2637532645 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790007b00 con 0x7ff79c0fffc0 2023-11-06T01:33:56.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff78effd700 1 -- 172.21.15.53:0/2637532645 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff79000ce70 con 0x7ff79c0fffc0 2023-11-06T01:33:56.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.404+0000 7ff7a4026700 1 -- 172.21.15.53:0/2637532645 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c1a0cf0 con 0x7ff79c0fffc0 2023-11-06T01:33:56.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.405+0000 7ff78effd700 1 -- 172.21.15.53:0/2637532645 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900054e0 con 0x7ff79c0fffc0 2023-11-06T01:33:56.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.405+0000 7ff7a4026700 1 -- 172.21.15.53:0/2637532645 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79c1a11f0 con 0x7ff79c0fffc0 2023-11-06T01:33:56.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.406+0000 7ff7a4026700 1 -- 172.21.15.53:0/2637532645 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79c10a440 con 0x7ff79c0fffc0 2023-11-06T01:33:56.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.406+0000 7ff78effd700 1 -- 172.21.15.53:0/2637532645 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff790004750 con 0x7ff79c0fffc0 2023-11-06T01:33:56.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.407+0000 7ff78effd700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff7880a05f0 0x7ff7880a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:56.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.407+0000 7ff7a1dc2700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff7880a05f0 0x7ff7880a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:56.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.407+0000 7ff78effd700 1 -- 172.21.15.53:0/2637532645 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff7900cb4c0 con 0x7ff79c0fffc0 2023-11-06T01:33:56.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.408+0000 7ff7a1dc2700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff7880a05f0 0x7ff7880a2ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff79800b5c0 tx=0x7ff798005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:56.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.412+0000 7ff78effd700 1 -- 172.21.15.53:0/2637532645 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff790095040 con 0x7ff79c0fffc0 2023-11-06T01:33:56.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:56 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3092090133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:56.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:56 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/140383426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:56.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.608+0000 7ff7a4026700 1 -- 172.21.15.53:0/2637532645 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff79c0611d0 con 0x7ff7880a05f0 2023-11-06T01:33:56.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.621+0000 7ff78effd700 1 -- 172.21.15.53:0/2637532645 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343607 (secure 0 0 0) 0x7ff79c0611d0 con 0x7ff7880a05f0 2023-11-06T01:33:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:56.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 -- 172.21.15.53:0/2637532645 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff7880a05f0 msgr2=0x7ff7880a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:56.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff7880a05f0 0x7ff7880a2ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff79800b5c0 tx=0x7ff798005fb0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 -- 172.21.15.53:0/2637532645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c0fffc0 msgr2=0x7ff79c199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:56.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c0fffc0 0x7ff79c199100 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c920 tx=0x7ff79000cc30 comp rx=0 tx=0).stop 2023-11-06T01:33:56.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 -- 172.21.15.53:0/2637532645 shutdown_connections 2023-11-06T01:33:56.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff7880a05f0 0x7ff7880a2ab0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff79c0ff660 0x7ff79c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff79c100980 0x7ff79c1a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 --2- 172.21.15.53:0/2637532645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c0fffc0 0x7ff79c199100 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 -- 172.21.15.53:0/2637532645 >> 172.21.15.53:0/2637532645 conn(0x7ff79c0fb1c0 msgr2=0x7ff79c0fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:56.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 -- 172.21.15.53:0/2637532645 shutdown_connections 2023-11-06T01:33:56.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.624+0000 7ff7a4026700 1 -- 172.21.15.53:0/2637532645 wait complete. 2023-11-06T01:33:56.636 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-06T01:33:56.638 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-11-06T01:33:55.498729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656232,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":981,"num_read_kb":605,"num_write":415,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1182566,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334636,"kb_used_data":11212,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607188,"statfs":{"total":1151940427776,"available":1151597760512,"internally_reserved":0,"allocated":11481088,"data_stored":7182927,"data_compressed":177927,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4956,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":72,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003045"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121147+0000","last_change":"2023-11-06T01:32:26.110793+0000","last_active":"2023-11-06T01:33:26.121147+0000","last_peered":"2023-11-06T01:33:26.121147+0000","last_clean":"2023-11-06T01:33:26.121147+0000","last_became_active":"2023-11-06T01:32:26.110489+0000","last_became_peered":"2023-11-06T01:32:26.110489+0000","last_unstale":"2023-11-06T01:33:26.121147+0000","last_undegraded":"2023-11-06T01:33:26.121147+0000","last_fullsized":"2023-11-06T01:33:26.121147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:01:08.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122964+0000","last_change":"2023-11-06T01:32:26.107273+0000","last_active":"2023-11-06T01:33:26.122964+0000","last_peered":"2023-11-06T01:33:26.122964+0000","last_clean":"2023-11-06T01:33:26.122964+0000","last_became_active":"2023-11-06T01:32:26.106961+0000","last_became_peered":"2023-11-06T01:32:26.106961+0000","last_unstale":"2023-11-06T01:33:26.122964+0000","last_undegraded":"2023-11-06T01:33:26.122964+0000","last_fullsized":"2023-11-06T01:33:26.122964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:40:39.073926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122761+0000","last_change":"2023-11-06T01:32:26.121316+0000","last_active":"2023-11-06T01:33:26.122761+0000","last_peered":"2023-11-06T01:33:26.122761+0000","last_clean":"2023-11-06T01:33:26.122761+0000","last_became_active":"2023-11-06T01:32:26.120756+0000","last_became_peered":"2023-11-06T01:32:26.120756+0000","last_unstale":"2023-11-06T01:33:26.122761+0000","last_undegraded":"2023-11-06T01:33:26.122761+0000","last_fullsized":"2023-11-06T01:33:26.122761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:02:45.814577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120866+0000","last_change":"2023-11-06T01:32:26.121951+0000","last_active":"2023-11-06T01:33:26.120866+0000","last_peered":"2023-11-06T01:33:26.120866+0000","last_clean":"2023-11-06T01:33:26.120866+0000","last_became_active":"2023-11-06T01:32:26.121636+0000","last_became_peered":"2023-11-06T01:32:26.121636+0000","last_unstale":"2023-11-06T01:33:26.120866+0000","last_undegraded":"2023-11-06T01:33:26.120866+0000","last_fullsized":"2023-11-06T01:33:26.120866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:02.381355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120004+0000","last_change":"2023-11-06T01:32:26.117788+0000","last_active":"2023-11-06T01:33:26.120004+0000","last_peered":"2023-11-06T01:33:26.120004+0000","last_clean":"2023-11-06T01:33:26.120004+0000","last_became_active":"2023-11-06T01:32:26.117161+0000","last_became_peered":"2023-11-06T01:32:26.117161+0000","last_unstale":"2023-11-06T01:33:26.120004+0000","last_undegraded":"2023-11-06T01:33:26.120004+0000","last_fullsized":"2023-11-06T01:33:26.120004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:31.665690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123916+0000","last_change":"2023-11-06T01:32:26.109532+0000","last_active":"2023-11-06T01:33:26.123916+0000","last_peered":"2023-11-06T01:33:26.123916+0000","last_clean":"2023-11-06T01:33:26.123916+0000","last_became_active":"2023-11-06T01:32:26.106144+0000","last_became_peered":"2023-11-06T01:32:26.106144+0000","last_unstale":"2023-11-06T01:33:26.123916+0000","last_undegraded":"2023-11-06T01:33:26.123916+0000","last_fullsized":"2023-11-06T01:33:26.123916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:54.220515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119514+0000","last_change":"2023-11-06T01:32:26.116280+0000","last_active":"2023-11-06T01:33:26.119514+0000","last_peered":"2023-11-06T01:33:26.119514+0000","last_clean":"2023-11-06T01:33:26.119514+0000","last_became_active":"2023-11-06T01:32:26.115957+0000","last_became_peered":"2023-11-06T01:32:26.115957+0000","last_unstale":"2023-11-06T01:33:26.119514+0000","last_undegraded":"2023-11-06T01:33:26.119514+0000","last_fullsized":"2023-11-06T01:33:26.119514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:25.956902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118127+0000","last_change":"2023-11-06T01:32:26.124818+0000","last_active":"2023-11-06T01:33:26.118127+0000","last_peered":"2023-11-06T01:33:26.118127+0000","last_clean":"2023-11-06T01:33:26.118127+0000","last_became_active":"2023-11-06T01:32:26.124511+0000","last_became_peered":"2023-11-06T01:32:26.124511+0000","last_unstale":"2023-11-06T01:33:26.118127+0000","last_undegraded":"2023-11-06T01:33:26.118127+0000","last_fullsized":"2023-11-06T01:33:26.118127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:35.853898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122869+0000","last_change":"2023-11-06T01:32:26.114243+0000","last_active":"2023-11-06T01:33:26.122869+0000","last_peered":"2023-11-06T01:33:26.122869+0000","last_clean":"2023-11-06T01:33:26.122869+0000","last_became_active":"2023-11-06T01:32:26.113923+0000","last_became_peered":"2023-11-06T01:32:26.113923+0000","last_unstale":"2023-11-06T01:33:26.122869+0000","last_undegraded":"2023-11-06T01:33:26.122869+0000","last_fullsized":"2023-11-06T01:33:26.122869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:48.197329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124318+0000","last_change":"2023-11-06T01:32:26.113819+0000","last_active":"2023-11-06T01:33:26.124318+0000","last_peered":"2023-11-06T01:33:26.124318+0000","last_clean":"2023-11-06T01:33:26.124318+0000","last_became_active":"2023-11-06T01:32:26.113506+0000","last_became_peered":"2023-11-06T01:32:26.113506+0000","last_unstale":"2023-11-06T01:33:26.124318+0000","last_undegraded":"2023-11-06T01:33:26.124318+0000","last_fullsized":"2023-11-06T01:33:26.124318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:33.122100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120607+0000","last_change":"2023-11-06T01:32:26.116504+0000","last_active":"2023-11-06T01:33:26.120607+0000","last_peered":"2023-11-06T01:33:26.120607+0000","last_clean":"2023-11-06T01:33:26.120607+0000","last_became_active":"2023-11-06T01:32:26.115220+0000","last_became_peered":"2023-11-06T01:32:26.115220+0000","last_unstale":"2023-11-06T01:33:26.120607+0000","last_undegraded":"2023-11-06T01:33:26.120607+0000","last_fullsized":"2023-11-06T01:33:26.120607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:34.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120346+0000","last_change":"2023-11-06T01:32:26.111596+0000","last_active":"2023-11-06T01:33:26.120346+0000","last_peered":"2023-11-06T01:33:26.120346+0000","last_clean":"2023-11-06T01:33:26.120346+0000","last_became_active":"2023-11-06T01:32:26.111309+0000","last_became_peered":"2023-11-06T01:32:26.111309+0000","last_unstale":"2023-11-06T01:33:26.120346+0000","last_undegraded":"2023-11-06T01:33:26.120346+0000","last_fullsized":"2023-11-06T01:33:26.120346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:34.917068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129435+0000","last_change":"2023-11-06T01:32:26.114536+0000","last_active":"2023-11-06T01:33:26.129435+0000","last_peered":"2023-11-06T01:33:26.129435+0000","last_clean":"2023-11-06T01:33:26.129435+0000","last_became_active":"2023-11-06T01:32:26.114124+0000","last_became_peered":"2023-11-06T01:32:26.114124+0000","last_unstale":"2023-11-06T01:33:26.129435+0000","last_undegraded":"2023-11-06T01:33:26.129435+0000","last_fullsized":"2023-11-06T01:33:26.129435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:05:10.855240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122520+0000","last_change":"2023-11-06T01:32:26.111139+0000","last_active":"2023-11-06T01:33:26.122520+0000","last_peered":"2023-11-06T01:33:26.122520+0000","last_clean":"2023-11-06T01:33:26.122520+0000","last_became_active":"2023-11-06T01:32:26.110764+0000","last_became_peered":"2023-11-06T01:32:26.110764+0000","last_unstale":"2023-11-06T01:33:26.122520+0000","last_undegraded":"2023-11-06T01:33:26.122520+0000","last_fullsized":"2023-11-06T01:33:26.122520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:46.422701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.127151+0000","last_change":"2023-11-06T01:33:27.127151+0000","last_active":"2023-11-06T01:33:27.127151+0000","last_peered":"2023-11-06T01:33:27.127151+0000","last_clean":"2023-11-06T01:33:27.127151+0000","last_became_active":"2023-11-06T01:33:27.126884+0000","last_became_peered":"2023-11-06T01:33:27.126884+0000","last_unstale":"2023-11-06T01:33:27.127151+0000","last_undegraded":"2023-11-06T01:33:27.127151+0000","last_fullsized":"2023-11-06T01:33:27.127151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:03:18.021947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122447+0000","last_change":"2023-11-06T01:32:26.118169+0000","last_active":"2023-11-06T01:33:26.122447+0000","last_peered":"2023-11-06T01:33:26.122447+0000","last_clean":"2023-11-06T01:33:26.122447+0000","last_became_active":"2023-11-06T01:32:26.117837+0000","last_became_peered":"2023-11-06T01:32:26.117837+0000","last_unstale":"2023-11-06T01:33:26.122447+0000","last_undegraded":"2023-11-06T01:33:26.122447+0000","last_fullsized":"2023-11-06T01:33:26.122447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:46.838927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121234+0000","last_change":"2023-11-06T01:32:26.110020+0000","last_active":"2023-11-06T01:33:26.121234+0000","last_peered":"2023-11-06T01:33:26.121234+0000","last_clean":"2023-11-06T01:33:26.121234+0000","last_became_active":"2023-11-06T01:32:26.109707+0000","last_became_peered":"2023-11-06T01:32:26.109707+0000","last_unstale":"2023-11-06T01:33:26.121234+0000","last_undegraded":"2023-11-06T01:33:26.121234+0000","last_fullsized":"2023-11-06T01:33:26.121234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:00.016258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.115738+0000","last_change":"2023-11-06T01:32:26.119949+0000","last_active":"2023-11-06T01:33:26.115738+0000","last_peered":"2023-11-06T01:33:26.115738+0000","last_clean":"2023-11-06T01:33:26.115738+0000","last_became_active":"2023-11-06T01:32:26.117594+0000","last_became_peered":"2023-11-06T01:32:26.117594+0000","last_unstale":"2023-11-06T01:33:26.115738+0000","last_undegraded":"2023-11-06T01:33:26.115738+0000","last_fullsized":"2023-11-06T01:33:26.115738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:17:22.757846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121991+0000","last_change":"2023-11-06T01:32:26.123176+0000","last_active":"2023-11-06T01:33:26.121991+0000","last_peered":"2023-11-06T01:33:26.121991+0000","last_clean":"2023-11-06T01:33:26.121991+0000","last_became_active":"2023-11-06T01:32:26.116687+0000","last_became_peered":"2023-11-06T01:32:26.116687+0000","last_unstale":"2023-11-06T01:33:26.121991+0000","last_undegraded":"2023-11-06T01:33:26.121991+0000","last_fullsized":"2023-11-06T01:33:26.121991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:30.422426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":131,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:51.423488+0000","last_change":"2023-11-06T01:33:27.401065+0000","last_active":"2023-11-06T01:33:51.423488+0000","last_peered":"2023-11-06T01:33:51.423488+0000","last_clean":"2023-11-06T01:33:51.423488+0000","last_became_active":"2023-11-06T01:33:27.391423+0000","last_became_peered":"2023-11-06T01:33:27.391423+0000","last_unstale":"2023-11-06T01:33:51.423488+0000","last_undegraded":"2023-11-06T01:33:51.423488+0000","last_fullsized":"2023-11-06T01:33:51.423488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:00:05.573076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118711+0000","last_change":"2023-11-06T01:32:26.126350+0000","last_active":"2023-11-06T01:33:26.118711+0000","last_peered":"2023-11-06T01:33:26.118711+0000","last_clean":"2023-11-06T01:33:26.118711+0000","last_became_active":"2023-11-06T01:32:26.124703+0000","last_became_peered":"2023-11-06T01:32:26.124703+0000","last_unstale":"2023-11-06T01:33:26.118711+0000","last_undegraded":"2023-11-06T01:33:26.118711+0000","last_fullsized":"2023-11-06T01:33:26.118711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:08:25.866045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118659+0000","last_change":"2023-11-06T01:32:25.680184+0000","last_active":"2023-11-06T01:33:26.118659+0000","last_peered":"2023-11-06T01:33:26.118659+0000","last_clean":"2023-11-06T01:33:26.118659+0000","last_became_active":"2023-11-06T01:32:25.679859+0000","last_became_peered":"2023-11-06T01:32:25.679859+0000","last_unstale":"2023-11-06T01:33:26.118659+0000","last_undegraded":"2023-11-06T01:33:26.118659+0000","last_fullsized":"2023-11-06T01:33:26.118659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:04:34.153562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124048+0000","last_change":"2023-11-06T01:32:26.114511+0000","last_active":"2023-11-06T01:33:26.124048+0000","last_peered":"2023-11-06T01:33:26.124048+0000","last_clean":"2023-11-06T01:33:26.124048+0000","last_became_active":"2023-11-06T01:32:26.114182+0000","last_became_peered":"2023-11-06T01:32:26.114182+0000","last_unstale":"2023-11-06T01:33:26.124048+0000","last_undegraded":"2023-11-06T01:33:26.124048+0000","last_fullsized":"2023-11-06T01:33:26.124048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:49:10.694358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120210+0000","last_change":"2023-11-06T01:32:25.681453+0000","last_active":"2023-11-06T01:33:26.120210+0000","last_peered":"2023-11-06T01:33:26.120210+0000","last_clean":"2023-11-06T01:33:26.120210+0000","last_became_active":"2023-11-06T01:32:25.681180+0000","last_became_peered":"2023-11-06T01:32:25.681180+0000","last_unstale":"2023-11-06T01:33:26.120210+0000","last_undegraded":"2023-11-06T01:33:26.120210+0000","last_fullsized":"2023-11-06T01:33:26.120210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:39:19.738479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122292+0000","last_change":"2023-11-06T01:32:26.118748+0000","last_active":"2023-11-06T01:33:26.122292+0000","last_peered":"2023-11-06T01:33:26.122292+0000","last_clean":"2023-11-06T01:33:26.122292+0000","last_became_active":"2023-11-06T01:32:26.118449+0000","last_became_peered":"2023-11-06T01:32:26.118449+0000","last_unstale":"2023-11-06T01:33:26.122292+0000","last_undegraded":"2023-11-06T01:33:26.122292+0000","last_fullsized":"2023-11-06T01:33:26.122292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:28.134553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119846+0000","last_change":"2023-11-06T01:32:25.680967+0000","last_active":"2023-11-06T01:33:26.119846+0000","last_peered":"2023-11-06T01:33:26.119846+0000","last_clean":"2023-11-06T01:33:26.119846+0000","last_became_active":"2023-11-06T01:32:25.678829+0000","last_became_peered":"2023-11-06T01:32:25.678829+0000","last_unstale":"2023-11-06T01:33:26.119846+0000","last_undegraded":"2023-11-06T01:33:26.119846+0000","last_fullsized":"2023-11-06T01:33:26.119846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:40:07.691852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123063+0000","last_change":"2023-11-06T01:32:26.122692+0000","last_active":"2023-11-06T01:33:26.123063+0000","last_peered":"2023-11-06T01:33:26.123063+0000","last_clean":"2023-11-06T01:33:26.123063+0000","last_became_active":"2023-11-06T01:32:26.121960+0000","last_became_peered":"2023-11-06T01:32:26.121960+0000","last_unstale":"2023-11-06T01:33:26.123063+0000","last_undegraded":"2023-11-06T01:33:26.123063+0000","last_fullsized":"2023-11-06T01:33:26.123063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:48.010467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:52.426673+0000","last_change":"2023-11-06T01:33:27.403594+0000","last_active":"2023-11-06T01:33:52.426673+0000","last_peered":"2023-11-06T01:33:52.426673+0000","last_clean":"2023-11-06T01:33:52.426673+0000","last_became_active":"2023-11-06T01:33:27.391131+0000","last_became_peered":"2023-11-06T01:33:27.391131+0000","last_unstale":"2023-11-06T01:33:52.426673+0000","last_undegraded":"2023-11-06T01:33:52.426673+0000","last_fullsized":"2023-11-06T01:33:52.426673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:55:10.577522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119781+0000","last_change":"2023-11-06T01:32:26.116839+0000","last_active":"2023-11-06T01:33:26.119781+0000","last_peered":"2023-11-06T01:33:26.119781+0000","last_clean":"2023-11-06T01:33:26.119781+0000","last_became_active":"2023-11-06T01:32:26.116535+0000","last_became_peered":"2023-11-06T01:32:26.116535+0000","last_unstale":"2023-11-06T01:33:26.119781+0000","last_undegraded":"2023-11-06T01:33:26.119781+0000","last_fullsized":"2023-11-06T01:33:26.119781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:11.219626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123841+0000","last_change":"2023-11-06T01:32:25.678426+0000","last_active":"2023-11-06T01:33:26.123841+0000","last_peered":"2023-11-06T01:33:26.123841+0000","last_clean":"2023-11-06T01:33:26.123841+0000","last_became_active":"2023-11-06T01:32:25.678072+0000","last_became_peered":"2023-11-06T01:32:25.678072+0000","last_unstale":"2023-11-06T01:33:26.123841+0000","last_undegraded":"2023-11-06T01:33:26.123841+0000","last_fullsized":"2023-11-06T01:33:26.123841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:09:41.630098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124752+0000","last_change":"2023-11-06T01:32:26.119184+0000","last_active":"2023-11-06T01:33:26.124752+0000","last_peered":"2023-11-06T01:33:26.124752+0000","last_clean":"2023-11-06T01:33:26.124752+0000","last_became_active":"2023-11-06T01:32:26.118896+0000","last_became_peered":"2023-11-06T01:32:26.118896+0000","last_unstale":"2023-11-06T01:33:26.124752+0000","last_undegraded":"2023-11-06T01:33:26.124752+0000","last_fullsized":"2023-11-06T01:33:26.124752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:09:17.351315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129133+0000","last_change":"2023-11-06T01:32:25.685621+0000","last_active":"2023-11-06T01:33:26.129133+0000","last_peered":"2023-11-06T01:33:26.129133+0000","last_clean":"2023-11-06T01:33:26.129133+0000","last_became_active":"2023-11-06T01:32:25.684965+0000","last_became_peered":"2023-11-06T01:32:25.684965+0000","last_unstale":"2023-11-06T01:33:26.129133+0000","last_undegraded":"2023-11-06T01:33:26.129133+0000","last_fullsized":"2023-11-06T01:33:26.129133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:28:00.922680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118211+0000","last_change":"2023-11-06T01:32:26.122750+0000","last_active":"2023-11-06T01:33:26.118211+0000","last_peered":"2023-11-06T01:33:26.118211+0000","last_clean":"2023-11-06T01:33:26.118211+0000","last_became_active":"2023-11-06T01:32:26.122241+0000","last_became_peered":"2023-11-06T01:32:26.122241+0000","last_unstale":"2023-11-06T01:33:26.118211+0000","last_undegraded":"2023-11-06T01:33:26.118211+0000","last_fullsized":"2023-11-06T01:33:26.118211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:04.402197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124078+0000","last_change":"2023-11-06T01:32:25.681594+0000","last_active":"2023-11-06T01:33:26.124078+0000","last_peered":"2023-11-06T01:33:26.124078+0000","last_clean":"2023-11-06T01:33:26.124078+0000","last_became_active":"2023-11-06T01:32:25.681303+0000","last_became_peered":"2023-11-06T01:32:25.681303+0000","last_unstale":"2023-11-06T01:33:26.124078+0000","last_undegraded":"2023-11-06T01:33:26.124078+0000","last_fullsized":"2023-11-06T01:33:26.124078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:33.141440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.115522+0000","last_change":"2023-11-06T01:32:26.117234+0000","last_active":"2023-11-06T01:33:26.115522+0000","last_peered":"2023-11-06T01:33:26.115522+0000","last_clean":"2023-11-06T01:33:26.115522+0000","last_became_active":"2023-11-06T01:32:26.116915+0000","last_became_peered":"2023-11-06T01:32:26.116915+0000","last_unstale":"2023-11-06T01:33:26.115522+0000","last_undegraded":"2023-11-06T01:33:26.115522+0000","last_fullsized":"2023-11-06T01:33:26.115522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:05:27.521552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":185,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:54.145360+0000","last_change":"2023-11-06T01:32:25.682827+0000","last_active":"2023-11-06T01:33:54.145360+0000","last_peered":"2023-11-06T01:33:54.145360+0000","last_clean":"2023-11-06T01:33:54.145360+0000","last_became_active":"2023-11-06T01:32:25.682470+0000","last_became_peered":"2023-11-06T01:32:25.682470+0000","last_unstale":"2023-11-06T01:33:54.145360+0000","last_undegraded":"2023-11-06T01:33:54.145360+0000","last_fullsized":"2023-11-06T01:33:54.145360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:43.632070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123248+0000","last_change":"2023-11-06T01:32:26.123918+0000","last_active":"2023-11-06T01:33:26.123248+0000","last_peered":"2023-11-06T01:33:26.123248+0000","last_clean":"2023-11-06T01:33:26.123248+0000","last_became_active":"2023-11-06T01:32:26.123450+0000","last_became_peered":"2023-11-06T01:32:26.123450+0000","last_unstale":"2023-11-06T01:33:26.123248+0000","last_undegraded":"2023-11-06T01:33:26.123248+0000","last_fullsized":"2023-11-06T01:33:26.123248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:38.227311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121215+0000","last_change":"2023-11-06T01:32:25.680051+0000","last_active":"2023-11-06T01:33:26.121215+0000","last_peered":"2023-11-06T01:33:26.121215+0000","last_clean":"2023-11-06T01:33:26.121215+0000","last_became_active":"2023-11-06T01:32:25.679705+0000","last_became_peered":"2023-11-06T01:32:25.679705+0000","last_unstale":"2023-11-06T01:33:26.121215+0000","last_undegraded":"2023-11-06T01:33:26.121215+0000","last_fullsized":"2023-11-06T01:33:26.121215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:18:22.620036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.126354+0000","last_change":"2023-11-06T01:33:27.126354+0000","last_active":"2023-11-06T01:33:27.126354+0000","last_peered":"2023-11-06T01:33:27.126354+0000","last_clean":"2023-11-06T01:33:27.126354+0000","last_became_active":"2023-11-06T01:33:27.125998+0000","last_became_peered":"2023-11-06T01:33:27.125998+0000","last_unstale":"2023-11-06T01:33:27.126354+0000","last_undegraded":"2023-11-06T01:33:27.126354+0000","last_fullsized":"2023-11-06T01:33:27.126354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:00:47.689894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123147+0000","last_change":"2023-11-06T01:32:25.678689+0000","last_active":"2023-11-06T01:33:26.123147+0000","last_peered":"2023-11-06T01:33:26.123147+0000","last_clean":"2023-11-06T01:33:26.123147+0000","last_became_active":"2023-11-06T01:32:25.678355+0000","last_became_peered":"2023-11-06T01:32:25.678355+0000","last_unstale":"2023-11-06T01:33:26.123147+0000","last_undegraded":"2023-11-06T01:33:26.123147+0000","last_fullsized":"2023-11-06T01:33:26.123147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:23.007596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122348+0000","last_change":"2023-11-06T01:32:26.118120+0000","last_active":"2023-11-06T01:33:26.122348+0000","last_peered":"2023-11-06T01:33:26.122348+0000","last_clean":"2023-11-06T01:33:26.122348+0000","last_became_active":"2023-11-06T01:32:26.117762+0000","last_became_peered":"2023-11-06T01:32:26.117762+0000","last_unstale":"2023-11-06T01:33:26.122348+0000","last_undegraded":"2023-11-06T01:33:26.122348+0000","last_fullsized":"2023-11-06T01:33:26.122348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:28:42.417463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'30","reported_seq":219,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:54.430124+0000","last_change":"2023-11-06T01:32:25.676036+0000","last_active":"2023-11-06T01:33:54.430124+0000","last_peered":"2023-11-06T01:33:54.430124+0000","last_clean":"2023-11-06T01:33:54.430124+0000","last_became_active":"2023-11-06T01:32:25.675672+0000","last_became_peered":"2023-11-06T01:32:25.675672+0000","last_unstale":"2023-11-06T01:33:54.430124+0000","last_undegraded":"2023-11-06T01:33:54.430124+0000","last_fullsized":"2023-11-06T01:33:54.430124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:30:54.060926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58677,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":175,"num_read_kb":140,"num_write":32,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120047+0000","last_change":"2023-11-06T01:32:26.118832+0000","last_active":"2023-11-06T01:33:26.120047+0000","last_peered":"2023-11-06T01:33:26.120047+0000","last_clean":"2023-11-06T01:33:26.120047+0000","last_became_active":"2023-11-06T01:32:26.118551+0000","last_became_peered":"2023-11-06T01:32:26.118551+0000","last_unstale":"2023-11-06T01:33:26.120047+0000","last_undegraded":"2023-11-06T01:33:26.120047+0000","last_fullsized":"2023-11-06T01:33:26.120047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:23.031203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.136439+0000","last_change":"2023-11-06T01:33:27.136439+0000","last_active":"2023-11-06T01:33:27.136439+0000","last_peered":"2023-11-06T01:33:27.136439+0000","last_clean":"2023-11-06T01:33:27.136439+0000","last_became_active":"2023-11-06T01:33:27.123153+0000","last_became_peered":"2023-11-06T01:33:27.123153+0000","last_unstale":"2023-11-06T01:33:27.136439+0000","last_undegraded":"2023-11-06T01:33:27.136439+0000","last_fullsized":"2023-11-06T01:33:27.136439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:52:29.385129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129312+0000","last_change":"2023-11-06T01:32:26.113421+0000","last_active":"2023-11-06T01:33:26.129312+0000","last_peered":"2023-11-06T01:33:26.129312+0000","last_clean":"2023-11-06T01:33:26.129312+0000","last_became_active":"2023-11-06T01:32:26.113085+0000","last_became_peered":"2023-11-06T01:32:26.113085+0000","last_unstale":"2023-11-06T01:33:26.129312+0000","last_undegraded":"2023-11-06T01:33:26.129312+0000","last_fullsized":"2023-11-06T01:33:26.129312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:15:04.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123170+0000","last_change":"2023-11-06T01:32:25.680293+0000","last_active":"2023-11-06T01:33:26.123170+0000","last_peered":"2023-11-06T01:33:26.123170+0000","last_clean":"2023-11-06T01:33:26.123170+0000","last_became_active":"2023-11-06T01:32:25.679803+0000","last_became_peered":"2023-11-06T01:32:25.679803+0000","last_unstale":"2023-11-06T01:33:26.123170+0000","last_undegraded":"2023-11-06T01:33:26.123170+0000","last_fullsized":"2023-11-06T01:33:26.123170+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:28.562797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120305+0000","last_change":"2023-11-06T01:32:26.115996+0000","last_active":"2023-11-06T01:33:26.120305+0000","last_peered":"2023-11-06T01:33:26.120305+0000","last_clean":"2023-11-06T01:33:26.120305+0000","last_became_active":"2023-11-06T01:32:26.115407+0000","last_became_peered":"2023-11-06T01:32:26.115407+0000","last_unstale":"2023-11-06T01:33:26.120305+0000","last_undegraded":"2023-11-06T01:33:26.120305+0000","last_fullsized":"2023-11-06T01:33:26.120305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:08.069698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":103,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:51.124706+0000","last_change":"2023-11-06T01:32:25.680958+0000","last_active":"2023-11-06T01:33:51.124706+0000","last_peered":"2023-11-06T01:33:51.124706+0000","last_clean":"2023-11-06T01:33:51.124706+0000","last_became_active":"2023-11-06T01:32:25.680616+0000","last_became_peered":"2023-11-06T01:32:25.680616+0000","last_unstale":"2023-11-06T01:33:51.124706+0000","last_undegraded":"2023-11-06T01:33:51.124706+0000","last_fullsized":"2023-11-06T01:33:51.124706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:02.631755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120959+0000","last_change":"2023-11-06T01:32:26.115464+0000","last_active":"2023-11-06T01:33:26.120959+0000","last_peered":"2023-11-06T01:33:26.120959+0000","last_clean":"2023-11-06T01:33:26.120959+0000","last_became_active":"2023-11-06T01:32:26.115090+0000","last_became_peered":"2023-11-06T01:32:26.115090+0000","last_unstale":"2023-11-06T01:33:26.120959+0000","last_undegraded":"2023-11-06T01:33:26.120959+0000","last_fullsized":"2023-11-06T01:33:26.120959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:08.018044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119958+0000","last_change":"2023-11-06T01:32:25.677787+0000","last_active":"2023-11-06T01:33:26.119958+0000","last_peered":"2023-11-06T01:33:26.119958+0000","last_clean":"2023-11-06T01:33:26.119958+0000","last_became_active":"2023-11-06T01:32:25.677527+0000","last_became_peered":"2023-11-06T01:32:25.677527+0000","last_unstale":"2023-11-06T01:33:26.119958+0000","last_undegraded":"2023-11-06T01:33:26.119958+0000","last_fullsized":"2023-11-06T01:33:26.119958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:34:09.452965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.391503+0000","last_change":"2023-11-06T01:33:27.391176+0000","last_active":"2023-11-06T01:33:27.391503+0000","last_peered":"2023-11-06T01:33:27.391503+0000","last_clean":"2023-11-06T01:33:27.391503+0000","last_became_active":"2023-11-06T01:33:27.390893+0000","last_became_peered":"2023-11-06T01:33:27.390893+0000","last_unstale":"2023-11-06T01:33:27.391503+0000","last_undegraded":"2023-11-06T01:33:27.391503+0000","last_fullsized":"2023-11-06T01:33:27.391503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:20:47.382152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120659+0000","last_change":"2023-11-06T01:32:25.677257+0000","last_active":"2023-11-06T01:33:26.120659+0000","last_peered":"2023-11-06T01:33:26.120659+0000","last_clean":"2023-11-06T01:33:26.120659+0000","last_became_active":"2023-11-06T01:32:25.676536+0000","last_became_peered":"2023-11-06T01:32:25.676536+0000","last_unstale":"2023-11-06T01:33:26.120659+0000","last_undegraded":"2023-11-06T01:33:26.120659+0000","last_fullsized":"2023-11-06T01:33:26.120659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:53:33.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119820+0000","last_change":"2023-11-06T01:32:26.111447+0000","last_active":"2023-11-06T01:33:26.119820+0000","last_peered":"2023-11-06T01:33:26.119820+0000","last_clean":"2023-11-06T01:33:26.119820+0000","last_became_active":"2023-11-06T01:32:26.111132+0000","last_became_peered":"2023-11-06T01:32:26.111132+0000","last_unstale":"2023-11-06T01:33:26.119820+0000","last_undegraded":"2023-11-06T01:33:26.119820+0000","last_fullsized":"2023-11-06T01:33:26.119820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:55.385003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.115705+0000","last_change":"2023-11-06T01:32:25.682562+0000","last_active":"2023-11-06T01:33:26.115705+0000","last_peered":"2023-11-06T01:33:26.115705+0000","last_clean":"2023-11-06T01:33:26.115705+0000","last_became_active":"2023-11-06T01:32:25.682227+0000","last_became_peered":"2023-11-06T01:32:25.682227+0000","last_unstale":"2023-11-06T01:33:26.115705+0000","last_undegraded":"2023-11-06T01:33:26.115705+0000","last_fullsized":"2023-11-06T01:33:26.115705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:40.113429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.129164+0000","last_change":"2023-11-06T01:33:27.129164+0000","last_active":"2023-11-06T01:33:27.129164+0000","last_peered":"2023-11-06T01:33:27.129164+0000","last_clean":"2023-11-06T01:33:27.129164+0000","last_became_active":"2023-11-06T01:33:27.123473+0000","last_became_peered":"2023-11-06T01:33:27.123473+0000","last_unstale":"2023-11-06T01:33:27.129164+0000","last_undegraded":"2023-11-06T01:33:27.129164+0000","last_fullsized":"2023-11-06T01:33:27.129164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:03:52.561653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:53.427221+0000","last_change":"2023-11-06T01:32:25.688176+0000","last_active":"2023-11-06T01:33:53.427221+0000","last_peered":"2023-11-06T01:33:53.427221+0000","last_clean":"2023-11-06T01:33:53.427221+0000","last_became_active":"2023-11-06T01:32:25.687745+0000","last_became_peered":"2023-11-06T01:32:25.687745+0000","last_unstale":"2023-11-06T01:33:53.427221+0000","last_undegraded":"2023-11-06T01:33:53.427221+0000","last_fullsized":"2023-11-06T01:33:53.427221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:21:23.566389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124027+0000","last_change":"2023-11-06T01:32:26.111700+0000","last_active":"2023-11-06T01:33:26.124027+0000","last_peered":"2023-11-06T01:33:26.124027+0000","last_clean":"2023-11-06T01:33:26.124027+0000","last_became_active":"2023-11-06T01:32:26.111380+0000","last_became_peered":"2023-11-06T01:32:26.111380+0000","last_unstale":"2023-11-06T01:33:26.124027+0000","last_undegraded":"2023-11-06T01:33:26.124027+0000","last_fullsized":"2023-11-06T01:33:26.124027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:56:47.874663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124206+0000","last_change":"2023-11-06T01:32:26.121906+0000","last_active":"2023-11-06T01:33:26.124206+0000","last_peered":"2023-11-06T01:33:26.124206+0000","last_clean":"2023-11-06T01:33:26.124206+0000","last_became_active":"2023-11-06T01:32:26.121292+0000","last_became_peered":"2023-11-06T01:32:26.121292+0000","last_unstale":"2023-11-06T01:33:26.124206+0000","last_undegraded":"2023-11-06T01:33:26.124206+0000","last_fullsized":"2023-11-06T01:33:26.124206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.532098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":103,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:52.125485+0000","last_change":"2023-11-06T01:32:25.685814+0000","last_active":"2023-11-06T01:33:52.125485+0000","last_peered":"2023-11-06T01:33:52.125485+0000","last_clean":"2023-11-06T01:33:52.125485+0000","last_became_active":"2023-11-06T01:32:25.684890+0000","last_became_peered":"2023-11-06T01:32:25.684890+0000","last_unstale":"2023-11-06T01:33:52.125485+0000","last_undegraded":"2023-11-06T01:33:52.125485+0000","last_fullsized":"2023-11-06T01:33:52.125485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:21.950945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118629+0000","last_change":"2023-11-06T01:32:26.121752+0000","last_active":"2023-11-06T01:33:26.118629+0000","last_peered":"2023-11-06T01:33:26.118629+0000","last_clean":"2023-11-06T01:33:26.118629+0000","last_became_active":"2023-11-06T01:32:26.121447+0000","last_became_peered":"2023-11-06T01:32:26.121447+0000","last_unstale":"2023-11-06T01:33:26.118629+0000","last_undegraded":"2023-11-06T01:33:26.118629+0000","last_fullsized":"2023-11-06T01:33:26.118629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:51:19.727335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121153+0000","last_change":"2023-11-06T01:32:25.682727+0000","last_active":"2023-11-06T01:33:26.121153+0000","last_peered":"2023-11-06T01:33:26.121153+0000","last_clean":"2023-11-06T01:33:26.121153+0000","last_became_active":"2023-11-06T01:32:25.682377+0000","last_became_peered":"2023-11-06T01:32:25.682377+0000","last_unstale":"2023-11-06T01:33:26.121153+0000","last_undegraded":"2023-11-06T01:33:26.121153+0000","last_fullsized":"2023-11-06T01:33:26.121153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:17:38.614350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119733+0000","last_change":"2023-11-06T01:31:27.931076+0000","last_active":"2023-11-06T01:33:26.119733+0000","last_peered":"2023-11-06T01:33:26.119733+0000","last_clean":"2023-11-06T01:33:26.119733+0000","last_became_active":"2023-11-06T01:31:27.930669+0000","last_became_peered":"2023-11-06T01:31:27.930669+0000","last_unstale":"2023-11-06T01:33:26.119733+0000","last_undegraded":"2023-11-06T01:33:26.119733+0000","last_fullsized":"2023-11-06T01:33:26.119733+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_clean_scrub_stamp":"2023-11-06T01:28:36.252549+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:47:48.729325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.128723+0000","last_change":"2023-11-06T01:32:26.110055+0000","last_active":"2023-11-06T01:33:26.128723+0000","last_peered":"2023-11-06T01:33:26.128723+0000","last_clean":"2023-11-06T01:33:26.128723+0000","last_became_active":"2023-11-06T01:32:26.109640+0000","last_became_peered":"2023-11-06T01:32:26.109640+0000","last_unstale":"2023-11-06T01:33:26.128723+0000","last_undegraded":"2023-11-06T01:33:26.128723+0000","last_fullsized":"2023-11-06T01:33:26.128723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:45:43.076910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119178+0000","last_change":"2023-11-06T01:32:25.679885+0000","last_active":"2023-11-06T01:33:26.119178+0000","last_peered":"2023-11-06T01:33:26.119178+0000","last_clean":"2023-11-06T01:33:26.119178+0000","last_became_active":"2023-11-06T01:32:25.679518+0000","last_became_peered":"2023-11-06T01:32:25.679518+0000","last_unstale":"2023-11-06T01:33:26.119178+0000","last_undegraded":"2023-11-06T01:33:26.119178+0000","last_fullsized":"2023-11-06T01:33:26.119178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:44.558534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120105+0000","last_change":"2023-11-06T01:32:25.682016+0000","last_active":"2023-11-06T01:33:26.120105+0000","last_peered":"2023-11-06T01:33:26.120105+0000","last_clean":"2023-11-06T01:33:26.120105+0000","last_became_active":"2023-11-06T01:32:25.681753+0000","last_became_peered":"2023-11-06T01:32:25.681753+0000","last_unstale":"2023-11-06T01:33:26.120105+0000","last_undegraded":"2023-11-06T01:33:26.120105+0000","last_fullsized":"2023-11-06T01:33:26.120105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:08.575728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129585+0000","last_change":"2023-11-06T01:32:26.117109+0000","last_active":"2023-11-06T01:33:26.129585+0000","last_peered":"2023-11-06T01:33:26.129585+0000","last_clean":"2023-11-06T01:33:26.129585+0000","last_became_active":"2023-11-06T01:32:26.116769+0000","last_became_peered":"2023-11-06T01:32:26.116769+0000","last_unstale":"2023-11-06T01:33:26.129585+0000","last_undegraded":"2023-11-06T01:33:26.129585+0000","last_fullsized":"2023-11-06T01:33:26.129585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:48:04.188806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.142547+0000","last_change":"2023-11-06T01:33:27.142547+0000","last_active":"2023-11-06T01:33:27.142547+0000","last_peered":"2023-11-06T01:33:27.142547+0000","last_clean":"2023-11-06T01:33:27.142547+0000","last_became_active":"2023-11-06T01:33:27.127408+0000","last_became_peered":"2023-11-06T01:33:27.127408+0000","last_unstale":"2023-11-06T01:33:27.142547+0000","last_undegraded":"2023-11-06T01:33:27.142547+0000","last_fullsized":"2023-11-06T01:33:27.142547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:08:10.084437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118671+0000","last_change":"2023-11-06T01:32:26.116351+0000","last_active":"2023-11-06T01:33:26.118671+0000","last_peered":"2023-11-06T01:33:26.118671+0000","last_clean":"2023-11-06T01:33:26.118671+0000","last_became_active":"2023-11-06T01:32:26.116044+0000","last_became_peered":"2023-11-06T01:32:26.116044+0000","last_unstale":"2023-11-06T01:33:26.118671+0000","last_undegraded":"2023-11-06T01:33:26.118671+0000","last_fullsized":"2023-11-06T01:33:26.118671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:20.816412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121520+0000","last_change":"2023-11-06T01:32:25.676764+0000","last_active":"2023-11-06T01:33:26.121520+0000","last_peered":"2023-11-06T01:33:26.121520+0000","last_clean":"2023-11-06T01:33:26.121520+0000","last_became_active":"2023-11-06T01:32:25.676418+0000","last_became_peered":"2023-11-06T01:32:25.676418+0000","last_unstale":"2023-11-06T01:33:26.121520+0000","last_undegraded":"2023-11-06T01:33:26.121520+0000","last_fullsized":"2023-11-06T01:33:26.121520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:49.320532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123031+0000","last_change":"2023-11-06T01:32:26.121107+0000","last_active":"2023-11-06T01:33:26.123031+0000","last_peered":"2023-11-06T01:33:26.123031+0000","last_clean":"2023-11-06T01:33:26.123031+0000","last_became_active":"2023-11-06T01:32:26.120790+0000","last_became_peered":"2023-11-06T01:32:26.120790+0000","last_unstale":"2023-11-06T01:33:26.123031+0000","last_undegraded":"2023-11-06T01:33:26.123031+0000","last_fullsized":"2023-11-06T01:33:26.123031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:34:46.660301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122657+0000","last_change":"2023-11-06T01:32:25.678959+0000","last_active":"2023-11-06T01:33:26.122657+0000","last_peered":"2023-11-06T01:33:26.122657+0000","last_clean":"2023-11-06T01:33:26.122657+0000","last_became_active":"2023-11-06T01:32:25.678564+0000","last_became_peered":"2023-11-06T01:32:25.678564+0000","last_unstale":"2023-11-06T01:33:26.122657+0000","last_undegraded":"2023-11-06T01:33:26.122657+0000","last_fullsized":"2023-11-06T01:33:26.122657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:30:06.304889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119508+0000","last_change":"2023-11-06T01:32:26.113222+0000","last_active":"2023-11-06T01:33:26.119508+0000","last_peered":"2023-11-06T01:33:26.119508+0000","last_clean":"2023-11-06T01:33:26.119508+0000","last_became_active":"2023-11-06T01:32:26.112926+0000","last_became_peered":"2023-11-06T01:32:26.112926+0000","last_unstale":"2023-11-06T01:33:26.119508+0000","last_undegraded":"2023-11-06T01:33:26.119508+0000","last_fullsized":"2023-11-06T01:33:26.119508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:09:51.042469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118301+0000","last_change":"2023-11-06T01:32:25.684985+0000","last_active":"2023-11-06T01:33:26.118301+0000","last_peered":"2023-11-06T01:33:26.118301+0000","last_clean":"2023-11-06T01:33:26.118301+0000","last_became_active":"2023-11-06T01:32:25.684656+0000","last_became_peered":"2023-11-06T01:32:25.684656+0000","last_unstale":"2023-11-06T01:33:26.118301+0000","last_undegraded":"2023-11-06T01:33:26.118301+0000","last_fullsized":"2023-11-06T01:33:26.118301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:42:29.406803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120058+0000","last_change":"2023-11-06T01:32:26.107484+0000","last_active":"2023-11-06T01:33:26.120058+0000","last_peered":"2023-11-06T01:33:26.120058+0000","last_clean":"2023-11-06T01:33:26.120058+0000","last_became_active":"2023-11-06T01:32:26.106092+0000","last_became_peered":"2023-11-06T01:32:26.106092+0000","last_unstale":"2023-11-06T01:33:26.120058+0000","last_undegraded":"2023-11-06T01:33:26.120058+0000","last_fullsized":"2023-11-06T01:33:26.120058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:43:39.481290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123047+0000","last_change":"2023-11-06T01:32:25.679531+0000","last_active":"2023-11-06T01:33:26.123047+0000","last_peered":"2023-11-06T01:33:26.123047+0000","last_clean":"2023-11-06T01:33:26.123047+0000","last_became_active":"2023-11-06T01:32:25.679076+0000","last_became_peered":"2023-11-06T01:32:25.679076+0000","last_unstale":"2023-11-06T01:33:26.123047+0000","last_undegraded":"2023-11-06T01:33:26.123047+0000","last_fullsized":"2023-11-06T01:33:26.123047+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:37.381547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118418+0000","last_change":"2023-11-06T01:32:26.118584+0000","last_active":"2023-11-06T01:33:26.118418+0000","last_peered":"2023-11-06T01:33:26.118418+0000","last_clean":"2023-11-06T01:33:26.118418+0000","last_became_active":"2023-11-06T01:32:26.118285+0000","last_became_peered":"2023-11-06T01:32:26.118285+0000","last_unstale":"2023-11-06T01:33:26.118418+0000","last_undegraded":"2023-11-06T01:33:26.118418+0000","last_fullsized":"2023-11-06T01:33:26.118418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:52:33.057377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119913+0000","last_change":"2023-11-06T01:32:25.681214+0000","last_active":"2023-11-06T01:33:26.119913+0000","last_peered":"2023-11-06T01:33:26.119913+0000","last_clean":"2023-11-06T01:33:26.119913+0000","last_became_active":"2023-11-06T01:32:25.680907+0000","last_became_peered":"2023-11-06T01:32:25.680907+0000","last_unstale":"2023-11-06T01:33:26.119913+0000","last_undegraded":"2023-11-06T01:33:26.119913+0000","last_fullsized":"2023-11-06T01:33:26.119913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:04:58.248787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119830+0000","last_change":"2023-11-06T01:32:26.120862+0000","last_active":"2023-11-06T01:33:26.119830+0000","last_peered":"2023-11-06T01:33:26.119830+0000","last_clean":"2023-11-06T01:33:26.119830+0000","last_became_active":"2023-11-06T01:32:26.120307+0000","last_became_peered":"2023-11-06T01:32:26.120307+0000","last_unstale":"2023-11-06T01:33:26.119830+0000","last_undegraded":"2023-11-06T01:33:26.119830+0000","last_fullsized":"2023-11-06T01:33:26.119830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:45:35.611541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":105,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:53.144644+0000","last_change":"2023-11-06T01:32:25.675607+0000","last_active":"2023-11-06T01:33:53.144644+0000","last_peered":"2023-11-06T01:33:53.144644+0000","last_clean":"2023-11-06T01:33:53.144644+0000","last_became_active":"2023-11-06T01:32:25.675293+0000","last_became_peered":"2023-11-06T01:32:25.675293+0000","last_unstale":"2023-11-06T01:33:53.144644+0000","last_undegraded":"2023-11-06T01:33:53.144644+0000","last_fullsized":"2023-11-06T01:33:53.144644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:05.190844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1440,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120568+0000","last_change":"2023-11-06T01:32:26.116987+0000","last_active":"2023-11-06T01:33:26.120568+0000","last_peered":"2023-11-06T01:33:26.120568+0000","last_clean":"2023-11-06T01:33:26.120568+0000","last_became_active":"2023-11-06T01:32:26.116756+0000","last_became_peered":"2023-11-06T01:32:26.116756+0000","last_unstale":"2023-11-06T01:33:26.120568+0000","last_undegraded":"2023-11-06T01:33:26.120568+0000","last_fullsized":"2023-11-06T01:33:26.120568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:44.039520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122980+0000","last_change":"2023-11-06T01:32:25.675483+0000","last_active":"2023-11-06T01:33:26.122980+0000","last_peered":"2023-11-06T01:33:26.122980+0000","last_clean":"2023-11-06T01:33:26.122980+0000","last_became_active":"2023-11-06T01:32:25.675137+0000","last_became_peered":"2023-11-06T01:32:25.675137+0000","last_unstale":"2023-11-06T01:33:26.122980+0000","last_undegraded":"2023-11-06T01:33:26.122980+0000","last_fullsized":"2023-11-06T01:33:26.122980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:42.415165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120363+0000","last_change":"2023-11-06T01:32:26.115575+0000","last_active":"2023-11-06T01:33:26.120363+0000","last_peered":"2023-11-06T01:33:26.120363+0000","last_clean":"2023-11-06T01:33:26.120363+0000","last_became_active":"2023-11-06T01:32:26.115142+0000","last_became_peered":"2023-11-06T01:32:26.115142+0000","last_unstale":"2023-11-06T01:33:26.120363+0000","last_undegraded":"2023-11-06T01:33:26.120363+0000","last_fullsized":"2023-11-06T01:33:26.120363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:52.859340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129692+0000","last_change":"2023-11-06T01:32:25.676972+0000","last_active":"2023-11-06T01:33:26.129692+0000","last_peered":"2023-11-06T01:33:26.129692+0000","last_clean":"2023-11-06T01:33:26.129692+0000","last_became_active":"2023-11-06T01:32:25.676685+0000","last_became_peered":"2023-11-06T01:32:25.676685+0000","last_unstale":"2023-11-06T01:33:26.129692+0000","last_undegraded":"2023-11-06T01:33:26.129692+0000","last_fullsized":"2023-11-06T01:33:26.129692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:30:21.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121970+0000","last_change":"2023-11-06T01:32:26.115164+0000","last_active":"2023-11-06T01:33:26.121970+0000","last_peered":"2023-11-06T01:33:26.121970+0000","last_clean":"2023-11-06T01:33:26.121970+0000","last_became_active":"2023-11-06T01:32:26.114868+0000","last_became_peered":"2023-11-06T01:32:26.114868+0000","last_unstale":"2023-11-06T01:33:26.121970+0000","last_undegraded":"2023-11-06T01:33:26.121970+0000","last_fullsized":"2023-11-06T01:33:26.121970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:01:34.149871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.127653+0000","last_change":"2023-11-06T01:33:27.127653+0000","last_active":"2023-11-06T01:33:27.127653+0000","last_peered":"2023-11-06T01:33:27.127653+0000","last_clean":"2023-11-06T01:33:27.127653+0000","last_became_active":"2023-11-06T01:33:27.127151+0000","last_became_peered":"2023-11-06T01:33:27.127151+0000","last_unstale":"2023-11-06T01:33:27.127653+0000","last_undegraded":"2023-11-06T01:33:27.127653+0000","last_fullsized":"2023-11-06T01:33:27.127653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:54.825681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122463+0000","last_change":"2023-11-06T01:32:26.115336+0000","last_active":"2023-11-06T01:33:26.122463+0000","last_peered":"2023-11-06T01:33:26.122463+0000","last_clean":"2023-11-06T01:33:26.122463+0000","last_became_active":"2023-11-06T01:32:26.114946+0000","last_became_peered":"2023-11-06T01:32:26.114946+0000","last_unstale":"2023-11-06T01:33:26.122463+0000","last_undegraded":"2023-11-06T01:33:26.122463+0000","last_fullsized":"2023-11-06T01:33:26.122463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:19:13.651494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122948+0000","last_change":"2023-11-06T01:32:26.117794+0000","last_active":"2023-11-06T01:33:26.122948+0000","last_peered":"2023-11-06T01:33:26.122948+0000","last_clean":"2023-11-06T01:33:26.122948+0000","last_became_active":"2023-11-06T01:32:26.117425+0000","last_became_peered":"2023-11-06T01:32:26.117425+0000","last_unstale":"2023-11-06T01:33:26.122948+0000","last_undegraded":"2023-11-06T01:33:26.122948+0000","last_fullsized":"2023-11-06T01:33:26.122948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:00:24.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.125881+0000","last_change":"2023-11-06T01:33:27.125881+0000","last_active":"2023-11-06T01:33:27.125881+0000","last_peered":"2023-11-06T01:33:27.125881+0000","last_clean":"2023-11-06T01:33:27.125881+0000","last_became_active":"2023-11-06T01:33:27.125445+0000","last_became_peered":"2023-11-06T01:33:27.125445+0000","last_unstale":"2023-11-06T01:33:27.125881+0000","last_undegraded":"2023-11-06T01:33:27.125881+0000","last_fullsized":"2023-11-06T01:33:27.125881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:31.654256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129629+0000","last_change":"2023-11-06T01:32:26.109910+0000","last_active":"2023-11-06T01:33:26.129629+0000","last_peered":"2023-11-06T01:33:26.129629+0000","last_clean":"2023-11-06T01:33:26.129629+0000","last_became_active":"2023-11-06T01:32:26.109568+0000","last_became_peered":"2023-11-06T01:32:26.109568+0000","last_unstale":"2023-11-06T01:33:26.129629+0000","last_undegraded":"2023-11-06T01:33:26.129629+0000","last_fullsized":"2023-11-06T01:33:26.129629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:11.049749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120801+0000","last_change":"2023-11-06T01:32:26.114824+0000","last_active":"2023-11-06T01:33:26.120801+0000","last_peered":"2023-11-06T01:33:26.120801+0000","last_clean":"2023-11-06T01:33:26.120801+0000","last_became_active":"2023-11-06T01:32:26.114539+0000","last_became_peered":"2023-11-06T01:32:26.114539+0000","last_unstale":"2023-11-06T01:33:26.120801+0000","last_undegraded":"2023-11-06T01:33:26.120801+0000","last_fullsized":"2023-11-06T01:33:26.120801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:03:02.018769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122658+0000","last_change":"2023-11-06T01:32:26.114743+0000","last_active":"2023-11-06T01:33:26.122658+0000","last_peered":"2023-11-06T01:33:26.122658+0000","last_clean":"2023-11-06T01:33:26.122658+0000","last_became_active":"2023-11-06T01:32:26.114432+0000","last_became_peered":"2023-11-06T01:32:26.114432+0000","last_unstale":"2023-11-06T01:33:26.122658+0000","last_undegraded":"2023-11-06T01:33:26.122658+0000","last_fullsized":"2023-11-06T01:33:26.122658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:27:18.209810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118548+0000","last_change":"2023-11-06T01:32:26.125724+0000","last_active":"2023-11-06T01:33:26.118548+0000","last_peered":"2023-11-06T01:33:26.118548+0000","last_clean":"2023-11-06T01:33:26.118548+0000","last_became_active":"2023-11-06T01:32:26.125455+0000","last_became_peered":"2023-11-06T01:32:26.125455+0000","last_unstale":"2023-11-06T01:33:26.118548+0000","last_undegraded":"2023-11-06T01:33:26.118548+0000","last_fullsized":"2023-11-06T01:33:26.118548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:49.099308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124151+0000","last_change":"2023-11-06T01:32:26.119328+0000","last_active":"2023-11-06T01:33:26.124151+0000","last_peered":"2023-11-06T01:33:26.124151+0000","last_clean":"2023-11-06T01:33:26.124151+0000","last_became_active":"2023-11-06T01:32:26.119042+0000","last_became_peered":"2023-11-06T01:32:26.119042+0000","last_unstale":"2023-11-06T01:33:26.124151+0000","last_undegraded":"2023-11-06T01:33:26.124151+0000","last_fullsized":"2023-11-06T01:33:26.124151+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:36.143788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118310+0000","last_change":"2023-11-06T01:32:26.126538+0000","last_active":"2023-11-06T01:33:26.118310+0000","last_peered":"2023-11-06T01:33:26.118310+0000","last_clean":"2023-11-06T01:33:26.118310+0000","last_became_active":"2023-11-06T01:32:26.124898+0000","last_became_peered":"2023-11-06T01:32:26.124898+0000","last_unstale":"2023-11-06T01:33:26.118310+0000","last_undegraded":"2023-11-06T01:33:26.118310+0000","last_fullsized":"2023-11-06T01:33:26.118310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:33.019326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122059+0000","last_change":"2023-11-06T01:32:26.118751+0000","last_active":"2023-11-06T01:33:26.122059+0000","last_peered":"2023-11-06T01:33:26.122059+0000","last_clean":"2023-11-06T01:33:26.122059+0000","last_became_active":"2023-11-06T01:32:26.118359+0000","last_became_peered":"2023-11-06T01:32:26.118359+0000","last_unstale":"2023-11-06T01:33:26.122059+0000","last_undegraded":"2023-11-06T01:33:26.122059+0000","last_fullsized":"2023-11-06T01:33:26.122059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:55.960400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129434+0000","last_change":"2023-11-06T01:32:26.110133+0000","last_active":"2023-11-06T01:33:26.129434+0000","last_peered":"2023-11-06T01:33:26.129434+0000","last_clean":"2023-11-06T01:33:26.129434+0000","last_became_active":"2023-11-06T01:32:26.109793+0000","last_became_peered":"2023-11-06T01:32:26.109793+0000","last_unstale":"2023-11-06T01:33:26.129434+0000","last_undegraded":"2023-11-06T01:33:26.129434+0000","last_fullsized":"2023-11-06T01:33:26.129434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:00:03.956947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120467+0000","last_change":"2023-11-06T01:32:26.109828+0000","last_active":"2023-11-06T01:33:26.120467+0000","last_peered":"2023-11-06T01:33:26.120467+0000","last_clean":"2023-11-06T01:33:26.120467+0000","last_became_active":"2023-11-06T01:32:26.109526+0000","last_became_peered":"2023-11-06T01:32:26.109526+0000","last_unstale":"2023-11-06T01:33:26.120467+0000","last_undegraded":"2023-11-06T01:33:26.120467+0000","last_fullsized":"2023-11-06T01:33:26.120467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:36.561908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250075,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65618,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":855,"num_read_kb":496,"num_write":232,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1707,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":192216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481821,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":436587,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95799999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":1.002,"15min":1.002},"last":0.59599999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51200000000000001,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.68100000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.75700000000000001,"15min":0.77300000000000002},"min":{"1min":0.48299999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.86499999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80100000000000005,"5min":2.137,"15min":2.137},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71399999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78000000000000003,"5min":1.056,"15min":1.056},"last":0.54100000000000004}]},{"osd":6,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.877}]},{"osd":7,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.51900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92100000000000004,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.81899999999999995,"15min":0.83499999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89900000000000002,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.70599999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.66300000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1970000000000001,"5min":1.343,"15min":1.343},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72099999999999997,"15min":0.71499999999999997},"min":{"1min":0.58899999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65000000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":0.877},"min":{"1min":0.626,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.085,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78200000000000003,"15min":0.78500000000000003},"min":{"1min":0.59499999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.94999999999999996,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.78900000000000003}]}]},{"osd":10,"up_from":70,"seq":300647710754,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":1081580,"data_compressed":41972,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.54000000000000004}]},{"osd":1,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55800000000000005,"15min":0.55700000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.443}]},{"osd":2,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.55000000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0629999999999999,"5min":1.071,"15min":1.071},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0900000000000001,"5min":1.169,"15min":1.169},"last":0.63600000000000001}]},{"osd":3,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76300000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.48399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84799999999999998,"5min":1.052,"15min":1.052},"last":0.60399999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61899999999999999,"15min":0.60999999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.88900000000000001,"5min":1.133,"15min":1.133},"last":0.58299999999999996}]},{"osd":5,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.50800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.105,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82899999999999996,"15min":0.82699999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.044,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77900000000000003,"15min":0.78000000000000003},"min":{"1min":0.51800000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95099999999999996,"5min":1.147,"15min":1.147},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78700000000000003,"15min":0.78900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.079,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.875,"15min":0.875},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.133,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91600000000000004},"min":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.179,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.82099999999999995}]},{"osd":9,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.57099999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.026,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.85199999999999998,"15min":0.86099999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0549999999999999,"5min":1.246,"15min":1.246},"last":0.754}]},{"osd":11,"last update":"Mon Nov 6 01:33:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.145,"5min":1.226,"15min":1.226},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.82599999999999996,"15min":0.82099999999999995},"min":{"1min":0.63,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.76200000000000001}]}]},{"osd":9,"up_from":65,"seq":279172874278,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":436620,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.49199999999999999,"15min":0.50700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.9780000000000002,"5min":3.9780000000000002,"15min":3.9780000000000002},"last":0.28000000000000003}]},{"osd":1,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.66900000000000004,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77800000000000002,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":6.6210000000000004,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":0.46600000000000003}]},{"osd":2,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.67700000000000005,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.476,"5min":6.476,"15min":6.476},"last":0.39800000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59799999999999998,"15min":0.58499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":4.0270000000000001,"5min":4.0270000000000001,"15min":4.0270000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82199999999999995,"5min":0.87,"15min":0.87},"last":0.34799999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.70899999999999996,"15min":0.68000000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.47699999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77200000000000002,"15min":0.77100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":4.1589999999999998,"5min":4.1589999999999998,"15min":4.1589999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.79300000000000004,"15min":0.81499999999999995},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.44700000000000001}]},{"osd":6,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.80100000000000005,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":6.056,"5min":6.056,"15min":6.056},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.3700000000000001,"5min":6.3700000000000001,"15min":6.3700000000000001},"last":0.45900000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85999999999999999,"15min":0.86199999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.97799999999999998,"5min":1.603,"15min":1.603},"last":0.57599999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.83899999999999997,"15min":0.81599999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.524,"5min":6.524,"15min":6.524},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.85999999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":4.1900000000000004,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":0.41999999999999998}]},{"osd":10,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.84199999999999997,"15min":0.81799999999999995},"min":{"1min":0.51900000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.85899999999999999,"15min":0.84499999999999997},"min":{"1min":0.56799999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":4.1280000000000001,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":0.55000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":8,"up_from":58,"seq":249108103209,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":435872,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80400000000000005,"5min":1.177,"15min":1.177},"last":0.497},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":1.196,"15min":1.196},"last":0.70499999999999996}]},{"osd":1,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.50800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84399999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71699999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.47199999999999998,"15min":0.46999999999999997},"min":{"1min":0.35299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.48699999999999999,"15min":0.48199999999999998},"min":{"1min":0.40500000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60399999999999998}]},{"osd":3,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78300000000000003,"5min":1.083,"15min":1.083},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.497,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.498,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":1.24,"15min":1.24},"last":0.627},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.60199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94299999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72599999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58099999999999996,"15min":0.56799999999999995},"min":{"1min":0.52900000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.95599999999999996,"5min":1.036,"15min":1.036},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.61199999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.754}]},{"osd":7,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.57399999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.0600000000000001,"5min":1.353,"15min":1.353},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.72299999999999998,"15min":0.70499999999999996},"min":{"1min":0.58699999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.0149999999999999,"5min":1.276,"15min":1.276},"last":0.78500000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81399999999999995}]},{"osd":10,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.77200000000000002,"15min":0.75900000000000001},"min":{"1min":0.61899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0840000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.78400000000000003,"15min":0.76400000000000001},"min":{"1min":0.60399999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.88700000000000001}]},{"osd":11,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86699999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":1029257,"data_compressed":41972,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.873,"5min":1.157,"15min":1.157},"last":0.376}]},{"osd":1,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.433,"15min":0.40899999999999997},"min":{"1min":0.35899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79500000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.307,"15min":0.307},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.41099999999999998}]},{"osd":2,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.60899999999999999,"15min":0.58299999999999996},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.60599999999999998,"15min":0.57699999999999996},"min":{"1min":0.44700000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.69599999999999995,"15min":0.67600000000000005},"min":{"1min":0.45900000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.8090000000000002,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.63200000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56699999999999995,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.53700000000000003,"15min":0.50600000000000001},"min":{"1min":0.372,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":0.379}]},{"osd":5,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.66800000000000004,"15min":0.64200000000000002},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.66000000000000003,"15min":0.626},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71899999999999997,"15min":0.69599999999999995},"min":{"1min":0.57299999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73099999999999998,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.48299999999999998}]},{"osd":8,"last update":"Mon Nov 6 01:33:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.75900000000000001,"15min":0.72799999999999998},"min":{"1min":0.624,"5min":0.442,"15min":0.442},"max":{"1min":1.0209999999999999,"5min":2.395,"15min":2.395},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78300000000000003,"15min":0.753},"min":{"1min":0.63,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.089,"5min":2.484,"15min":2.484},"last":0.73699999999999999}]},{"osd":9,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.79400000000000004,"15min":0.78200000000000003},"min":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.78400000000000003}]},{"osd":10,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.51900000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.84899999999999998}]},{"osd":11,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.77300000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":436172,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.52100000000000002,"15min":0.52600000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.39800000000000002}]},{"osd":1,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.499},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.47199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.68899999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.53500000000000003}]},{"osd":2,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.60699999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70899999999999996,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52300000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.57399999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.47799999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.61199999999999999,"15min":0.62},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78800000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.44900000000000001}]},{"osd":5,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86799999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46899999999999997}]},{"osd":7,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.75,"15min":0.76800000000000002},"min":{"1min":0.496,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.78200000000000003,"5min":1.042,"15min":1.042},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.68899999999999995,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.879,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.622}]},{"osd":8,"last update":"Mon Nov 6 01:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.78900000000000003,"15min":0.82999999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.74199999999999999,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.54900000000000004}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.42999999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88900000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.82399999999999995,"15min":0.84799999999999998},"min":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64600000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.76900000000000002,"15min":0.78700000000000003},"min":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.93500000000000005,"5min":1.238,"15min":1.238},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.82999999999999996,"15min":0.84899999999999998},"min":{"1min":0.58699999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.86299999999999999,"5min":1.141,"15min":1.141},"last":0.68000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":492403,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82699999999999996,"5min":1.093,"15min":1.093},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53100000000000003,"15min":0.498},"min":{"1min":0.31,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.85999999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.751}]},{"osd":2,"last update":"Mon Nov 6 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.60899999999999999,"15min":0.53700000000000003},"min":{"1min":0.51500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.95699999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.58599999999999997,"15min":0.52100000000000002},"min":{"1min":0.48099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.874},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.61699999999999999,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1459999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.52000000000000002,"15min":0.47499999999999998},"min":{"1min":0.308,"5min":0.252,"15min":0.252},"max":{"1min":1.0580000000000001,"5min":1.617,"15min":1.617},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.57799999999999996,"15min":0.52400000000000002},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.93400000000000005,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.56599999999999995}]},{"osd":5,"last update":"Mon Nov 6 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.65900000000000003,"15min":0.60399999999999998},"min":{"1min":0.59099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0189999999999999,"5min":1.056,"15min":1.056},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.66400000000000003,"15min":0.61799999999999999},"min":{"1min":0.54800000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82399999999999995}]},{"osd":6,"last update":"Mon Nov 6 01:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.64700000000000002,"15min":0.61499999999999999},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96699999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.69899999999999995,"15min":0.66600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.64500000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.66400000000000003,"15min":0.58799999999999997},"min":{"1min":0.49099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.137,"5min":2.27,"15min":2.27},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.68400000000000005,"15min":0.59499999999999997},"min":{"1min":0.53300000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1739999999999999,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.65900000000000003}]},{"osd":8,"last update":"Mon Nov 6 01:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.75800000000000001,"15min":0.70599999999999996},"min":{"1min":0.68999999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.79600000000000004,"15min":0.751},"min":{"1min":0.68799999999999994,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.194,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.78900000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.45300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.82799999999999996,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":0.96599999999999997}]},{"osd":10,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.88300000000000001,"15min":0.875},"min":{"1min":0.56200000000000006,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.23,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.876,"15min":0.86699999999999999},"min":{"1min":0.57999999999999996,"5min":0.501,"15min":0.501},"max":{"1min":1.1579999999999999,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.996},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.94399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":437505,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 01:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.53500000000000003},"min":{"1min":0.434,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74099999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.32200000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.42499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72099999999999997,"5min":1.052,"15min":1.052},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55600000000000005}]},{"osd":5,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70999999999999996,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":1.198,"15min":1.198},"last":0.58499999999999996}]},{"osd":6,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52800000000000002,"15min":0.55300000000000005},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66300000000000003,"5min":1.075,"15min":1.075},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.33500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.68000000000000005,"5min":1.222,"15min":1.222},"last":0.39200000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.52400000000000002}]},{"osd":8,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.71299999999999997,"15min":0.75700000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80000000000000004,"5min":1.292,"15min":1.292},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.71299999999999997,"15min":0.753},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76400000000000001,"5min":1.341,"15min":1.341},"last":0.66300000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.66000000000000003,"15min":0.67500000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":1.123,"15min":1.123},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76000000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.57399999999999995}]},{"osd":10,"last update":"Mon Nov 6 01:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.71299999999999997,"15min":0.72499999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86899999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86799999999999999,"5min":1.165,"15min":1.165},"last":0.61599999999999999}]},{"osd":11,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.56299999999999994,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.51000000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79600000000000004,"5min":0.878,"15min":0.878},"last":0.624}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":495604,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.497},"min":{"1min":0.46100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53800000000000003,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Nov 6 01:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81200000000000006,"5min":1.044,"15min":1.044},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.44500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81499999999999995}]},{"osd":3,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.75},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60399999999999998,"15min":0.57599999999999996},"min":{"1min":0.443,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67000000000000004}]},{"osd":5,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.49199999999999999,"15min":0.45200000000000001},"min":{"1min":0.34499999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56100000000000005,"15min":0.52700000000000002},"min":{"1min":0.499,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76100000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.73699999999999999}]},{"osd":6,"last update":"Mon Nov 6 01:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.46200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.86199999999999999}]},{"osd":7,"last update":"Mon Nov 6 01:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.54700000000000004}]},{"osd":8,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.46700000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76900000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.48599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":1,"15min":1},"last":0.79400000000000004}]},{"osd":9,"last update":"Mon Nov 6 01:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76500000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.89600000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.60699999999999998,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.022,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.96199999999999997,"5min":1.264,"15min":1.264},"last":0.78100000000000003}]},{"osd":11,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.83399999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149759,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":435928,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.36399999999999999},"min":{"1min":0.27300000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51500000000000001,"5min":0.627,"15min":0.627},"last":0.443},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.31,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":1.119,"15min":1.119},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54500000000000004,"15min":0.496},"min":{"1min":0.42099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60399999999999998,"15min":0.55800000000000005},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83099999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61299999999999999,"15min":0.59299999999999997},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.72499999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.39000000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.46300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.66800000000000004,"5min":1.159,"15min":1.159},"last":0.73199999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.68100000000000005,"5min":1.042,"15min":1.042},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59799999999999998,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76100000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.318}]},{"osd":6,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89800000000000002,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.498}]},{"osd":7,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65900000000000003,"15min":0.64700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83999999999999997,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67200000000000004,"15min":0.66000000000000003},"min":{"1min":0.497,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93400000000000005,"5min":1.149,"15min":1.149},"last":0.64800000000000002}]},{"osd":8,"last update":"Mon Nov 6 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.622},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.0430000000000001,"5min":6.0430000000000001,"15min":6.0430000000000001},"last":0.76500000000000001}]},{"osd":9,"last update":"Mon Nov 6 01:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.74199999999999999,"15min":0.76900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85899999999999999,"5min":1.319,"15min":1.319},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.46400000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82199999999999995,"5min":1.214,"15min":1.214},"last":0.54500000000000004}]},{"osd":10,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.79100000000000004,"15min":0.81200000000000006},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.83699999999999997,"5min":1.173,"15min":1.173},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88700000000000001,"5min":1.286,"15min":1.286},"last":0.747}]},{"osd":11,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58999999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888122,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":436699,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.45300000000000001},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.49399999999999999,"15min":0.47799999999999998},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.749,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42999999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.36499999999999999},"min":{"1min":0.34499999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58499999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.41099999999999998,"15min":0.375},"min":{"1min":0.38900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.38600000000000001}]},{"osd":2,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55000000000000004,"15min":0.51600000000000001},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76400000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.504},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58899999999999997,"15min":0.53300000000000003},"min":{"1min":0.46600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81100000000000005,"5min":1.028,"15min":1.028},"last":0.55100000000000005}]},{"osd":3,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":0.63400000000000001,"15min":0.53100000000000003},"min":{"1min":0.53000000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":6.9050000000000002,"5min":6.9050000000000002,"15min":6.9050000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.4770000000000001,"5min":0.69999999999999996,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":12.869,"5min":12.869,"15min":12.869},"last":0.72799999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61499999999999999,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92900000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.624,"15min":0.57799999999999996},"min":{"1min":0.53700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.623}]},{"osd":6,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.68999999999999995,"15min":0.64800000000000002},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":5.3949999999999996,"5min":5.3949999999999996,"15min":5.3949999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.127,"5min":0.73399999999999999,"15min":0.66800000000000004},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.64100000000000001}]},{"osd":7,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.65100000000000002,"15min":0.623},"min":{"1min":0.51500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999}]},{"osd":8,"last update":"Mon Nov 6 01:33:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66400000000000003,"15min":0.63900000000000001},"min":{"1min":0.51100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.92700000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76100000000000001,"15min":0.73499999999999999},"min":{"1min":0.65000000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77900000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":0.82999999999999996,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.0659999999999998,"5min":7.0659999999999998,"15min":7.0659999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.73699999999999999,"15min":0.71599999999999997},"min":{"1min":0.63900000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.68999999999999995}]},{"osd":10,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.70199999999999996,"15min":0.68600000000000005},"min":{"1min":0.55000000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.65300000000000002,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.75800000000000001}]},{"osd":11,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.84399999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":1028700,"data_compressed":41972,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.53700000000000003},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76400000000000001,"5min":1.01,"15min":1.01},"last":0.48499999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56999999999999995,"15min":0.53800000000000003},"min":{"1min":0.436,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81699999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57899999999999996,"15min":0.55000000000000004},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66300000000000003}]},{"osd":2,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.41499999999999998,"15min":0.39100000000000001},"min":{"1min":0.32700000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.41899999999999998,"15min":0.38300000000000001},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63500000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.31900000000000001}]},{"osd":3,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79600000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.439,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.74199999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.49199999999999999}]},{"osd":4,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59999999999999998,"15min":0.57199999999999995},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.61399999999999999,"15min":0.57199999999999995},"min":{"1min":0.47599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86099999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.80400000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55400000000000005}]},{"osd":7,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.67100000000000004,"15min":0.66000000000000003},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92100000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75}]},{"osd":8,"last update":"Mon Nov 6 01:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.59699999999999998},"min":{"1min":0.45800000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93100000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.42099999999999999}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.85899999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59399999999999997}]},{"osd":10,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.55400000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94499999999999995,"5min":1.024,"15min":1.024},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96799999999999997,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T01:33:56.646 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-11-06T01:33:56.646 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T01:33:56.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:56 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3092090133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:56.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:56 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/140383426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:56.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:56 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3092090133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:56.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:56 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/140383426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:56.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.803+0000 7f1f4ca84700 1 -- 172.21.15.53:0/1441915837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f480ff620 msgr2=0x7f1f480ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:56.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.803+0000 7f1f4ca84700 1 --2- 172.21.15.53:0/1441915837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f480ff620 0x7f1f480ffa40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009990 tx=0x7f1f38009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.803+0000 7f1f4ca84700 1 -- 172.21.15.53:0/1441915837 shutdown_connections 2023-11-06T01:33:56.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.803+0000 7f1f4ca84700 1 --2- 172.21.15.53:0/1441915837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f48100940 0x7f1f48100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.803+0000 7f1f4ca84700 1 --2- 172.21.15.53:0/1441915837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f480ff620 0x7f1f480ffa40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.803+0000 7f1f4ca84700 1 --2- 172.21.15.53:0/1441915837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f480fff80 0x7f1f48100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.803+0000 7f1f4ca84700 1 -- 172.21.15.53:0/1441915837 >> 172.21.15.53:0/1441915837 conn(0x7f1f480fb1c0 msgr2=0x7f1f480fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:56.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.803+0000 7f1f4ca84700 1 -- 172.21.15.53:0/1441915837 shutdown_connections 2023-11-06T01:33:56.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.803+0000 7f1f4ca84700 1 -- 172.21.15.53:0/1441915837 wait complete. 2023-11-06T01:33:56.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f4ca84700 1 Processor -- start 2023-11-06T01:33:56.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f4ca84700 1 -- start start 2023-11-06T01:33:56.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f4ca84700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f480ff620 0x7f1f48198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:56.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f4ca84700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f480fff80 0x7f1f481991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:56.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f4ca84700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f48100940 0x7f1f481a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:56.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f4ca84700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f481a07a0 con 0x7f1f480fff80 2023-11-06T01:33:56.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f4ca84700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f481a0910 con 0x7f1f480ff620 2023-11-06T01:33:56.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f4ca84700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f481a0ad0 con 0x7f1f48100940 2023-11-06T01:33:56.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f45d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f480fff80 0x7f1f481991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:56.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f45d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f480fff80 0x7f1f481991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39628/0 (socket says 172.21.15.53:39628) 2023-11-06T01:33:56.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f45d9b700 1 -- 172.21.15.53:0/607410316 learned_addr learned my addr 172.21.15.53:0/607410316 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:56.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.804+0000 7f1f4659c700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f480ff620 0x7f1f48198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:56.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f46d9d700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f48100940 0x7f1f481a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:56.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f45d9b700 1 -- 172.21.15.53:0/607410316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f48100940 msgr2=0x7f1f481a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:56.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f45d9b700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f48100940 0x7f1f481a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f45d9b700 1 -- 172.21.15.53:0/607410316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f480ff620 msgr2=0x7f1f48198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:56.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f45d9b700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f480ff620 0x7f1f48198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:56.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f45d9b700 1 -- 172.21.15.53:0/607410316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f40009c40 con 0x7f1f480fff80 2023-11-06T01:33:56.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f4659c700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f480ff620 0x7f1f48198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:33:56.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f45d9b700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f480fff80 0x7f1f481991a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00e9a0 tx=0x7f1f3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:56.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f377fe700 1 -- 172.21.15.53:0/607410316 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c009790 con 0x7f1f480fff80 2023-11-06T01:33:56.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f377fe700 1 -- 172.21.15.53:0/607410316 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f3c00cbd0 con 0x7f1f480fff80 2023-11-06T01:33:56.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f4ca84700 1 -- 172.21.15.53:0/607410316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f380095f0 con 0x7f1f480fff80 2023-11-06T01:33:56.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f377fe700 1 -- 172.21.15.53:0/607410316 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c018680 con 0x7f1f480fff80 2023-11-06T01:33:56.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.805+0000 7f1f4ca84700 1 -- 172.21.15.53:0/607410316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f481a1130 con 0x7f1f480fff80 2023-11-06T01:33:56.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.806+0000 7f1f4ca84700 1 -- 172.21.15.53:0/607410316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4810a400 con 0x7f1f480fff80 2023-11-06T01:33:56.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.807+0000 7f1f377fe700 1 -- 172.21.15.53:0/607410316 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f1f3c00cd40 con 0x7f1f480fff80 2023-11-06T01:33:56.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.808+0000 7f1f377fe700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f300a0570 0x7f1f300a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:56.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.808+0000 7f1f377fe700 1 -- 172.21.15.53:0/607410316 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1f3c014070 con 0x7f1f480fff80 2023-11-06T01:33:56.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.808+0000 7f1f4659c700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f300a0570 0x7f1f300a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:56.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.808+0000 7f1f4659c700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f300a0570 0x7f1f300a2a30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800b2e0 tx=0x7f1f38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:56.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:56.812+0000 7f1f377fe700 1 -- 172.21.15.53:0/607410316 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1f3c0520e0 con 0x7f1f480fff80 2023-11-06T01:33:57.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.008+0000 7f1f4ca84700 1 -- 172.21.15.53:0/607410316 --> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f480611d0 con 0x7f1f300a0570 2023-11-06T01:33:57.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.020+0000 7f1f377fe700 1 -- 172.21.15.53:0/607410316 <== mgr.14152 v2:172.21.15.53:6800/1491433716 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343607 (secure 0 0 0) 0x7f1f480611d0 con 0x7f1f300a0570 2023-11-06T01:33:57.025 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:57.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.023+0000 7f1f4ca84700 1 -- 172.21.15.53:0/607410316 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f300a0570 msgr2=0x7f1f300a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.023+0000 7f1f4ca84700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f300a0570 0x7f1f300a2a30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800b2e0 tx=0x7f1f38005fb0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.023+0000 7f1f4ca84700 1 -- 172.21.15.53:0/607410316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f480fff80 msgr2=0x7f1f481991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.023+0000 7f1f4ca84700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f480fff80 0x7f1f481991a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00e9a0 tx=0x7f1f3c00ecb0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.024+0000 7f1f4ca84700 1 -- 172.21.15.53:0/607410316 shutdown_connections 2023-11-06T01:33:57.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.024+0000 7f1f4ca84700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f300a0570 0x7f1f300a2a30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.024+0000 7f1f4ca84700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f480ff620 0x7f1f48198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.024+0000 7f1f4ca84700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f48100940 0x7f1f481a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.024+0000 7f1f4ca84700 1 --2- 172.21.15.53:0/607410316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f480fff80 0x7f1f481991a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.024+0000 7f1f4ca84700 1 -- 172.21.15.53:0/607410316 >> 172.21.15.53:0/607410316 conn(0x7f1f480fb1c0 msgr2=0x7f1f480fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:57.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.024+0000 7f1f4ca84700 1 -- 172.21.15.53:0/607410316 shutdown_connections 2023-11-06T01:33:57.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.024+0000 7f1f4ca84700 1 -- 172.21.15.53:0/607410316 wait complete. 2023-11-06T01:33:57.031 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-11-06T01:33:57.037 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-11-06T01:33:55.498729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656232,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":981,"num_read_kb":605,"num_write":415,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1182566,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334636,"kb_used_data":11212,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607188,"statfs":{"total":1151940427776,"available":1151597760512,"internally_reserved":0,"allocated":11481088,"data_stored":7182927,"data_compressed":177927,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4956,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":72,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003045"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121147+0000","last_change":"2023-11-06T01:32:26.110793+0000","last_active":"2023-11-06T01:33:26.121147+0000","last_peered":"2023-11-06T01:33:26.121147+0000","last_clean":"2023-11-06T01:33:26.121147+0000","last_became_active":"2023-11-06T01:32:26.110489+0000","last_became_peered":"2023-11-06T01:32:26.110489+0000","last_unstale":"2023-11-06T01:33:26.121147+0000","last_undegraded":"2023-11-06T01:33:26.121147+0000","last_fullsized":"2023-11-06T01:33:26.121147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:01:08.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122964+0000","last_change":"2023-11-06T01:32:26.107273+0000","last_active":"2023-11-06T01:33:26.122964+0000","last_peered":"2023-11-06T01:33:26.122964+0000","last_clean":"2023-11-06T01:33:26.122964+0000","last_became_active":"2023-11-06T01:32:26.106961+0000","last_became_peered":"2023-11-06T01:32:26.106961+0000","last_unstale":"2023-11-06T01:33:26.122964+0000","last_undegraded":"2023-11-06T01:33:26.122964+0000","last_fullsized":"2023-11-06T01:33:26.122964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:40:39.073926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122761+0000","last_change":"2023-11-06T01:32:26.121316+0000","last_active":"2023-11-06T01:33:26.122761+0000","last_peered":"2023-11-06T01:33:26.122761+0000","last_clean":"2023-11-06T01:33:26.122761+0000","last_became_active":"2023-11-06T01:32:26.120756+0000","last_became_peered":"2023-11-06T01:32:26.120756+0000","last_unstale":"2023-11-06T01:33:26.122761+0000","last_undegraded":"2023-11-06T01:33:26.122761+0000","last_fullsized":"2023-11-06T01:33:26.122761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:02:45.814577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120866+0000","last_change":"2023-11-06T01:32:26.121951+0000","last_active":"2023-11-06T01:33:26.120866+0000","last_peered":"2023-11-06T01:33:26.120866+0000","last_clean":"2023-11-06T01:33:26.120866+0000","last_became_active":"2023-11-06T01:32:26.121636+0000","last_became_peered":"2023-11-06T01:32:26.121636+0000","last_unstale":"2023-11-06T01:33:26.120866+0000","last_undegraded":"2023-11-06T01:33:26.120866+0000","last_fullsized":"2023-11-06T01:33:26.120866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:02.381355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120004+0000","last_change":"2023-11-06T01:32:26.117788+0000","last_active":"2023-11-06T01:33:26.120004+0000","last_peered":"2023-11-06T01:33:26.120004+0000","last_clean":"2023-11-06T01:33:26.120004+0000","last_became_active":"2023-11-06T01:32:26.117161+0000","last_became_peered":"2023-11-06T01:32:26.117161+0000","last_unstale":"2023-11-06T01:33:26.120004+0000","last_undegraded":"2023-11-06T01:33:26.120004+0000","last_fullsized":"2023-11-06T01:33:26.120004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:31.665690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123916+0000","last_change":"2023-11-06T01:32:26.109532+0000","last_active":"2023-11-06T01:33:26.123916+0000","last_peered":"2023-11-06T01:33:26.123916+0000","last_clean":"2023-11-06T01:33:26.123916+0000","last_became_active":"2023-11-06T01:32:26.106144+0000","last_became_peered":"2023-11-06T01:32:26.106144+0000","last_unstale":"2023-11-06T01:33:26.123916+0000","last_undegraded":"2023-11-06T01:33:26.123916+0000","last_fullsized":"2023-11-06T01:33:26.123916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:54.220515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119514+0000","last_change":"2023-11-06T01:32:26.116280+0000","last_active":"2023-11-06T01:33:26.119514+0000","last_peered":"2023-11-06T01:33:26.119514+0000","last_clean":"2023-11-06T01:33:26.119514+0000","last_became_active":"2023-11-06T01:32:26.115957+0000","last_became_peered":"2023-11-06T01:32:26.115957+0000","last_unstale":"2023-11-06T01:33:26.119514+0000","last_undegraded":"2023-11-06T01:33:26.119514+0000","last_fullsized":"2023-11-06T01:33:26.119514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:25.956902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118127+0000","last_change":"2023-11-06T01:32:26.124818+0000","last_active":"2023-11-06T01:33:26.118127+0000","last_peered":"2023-11-06T01:33:26.118127+0000","last_clean":"2023-11-06T01:33:26.118127+0000","last_became_active":"2023-11-06T01:32:26.124511+0000","last_became_peered":"2023-11-06T01:32:26.124511+0000","last_unstale":"2023-11-06T01:33:26.118127+0000","last_undegraded":"2023-11-06T01:33:26.118127+0000","last_fullsized":"2023-11-06T01:33:26.118127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:35.853898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122869+0000","last_change":"2023-11-06T01:32:26.114243+0000","last_active":"2023-11-06T01:33:26.122869+0000","last_peered":"2023-11-06T01:33:26.122869+0000","last_clean":"2023-11-06T01:33:26.122869+0000","last_became_active":"2023-11-06T01:32:26.113923+0000","last_became_peered":"2023-11-06T01:32:26.113923+0000","last_unstale":"2023-11-06T01:33:26.122869+0000","last_undegraded":"2023-11-06T01:33:26.122869+0000","last_fullsized":"2023-11-06T01:33:26.122869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:48.197329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124318+0000","last_change":"2023-11-06T01:32:26.113819+0000","last_active":"2023-11-06T01:33:26.124318+0000","last_peered":"2023-11-06T01:33:26.124318+0000","last_clean":"2023-11-06T01:33:26.124318+0000","last_became_active":"2023-11-06T01:32:26.113506+0000","last_became_peered":"2023-11-06T01:32:26.113506+0000","last_unstale":"2023-11-06T01:33:26.124318+0000","last_undegraded":"2023-11-06T01:33:26.124318+0000","last_fullsized":"2023-11-06T01:33:26.124318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:33.122100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120607+0000","last_change":"2023-11-06T01:32:26.116504+0000","last_active":"2023-11-06T01:33:26.120607+0000","last_peered":"2023-11-06T01:33:26.120607+0000","last_clean":"2023-11-06T01:33:26.120607+0000","last_became_active":"2023-11-06T01:32:26.115220+0000","last_became_peered":"2023-11-06T01:32:26.115220+0000","last_unstale":"2023-11-06T01:33:26.120607+0000","last_undegraded":"2023-11-06T01:33:26.120607+0000","last_fullsized":"2023-11-06T01:33:26.120607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:34.447899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120346+0000","last_change":"2023-11-06T01:32:26.111596+0000","last_active":"2023-11-06T01:33:26.120346+0000","last_peered":"2023-11-06T01:33:26.120346+0000","last_clean":"2023-11-06T01:33:26.120346+0000","last_became_active":"2023-11-06T01:32:26.111309+0000","last_became_peered":"2023-11-06T01:32:26.111309+0000","last_unstale":"2023-11-06T01:33:26.120346+0000","last_undegraded":"2023-11-06T01:33:26.120346+0000","last_fullsized":"2023-11-06T01:33:26.120346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:34.917068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129435+0000","last_change":"2023-11-06T01:32:26.114536+0000","last_active":"2023-11-06T01:33:26.129435+0000","last_peered":"2023-11-06T01:33:26.129435+0000","last_clean":"2023-11-06T01:33:26.129435+0000","last_became_active":"2023-11-06T01:32:26.114124+0000","last_became_peered":"2023-11-06T01:32:26.114124+0000","last_unstale":"2023-11-06T01:33:26.129435+0000","last_undegraded":"2023-11-06T01:33:26.129435+0000","last_fullsized":"2023-11-06T01:33:26.129435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:05:10.855240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122520+0000","last_change":"2023-11-06T01:32:26.111139+0000","last_active":"2023-11-06T01:33:26.122520+0000","last_peered":"2023-11-06T01:33:26.122520+0000","last_clean":"2023-11-06T01:33:26.122520+0000","last_became_active":"2023-11-06T01:32:26.110764+0000","last_became_peered":"2023-11-06T01:32:26.110764+0000","last_unstale":"2023-11-06T01:33:26.122520+0000","last_undegraded":"2023-11-06T01:33:26.122520+0000","last_fullsized":"2023-11-06T01:33:26.122520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:46.422701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.127151+0000","last_change":"2023-11-06T01:33:27.127151+0000","last_active":"2023-11-06T01:33:27.127151+0000","last_peered":"2023-11-06T01:33:27.127151+0000","last_clean":"2023-11-06T01:33:27.127151+0000","last_became_active":"2023-11-06T01:33:27.126884+0000","last_became_peered":"2023-11-06T01:33:27.126884+0000","last_unstale":"2023-11-06T01:33:27.127151+0000","last_undegraded":"2023-11-06T01:33:27.127151+0000","last_fullsized":"2023-11-06T01:33:27.127151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:03:18.021947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122447+0000","last_change":"2023-11-06T01:32:26.118169+0000","last_active":"2023-11-06T01:33:26.122447+0000","last_peered":"2023-11-06T01:33:26.122447+0000","last_clean":"2023-11-06T01:33:26.122447+0000","last_became_active":"2023-11-06T01:32:26.117837+0000","last_became_peered":"2023-11-06T01:32:26.117837+0000","last_unstale":"2023-11-06T01:33:26.122447+0000","last_undegraded":"2023-11-06T01:33:26.122447+0000","last_fullsized":"2023-11-06T01:33:26.122447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:46.838927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121234+0000","last_change":"2023-11-06T01:32:26.110020+0000","last_active":"2023-11-06T01:33:26.121234+0000","last_peered":"2023-11-06T01:33:26.121234+0000","last_clean":"2023-11-06T01:33:26.121234+0000","last_became_active":"2023-11-06T01:32:26.109707+0000","last_became_peered":"2023-11-06T01:32:26.109707+0000","last_unstale":"2023-11-06T01:33:26.121234+0000","last_undegraded":"2023-11-06T01:33:26.121234+0000","last_fullsized":"2023-11-06T01:33:26.121234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:00.016258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.115738+0000","last_change":"2023-11-06T01:32:26.119949+0000","last_active":"2023-11-06T01:33:26.115738+0000","last_peered":"2023-11-06T01:33:26.115738+0000","last_clean":"2023-11-06T01:33:26.115738+0000","last_became_active":"2023-11-06T01:32:26.117594+0000","last_became_peered":"2023-11-06T01:32:26.117594+0000","last_unstale":"2023-11-06T01:33:26.115738+0000","last_undegraded":"2023-11-06T01:33:26.115738+0000","last_fullsized":"2023-11-06T01:33:26.115738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:17:22.757846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121991+0000","last_change":"2023-11-06T01:32:26.123176+0000","last_active":"2023-11-06T01:33:26.121991+0000","last_peered":"2023-11-06T01:33:26.121991+0000","last_clean":"2023-11-06T01:33:26.121991+0000","last_became_active":"2023-11-06T01:32:26.116687+0000","last_became_peered":"2023-11-06T01:32:26.116687+0000","last_unstale":"2023-11-06T01:33:26.121991+0000","last_undegraded":"2023-11-06T01:33:26.121991+0000","last_fullsized":"2023-11-06T01:33:26.121991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:30.422426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":131,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:51.423488+0000","last_change":"2023-11-06T01:33:27.401065+0000","last_active":"2023-11-06T01:33:51.423488+0000","last_peered":"2023-11-06T01:33:51.423488+0000","last_clean":"2023-11-06T01:33:51.423488+0000","last_became_active":"2023-11-06T01:33:27.391423+0000","last_became_peered":"2023-11-06T01:33:27.391423+0000","last_unstale":"2023-11-06T01:33:51.423488+0000","last_undegraded":"2023-11-06T01:33:51.423488+0000","last_fullsized":"2023-11-06T01:33:51.423488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:00:05.573076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118711+0000","last_change":"2023-11-06T01:32:26.126350+0000","last_active":"2023-11-06T01:33:26.118711+0000","last_peered":"2023-11-06T01:33:26.118711+0000","last_clean":"2023-11-06T01:33:26.118711+0000","last_became_active":"2023-11-06T01:32:26.124703+0000","last_became_peered":"2023-11-06T01:32:26.124703+0000","last_unstale":"2023-11-06T01:33:26.118711+0000","last_undegraded":"2023-11-06T01:33:26.118711+0000","last_fullsized":"2023-11-06T01:33:26.118711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:08:25.866045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118659+0000","last_change":"2023-11-06T01:32:25.680184+0000","last_active":"2023-11-06T01:33:26.118659+0000","last_peered":"2023-11-06T01:33:26.118659+0000","last_clean":"2023-11-06T01:33:26.118659+0000","last_became_active":"2023-11-06T01:32:25.679859+0000","last_became_peered":"2023-11-06T01:32:25.679859+0000","last_unstale":"2023-11-06T01:33:26.118659+0000","last_undegraded":"2023-11-06T01:33:26.118659+0000","last_fullsized":"2023-11-06T01:33:26.118659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:04:34.153562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124048+0000","last_change":"2023-11-06T01:32:26.114511+0000","last_active":"2023-11-06T01:33:26.124048+0000","last_peered":"2023-11-06T01:33:26.124048+0000","last_clean":"2023-11-06T01:33:26.124048+0000","last_became_active":"2023-11-06T01:32:26.114182+0000","last_became_peered":"2023-11-06T01:32:26.114182+0000","last_unstale":"2023-11-06T01:33:26.124048+0000","last_undegraded":"2023-11-06T01:33:26.124048+0000","last_fullsized":"2023-11-06T01:33:26.124048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:49:10.694358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120210+0000","last_change":"2023-11-06T01:32:25.681453+0000","last_active":"2023-11-06T01:33:26.120210+0000","last_peered":"2023-11-06T01:33:26.120210+0000","last_clean":"2023-11-06T01:33:26.120210+0000","last_became_active":"2023-11-06T01:32:25.681180+0000","last_became_peered":"2023-11-06T01:32:25.681180+0000","last_unstale":"2023-11-06T01:33:26.120210+0000","last_undegraded":"2023-11-06T01:33:26.120210+0000","last_fullsized":"2023-11-06T01:33:26.120210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:39:19.738479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122292+0000","last_change":"2023-11-06T01:32:26.118748+0000","last_active":"2023-11-06T01:33:26.122292+0000","last_peered":"2023-11-06T01:33:26.122292+0000","last_clean":"2023-11-06T01:33:26.122292+0000","last_became_active":"2023-11-06T01:32:26.118449+0000","last_became_peered":"2023-11-06T01:32:26.118449+0000","last_unstale":"2023-11-06T01:33:26.122292+0000","last_undegraded":"2023-11-06T01:33:26.122292+0000","last_fullsized":"2023-11-06T01:33:26.122292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:28.134553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119846+0000","last_change":"2023-11-06T01:32:25.680967+0000","last_active":"2023-11-06T01:33:26.119846+0000","last_peered":"2023-11-06T01:33:26.119846+0000","last_clean":"2023-11-06T01:33:26.119846+0000","last_became_active":"2023-11-06T01:32:25.678829+0000","last_became_peered":"2023-11-06T01:32:25.678829+0000","last_unstale":"2023-11-06T01:33:26.119846+0000","last_undegraded":"2023-11-06T01:33:26.119846+0000","last_fullsized":"2023-11-06T01:33:26.119846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:40:07.691852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123063+0000","last_change":"2023-11-06T01:32:26.122692+0000","last_active":"2023-11-06T01:33:26.123063+0000","last_peered":"2023-11-06T01:33:26.123063+0000","last_clean":"2023-11-06T01:33:26.123063+0000","last_became_active":"2023-11-06T01:32:26.121960+0000","last_became_peered":"2023-11-06T01:32:26.121960+0000","last_unstale":"2023-11-06T01:33:26.123063+0000","last_undegraded":"2023-11-06T01:33:26.123063+0000","last_fullsized":"2023-11-06T01:33:26.123063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:48.010467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:52.426673+0000","last_change":"2023-11-06T01:33:27.403594+0000","last_active":"2023-11-06T01:33:52.426673+0000","last_peered":"2023-11-06T01:33:52.426673+0000","last_clean":"2023-11-06T01:33:52.426673+0000","last_became_active":"2023-11-06T01:33:27.391131+0000","last_became_peered":"2023-11-06T01:33:27.391131+0000","last_unstale":"2023-11-06T01:33:52.426673+0000","last_undegraded":"2023-11-06T01:33:52.426673+0000","last_fullsized":"2023-11-06T01:33:52.426673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:55:10.577522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119781+0000","last_change":"2023-11-06T01:32:26.116839+0000","last_active":"2023-11-06T01:33:26.119781+0000","last_peered":"2023-11-06T01:33:26.119781+0000","last_clean":"2023-11-06T01:33:26.119781+0000","last_became_active":"2023-11-06T01:32:26.116535+0000","last_became_peered":"2023-11-06T01:32:26.116535+0000","last_unstale":"2023-11-06T01:33:26.119781+0000","last_undegraded":"2023-11-06T01:33:26.119781+0000","last_fullsized":"2023-11-06T01:33:26.119781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:11.219626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123841+0000","last_change":"2023-11-06T01:32:25.678426+0000","last_active":"2023-11-06T01:33:26.123841+0000","last_peered":"2023-11-06T01:33:26.123841+0000","last_clean":"2023-11-06T01:33:26.123841+0000","last_became_active":"2023-11-06T01:32:25.678072+0000","last_became_peered":"2023-11-06T01:32:25.678072+0000","last_unstale":"2023-11-06T01:33:26.123841+0000","last_undegraded":"2023-11-06T01:33:26.123841+0000","last_fullsized":"2023-11-06T01:33:26.123841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:09:41.630098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124752+0000","last_change":"2023-11-06T01:32:26.119184+0000","last_active":"2023-11-06T01:33:26.124752+0000","last_peered":"2023-11-06T01:33:26.124752+0000","last_clean":"2023-11-06T01:33:26.124752+0000","last_became_active":"2023-11-06T01:32:26.118896+0000","last_became_peered":"2023-11-06T01:32:26.118896+0000","last_unstale":"2023-11-06T01:33:26.124752+0000","last_undegraded":"2023-11-06T01:33:26.124752+0000","last_fullsized":"2023-11-06T01:33:26.124752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:09:17.351315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129133+0000","last_change":"2023-11-06T01:32:25.685621+0000","last_active":"2023-11-06T01:33:26.129133+0000","last_peered":"2023-11-06T01:33:26.129133+0000","last_clean":"2023-11-06T01:33:26.129133+0000","last_became_active":"2023-11-06T01:32:25.684965+0000","last_became_peered":"2023-11-06T01:32:25.684965+0000","last_unstale":"2023-11-06T01:33:26.129133+0000","last_undegraded":"2023-11-06T01:33:26.129133+0000","last_fullsized":"2023-11-06T01:33:26.129133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:28:00.922680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118211+0000","last_change":"2023-11-06T01:32:26.122750+0000","last_active":"2023-11-06T01:33:26.118211+0000","last_peered":"2023-11-06T01:33:26.118211+0000","last_clean":"2023-11-06T01:33:26.118211+0000","last_became_active":"2023-11-06T01:32:26.122241+0000","last_became_peered":"2023-11-06T01:32:26.122241+0000","last_unstale":"2023-11-06T01:33:26.118211+0000","last_undegraded":"2023-11-06T01:33:26.118211+0000","last_fullsized":"2023-11-06T01:33:26.118211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:04.402197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124078+0000","last_change":"2023-11-06T01:32:25.681594+0000","last_active":"2023-11-06T01:33:26.124078+0000","last_peered":"2023-11-06T01:33:26.124078+0000","last_clean":"2023-11-06T01:33:26.124078+0000","last_became_active":"2023-11-06T01:32:25.681303+0000","last_became_peered":"2023-11-06T01:32:25.681303+0000","last_unstale":"2023-11-06T01:33:26.124078+0000","last_undegraded":"2023-11-06T01:33:26.124078+0000","last_fullsized":"2023-11-06T01:33:26.124078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:33.141440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.115522+0000","last_change":"2023-11-06T01:32:26.117234+0000","last_active":"2023-11-06T01:33:26.115522+0000","last_peered":"2023-11-06T01:33:26.115522+0000","last_clean":"2023-11-06T01:33:26.115522+0000","last_became_active":"2023-11-06T01:32:26.116915+0000","last_became_peered":"2023-11-06T01:32:26.116915+0000","last_unstale":"2023-11-06T01:33:26.115522+0000","last_undegraded":"2023-11-06T01:33:26.115522+0000","last_fullsized":"2023-11-06T01:33:26.115522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:05:27.521552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":185,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:54.145360+0000","last_change":"2023-11-06T01:32:25.682827+0000","last_active":"2023-11-06T01:33:54.145360+0000","last_peered":"2023-11-06T01:33:54.145360+0000","last_clean":"2023-11-06T01:33:54.145360+0000","last_became_active":"2023-11-06T01:32:25.682470+0000","last_became_peered":"2023-11-06T01:32:25.682470+0000","last_unstale":"2023-11-06T01:33:54.145360+0000","last_undegraded":"2023-11-06T01:33:54.145360+0000","last_fullsized":"2023-11-06T01:33:54.145360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:43.632070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123248+0000","last_change":"2023-11-06T01:32:26.123918+0000","last_active":"2023-11-06T01:33:26.123248+0000","last_peered":"2023-11-06T01:33:26.123248+0000","last_clean":"2023-11-06T01:33:26.123248+0000","last_became_active":"2023-11-06T01:32:26.123450+0000","last_became_peered":"2023-11-06T01:32:26.123450+0000","last_unstale":"2023-11-06T01:33:26.123248+0000","last_undegraded":"2023-11-06T01:33:26.123248+0000","last_fullsized":"2023-11-06T01:33:26.123248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:38.227311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121215+0000","last_change":"2023-11-06T01:32:25.680051+0000","last_active":"2023-11-06T01:33:26.121215+0000","last_peered":"2023-11-06T01:33:26.121215+0000","last_clean":"2023-11-06T01:33:26.121215+0000","last_became_active":"2023-11-06T01:32:25.679705+0000","last_became_peered":"2023-11-06T01:32:25.679705+0000","last_unstale":"2023-11-06T01:33:26.121215+0000","last_undegraded":"2023-11-06T01:33:26.121215+0000","last_fullsized":"2023-11-06T01:33:26.121215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:18:22.620036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.126354+0000","last_change":"2023-11-06T01:33:27.126354+0000","last_active":"2023-11-06T01:33:27.126354+0000","last_peered":"2023-11-06T01:33:27.126354+0000","last_clean":"2023-11-06T01:33:27.126354+0000","last_became_active":"2023-11-06T01:33:27.125998+0000","last_became_peered":"2023-11-06T01:33:27.125998+0000","last_unstale":"2023-11-06T01:33:27.126354+0000","last_undegraded":"2023-11-06T01:33:27.126354+0000","last_fullsized":"2023-11-06T01:33:27.126354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:00:47.689894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123147+0000","last_change":"2023-11-06T01:32:25.678689+0000","last_active":"2023-11-06T01:33:26.123147+0000","last_peered":"2023-11-06T01:33:26.123147+0000","last_clean":"2023-11-06T01:33:26.123147+0000","last_became_active":"2023-11-06T01:32:25.678355+0000","last_became_peered":"2023-11-06T01:32:25.678355+0000","last_unstale":"2023-11-06T01:33:26.123147+0000","last_undegraded":"2023-11-06T01:33:26.123147+0000","last_fullsized":"2023-11-06T01:33:26.123147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:23.007596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122348+0000","last_change":"2023-11-06T01:32:26.118120+0000","last_active":"2023-11-06T01:33:26.122348+0000","last_peered":"2023-11-06T01:33:26.122348+0000","last_clean":"2023-11-06T01:33:26.122348+0000","last_became_active":"2023-11-06T01:32:26.117762+0000","last_became_peered":"2023-11-06T01:32:26.117762+0000","last_unstale":"2023-11-06T01:33:26.122348+0000","last_undegraded":"2023-11-06T01:33:26.122348+0000","last_fullsized":"2023-11-06T01:33:26.122348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:28:42.417463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'30","reported_seq":219,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:54.430124+0000","last_change":"2023-11-06T01:32:25.676036+0000","last_active":"2023-11-06T01:33:54.430124+0000","last_peered":"2023-11-06T01:33:54.430124+0000","last_clean":"2023-11-06T01:33:54.430124+0000","last_became_active":"2023-11-06T01:32:25.675672+0000","last_became_peered":"2023-11-06T01:32:25.675672+0000","last_unstale":"2023-11-06T01:33:54.430124+0000","last_undegraded":"2023-11-06T01:33:54.430124+0000","last_fullsized":"2023-11-06T01:33:54.430124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:30:54.060926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58677,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":175,"num_read_kb":140,"num_write":32,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120047+0000","last_change":"2023-11-06T01:32:26.118832+0000","last_active":"2023-11-06T01:33:26.120047+0000","last_peered":"2023-11-06T01:33:26.120047+0000","last_clean":"2023-11-06T01:33:26.120047+0000","last_became_active":"2023-11-06T01:32:26.118551+0000","last_became_peered":"2023-11-06T01:32:26.118551+0000","last_unstale":"2023-11-06T01:33:26.120047+0000","last_undegraded":"2023-11-06T01:33:26.120047+0000","last_fullsized":"2023-11-06T01:33:26.120047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:23.031203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.136439+0000","last_change":"2023-11-06T01:33:27.136439+0000","last_active":"2023-11-06T01:33:27.136439+0000","last_peered":"2023-11-06T01:33:27.136439+0000","last_clean":"2023-11-06T01:33:27.136439+0000","last_became_active":"2023-11-06T01:33:27.123153+0000","last_became_peered":"2023-11-06T01:33:27.123153+0000","last_unstale":"2023-11-06T01:33:27.136439+0000","last_undegraded":"2023-11-06T01:33:27.136439+0000","last_fullsized":"2023-11-06T01:33:27.136439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:52:29.385129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129312+0000","last_change":"2023-11-06T01:32:26.113421+0000","last_active":"2023-11-06T01:33:26.129312+0000","last_peered":"2023-11-06T01:33:26.129312+0000","last_clean":"2023-11-06T01:33:26.129312+0000","last_became_active":"2023-11-06T01:32:26.113085+0000","last_became_peered":"2023-11-06T01:32:26.113085+0000","last_unstale":"2023-11-06T01:33:26.129312+0000","last_undegraded":"2023-11-06T01:33:26.129312+0000","last_fullsized":"2023-11-06T01:33:26.129312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:15:04.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123170+0000","last_change":"2023-11-06T01:32:25.680293+0000","last_active":"2023-11-06T01:33:26.123170+0000","last_peered":"2023-11-06T01:33:26.123170+0000","last_clean":"2023-11-06T01:33:26.123170+0000","last_became_active":"2023-11-06T01:32:25.679803+0000","last_became_peered":"2023-11-06T01:32:25.679803+0000","last_unstale":"2023-11-06T01:33:26.123170+0000","last_undegraded":"2023-11-06T01:33:26.123170+0000","last_fullsized":"2023-11-06T01:33:26.123170+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:28.562797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120305+0000","last_change":"2023-11-06T01:32:26.115996+0000","last_active":"2023-11-06T01:33:26.120305+0000","last_peered":"2023-11-06T01:33:26.120305+0000","last_clean":"2023-11-06T01:33:26.120305+0000","last_became_active":"2023-11-06T01:32:26.115407+0000","last_became_peered":"2023-11-06T01:32:26.115407+0000","last_unstale":"2023-11-06T01:33:26.120305+0000","last_undegraded":"2023-11-06T01:33:26.120305+0000","last_fullsized":"2023-11-06T01:33:26.120305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:08.069698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":103,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:51.124706+0000","last_change":"2023-11-06T01:32:25.680958+0000","last_active":"2023-11-06T01:33:51.124706+0000","last_peered":"2023-11-06T01:33:51.124706+0000","last_clean":"2023-11-06T01:33:51.124706+0000","last_became_active":"2023-11-06T01:32:25.680616+0000","last_became_peered":"2023-11-06T01:32:25.680616+0000","last_unstale":"2023-11-06T01:33:51.124706+0000","last_undegraded":"2023-11-06T01:33:51.124706+0000","last_fullsized":"2023-11-06T01:33:51.124706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:02.631755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120959+0000","last_change":"2023-11-06T01:32:26.115464+0000","last_active":"2023-11-06T01:33:26.120959+0000","last_peered":"2023-11-06T01:33:26.120959+0000","last_clean":"2023-11-06T01:33:26.120959+0000","last_became_active":"2023-11-06T01:32:26.115090+0000","last_became_peered":"2023-11-06T01:32:26.115090+0000","last_unstale":"2023-11-06T01:33:26.120959+0000","last_undegraded":"2023-11-06T01:33:26.120959+0000","last_fullsized":"2023-11-06T01:33:26.120959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:08.018044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119958+0000","last_change":"2023-11-06T01:32:25.677787+0000","last_active":"2023-11-06T01:33:26.119958+0000","last_peered":"2023-11-06T01:33:26.119958+0000","last_clean":"2023-11-06T01:33:26.119958+0000","last_became_active":"2023-11-06T01:32:25.677527+0000","last_became_peered":"2023-11-06T01:32:25.677527+0000","last_unstale":"2023-11-06T01:33:26.119958+0000","last_undegraded":"2023-11-06T01:33:26.119958+0000","last_fullsized":"2023-11-06T01:33:26.119958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:34:09.452965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.391503+0000","last_change":"2023-11-06T01:33:27.391176+0000","last_active":"2023-11-06T01:33:27.391503+0000","last_peered":"2023-11-06T01:33:27.391503+0000","last_clean":"2023-11-06T01:33:27.391503+0000","last_became_active":"2023-11-06T01:33:27.390893+0000","last_became_peered":"2023-11-06T01:33:27.390893+0000","last_unstale":"2023-11-06T01:33:27.391503+0000","last_undegraded":"2023-11-06T01:33:27.391503+0000","last_fullsized":"2023-11-06T01:33:27.391503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:20:47.382152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120659+0000","last_change":"2023-11-06T01:32:25.677257+0000","last_active":"2023-11-06T01:33:26.120659+0000","last_peered":"2023-11-06T01:33:26.120659+0000","last_clean":"2023-11-06T01:33:26.120659+0000","last_became_active":"2023-11-06T01:32:25.676536+0000","last_became_peered":"2023-11-06T01:32:25.676536+0000","last_unstale":"2023-11-06T01:33:26.120659+0000","last_undegraded":"2023-11-06T01:33:26.120659+0000","last_fullsized":"2023-11-06T01:33:26.120659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:53:33.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119820+0000","last_change":"2023-11-06T01:32:26.111447+0000","last_active":"2023-11-06T01:33:26.119820+0000","last_peered":"2023-11-06T01:33:26.119820+0000","last_clean":"2023-11-06T01:33:26.119820+0000","last_became_active":"2023-11-06T01:32:26.111132+0000","last_became_peered":"2023-11-06T01:32:26.111132+0000","last_unstale":"2023-11-06T01:33:26.119820+0000","last_undegraded":"2023-11-06T01:33:26.119820+0000","last_fullsized":"2023-11-06T01:33:26.119820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:55.385003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.115705+0000","last_change":"2023-11-06T01:32:25.682562+0000","last_active":"2023-11-06T01:33:26.115705+0000","last_peered":"2023-11-06T01:33:26.115705+0000","last_clean":"2023-11-06T01:33:26.115705+0000","last_became_active":"2023-11-06T01:32:25.682227+0000","last_became_peered":"2023-11-06T01:32:25.682227+0000","last_unstale":"2023-11-06T01:33:26.115705+0000","last_undegraded":"2023-11-06T01:33:26.115705+0000","last_fullsized":"2023-11-06T01:33:26.115705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:40.113429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.129164+0000","last_change":"2023-11-06T01:33:27.129164+0000","last_active":"2023-11-06T01:33:27.129164+0000","last_peered":"2023-11-06T01:33:27.129164+0000","last_clean":"2023-11-06T01:33:27.129164+0000","last_became_active":"2023-11-06T01:33:27.123473+0000","last_became_peered":"2023-11-06T01:33:27.123473+0000","last_unstale":"2023-11-06T01:33:27.129164+0000","last_undegraded":"2023-11-06T01:33:27.129164+0000","last_fullsized":"2023-11-06T01:33:27.129164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:03:52.561653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:53.427221+0000","last_change":"2023-11-06T01:32:25.688176+0000","last_active":"2023-11-06T01:33:53.427221+0000","last_peered":"2023-11-06T01:33:53.427221+0000","last_clean":"2023-11-06T01:33:53.427221+0000","last_became_active":"2023-11-06T01:32:25.687745+0000","last_became_peered":"2023-11-06T01:32:25.687745+0000","last_unstale":"2023-11-06T01:33:53.427221+0000","last_undegraded":"2023-11-06T01:33:53.427221+0000","last_fullsized":"2023-11-06T01:33:53.427221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:21:23.566389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124027+0000","last_change":"2023-11-06T01:32:26.111700+0000","last_active":"2023-11-06T01:33:26.124027+0000","last_peered":"2023-11-06T01:33:26.124027+0000","last_clean":"2023-11-06T01:33:26.124027+0000","last_became_active":"2023-11-06T01:32:26.111380+0000","last_became_peered":"2023-11-06T01:32:26.111380+0000","last_unstale":"2023-11-06T01:33:26.124027+0000","last_undegraded":"2023-11-06T01:33:26.124027+0000","last_fullsized":"2023-11-06T01:33:26.124027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:56:47.874663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124206+0000","last_change":"2023-11-06T01:32:26.121906+0000","last_active":"2023-11-06T01:33:26.124206+0000","last_peered":"2023-11-06T01:33:26.124206+0000","last_clean":"2023-11-06T01:33:26.124206+0000","last_became_active":"2023-11-06T01:32:26.121292+0000","last_became_peered":"2023-11-06T01:32:26.121292+0000","last_unstale":"2023-11-06T01:33:26.124206+0000","last_undegraded":"2023-11-06T01:33:26.124206+0000","last_fullsized":"2023-11-06T01:33:26.124206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.532098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":103,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:52.125485+0000","last_change":"2023-11-06T01:32:25.685814+0000","last_active":"2023-11-06T01:33:52.125485+0000","last_peered":"2023-11-06T01:33:52.125485+0000","last_clean":"2023-11-06T01:33:52.125485+0000","last_became_active":"2023-11-06T01:32:25.684890+0000","last_became_peered":"2023-11-06T01:32:25.684890+0000","last_unstale":"2023-11-06T01:33:52.125485+0000","last_undegraded":"2023-11-06T01:33:52.125485+0000","last_fullsized":"2023-11-06T01:33:52.125485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:21.950945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118629+0000","last_change":"2023-11-06T01:32:26.121752+0000","last_active":"2023-11-06T01:33:26.118629+0000","last_peered":"2023-11-06T01:33:26.118629+0000","last_clean":"2023-11-06T01:33:26.118629+0000","last_became_active":"2023-11-06T01:32:26.121447+0000","last_became_peered":"2023-11-06T01:32:26.121447+0000","last_unstale":"2023-11-06T01:33:26.118629+0000","last_undegraded":"2023-11-06T01:33:26.118629+0000","last_fullsized":"2023-11-06T01:33:26.118629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:51:19.727335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121153+0000","last_change":"2023-11-06T01:32:25.682727+0000","last_active":"2023-11-06T01:33:26.121153+0000","last_peered":"2023-11-06T01:33:26.121153+0000","last_clean":"2023-11-06T01:33:26.121153+0000","last_became_active":"2023-11-06T01:32:25.682377+0000","last_became_peered":"2023-11-06T01:32:25.682377+0000","last_unstale":"2023-11-06T01:33:26.121153+0000","last_undegraded":"2023-11-06T01:33:26.121153+0000","last_fullsized":"2023-11-06T01:33:26.121153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:17:38.614350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119733+0000","last_change":"2023-11-06T01:31:27.931076+0000","last_active":"2023-11-06T01:33:26.119733+0000","last_peered":"2023-11-06T01:33:26.119733+0000","last_clean":"2023-11-06T01:33:26.119733+0000","last_became_active":"2023-11-06T01:31:27.930669+0000","last_became_peered":"2023-11-06T01:31:27.930669+0000","last_unstale":"2023-11-06T01:33:26.119733+0000","last_undegraded":"2023-11-06T01:33:26.119733+0000","last_fullsized":"2023-11-06T01:33:26.119733+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:28:36.252549+0000","last_clean_scrub_stamp":"2023-11-06T01:28:36.252549+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:47:48.729325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.128723+0000","last_change":"2023-11-06T01:32:26.110055+0000","last_active":"2023-11-06T01:33:26.128723+0000","last_peered":"2023-11-06T01:33:26.128723+0000","last_clean":"2023-11-06T01:33:26.128723+0000","last_became_active":"2023-11-06T01:32:26.109640+0000","last_became_peered":"2023-11-06T01:32:26.109640+0000","last_unstale":"2023-11-06T01:33:26.128723+0000","last_undegraded":"2023-11-06T01:33:26.128723+0000","last_fullsized":"2023-11-06T01:33:26.128723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:45:43.076910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119178+0000","last_change":"2023-11-06T01:32:25.679885+0000","last_active":"2023-11-06T01:33:26.119178+0000","last_peered":"2023-11-06T01:33:26.119178+0000","last_clean":"2023-11-06T01:33:26.119178+0000","last_became_active":"2023-11-06T01:32:25.679518+0000","last_became_peered":"2023-11-06T01:32:25.679518+0000","last_unstale":"2023-11-06T01:33:26.119178+0000","last_undegraded":"2023-11-06T01:33:26.119178+0000","last_fullsized":"2023-11-06T01:33:26.119178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:44.558534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120105+0000","last_change":"2023-11-06T01:32:25.682016+0000","last_active":"2023-11-06T01:33:26.120105+0000","last_peered":"2023-11-06T01:33:26.120105+0000","last_clean":"2023-11-06T01:33:26.120105+0000","last_became_active":"2023-11-06T01:32:25.681753+0000","last_became_peered":"2023-11-06T01:32:25.681753+0000","last_unstale":"2023-11-06T01:33:26.120105+0000","last_undegraded":"2023-11-06T01:33:26.120105+0000","last_fullsized":"2023-11-06T01:33:26.120105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:08.575728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129585+0000","last_change":"2023-11-06T01:32:26.117109+0000","last_active":"2023-11-06T01:33:26.129585+0000","last_peered":"2023-11-06T01:33:26.129585+0000","last_clean":"2023-11-06T01:33:26.129585+0000","last_became_active":"2023-11-06T01:32:26.116769+0000","last_became_peered":"2023-11-06T01:32:26.116769+0000","last_unstale":"2023-11-06T01:33:26.129585+0000","last_undegraded":"2023-11-06T01:33:26.129585+0000","last_fullsized":"2023-11-06T01:33:26.129585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:48:04.188806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.142547+0000","last_change":"2023-11-06T01:33:27.142547+0000","last_active":"2023-11-06T01:33:27.142547+0000","last_peered":"2023-11-06T01:33:27.142547+0000","last_clean":"2023-11-06T01:33:27.142547+0000","last_became_active":"2023-11-06T01:33:27.127408+0000","last_became_peered":"2023-11-06T01:33:27.127408+0000","last_unstale":"2023-11-06T01:33:27.142547+0000","last_undegraded":"2023-11-06T01:33:27.142547+0000","last_fullsized":"2023-11-06T01:33:27.142547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:08:10.084437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118671+0000","last_change":"2023-11-06T01:32:26.116351+0000","last_active":"2023-11-06T01:33:26.118671+0000","last_peered":"2023-11-06T01:33:26.118671+0000","last_clean":"2023-11-06T01:33:26.118671+0000","last_became_active":"2023-11-06T01:32:26.116044+0000","last_became_peered":"2023-11-06T01:32:26.116044+0000","last_unstale":"2023-11-06T01:33:26.118671+0000","last_undegraded":"2023-11-06T01:33:26.118671+0000","last_fullsized":"2023-11-06T01:33:26.118671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:20.816412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121520+0000","last_change":"2023-11-06T01:32:25.676764+0000","last_active":"2023-11-06T01:33:26.121520+0000","last_peered":"2023-11-06T01:33:26.121520+0000","last_clean":"2023-11-06T01:33:26.121520+0000","last_became_active":"2023-11-06T01:32:25.676418+0000","last_became_peered":"2023-11-06T01:32:25.676418+0000","last_unstale":"2023-11-06T01:33:26.121520+0000","last_undegraded":"2023-11-06T01:33:26.121520+0000","last_fullsized":"2023-11-06T01:33:26.121520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:49.320532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123031+0000","last_change":"2023-11-06T01:32:26.121107+0000","last_active":"2023-11-06T01:33:26.123031+0000","last_peered":"2023-11-06T01:33:26.123031+0000","last_clean":"2023-11-06T01:33:26.123031+0000","last_became_active":"2023-11-06T01:32:26.120790+0000","last_became_peered":"2023-11-06T01:32:26.120790+0000","last_unstale":"2023-11-06T01:33:26.123031+0000","last_undegraded":"2023-11-06T01:33:26.123031+0000","last_fullsized":"2023-11-06T01:33:26.123031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:34:46.660301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122657+0000","last_change":"2023-11-06T01:32:25.678959+0000","last_active":"2023-11-06T01:33:26.122657+0000","last_peered":"2023-11-06T01:33:26.122657+0000","last_clean":"2023-11-06T01:33:26.122657+0000","last_became_active":"2023-11-06T01:32:25.678564+0000","last_became_peered":"2023-11-06T01:32:25.678564+0000","last_unstale":"2023-11-06T01:33:26.122657+0000","last_undegraded":"2023-11-06T01:33:26.122657+0000","last_fullsized":"2023-11-06T01:33:26.122657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:30:06.304889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119508+0000","last_change":"2023-11-06T01:32:26.113222+0000","last_active":"2023-11-06T01:33:26.119508+0000","last_peered":"2023-11-06T01:33:26.119508+0000","last_clean":"2023-11-06T01:33:26.119508+0000","last_became_active":"2023-11-06T01:32:26.112926+0000","last_became_peered":"2023-11-06T01:32:26.112926+0000","last_unstale":"2023-11-06T01:33:26.119508+0000","last_undegraded":"2023-11-06T01:33:26.119508+0000","last_fullsized":"2023-11-06T01:33:26.119508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:09:51.042469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118301+0000","last_change":"2023-11-06T01:32:25.684985+0000","last_active":"2023-11-06T01:33:26.118301+0000","last_peered":"2023-11-06T01:33:26.118301+0000","last_clean":"2023-11-06T01:33:26.118301+0000","last_became_active":"2023-11-06T01:32:25.684656+0000","last_became_peered":"2023-11-06T01:32:25.684656+0000","last_unstale":"2023-11-06T01:33:26.118301+0000","last_undegraded":"2023-11-06T01:33:26.118301+0000","last_fullsized":"2023-11-06T01:33:26.118301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:42:29.406803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120058+0000","last_change":"2023-11-06T01:32:26.107484+0000","last_active":"2023-11-06T01:33:26.120058+0000","last_peered":"2023-11-06T01:33:26.120058+0000","last_clean":"2023-11-06T01:33:26.120058+0000","last_became_active":"2023-11-06T01:32:26.106092+0000","last_became_peered":"2023-11-06T01:32:26.106092+0000","last_unstale":"2023-11-06T01:33:26.120058+0000","last_undegraded":"2023-11-06T01:33:26.120058+0000","last_fullsized":"2023-11-06T01:33:26.120058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:43:39.481290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.123047+0000","last_change":"2023-11-06T01:32:25.679531+0000","last_active":"2023-11-06T01:33:26.123047+0000","last_peered":"2023-11-06T01:33:26.123047+0000","last_clean":"2023-11-06T01:33:26.123047+0000","last_became_active":"2023-11-06T01:32:25.679076+0000","last_became_peered":"2023-11-06T01:32:25.679076+0000","last_unstale":"2023-11-06T01:33:26.123047+0000","last_undegraded":"2023-11-06T01:33:26.123047+0000","last_fullsized":"2023-11-06T01:33:26.123047+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:37.381547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118418+0000","last_change":"2023-11-06T01:32:26.118584+0000","last_active":"2023-11-06T01:33:26.118418+0000","last_peered":"2023-11-06T01:33:26.118418+0000","last_clean":"2023-11-06T01:33:26.118418+0000","last_became_active":"2023-11-06T01:32:26.118285+0000","last_became_peered":"2023-11-06T01:32:26.118285+0000","last_unstale":"2023-11-06T01:33:26.118418+0000","last_undegraded":"2023-11-06T01:33:26.118418+0000","last_fullsized":"2023-11-06T01:33:26.118418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T01:52:33.057377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119913+0000","last_change":"2023-11-06T01:32:25.681214+0000","last_active":"2023-11-06T01:33:26.119913+0000","last_peered":"2023-11-06T01:33:26.119913+0000","last_clean":"2023-11-06T01:33:26.119913+0000","last_became_active":"2023-11-06T01:32:25.680907+0000","last_became_peered":"2023-11-06T01:32:25.680907+0000","last_unstale":"2023-11-06T01:33:26.119913+0000","last_undegraded":"2023-11-06T01:33:26.119913+0000","last_fullsized":"2023-11-06T01:33:26.119913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:04:58.248787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.119830+0000","last_change":"2023-11-06T01:32:26.120862+0000","last_active":"2023-11-06T01:33:26.119830+0000","last_peered":"2023-11-06T01:33:26.119830+0000","last_clean":"2023-11-06T01:33:26.119830+0000","last_became_active":"2023-11-06T01:32:26.120307+0000","last_became_peered":"2023-11-06T01:32:26.120307+0000","last_unstale":"2023-11-06T01:33:26.119830+0000","last_undegraded":"2023-11-06T01:33:26.119830+0000","last_fullsized":"2023-11-06T01:33:26.119830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:45:35.611541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":105,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:53.144644+0000","last_change":"2023-11-06T01:32:25.675607+0000","last_active":"2023-11-06T01:33:53.144644+0000","last_peered":"2023-11-06T01:33:53.144644+0000","last_clean":"2023-11-06T01:33:53.144644+0000","last_became_active":"2023-11-06T01:32:25.675293+0000","last_became_peered":"2023-11-06T01:32:25.675293+0000","last_unstale":"2023-11-06T01:33:53.144644+0000","last_undegraded":"2023-11-06T01:33:53.144644+0000","last_fullsized":"2023-11-06T01:33:53.144644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:05.190844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1440,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120568+0000","last_change":"2023-11-06T01:32:26.116987+0000","last_active":"2023-11-06T01:33:26.120568+0000","last_peered":"2023-11-06T01:33:26.120568+0000","last_clean":"2023-11-06T01:33:26.120568+0000","last_became_active":"2023-11-06T01:32:26.116756+0000","last_became_peered":"2023-11-06T01:32:26.116756+0000","last_unstale":"2023-11-06T01:33:26.120568+0000","last_undegraded":"2023-11-06T01:33:26.120568+0000","last_fullsized":"2023-11-06T01:33:26.120568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:44.039520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122980+0000","last_change":"2023-11-06T01:32:25.675483+0000","last_active":"2023-11-06T01:33:26.122980+0000","last_peered":"2023-11-06T01:33:26.122980+0000","last_clean":"2023-11-06T01:33:26.122980+0000","last_became_active":"2023-11-06T01:32:25.675137+0000","last_became_peered":"2023-11-06T01:32:25.675137+0000","last_unstale":"2023-11-06T01:33:26.122980+0000","last_undegraded":"2023-11-06T01:33:26.122980+0000","last_fullsized":"2023-11-06T01:33:26.122980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:42.415165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120363+0000","last_change":"2023-11-06T01:32:26.115575+0000","last_active":"2023-11-06T01:33:26.120363+0000","last_peered":"2023-11-06T01:33:26.120363+0000","last_clean":"2023-11-06T01:33:26.120363+0000","last_became_active":"2023-11-06T01:32:26.115142+0000","last_became_peered":"2023-11-06T01:32:26.115142+0000","last_unstale":"2023-11-06T01:33:26.120363+0000","last_undegraded":"2023-11-06T01:33:26.120363+0000","last_fullsized":"2023-11-06T01:33:26.120363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:52.859340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129692+0000","last_change":"2023-11-06T01:32:25.676972+0000","last_active":"2023-11-06T01:33:26.129692+0000","last_peered":"2023-11-06T01:33:26.129692+0000","last_clean":"2023-11-06T01:33:26.129692+0000","last_became_active":"2023-11-06T01:32:25.676685+0000","last_became_peered":"2023-11-06T01:32:25.676685+0000","last_unstale":"2023-11-06T01:33:26.129692+0000","last_undegraded":"2023-11-06T01:33:26.129692+0000","last_fullsized":"2023-11-06T01:33:26.129692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:24.654794+0000","last_clean_scrub_stamp":"2023-11-06T01:32:24.654794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:30:21.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.121970+0000","last_change":"2023-11-06T01:32:26.115164+0000","last_active":"2023-11-06T01:33:26.121970+0000","last_peered":"2023-11-06T01:33:26.121970+0000","last_clean":"2023-11-06T01:33:26.121970+0000","last_became_active":"2023-11-06T01:32:26.114868+0000","last_became_peered":"2023-11-06T01:32:26.114868+0000","last_unstale":"2023-11-06T01:33:26.121970+0000","last_undegraded":"2023-11-06T01:33:26.121970+0000","last_fullsized":"2023-11-06T01:33:26.121970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:01:34.149871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.127653+0000","last_change":"2023-11-06T01:33:27.127653+0000","last_active":"2023-11-06T01:33:27.127653+0000","last_peered":"2023-11-06T01:33:27.127653+0000","last_clean":"2023-11-06T01:33:27.127653+0000","last_became_active":"2023-11-06T01:33:27.127151+0000","last_became_peered":"2023-11-06T01:33:27.127151+0000","last_unstale":"2023-11-06T01:33:27.127653+0000","last_undegraded":"2023-11-06T01:33:27.127653+0000","last_fullsized":"2023-11-06T01:33:27.127653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:54.825681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122463+0000","last_change":"2023-11-06T01:32:26.115336+0000","last_active":"2023-11-06T01:33:26.122463+0000","last_peered":"2023-11-06T01:33:26.122463+0000","last_clean":"2023-11-06T01:33:26.122463+0000","last_became_active":"2023-11-06T01:32:26.114946+0000","last_became_peered":"2023-11-06T01:32:26.114946+0000","last_unstale":"2023-11-06T01:33:26.122463+0000","last_undegraded":"2023-11-06T01:33:26.122463+0000","last_fullsized":"2023-11-06T01:33:26.122463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:19:13.651494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122948+0000","last_change":"2023-11-06T01:32:26.117794+0000","last_active":"2023-11-06T01:33:26.122948+0000","last_peered":"2023-11-06T01:33:26.122948+0000","last_clean":"2023-11-06T01:33:26.122948+0000","last_became_active":"2023-11-06T01:32:26.117425+0000","last_became_peered":"2023-11-06T01:32:26.117425+0000","last_unstale":"2023-11-06T01:33:26.122948+0000","last_undegraded":"2023-11-06T01:33:26.122948+0000","last_fullsized":"2023-11-06T01:33:26.122948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:00:24.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T01:33:27.125881+0000","last_change":"2023-11-06T01:33:27.125881+0000","last_active":"2023-11-06T01:33:27.125881+0000","last_peered":"2023-11-06T01:33:27.125881+0000","last_clean":"2023-11-06T01:33:27.125881+0000","last_became_active":"2023-11-06T01:33:27.125445+0000","last_became_peered":"2023-11-06T01:33:27.125445+0000","last_unstale":"2023-11-06T01:33:27.125881+0000","last_undegraded":"2023-11-06T01:33:27.125881+0000","last_fullsized":"2023-11-06T01:33:27.125881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:31.654256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129629+0000","last_change":"2023-11-06T01:32:26.109910+0000","last_active":"2023-11-06T01:33:26.129629+0000","last_peered":"2023-11-06T01:33:26.129629+0000","last_clean":"2023-11-06T01:33:26.129629+0000","last_became_active":"2023-11-06T01:32:26.109568+0000","last_became_peered":"2023-11-06T01:32:26.109568+0000","last_unstale":"2023-11-06T01:33:26.129629+0000","last_undegraded":"2023-11-06T01:33:26.129629+0000","last_fullsized":"2023-11-06T01:33:26.129629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:11.049749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120801+0000","last_change":"2023-11-06T01:32:26.114824+0000","last_active":"2023-11-06T01:33:26.120801+0000","last_peered":"2023-11-06T01:33:26.120801+0000","last_clean":"2023-11-06T01:33:26.120801+0000","last_became_active":"2023-11-06T01:32:26.114539+0000","last_became_peered":"2023-11-06T01:32:26.114539+0000","last_unstale":"2023-11-06T01:33:26.120801+0000","last_undegraded":"2023-11-06T01:33:26.120801+0000","last_fullsized":"2023-11-06T01:33:26.120801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:03:02.018769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122658+0000","last_change":"2023-11-06T01:32:26.114743+0000","last_active":"2023-11-06T01:33:26.122658+0000","last_peered":"2023-11-06T01:33:26.122658+0000","last_clean":"2023-11-06T01:33:26.122658+0000","last_became_active":"2023-11-06T01:32:26.114432+0000","last_became_peered":"2023-11-06T01:32:26.114432+0000","last_unstale":"2023-11-06T01:33:26.122658+0000","last_undegraded":"2023-11-06T01:33:26.122658+0000","last_fullsized":"2023-11-06T01:33:26.122658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:27:18.209810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118548+0000","last_change":"2023-11-06T01:32:26.125724+0000","last_active":"2023-11-06T01:33:26.118548+0000","last_peered":"2023-11-06T01:33:26.118548+0000","last_clean":"2023-11-06T01:33:26.118548+0000","last_became_active":"2023-11-06T01:32:26.125455+0000","last_became_peered":"2023-11-06T01:32:26.125455+0000","last_unstale":"2023-11-06T01:33:26.118548+0000","last_undegraded":"2023-11-06T01:33:26.118548+0000","last_fullsized":"2023-11-06T01:33:26.118548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:49.099308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.124151+0000","last_change":"2023-11-06T01:32:26.119328+0000","last_active":"2023-11-06T01:33:26.124151+0000","last_peered":"2023-11-06T01:33:26.124151+0000","last_clean":"2023-11-06T01:33:26.124151+0000","last_became_active":"2023-11-06T01:32:26.119042+0000","last_became_peered":"2023-11-06T01:32:26.119042+0000","last_unstale":"2023-11-06T01:33:26.124151+0000","last_undegraded":"2023-11-06T01:33:26.124151+0000","last_fullsized":"2023-11-06T01:33:26.124151+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:04:36.143788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.118310+0000","last_change":"2023-11-06T01:32:26.126538+0000","last_active":"2023-11-06T01:33:26.118310+0000","last_peered":"2023-11-06T01:33:26.118310+0000","last_clean":"2023-11-06T01:33:26.118310+0000","last_became_active":"2023-11-06T01:32:26.124898+0000","last_became_peered":"2023-11-06T01:32:26.124898+0000","last_unstale":"2023-11-06T01:33:26.118310+0000","last_undegraded":"2023-11-06T01:33:26.118310+0000","last_fullsized":"2023-11-06T01:33:26.118310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:33.019326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.122059+0000","last_change":"2023-11-06T01:32:26.118751+0000","last_active":"2023-11-06T01:33:26.122059+0000","last_peered":"2023-11-06T01:33:26.122059+0000","last_clean":"2023-11-06T01:33:26.122059+0000","last_became_active":"2023-11-06T01:32:26.118359+0000","last_became_peered":"2023-11-06T01:32:26.118359+0000","last_unstale":"2023-11-06T01:33:26.122059+0000","last_undegraded":"2023-11-06T01:33:26.122059+0000","last_fullsized":"2023-11-06T01:33:26.122059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:23:55.960400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.129434+0000","last_change":"2023-11-06T01:32:26.110133+0000","last_active":"2023-11-06T01:33:26.129434+0000","last_peered":"2023-11-06T01:33:26.129434+0000","last_clean":"2023-11-06T01:33:26.129434+0000","last_became_active":"2023-11-06T01:32:26.109793+0000","last_became_peered":"2023-11-06T01:32:26.109793+0000","last_unstale":"2023-11-06T01:33:26.129434+0000","last_undegraded":"2023-11-06T01:33:26.129434+0000","last_fullsized":"2023-11-06T01:33:26.129434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:00:03.956947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T01:33:26.120467+0000","last_change":"2023-11-06T01:32:26.109828+0000","last_active":"2023-11-06T01:33:26.120467+0000","last_peered":"2023-11-06T01:33:26.120467+0000","last_clean":"2023-11-06T01:33:26.120467+0000","last_became_active":"2023-11-06T01:32:26.109526+0000","last_became_peered":"2023-11-06T01:32:26.109526+0000","last_unstale":"2023-11-06T01:33:26.120467+0000","last_undegraded":"2023-11-06T01:33:26.120467+0000","last_fullsized":"2023-11-06T01:33:26.120467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T01:32:25.656301+0000","last_clean_scrub_stamp":"2023-11-06T01:32:25.656301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:36.561908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250075,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65618,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":855,"num_read_kb":496,"num_write":232,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1707,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":192216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481821,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":436587,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95799999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":1.002,"15min":1.002},"last":0.59599999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51200000000000001,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.68100000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.75700000000000001,"15min":0.77300000000000002},"min":{"1min":0.48299999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.86499999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80100000000000005,"5min":2.137,"15min":2.137},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71399999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78000000000000003,"5min":1.056,"15min":1.056},"last":0.54100000000000004}]},{"osd":6,"last update":"Mon Nov 6 01:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.877}]},{"osd":7,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.51900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92100000000000004,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.81899999999999995,"15min":0.83499999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89900000000000002,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.70599999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.66300000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1970000000000001,"5min":1.343,"15min":1.343},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72099999999999997,"15min":0.71499999999999997},"min":{"1min":0.58899999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65000000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":0.877},"min":{"1min":0.626,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.085,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78200000000000003,"15min":0.78500000000000003},"min":{"1min":0.59499999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.94999999999999996,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.78900000000000003}]}]},{"osd":10,"up_from":70,"seq":300647710754,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":1081580,"data_compressed":41972,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.54000000000000004}]},{"osd":1,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55800000000000005,"15min":0.55700000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.443}]},{"osd":2,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.55000000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0629999999999999,"5min":1.071,"15min":1.071},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0900000000000001,"5min":1.169,"15min":1.169},"last":0.63600000000000001}]},{"osd":3,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76300000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.48399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84799999999999998,"5min":1.052,"15min":1.052},"last":0.60399999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61899999999999999,"15min":0.60999999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.88900000000000001,"5min":1.133,"15min":1.133},"last":0.58299999999999996}]},{"osd":5,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.50800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.105,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82899999999999996,"15min":0.82699999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.044,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77900000000000003,"15min":0.78000000000000003},"min":{"1min":0.51800000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95099999999999996,"5min":1.147,"15min":1.147},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78700000000000003,"15min":0.78900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.079,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.875,"15min":0.875},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.133,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91600000000000004},"min":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.179,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.82099999999999995}]},{"osd":9,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.57099999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.026,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.85199999999999998,"15min":0.86099999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0549999999999999,"5min":1.246,"15min":1.246},"last":0.754}]},{"osd":11,"last update":"Mon Nov 6 01:33:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.145,"5min":1.226,"15min":1.226},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.82599999999999996,"15min":0.82099999999999995},"min":{"1min":0.63,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.76200000000000001}]}]},{"osd":9,"up_from":65,"seq":279172874278,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":436620,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.49199999999999999,"15min":0.50700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.9780000000000002,"5min":3.9780000000000002,"15min":3.9780000000000002},"last":0.28000000000000003}]},{"osd":1,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.66900000000000004,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77800000000000002,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":6.6210000000000004,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":0.46600000000000003}]},{"osd":2,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.67700000000000005,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.476,"5min":6.476,"15min":6.476},"last":0.39800000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59799999999999998,"15min":0.58499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":4.0270000000000001,"5min":4.0270000000000001,"15min":4.0270000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82199999999999995,"5min":0.87,"15min":0.87},"last":0.34799999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.70899999999999996,"15min":0.68000000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.47699999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77200000000000002,"15min":0.77100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":4.1589999999999998,"5min":4.1589999999999998,"15min":4.1589999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.79300000000000004,"15min":0.81499999999999995},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.44700000000000001}]},{"osd":6,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.80100000000000005,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":6.056,"5min":6.056,"15min":6.056},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.3700000000000001,"5min":6.3700000000000001,"15min":6.3700000000000001},"last":0.45900000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85999999999999999,"15min":0.86199999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.97799999999999998,"5min":1.603,"15min":1.603},"last":0.57599999999999996}]},{"osd":8,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.83899999999999997,"15min":0.81599999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.524,"5min":6.524,"15min":6.524},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.85999999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":4.1900000000000004,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":0.41999999999999998}]},{"osd":10,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.84199999999999997,"15min":0.81799999999999995},"min":{"1min":0.51900000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.85899999999999999,"15min":0.84499999999999997},"min":{"1min":0.56799999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":4.1280000000000001,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":0.55000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":8,"up_from":58,"seq":249108103209,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":435872,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80400000000000005,"5min":1.177,"15min":1.177},"last":0.497},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":1.196,"15min":1.196},"last":0.70499999999999996}]},{"osd":1,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.50800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84399999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71699999999999997}]},{"osd":2,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.47199999999999998,"15min":0.46999999999999997},"min":{"1min":0.35299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.48699999999999999,"15min":0.48199999999999998},"min":{"1min":0.40500000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60399999999999998}]},{"osd":3,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78300000000000003,"5min":1.083,"15min":1.083},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.497,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.498,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":1.24,"15min":1.24},"last":0.627},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.60199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94299999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72599999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58099999999999996,"15min":0.56799999999999995},"min":{"1min":0.52900000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.95599999999999996,"5min":1.036,"15min":1.036},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.61199999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.754}]},{"osd":7,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.57399999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.0600000000000001,"5min":1.353,"15min":1.353},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.72299999999999998,"15min":0.70499999999999996},"min":{"1min":0.58699999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.0149999999999999,"5min":1.276,"15min":1.276},"last":0.78500000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81399999999999995}]},{"osd":10,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.77200000000000002,"15min":0.75900000000000001},"min":{"1min":0.61899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0840000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.78400000000000003,"15min":0.76400000000000001},"min":{"1min":0.60399999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.88700000000000001}]},{"osd":11,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86699999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":1029257,"data_compressed":41972,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.873,"5min":1.157,"15min":1.157},"last":0.376}]},{"osd":1,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.433,"15min":0.40899999999999997},"min":{"1min":0.35899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79500000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.307,"15min":0.307},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.41099999999999998}]},{"osd":2,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.60899999999999999,"15min":0.58299999999999996},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.60599999999999998,"15min":0.57699999999999996},"min":{"1min":0.44700000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.69599999999999995,"15min":0.67600000000000005},"min":{"1min":0.45900000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.8090000000000002,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.63200000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56699999999999995,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.53700000000000003,"15min":0.50600000000000001},"min":{"1min":0.372,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":0.379}]},{"osd":5,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.66800000000000004,"15min":0.64200000000000002},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.66000000000000003,"15min":0.626},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71899999999999997,"15min":0.69599999999999995},"min":{"1min":0.57299999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73099999999999998,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.48299999999999998}]},{"osd":8,"last update":"Mon Nov 6 01:33:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.75900000000000001,"15min":0.72799999999999998},"min":{"1min":0.624,"5min":0.442,"15min":0.442},"max":{"1min":1.0209999999999999,"5min":2.395,"15min":2.395},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78300000000000003,"15min":0.753},"min":{"1min":0.63,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.089,"5min":2.484,"15min":2.484},"last":0.73699999999999999}]},{"osd":9,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.79400000000000004,"15min":0.78200000000000003},"min":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.78400000000000003}]},{"osd":10,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.51900000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.84899999999999998}]},{"osd":11,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.77300000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":436172,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.52100000000000002,"15min":0.52600000000000002},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.39800000000000002}]},{"osd":1,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.499},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.47199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.68899999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.53500000000000003}]},{"osd":2,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.60699999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70899999999999996,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52300000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.57399999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.47799999999999998}]},{"osd":4,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.61199999999999999,"15min":0.62},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78800000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.44900000000000001}]},{"osd":5,"last update":"Mon Nov 6 01:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86799999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46899999999999997}]},{"osd":7,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.75,"15min":0.76800000000000002},"min":{"1min":0.496,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.78200000000000003,"5min":1.042,"15min":1.042},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.68899999999999995,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.879,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.622}]},{"osd":8,"last update":"Mon Nov 6 01:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.78900000000000003,"15min":0.82999999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.74199999999999999,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.54900000000000004}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.42999999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88900000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.82399999999999995,"15min":0.84799999999999998},"min":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64600000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.76900000000000002,"15min":0.78700000000000003},"min":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.93500000000000005,"5min":1.238,"15min":1.238},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.82999999999999996,"15min":0.84899999999999998},"min":{"1min":0.58699999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.86299999999999999,"5min":1.141,"15min":1.141},"last":0.68000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":492403,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82699999999999996,"5min":1.093,"15min":1.093},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53100000000000003,"15min":0.498},"min":{"1min":0.31,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.85999999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.751}]},{"osd":2,"last update":"Mon Nov 6 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.60899999999999999,"15min":0.53700000000000003},"min":{"1min":0.51500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.95699999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.58599999999999997,"15min":0.52100000000000002},"min":{"1min":0.48099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.874},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.61699999999999999,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1459999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.52000000000000002,"15min":0.47499999999999998},"min":{"1min":0.308,"5min":0.252,"15min":0.252},"max":{"1min":1.0580000000000001,"5min":1.617,"15min":1.617},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.57799999999999996,"15min":0.52400000000000002},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.93400000000000005,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.56599999999999995}]},{"osd":5,"last update":"Mon Nov 6 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.65900000000000003,"15min":0.60399999999999998},"min":{"1min":0.59099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0189999999999999,"5min":1.056,"15min":1.056},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.66400000000000003,"15min":0.61799999999999999},"min":{"1min":0.54800000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82399999999999995}]},{"osd":6,"last update":"Mon Nov 6 01:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.64700000000000002,"15min":0.61499999999999999},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96699999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.69899999999999995,"15min":0.66600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.64500000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.66400000000000003,"15min":0.58799999999999997},"min":{"1min":0.49099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.137,"5min":2.27,"15min":2.27},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.68400000000000005,"15min":0.59499999999999997},"min":{"1min":0.53300000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1739999999999999,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.65900000000000003}]},{"osd":8,"last update":"Mon Nov 6 01:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.75800000000000001,"15min":0.70599999999999996},"min":{"1min":0.68999999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.79600000000000004,"15min":0.751},"min":{"1min":0.68799999999999994,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.194,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.78900000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.45300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.82799999999999996,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":0.96599999999999997}]},{"osd":10,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.88300000000000001,"15min":0.875},"min":{"1min":0.56200000000000006,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.23,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.876,"15min":0.86699999999999999},"min":{"1min":0.57999999999999996,"5min":0.501,"15min":0.501},"max":{"1min":1.1579999999999999,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.996},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.94399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":437505,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 01:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.53500000000000003},"min":{"1min":0.434,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74099999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.32200000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.42499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72099999999999997,"5min":1.052,"15min":1.052},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55600000000000005}]},{"osd":5,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70999999999999996,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":1.198,"15min":1.198},"last":0.58499999999999996}]},{"osd":6,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52800000000000002,"15min":0.55300000000000005},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66300000000000003,"5min":1.075,"15min":1.075},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.33500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.68000000000000005,"5min":1.222,"15min":1.222},"last":0.39200000000000002}]},{"osd":7,"last update":"Mon Nov 6 01:32:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.52400000000000002}]},{"osd":8,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.71299999999999997,"15min":0.75700000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80000000000000004,"5min":1.292,"15min":1.292},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.71299999999999997,"15min":0.753},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76400000000000001,"5min":1.341,"15min":1.341},"last":0.66300000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.66000000000000003,"15min":0.67500000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.751,"5min":1.123,"15min":1.123},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76000000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.57399999999999995}]},{"osd":10,"last update":"Mon Nov 6 01:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.71299999999999997,"15min":0.72499999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86899999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86799999999999999,"5min":1.165,"15min":1.165},"last":0.61599999999999999}]},{"osd":11,"last update":"Mon Nov 6 01:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.56299999999999994,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.51000000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79600000000000004,"5min":0.878,"15min":0.878},"last":0.624}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":495604,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.497},"min":{"1min":0.46100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53800000000000003,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Nov 6 01:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81200000000000006,"5min":1.044,"15min":1.044},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.44500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81499999999999995}]},{"osd":3,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.75},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60399999999999998,"15min":0.57599999999999996},"min":{"1min":0.443,"5min":0.374,"15min":0.374},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67000000000000004}]},{"osd":5,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.49199999999999999,"15min":0.45200000000000001},"min":{"1min":0.34499999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56100000000000005,"15min":0.52700000000000002},"min":{"1min":0.499,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76100000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.73699999999999999}]},{"osd":6,"last update":"Mon Nov 6 01:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.46200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.86199999999999999}]},{"osd":7,"last update":"Mon Nov 6 01:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.54700000000000004}]},{"osd":8,"last update":"Mon Nov 6 01:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.46700000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76900000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.48599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":1,"15min":1},"last":0.79400000000000004}]},{"osd":9,"last update":"Mon Nov 6 01:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76500000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.89600000000000002}]},{"osd":10,"last update":"Mon Nov 6 01:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.60699999999999998,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.022,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.96199999999999997,"5min":1.264,"15min":1.264},"last":0.78100000000000003}]},{"osd":11,"last update":"Mon Nov 6 01:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.83399999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149759,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":435928,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.36399999999999999},"min":{"1min":0.27300000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51500000000000001,"5min":0.627,"15min":0.627},"last":0.443},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.31,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":1.119,"15min":1.119},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54500000000000004,"15min":0.496},"min":{"1min":0.42099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60399999999999998,"15min":0.55800000000000005},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83099999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61299999999999999,"15min":0.59299999999999997},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.72499999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.39000000000000001}]},{"osd":4,"last update":"Mon Nov 6 01:33:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.46300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.66800000000000004,"5min":1.159,"15min":1.159},"last":0.73199999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.68100000000000005,"5min":1.042,"15min":1.042},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59799999999999998,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76100000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.318}]},{"osd":6,"last update":"Mon Nov 6 01:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89800000000000002,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.498}]},{"osd":7,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65900000000000003,"15min":0.64700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83999999999999997,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67200000000000004,"15min":0.66000000000000003},"min":{"1min":0.497,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93400000000000005,"5min":1.149,"15min":1.149},"last":0.64800000000000002}]},{"osd":8,"last update":"Mon Nov 6 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.622},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.0430000000000001,"5min":6.0430000000000001,"15min":6.0430000000000001},"last":0.76500000000000001}]},{"osd":9,"last update":"Mon Nov 6 01:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.74199999999999999,"15min":0.76900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85899999999999999,"5min":1.319,"15min":1.319},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.46400000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82199999999999995,"5min":1.214,"15min":1.214},"last":0.54500000000000004}]},{"osd":10,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.79100000000000004,"15min":0.81200000000000006},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.83699999999999997,"5min":1.173,"15min":1.173},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88700000000000001,"5min":1.286,"15min":1.286},"last":0.747}]},{"osd":11,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58999999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888122,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":436699,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.45300000000000001},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.49399999999999999,"15min":0.47799999999999998},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.749,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42999999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.36499999999999999},"min":{"1min":0.34499999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58499999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.41099999999999998,"15min":0.375},"min":{"1min":0.38900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.38600000000000001}]},{"osd":2,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55000000000000004,"15min":0.51600000000000001},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76400000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.504},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58899999999999997,"15min":0.53300000000000003},"min":{"1min":0.46600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81100000000000005,"5min":1.028,"15min":1.028},"last":0.55100000000000005}]},{"osd":3,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":0.63400000000000001,"15min":0.53100000000000003},"min":{"1min":0.53000000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":6.9050000000000002,"5min":6.9050000000000002,"15min":6.9050000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.4770000000000001,"5min":0.69999999999999996,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":12.869,"5min":12.869,"15min":12.869},"last":0.72799999999999998}]},{"osd":5,"last update":"Mon Nov 6 01:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61499999999999999,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92900000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.624,"15min":0.57799999999999996},"min":{"1min":0.53700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.623}]},{"osd":6,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.68999999999999995,"15min":0.64800000000000002},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":5.3949999999999996,"5min":5.3949999999999996,"15min":5.3949999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.127,"5min":0.73399999999999999,"15min":0.66800000000000004},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.64100000000000001}]},{"osd":7,"last update":"Mon Nov 6 01:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.65100000000000002,"15min":0.623},"min":{"1min":0.51500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999}]},{"osd":8,"last update":"Mon Nov 6 01:33:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66400000000000003,"15min":0.63900000000000001},"min":{"1min":0.51100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.92700000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76100000000000001,"15min":0.73499999999999999},"min":{"1min":0.65000000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77900000000000003}]},{"osd":9,"last update":"Mon Nov 6 01:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":0.82999999999999996,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":7.0659999999999998,"5min":7.0659999999999998,"15min":7.0659999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.73699999999999999,"15min":0.71599999999999997},"min":{"1min":0.63900000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.68999999999999995}]},{"osd":10,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.70199999999999996,"15min":0.68600000000000005},"min":{"1min":0.55000000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.65300000000000002,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.75800000000000001}]},{"osd":11,"last update":"Mon Nov 6 01:33:27 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.84399999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":1028700,"data_compressed":41972,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.53700000000000003},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76400000000000001,"5min":1.01,"15min":1.01},"last":0.48499999999999999}]},{"osd":1,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56999999999999995,"15min":0.53800000000000003},"min":{"1min":0.436,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81699999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57899999999999996,"15min":0.55000000000000004},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66300000000000003}]},{"osd":2,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.41499999999999998,"15min":0.39100000000000001},"min":{"1min":0.32700000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.41899999999999998,"15min":0.38300000000000001},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63500000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.31900000000000001}]},{"osd":3,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79600000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.439,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.74199999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.49199999999999999}]},{"osd":4,"last update":"Mon Nov 6 01:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59999999999999998,"15min":0.57199999999999995},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.61399999999999999,"15min":0.57199999999999995},"min":{"1min":0.47599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Mon Nov 6 01:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86099999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.80400000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55400000000000005}]},{"osd":7,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.67100000000000004,"15min":0.66000000000000003},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92100000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75}]},{"osd":8,"last update":"Mon Nov 6 01:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.59699999999999998},"min":{"1min":0.45800000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93100000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.42099999999999999}]},{"osd":9,"last update":"Mon Nov 6 01:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.85899999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59399999999999997}]},{"osd":10,"last update":"Mon Nov 6 01:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.55400000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94499999999999995,"5min":1.024,"15min":1.024},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96799999999999997,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34555,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36193,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T01:33:57.045 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:33:57.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.203+0000 7ff5cab46700 1 -- 172.21.15.53:0/2925231435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c4101800 msgr2=0x7ff5c41080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.203+0000 7ff5cab46700 1 --2- 172.21.15.53:0/2925231435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c4101800 0x7ff5c41080b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00b0d0 tx=0x7ff5bc00b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.203+0000 7ff5cab46700 1 -- 172.21.15.53:0/2925231435 shutdown_connections 2023-11-06T01:33:57.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.203+0000 7ff5cab46700 1 --2- 172.21.15.53:0/2925231435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5c40ffc70 0x7ff5c4100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.203+0000 7ff5cab46700 1 --2- 172.21.15.53:0/2925231435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c4101800 0x7ff5c41080b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.203+0000 7ff5cab46700 1 --2- 172.21.15.53:0/2925231435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c4100e60 0x7ff5c41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.203+0000 7ff5cab46700 1 -- 172.21.15.53:0/2925231435 >> 172.21.15.53:0/2925231435 conn(0x7ff5c40fb1c0 msgr2=0x7ff5c40fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:57.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.203+0000 7ff5cab46700 1 -- 172.21.15.53:0/2925231435 shutdown_connections 2023-11-06T01:33:57.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.203+0000 7ff5cab46700 1 -- 172.21.15.53:0/2925231435 wait complete. 2023-11-06T01:33:57.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.204+0000 7ff5cab46700 1 Processor -- start 2023-11-06T01:33:57.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.204+0000 7ff5cab46700 1 -- start start 2023-11-06T01:33:57.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.204+0000 7ff5cab46700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c40ffc70 0x7ff5c4195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.204+0000 7ff5cab46700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c4100e60 0x7ff5c4195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.204+0000 7ff5cab46700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5c4101800 0x7ff5c419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.204+0000 7ff5cab46700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c419d5c0 con 0x7ff5c40ffc70 2023-11-06T01:33:57.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.204+0000 7ff5cab46700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c419d700 con 0x7ff5c4101800 2023-11-06T01:33:57.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.204+0000 7ff5cab46700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c419d890 con 0x7ff5c4100e60 2023-11-06T01:33:57.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.204+0000 7ff5c88e2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c40ffc70 0x7ff5c4195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c88e2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c40ffc70 0x7ff5c4195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39640/0 (socket says 172.21.15.53:39640) 2023-11-06T01:33:57.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c88e2700 1 -- 172.21.15.53:0/1756391979 learned_addr learned my addr 172.21.15.53:0/1756391979 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:57.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c90e3700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5c4101800 0x7ff5c419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c3fff700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c4100e60 0x7ff5c4195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c88e2700 1 -- 172.21.15.53:0/1756391979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c4100e60 msgr2=0x7ff5c4195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c88e2700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c4100e60 0x7ff5c4195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c88e2700 1 -- 172.21.15.53:0/1756391979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5c4101800 msgr2=0x7ff5c419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c88e2700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5c4101800 0x7ff5c419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c88e2700 1 -- 172.21.15.53:0/1756391979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc009d00 con 0x7ff5c40ffc70 2023-11-06T01:33:57.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c3fff700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c4100e60 0x7ff5c4195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:57.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c88e2700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c40ffc70 0x7ff5c4195a40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800c970 tx=0x7ff5b800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:57.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c1ffb700 1 -- 172.21.15.53:0/1756391979 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8004220 con 0x7ff5c40ffc70 2023-11-06T01:33:57.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c1ffb700 1 -- 172.21.15.53:0/1756391979 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b8007d60 con 0x7ff5c40ffc70 2023-11-06T01:33:57.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5c1ffb700 1 -- 172.21.15.53:0/1756391979 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8003c90 con 0x7ff5c40ffc70 2023-11-06T01:33:57.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.205+0000 7ff5cab46700 1 -- 172.21.15.53:0/1756391979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c419db70 con 0x7ff5c40ffc70 2023-11-06T01:33:57.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.206+0000 7ff5cab46700 1 -- 172.21.15.53:0/1756391979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c419e030 con 0x7ff5c40ffc70 2023-11-06T01:33:57.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.206+0000 7ff5cab46700 1 -- 172.21.15.53:0/1756391979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c4107170 con 0x7ff5c40ffc70 2023-11-06T01:33:57.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.208+0000 7ff5c1ffb700 1 -- 172.21.15.53:0/1756391979 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff5b8004380 con 0x7ff5c40ffc70 2023-11-06T01:33:57.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.208+0000 7ff5c1ffb700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff5b40a05f0 0x7ff5b40a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.208+0000 7ff5c1ffb700 1 -- 172.21.15.53:0/1756391979 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff5b80cb890 con 0x7ff5c40ffc70 2023-11-06T01:33:57.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.208+0000 7ff5c3fff700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff5b40a05f0 0x7ff5b40a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.209+0000 7ff5c3fff700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff5b40a05f0 0x7ff5b40a2ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0007900 tx=0x7ff5b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:57.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.212+0000 7ff5c1ffb700 1 -- 172.21.15.53:0/1756391979 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff5b8095490 con 0x7ff5c40ffc70 2023-11-06T01:33:57.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:57 smithi053 ceph-mon[99472]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 12 op/s 2023-11-06T01:33:57.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:57 smithi053 ceph-mon[99472]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:57.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:57 smithi053 ceph-mon[99472]: mgrmap e18: x(active, since 7m), standbys: y, z 2023-11-06T01:33:57.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.419+0000 7ff5cab46700 1 -- 172.21.15.53:0/1756391979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5c402cc70 con 0x7ff5c40ffc70 2023-11-06T01:33:57.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.420+0000 7ff5c1ffb700 1 -- 172.21.15.53:0/1756391979 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22543 (secure 0 0 0) 0x7ff5b801d020 con 0x7ff5c40ffc70 2023-11-06T01:33:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:57.423 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:33:27.113678+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:33:57.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 -- 172.21.15.53:0/1756391979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff5b40a05f0 msgr2=0x7ff5b40a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff5b40a05f0 0x7ff5b40a2ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0007900 tx=0x7ff5b0008040 comp rx=0 tx=0).stop 2023-11-06T01:33:57.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 -- 172.21.15.53:0/1756391979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c40ffc70 msgr2=0x7ff5c4195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c40ffc70 0x7ff5c4195a40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800c970 tx=0x7ff5b800cc80 comp rx=0 tx=0).stop 2023-11-06T01:33:57.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 -- 172.21.15.53:0/1756391979 shutdown_connections 2023-11-06T01:33:57.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff5b40a05f0 0x7ff5b40a2ab0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5c4101800 0x7ff5c419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c40ffc70 0x7ff5c4195a40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 --2- 172.21.15.53:0/1756391979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c4100e60 0x7ff5c4195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 -- 172.21.15.53:0/1756391979 >> 172.21.15.53:0/1756391979 conn(0x7ff5c40fb1c0 msgr2=0x7ff5c41042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:57.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 -- 172.21.15.53:0/1756391979 shutdown_connections 2023-11-06T01:33:57.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.422+0000 7ff5cab46700 1 -- 172.21.15.53:0/1756391979 wait complete. 2023-11-06T01:33:57.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T01:32:24.604601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T01:32:25.457741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:33:57.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T01:33:57.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.564+0000 7f1b4beab700 1 -- 172.21.15.53:0/3869399445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44102910 msgr2=0x7f1b441091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.564+0000 7f1b4beab700 1 --2- 172.21.15.53:0/3869399445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44102910 0x7f1b441091c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00b0d0 tx=0x7f1b3c00b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 -- 172.21.15.53:0/3869399445 shutdown_connections 2023-11-06T01:33:57.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 --2- 172.21.15.53:0/3869399445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b44100d80 0x7f1b441011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 --2- 172.21.15.53:0/3869399445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44102910 0x7f1b441091c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 --2- 172.21.15.53:0/3869399445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b44101f70 0x7f1b441023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 -- 172.21.15.53:0/3869399445 >> 172.21.15.53:0/3869399445 conn(0x7f1b440fc2d0 msgr2=0x7f1b440fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:57.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 -- 172.21.15.53:0/3869399445 shutdown_connections 2023-11-06T01:33:57.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 -- 172.21.15.53:0/3869399445 wait complete. 2023-11-06T01:33:57.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 Processor -- start 2023-11-06T01:33:57.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 -- start start 2023-11-06T01:33:57.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b44100d80 0x7f1b44196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b44101f70 0x7f1b44196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44102910 0x7f1b4419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4419e620 con 0x7f1b44102910 2023-11-06T01:33:57.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4419e760 con 0x7f1b44100d80 2023-11-06T01:33:57.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4beab700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4419e8f0 con 0x7f1b44101f70 2023-11-06T01:33:57.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.565+0000 7f1b4a448700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44102910 0x7f1b4419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b4a448700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44102910 0x7f1b4419e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39658/0 (socket says 172.21.15.53:39658) 2023-11-06T01:33:57.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b4a448700 1 -- 172.21.15.53:0/586786787 learned_addr learned my addr 172.21.15.53:0/586786787 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:57.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b49446700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b44101f70 0x7f1b44196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b49c47700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b44100d80 0x7f1b44196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b4a448700 1 -- 172.21.15.53:0/586786787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b44101f70 msgr2=0x7f1b44196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b4a448700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b44101f70 0x7f1b44196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b4a448700 1 -- 172.21.15.53:0/586786787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b44100d80 msgr2=0x7f1b44196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b4a448700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b44100d80 0x7f1b44196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b4a448700 1 -- 172.21.15.53:0/586786787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c009d00 con 0x7f1b44102910 2023-11-06T01:33:57.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b49446700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b44101f70 0x7f1b44196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:57.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b4a448700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44102910 0x7f1b4419e050 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c007a00 tx=0x7f1b3c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:57.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b3affd700 1 -- 172.21.15.53:0/586786787 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00e040 con 0x7f1b44102910 2023-11-06T01:33:57.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b3affd700 1 -- 172.21.15.53:0/586786787 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b3c003e10 con 0x7f1b44102910 2023-11-06T01:33:57.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b4beab700 1 -- 172.21.15.53:0/586786787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4419eb70 con 0x7f1b44102910 2023-11-06T01:33:57.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b3affd700 1 -- 172.21.15.53:0/586786787 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c005610 con 0x7f1b44102910 2023-11-06T01:33:57.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.566+0000 7f1b4beab700 1 -- 172.21.15.53:0/586786787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4419eff0 con 0x7f1b44102910 2023-11-06T01:33:57.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.567+0000 7f1b4beab700 1 -- 172.21.15.53:0/586786787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b44108250 con 0x7f1b44102910 2023-11-06T01:33:57.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.568+0000 7f1b3affd700 1 -- 172.21.15.53:0/586786787 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f1b3c005770 con 0x7f1b44102910 2023-11-06T01:33:57.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.569+0000 7f1b3affd700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1b300a05a0 0x7f1b300a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.569+0000 7f1b49c47700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1b300a05a0 0x7f1b300a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.569+0000 7f1b3affd700 1 -- 172.21.15.53:0/586786787 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1b3c0cd000 con 0x7f1b44102910 2023-11-06T01:33:57.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.570+0000 7f1b49c47700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1b300a05a0 0x7f1b300a2a60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1b40005fd0 tx=0x7f1b40005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:57.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.574+0000 7f1b3affd700 1 -- 172.21.15.53:0/586786787 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1b3c096cb0 con 0x7f1b44102910 2023-11-06T01:33:57.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:57 smithi062 ceph-mon[103924]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 12 op/s 2023-11-06T01:33:57.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:57 smithi062 ceph-mon[103924]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:57.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:57 smithi062 ceph-mon[103924]: mgrmap e18: x(active, since 7m), standbys: y, z 2023-11-06T01:33:57.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:57 smithi163 ceph-mon[104416]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 12 op/s 2023-11-06T01:33:57.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:57 smithi163 ceph-mon[104416]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:57.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:57 smithi163 ceph-mon[104416]: mgrmap e18: x(active, since 7m), standbys: y, z 2023-11-06T01:33:57.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.773+0000 7f1b4beab700 1 -- 172.21.15.53:0/586786787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b4402cec0 con 0x7f1b44102910 2023-11-06T01:33:57.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.773+0000 7f1b3affd700 1 -- 172.21.15.53:0/586786787 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b3c09a2d0 con 0x7f1b44102910 2023-11-06T01:33:57.776 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:33:57.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 -- 172.21.15.53:0/586786787 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1b300a05a0 msgr2=0x7f1b300a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1b300a05a0 0x7f1b300a2a60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1b40005fd0 tx=0x7f1b40005e50 comp rx=0 tx=0).stop 2023-11-06T01:33:57.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 -- 172.21.15.53:0/586786787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44102910 msgr2=0x7f1b4419e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44102910 0x7f1b4419e050 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c007a00 tx=0x7f1b3c009570 comp rx=0 tx=0).stop 2023-11-06T01:33:57.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 -- 172.21.15.53:0/586786787 shutdown_connections 2023-11-06T01:33:57.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1b300a05a0 0x7f1b300a2a60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b44100d80 0x7f1b44196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44102910 0x7f1b4419e050 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 --2- 172.21.15.53:0/586786787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b44101f70 0x7f1b44196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 -- 172.21.15.53:0/586786787 >> 172.21.15.53:0/586786787 conn(0x7f1b440fc2d0 msgr2=0x7f1b44105390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:57.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 -- 172.21.15.53:0/586786787 shutdown_connections 2023-11-06T01:33:57.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.775+0000 7f1b4beab700 1 -- 172.21.15.53:0/586786787 wait complete. 2023-11-06T01:33:57.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T01:33:57.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.917+0000 7f6ce6d73700 1 -- 172.21.15.53:0/1862993875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ce01008d0 msgr2=0x7f6ce0108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 --2- 172.21.15.53:0/1862993875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ce01008d0 0x7f6ce0108190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800b0a0 tx=0x7f6cd800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 -- 172.21.15.53:0/1862993875 shutdown_connections 2023-11-06T01:33:57.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 --2- 172.21.15.53:0/1862993875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ce01008d0 0x7f6ce0108190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 --2- 172.21.15.53:0/1862993875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce0101200 0x7f6ce0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 --2- 172.21.15.53:0/1862993875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ce00fff30 0x7f6ce0100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 -- 172.21.15.53:0/1862993875 >> 172.21.15.53:0/1862993875 conn(0x7f6ce00fa960 msgr2=0x7f6ce00fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:57.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 -- 172.21.15.53:0/1862993875 shutdown_connections 2023-11-06T01:33:57.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 -- 172.21.15.53:0/1862993875 wait complete. 2023-11-06T01:33:57.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 Processor -- start 2023-11-06T01:33:57.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 -- start start 2023-11-06T01:33:57.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ce00fff30 0x7f6ce00ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce01008d0 0x7f6ce019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ce0101200 0x7f6ce019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce019e6a0 con 0x7f6ce0101200 2023-11-06T01:33:57.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce019e810 con 0x7f6ce00fff30 2023-11-06T01:33:57.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.918+0000 7f6ce6d73700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce019e9d0 con 0x7f6ce01008d0 2023-11-06T01:33:57.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce5310700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ce0101200 0x7f6ce019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce5310700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ce0101200 0x7f6ce019ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39672/0 (socket says 172.21.15.53:39672) 2023-11-06T01:33:57.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce5310700 1 -- 172.21.15.53:0/4031924608 learned_addr learned my addr 172.21.15.53:0/4031924608 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:57.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce4b0f700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ce00fff30 0x7f6ce00ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6cdffff700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce01008d0 0x7f6ce019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce5310700 1 -- 172.21.15.53:0/4031924608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce01008d0 msgr2=0x7f6ce019b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce5310700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce01008d0 0x7f6ce019b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce5310700 1 -- 172.21.15.53:0/4031924608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ce00fff30 msgr2=0x7f6ce00ff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:57.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce5310700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ce00fff30 0x7f6ce00ff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:57.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce5310700 1 -- 172.21.15.53:0/4031924608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd8009d00 con 0x7f6ce0101200 2023-11-06T01:33:57.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6cdffff700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce01008d0 0x7f6ce019b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:57.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce4b0f700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ce00fff30 0x7f6ce00ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:33:57.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce5310700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ce0101200 0x7f6ce019ded0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800b780 tx=0x7f6cd800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:57.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6cddffb700 1 -- 172.21.15.53:0/4031924608 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd800e040 con 0x7f6ce0101200 2023-11-06T01:33:57.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6cddffb700 1 -- 172.21.15.53:0/4031924608 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cd80094a0 con 0x7f6ce0101200 2023-11-06T01:33:57.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6cddffb700 1 -- 172.21.15.53:0/4031924608 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8005330 con 0x7f6ce0101200 2023-11-06T01:33:57.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.919+0000 7f6ce6d73700 1 -- 172.21.15.53:0/4031924608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce019ec50 con 0x7f6ce0101200 2023-11-06T01:33:57.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.920+0000 7f6ce6d73700 1 -- 172.21.15.53:0/4031924608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce019f0d0 con 0x7f6ce0101200 2023-11-06T01:33:57.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.920+0000 7f6ce6d73700 1 -- 172.21.15.53:0/4031924608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce01072f0 con 0x7f6ce0101200 2023-11-06T01:33:57.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.922+0000 7f6cddffb700 1 -- 172.21.15.53:0/4031924608 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f6cd801da30 con 0x7f6ce0101200 2023-11-06T01:33:57.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.922+0000 7f6cddffb700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6cd00a05a0 0x7f6cd00a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:57.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.922+0000 7f6cddffb700 1 -- 172.21.15.53:0/4031924608 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6cd80cd2f0 con 0x7f6ce0101200 2023-11-06T01:33:57.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.922+0000 7f6ce4b0f700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6cd00a05a0 0x7f6cd00a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:57.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.923+0000 7f6ce4b0f700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6cd00a05a0 0x7f6cd00a2a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0067c70 tx=0x7f6cd4007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:57.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:57.926+0000 7f6cddffb700 1 -- 172.21.15.53:0/4031924608 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6cd8096f00 con 0x7f6ce0101200 2023-11-06T01:33:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.131+0000 7f6ce6d73700 1 -- 172.21.15.53:0/4031924608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ce002cec0 con 0x7f6ce0101200 2023-11-06T01:33:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.131+0000 7f6cddffb700 1 -- 172.21.15.53:0/4031924608 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6cd809a520 con 0x7f6ce0101200 2023-11-06T01:33:58.133 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-06T01:33:58.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.133+0000 7f6ce6d73700 1 -- 172.21.15.53:0/4031924608 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6cd00a05a0 msgr2=0x7f6cd00a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.133+0000 7f6ce6d73700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6cd00a05a0 0x7f6cd00a2a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0067c70 tx=0x7f6cd4007400 comp rx=0 tx=0).stop 2023-11-06T01:33:58.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.133+0000 7f6ce6d73700 1 -- 172.21.15.53:0/4031924608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ce0101200 msgr2=0x7f6ce019ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.133+0000 7f6ce6d73700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ce0101200 0x7f6ce019ded0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800b780 tx=0x7f6cd800bd60 comp rx=0 tx=0).stop 2023-11-06T01:33:58.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.133+0000 7f6ce6d73700 1 -- 172.21.15.53:0/4031924608 shutdown_connections 2023-11-06T01:33:58.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.133+0000 7f6ce6d73700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6cd00a05a0 0x7f6cd00a2a60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.133+0000 7f6ce6d73700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ce00fff30 0x7f6ce00ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.133+0000 7f6ce6d73700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ce0101200 0x7f6ce019ded0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.133+0000 7f6ce6d73700 1 --2- 172.21.15.53:0/4031924608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce01008d0 0x7f6ce019b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.133+0000 7f6ce6d73700 1 -- 172.21.15.53:0/4031924608 >> 172.21.15.53:0/4031924608 conn(0x7f6ce00fa960 msgr2=0x7f6ce0104430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:58.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.134+0000 7f6ce6d73700 1 -- 172.21.15.53:0/4031924608 shutdown_connections 2023-11-06T01:33:58.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.134+0000 7f6ce6d73700 1 -- 172.21.15.53:0/4031924608 wait complete. 2023-11-06T01:33:58.147 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T01:33:58.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.276+0000 7fba1f6e6700 1 -- 172.21.15.53:0/3204562806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba18101fb0 msgr2=0x7fba18102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.276+0000 7fba1f6e6700 1 --2- 172.21.15.53:0/3204562806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba18101fb0 0x7fba18102410 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fba140099c0 tx=0x7fba14009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.276+0000 7fba1f6e6700 1 -- 172.21.15.53:0/3204562806 shutdown_connections 2023-11-06T01:33:58.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.276+0000 7fba1f6e6700 1 --2- 172.21.15.53:0/3204562806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba18100dc0 0x7fba181011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.276+0000 7fba1f6e6700 1 --2- 172.21.15.53:0/3204562806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba18102950 0x7fba18109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.276+0000 7fba1f6e6700 1 --2- 172.21.15.53:0/3204562806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba18101fb0 0x7fba18102410 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.276+0000 7fba1f6e6700 1 -- 172.21.15.53:0/3204562806 >> 172.21.15.53:0/3204562806 conn(0x7fba180fc310 msgr2=0x7fba180fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:58.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.276+0000 7fba1f6e6700 1 -- 172.21.15.53:0/3204562806 shutdown_connections 2023-11-06T01:33:58.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.276+0000 7fba1f6e6700 1 -- 172.21.15.53:0/3204562806 wait complete. 2023-11-06T01:33:58.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1f6e6700 1 Processor -- start 2023-11-06T01:33:58.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1f6e6700 1 -- start start 2023-11-06T01:33:58.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1f6e6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba18100dc0 0x7fba18074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:58.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1f6e6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba18101fb0 0x7fba18073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:58.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1f6e6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba18102950 0x7fba180736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:58.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1f6e6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba18073e60 con 0x7fba18100dc0 2023-11-06T01:33:58.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1f6e6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba18073fd0 con 0x7fba18101fb0 2023-11-06T01:33:58.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1f6e6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba181a0af0 con 0x7fba18102950 2023-11-06T01:33:58.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1d482700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba18100dc0 0x7fba18074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:58.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1d482700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba18100dc0 0x7fba18074b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39682/0 (socket says 172.21.15.53:39682) 2023-11-06T01:33:58.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1d482700 1 -- 172.21.15.53:0/2025882919 learned_addr learned my addr 172.21.15.53:0/2025882919 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:58.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1cc81700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba18101fb0 0x7fba18073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:58.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1dc83700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba18102950 0x7fba180736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:58.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1d482700 1 -- 172.21.15.53:0/2025882919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba18102950 msgr2=0x7fba180736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1d482700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba18102950 0x7fba180736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1d482700 1 -- 172.21.15.53:0/2025882919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba18101fb0 msgr2=0x7fba18073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.277+0000 7fba1d482700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba18101fb0 0x7fba18073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.278+0000 7fba1d482700 1 -- 172.21.15.53:0/2025882919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba140095f0 con 0x7fba18100dc0 2023-11-06T01:33:58.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.278+0000 7fba1d482700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba18100dc0 0x7fba18074b40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fba0800e970 tx=0x7fba0800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:58.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.278+0000 7fba0e7fc700 1 -- 172.21.15.53:0/2025882919 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba08009790 con 0x7fba18100dc0 2023-11-06T01:33:58.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.278+0000 7fba0e7fc700 1 -- 172.21.15.53:0/2025882919 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba0800c7e0 con 0x7fba18100dc0 2023-11-06T01:33:58.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.278+0000 7fba1f6e6700 1 -- 172.21.15.53:0/2025882919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba181a0dd0 con 0x7fba18100dc0 2023-11-06T01:33:58.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.278+0000 7fba0e7fc700 1 -- 172.21.15.53:0/2025882919 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba08003c70 con 0x7fba18100dc0 2023-11-06T01:33:58.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.278+0000 7fba1f6e6700 1 -- 172.21.15.53:0/2025882919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba181a1220 con 0x7fba18100dc0 2023-11-06T01:33:58.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.279+0000 7fba1f6e6700 1 -- 172.21.15.53:0/2025882919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1810c5d0 con 0x7fba18100dc0 2023-11-06T01:33:58.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.280+0000 7fba0e7fc700 1 -- 172.21.15.53:0/2025882919 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fba0800cc50 con 0x7fba18100dc0 2023-11-06T01:33:58.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.281+0000 7fba0e7fc700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fba040a0520 0x7fba040a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:58.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.281+0000 7fba0e7fc700 1 -- 172.21.15.53:0/2025882919 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fba08014070 con 0x7fba18100dc0 2023-11-06T01:33:58.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.281+0000 7fba1cc81700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fba040a0520 0x7fba040a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:58.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.281+0000 7fba1cc81700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fba040a0520 0x7fba040a29e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fba1400b580 tx=0x7fba14005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:58.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.285+0000 7fba0e7fc700 1 -- 172.21.15.53:0/2025882919 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fba080956f0 con 0x7fba18100dc0 2023-11-06T01:33:58.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.486+0000 7fba1f6e6700 1 -- 172.21.15.53:0/2025882919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba18107d10 con 0x7fba18100dc0 2023-11-06T01:33:58.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.487+0000 7fba0e7fc700 1 -- 172.21.15.53:0/2025882919 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba08003dd0 con 0x7fba18100dc0 2023-11-06T01:33:58.490 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-06T01:33:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 -- 172.21.15.53:0/2025882919 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fba040a0520 msgr2=0x7fba040a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fba040a0520 0x7fba040a29e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fba1400b580 tx=0x7fba14005fb0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 -- 172.21.15.53:0/2025882919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba18100dc0 msgr2=0x7fba18074b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba18100dc0 0x7fba18074b40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fba0800e970 tx=0x7fba0800ec80 comp rx=0 tx=0).stop 2023-11-06T01:33:58.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 -- 172.21.15.53:0/2025882919 shutdown_connections 2023-11-06T01:33:58.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fba040a0520 0x7fba040a29e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba18102950 0x7fba180736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba18100dc0 0x7fba18074b40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 --2- 172.21.15.53:0/2025882919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba18101fb0 0x7fba18073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 -- 172.21.15.53:0/2025882919 >> 172.21.15.53:0/2025882919 conn(0x7fba180fc310 msgr2=0x7fba180fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:58.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 -- 172.21.15.53:0/2025882919 shutdown_connections 2023-11-06T01:33:58.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.489+0000 7fba1f6e6700 1 -- 172.21.15.53:0/2025882919 wait complete. 2023-11-06T01:33:58.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:58 smithi053 ceph-mon[99472]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:58.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:58 smithi053 ceph-mon[99472]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:58.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:58 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1756391979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:58.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:58 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/586786787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:58.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:58 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/4031924608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:58.504 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:33:58.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.628+0000 7fb7cb942700 1 -- 172.21.15.53:0/667624801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c40fffc0 msgr2=0x7fb7c4100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.628+0000 7fb7cb942700 1 --2- 172.21.15.53:0/667624801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c40fffc0 0x7fb7c4100440 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009990 tx=0x7fb7b4009ca0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.628+0000 7fb7cb942700 1 -- 172.21.15.53:0/667624801 shutdown_connections 2023-11-06T01:33:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.628+0000 7fb7cb942700 1 --2- 172.21.15.53:0/667624801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c40ff660 0x7fb7c40ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.628+0000 7fb7cb942700 1 --2- 172.21.15.53:0/667624801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c4100980 0x7fb7c4100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.628+0000 7fb7cb942700 1 --2- 172.21.15.53:0/667624801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c40fffc0 0x7fb7c4100440 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.628+0000 7fb7cb942700 1 -- 172.21.15.53:0/667624801 >> 172.21.15.53:0/667624801 conn(0x7fb7c40fb1c0 msgr2=0x7fb7c40fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:58.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.628+0000 7fb7cb942700 1 -- 172.21.15.53:0/667624801 shutdown_connections 2023-11-06T01:33:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.628+0000 7fb7cb942700 1 -- 172.21.15.53:0/667624801 wait complete. 2023-11-06T01:33:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7cb942700 1 Processor -- start 2023-11-06T01:33:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7cb942700 1 -- start start 2023-11-06T01:33:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7cb942700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c40ff660 0x7fb7c4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7cb942700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c40fffc0 0x7fb7c4199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7cb942700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4100980 0x7fb7c41a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7cb942700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c41a0780 con 0x7fb7c4100980 2023-11-06T01:33:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7cb942700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c41a08f0 con 0x7fb7c40ff660 2023-11-06T01:33:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7cb942700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c41a0ab0 con 0x7fb7c40fffc0 2023-11-06T01:33:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7c9edf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4100980 0x7fb7c41a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7c9edf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4100980 0x7fb7c41a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39700/0 (socket says 172.21.15.53:39700) 2023-11-06T01:33:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7c9edf700 1 -- 172.21.15.53:0/2859091038 learned_addr learned my addr 172.21.15.53:0/2859091038 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7c8edd700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c40fffc0 0x7fb7c4199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7c96de700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c40ff660 0x7fb7c4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7c9edf700 1 -- 172.21.15.53:0/2859091038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c40fffc0 msgr2=0x7fb7c4199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7c9edf700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c40fffc0 0x7fb7c4199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7c9edf700 1 -- 172.21.15.53:0/2859091038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c40ff660 msgr2=0x7fb7c4198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7c9edf700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c40ff660 0x7fb7c4198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7c9edf700 1 -- 172.21.15.53:0/2859091038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b40095f0 con 0x7fb7c4100980 2023-11-06T01:33:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.629+0000 7fb7c9edf700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4100980 0x7fb7c41a01b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00b970 tx=0x7fb7bc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:58.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.630+0000 7fb7ba7fc700 1 -- 172.21.15.53:0/2859091038 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00c930 con 0x7fb7c4100980 2023-11-06T01:33:58.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.630+0000 7fb7ba7fc700 1 -- 172.21.15.53:0/2859091038 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7bc014440 con 0x7fb7c4100980 2023-11-06T01:33:58.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.630+0000 7fb7cb942700 1 -- 172.21.15.53:0/2859091038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c41a0d90 con 0x7fb7c4100980 2023-11-06T01:33:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.630+0000 7fb7ba7fc700 1 -- 172.21.15.53:0/2859091038 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0075d0 con 0x7fb7c4100980 2023-11-06T01:33:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.630+0000 7fb7cb942700 1 -- 172.21.15.53:0/2859091038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c41a1210 con 0x7fb7c4100980 2023-11-06T01:33:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.631+0000 7fb7cb942700 1 -- 172.21.15.53:0/2859091038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c410a440 con 0x7fb7c4100980 2023-11-06T01:33:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.632+0000 7fb7ba7fc700 1 -- 172.21.15.53:0/2859091038 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fb7bc0145b0 con 0x7fb7c4100980 2023-11-06T01:33:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.632+0000 7fb7ba7fc700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb7b00a0570 0x7fb7b00a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.633+0000 7fb7c96de700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb7b00a0570 0x7fb7b00a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.633+0000 7fb7ba7fc700 1 -- 172.21.15.53:0/2859091038 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb7bc006e40 con 0x7fb7c4100980 2023-11-06T01:33:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.633+0000 7fb7c96de700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb7b00a0570 0x7fb7b00a2a30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0005950 tx=0x7fb7c00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.637+0000 7fb7ba7fc700 1 -- 172.21.15.53:0/2859091038 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb7bc094bc0 con 0x7fb7c4100980 2023-11-06T01:33:58.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:58 smithi062 ceph-mon[103924]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:58.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:58 smithi062 ceph-mon[103924]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:58.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:58 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1756391979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:58.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:58 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/586786787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:58.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:58 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/4031924608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:58.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:58 smithi163 ceph-mon[104416]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:58.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:58 smithi163 ceph-mon[104416]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-06T01:33:58.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:58 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1756391979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:58.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:58 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/586786787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:33:58.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:58 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/4031924608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:33:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.890+0000 7fb7cb942700 1 -- 172.21.15.53:0/2859091038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7c402cc70 con 0x7fb7c4100980 2023-11-06T01:33:58.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.891+0000 7fb7ba7fc700 1 -- 172.21.15.53:0/2859091038 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10815 (secure 0 0 0) 0x7fb7bc0981e0 con 0x7fb7c4100980 2023-11-06T01:33:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:58.894 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-06T01:32:26.093194+0000","modified":"2023-11-06T01:32:33.006369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24472,"mds_2":24464,"mds_3":24422,"mds_4":24395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24395":{"gid":24395,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T01:33:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.893+0000 7fb7cb942700 1 -- 172.21.15.53:0/2859091038 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb7b00a0570 msgr2=0x7fb7b00a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.893+0000 7fb7cb942700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb7b00a0570 0x7fb7b00a2a30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0005950 tx=0x7fb7c00058e0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.893+0000 7fb7cb942700 1 -- 172.21.15.53:0/2859091038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4100980 msgr2=0x7fb7c41a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.893+0000 7fb7cb942700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4100980 0x7fb7c41a01b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00b970 tx=0x7fb7bc00bc80 comp rx=0 tx=0).stop 2023-11-06T01:33:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.893+0000 7fb7cb942700 1 -- 172.21.15.53:0/2859091038 shutdown_connections 2023-11-06T01:33:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.893+0000 7fb7cb942700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb7b00a0570 0x7fb7b00a2a30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.893+0000 7fb7cb942700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c40ff660 0x7fb7c4198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.893+0000 7fb7cb942700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4100980 0x7fb7c41a01b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.893+0000 7fb7cb942700 1 --2- 172.21.15.53:0/2859091038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c40fffc0 0x7fb7c4199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:58.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.893+0000 7fb7cb942700 1 -- 172.21.15.53:0/2859091038 >> 172.21.15.53:0/2859091038 conn(0x7fb7c40fb1c0 msgr2=0x7fb7c40fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:58.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.893+0000 7fb7cb942700 1 -- 172.21.15.53:0/2859091038 shutdown_connections 2023-11-06T01:33:58.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:58.894+0000 7fb7cb942700 1 -- 172.21.15.53:0/2859091038 wait complete. 2023-11-06T01:33:58.899 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-11-06T01:33:58.908 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:33:59.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.032+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2687809511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddbc0ffcb0 msgr2=0x7fddbc1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.032+0000 7fddc2ee2700 1 --2- 172.21.15.53:0/2687809511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddbc0ffcb0 0x7fddbc1000d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fddb00099c0 tx=0x7fddb0009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.032+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2687809511 shutdown_connections 2023-11-06T01:33:59.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.032+0000 7fddc2ee2700 1 --2- 172.21.15.53:0/2687809511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddbc101840 0x7fddbc1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.032+0000 7fddc2ee2700 1 --2- 172.21.15.53:0/2687809511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddbc0ffcb0 0x7fddbc1000d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.032+0000 7fddc2ee2700 1 --2- 172.21.15.53:0/2687809511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddbc100ea0 0x7fddbc101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.032+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2687809511 >> 172.21.15.53:0/2687809511 conn(0x7fddbc0fb220 msgr2=0x7fddbc0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.032+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2687809511 shutdown_connections 2023-11-06T01:33:59.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.032+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2687809511 wait complete. 2023-11-06T01:33:59.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc2ee2700 1 Processor -- start 2023-11-06T01:33:59.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc2ee2700 1 -- start start 2023-11-06T01:33:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc2ee2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddbc0ffcb0 0x7fddbc069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc2ee2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddbc100ea0 0x7fddbc19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc2ee2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddbc101840 0x7fddbc19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc2ee2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc19e5f0 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc2ee2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc19e760 con 0x7fddbc100ea0 2023-11-06T01:33:59.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc2ee2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc19e920 con 0x7fddbc101840 2023-11-06T01:33:59.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc0c7e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddbc0ffcb0 0x7fddbc069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc0c7e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddbc0ffcb0 0x7fddbc069a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39710/0 (socket says 172.21.15.53:39710) 2023-11-06T01:33:59.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc0c7e700 1 -- 172.21.15.53:0/2721552728 learned_addr learned my addr 172.21.15.53:0/2721552728 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:59.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddbbfff700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddbc100ea0 0x7fddbc19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc147f700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddbc101840 0x7fddbc19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc0c7e700 1 -- 172.21.15.53:0/2721552728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddbc101840 msgr2=0x7fddbc19de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc0c7e700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddbc101840 0x7fddbc19de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc0c7e700 1 -- 172.21.15.53:0/2721552728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddbc100ea0 msgr2=0x7fddbc19b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc0c7e700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddbc100ea0 0x7fddbc19b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddc0c7e700 1 -- 172.21.15.53:0/2721552728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb00095f0 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.033+0000 7fddbbfff700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddbc100ea0 0x7fddbc19b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.034+0000 7fddc0c7e700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddbc0ffcb0 0x7fddbc069a50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009990 tx=0x7fddb0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.034+0000 7fddb9ffb700 1 -- 172.21.15.53:0/2721552728 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb001d070 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.034+0000 7fddb9ffb700 1 -- 172.21.15.53:0/2721552728 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddb0004200 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.034+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2721552728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddbc19eba0 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.034+0000 7fddb9ffb700 1 -- 172.21.15.53:0/2721552728 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0022e30 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.034+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2721552728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddbc19f0a0 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.035+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2721552728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddbc10b540 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.036+0000 7fddb9ffb700 1 -- 172.21.15.53:0/2721552728 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fddb0003d40 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.037+0000 7fddb9ffb700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fddac0a05f0 0x7fddac0a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.037+0000 7fddb9ffb700 1 -- 172.21.15.53:0/2721552728 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fddb00cd060 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.037+0000 7fddbbfff700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fddac0a05f0 0x7fddac0a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.038+0000 7fddbbfff700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fddac0a05f0 0x7fddac0a2ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0697c0 tx=0x7fdda8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.040+0000 7fddb9ffb700 1 -- 172.21.15.53:0/2721552728 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fddb0096c10 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.248+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2721552728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddbc10b730 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.249+0000 7fddb9ffb700 1 -- 172.21.15.53:0/2721552728 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22543 (secure 0 0 0) 0x7fddb009a230 con 0x7fddbc0ffcb0 2023-11-06T01:33:59.251 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:33:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:33:27.113678+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:33:59.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2721552728 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fddac0a05f0 msgr2=0x7fddac0a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fddac0a05f0 0x7fddac0a2ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0697c0 tx=0x7fdda8008040 comp rx=0 tx=0).stop 2023-11-06T01:33:59.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2721552728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddbc0ffcb0 msgr2=0x7fddbc069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddbc0ffcb0 0x7fddbc069a50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009990 tx=0x7fddb0005e80 comp rx=0 tx=0).stop 2023-11-06T01:33:59.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2721552728 shutdown_connections 2023-11-06T01:33:59.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fddac0a05f0 0x7fddac0a2ab0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddbc101840 0x7fddbc19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddbc0ffcb0 0x7fddbc069a50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 --2- 172.21.15.53:0/2721552728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddbc100ea0 0x7fddbc19b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2721552728 >> 172.21.15.53:0/2721552728 conn(0x7fddbc0fb220 msgr2=0x7fddbc0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2721552728 shutdown_connections 2023-11-06T01:33:59.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.251+0000 7fddc2ee2700 1 -- 172.21.15.53:0/2721552728 wait complete. 2023-11-06T01:33:59.266 DEBUG:tasks.fwd_scrub:Yielding 2023-11-06T01:33:59.266 INFO:teuthology.run_tasks:Running task background_exec... 2023-11-06T01:33:59.279 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi053.front.sepia.ceph.com 2023-11-06T01:33:59.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-11-06T01:33:59.281 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-11-06T01:33:59.281 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-06T01:33:59.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-06T01:33:59.283 INFO:teuthology.run_tasks:Running task exec... 2023-11-06T01:33:59.324 INFO:teuthology.task.exec:Executing custom commands... 2023-11-06T01:33:59.324 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi053.front.sepia.ceph.com 2023-11-06T01:33:59.325 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-11-06T01:33:59.366 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:33:59.403 INFO:teuthology.orchestra.run.smithi053.stdout:Filesystem Size Used Avail Use% Mounted on 2023-11-06T01:33:59.403 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:3300,172.21.15.62:3300,172.21.15.163:3300:/volumes/_nogroup/sv_1/9b213191-048f-4ff8-846e-a578c140694d 340G 0 340G 0% /tmp/cephfs 2023-11-06T01:33:59.405 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-11-06T01:33:59.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.424+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/1768532091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f880ffc80 msgr2=0x7f1f881000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.424+0000 7f1f8ca8a700 1 --2- 172.21.15.53:0/1768532091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f880ffc80 0x7f1f881000a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1f780099c0 tx=0x7f1f78009cd0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.424+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/1768532091 shutdown_connections 2023-11-06T01:33:59.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.424+0000 7f1f8ca8a700 1 --2- 172.21.15.53:0/1768532091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f88101810 0x7f1f881080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.424+0000 7f1f8ca8a700 1 --2- 172.21.15.53:0/1768532091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f880ffc80 0x7f1f881000a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.424+0000 7f1f8ca8a700 1 --2- 172.21.15.53:0/1768532091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f88100e70 0x7f1f881012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.424+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/1768532091 >> 172.21.15.53:0/1768532091 conn(0x7f1f880fb1f0 msgr2=0x7f1f880fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.424+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/1768532091 shutdown_connections 2023-11-06T01:33:59.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.424+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/1768532091 wait complete. 2023-11-06T01:33:59.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.425+0000 7f1f8ca8a700 1 Processor -- start 2023-11-06T01:33:59.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.425+0000 7f1f8ca8a700 1 -- start start 2023-11-06T01:33:59.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8ca8a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f880ffc80 0x7f1f88068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8ca8a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f88100e70 0x7f1f8819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8ca8a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f88101810 0x7f1f8819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8ca8a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8819e5a0 con 0x7f1f880ffc80 2023-11-06T01:33:59.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8ca8a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8819e710 con 0x7f1f88100e70 2023-11-06T01:33:59.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8ca8a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8819e8d0 con 0x7f1f88101810 2023-11-06T01:33:59.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f880ffc80 0x7f1f88068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f880ffc80 0x7f1f88068ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39726/0 (socket says 172.21.15.53:39726) 2023-11-06T01:33:59.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8659c700 1 -- 172.21.15.53:0/307248152 learned_addr learned my addr 172.21.15.53:0/307248152 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:59.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f86d9d700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f88101810 0x7f1f8819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f85d9b700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f88100e70 0x7f1f8819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8659c700 1 -- 172.21.15.53:0/307248152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f88101810 msgr2=0x7f1f8819de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8659c700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f88101810 0x7f1f8819de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8659c700 1 -- 172.21.15.53:0/307248152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f88100e70 msgr2=0x7f1f8819b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8659c700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f88100e70 0x7f1f8819b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8659c700 1 -- 172.21.15.53:0/307248152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f780095f0 con 0x7f1f880ffc80 2023-11-06T01:33:59.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f86d9d700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f88101810 0x7f1f8819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:59.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f85d9b700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f88100e70 0x7f1f8819b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:59.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.426+0000 7f1f8659c700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f880ffc80 0x7f1f88068ac0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009fd0 tx=0x7f1f7800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.427+0000 7f1f777fe700 1 -- 172.21.15.53:0/307248152 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7801d070 con 0x7f1f880ffc80 2023-11-06T01:33:59.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.427+0000 7f1f777fe700 1 -- 172.21.15.53:0/307248152 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f780054f0 con 0x7f1f880ffc80 2023-11-06T01:33:59.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.427+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/307248152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8819eb50 con 0x7f1f880ffc80 2023-11-06T01:33:59.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.427+0000 7f1f777fe700 1 -- 172.21.15.53:0/307248152 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78004c40 con 0x7f1f880ffc80 2023-11-06T01:33:59.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.427+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/307248152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8819ef70 con 0x7f1f880ffc80 2023-11-06T01:33:59.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.430+0000 7f1f777fe700 1 -- 172.21.15.53:0/307248152 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f1f7800fc30 con 0x7f1f880ffc80 2023-11-06T01:33:59.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.430+0000 7f1f777fe700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f700a0520 0x7f1f700a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.430+0000 7f1f85d9b700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f700a0520 0x7f1f700a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.430+0000 7f1f777fe700 1 -- 172.21.15.53:0/307248152 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1f780cc890 con 0x7f1f880ffc80 2023-11-06T01:33:59.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f98c1ab80 1 Processor -- start 2023-11-06T01:33:59.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f98c1ab80 1 -- start start 2023-11-06T01:33:59.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f85d9b700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f700a0520 0x7f1f700a29e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009be0 tx=0x7f1f7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981eaa30 0x55f9981f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981f8040 0x55f9981fa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f998141a90 con 0x55f9981ea0d0 2023-11-06T01:33:59.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99813f450 con 0x55f9981eaa30 2023-11-06T01:33:59.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99813ad40 con 0x55f9981f8040 2023-11-06T01:33:59.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981ea4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39734/0 (socket says 172.21.15.53:39734) 2023-11-06T01:33:59.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f8659c700 1 -- 172.21.15.53:0/489950812 learned_addr learned my addr 172.21.15.53:0/489950812 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:59.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f85d9b700 1 --2- 172.21.15.53:0/489950812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981eaa30 0x55f9981f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.431+0000 7f1f86d9d700 1 --2- 172.21.15.53:0/489950812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981f8040 0x55f9981fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.432+0000 7f1f8659c700 1 -- 172.21.15.53:0/489950812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981f8040 msgr2=0x55f9981fa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.432+0000 7f1f8659c700 1 --2- 172.21.15.53:0/489950812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981f8040 0x55f9981fa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.432+0000 7f1f8659c700 1 -- 172.21.15.53:0/489950812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981eaa30 msgr2=0x55f9981f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.432+0000 7f1f8659c700 1 --2- 172.21.15.53:0/489950812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981eaa30 0x55f9981f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.432+0000 7f1f8659c700 1 -- 172.21.15.53:0/489950812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f997f68e60 con 0x55f9981ea0d0 2023-11-06T01:33:59.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.432+0000 7f1f8659c700 1 --2- 172.21.15.53:0/489950812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981ea4f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1f7809a210 tx=0x7f1f78005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581036b9c0b260b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.432+0000 7f1f74ff9700 1 -- 172.21.15.53:0/489950812 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7809a280 con 0x55f9981ea0d0 2023-11-06T01:33:59.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.432+0000 7f1f74ff9700 1 -- 172.21.15.53:0/489950812 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f78094e90 con 0x55f9981ea0d0 2023-11-06T01:33:59.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.432+0000 7f1f74ff9700 1 -- 172.21.15.53:0/489950812 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78098740 con 0x55f9981ea0d0 2023-11-06T01:33:59.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.433+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/489950812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 msgr2=0x55f9981ea4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.433+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/489950812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981ea4f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1f7809a210 tx=0x7f1f78005ef0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.433+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/489950812 shutdown_connections 2023-11-06T01:33:59.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.433+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/489950812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981f8040 0x55f9981fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.433+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/489950812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981ea4f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.433+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/489950812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981eaa30 0x55f9981f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.433+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/489950812 >> 172.21.15.53:0/489950812 conn(0x55f998149150 msgr2=0x55f9981e49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.433+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/489950812 shutdown_connections 2023-11-06T01:33:59.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.433+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/489950812 wait complete. 2023-11-06T01:33:59.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.433+0000 7f1f98c1ab80 1 -- start start 2023-11-06T01:33:59.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f98c1ab80 1 Processor -- start 2023-11-06T01:33:59.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981eaa30 0x55f9981fc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981f8040 0x55f998203490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9980d7590 con 0x55f9981ea0d0 2023-11-06T01:33:59.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9980d7230 con 0x55f9981eaa30 2023-11-06T01:33:59.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f997fbe9a0 con 0x55f9981f8040 2023-11-06T01:33:59.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f98c1ab80 10 client.0 _ll_register_callbacks cb 0x55f998024b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:59.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981fbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39740/0 (socket says 172.21.15.53:39740) 2023-11-06T01:33:59.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f8659c700 1 -- 172.21.15.53:0/1250917053 learned_addr learned my addr 172.21.15.53:0/1250917053 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:59.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f85d9b700 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981eaa30 0x55f9981fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f86d9d700 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981f8040 0x55f998203490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f8659c700 1 -- 172.21.15.53:0/1250917053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981f8040 msgr2=0x55f998203490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f8659c700 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981f8040 0x55f998203490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f8659c700 1 -- 172.21.15.53:0/1250917053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981eaa30 msgr2=0x55f9981fc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f8659c700 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981eaa30 0x55f9981fc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.434+0000 7f1f8659c700 1 -- 172.21.15.53:0/1250917053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8819ef70 con 0x55f9981ea0d0 2023-11-06T01:33:59.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f8659c700 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981fbf10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1f78096cf0 tx=0x7f1f7800ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f71e1a77d99fbd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:59.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f6dffb700 1 -- 172.21.15.53:0/1250917053 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7809e0b0 con 0x55f9981ea0d0 2023-11-06T01:33:59.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f6dffb700 1 -- 172.21.15.53:0/1250917053 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1f780a2260 con 0x55f9981ea0d0 2023-11-06T01:33:59.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/1250917053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f998165d90 con 0x55f9981ea0d0 2023-11-06T01:33:59.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f6dffb700 1 -- 172.21.15.53:0/1250917053 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f78098720 con 0x55f9981ea0d0 2023-11-06T01:33:59.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f98c1ab80 10 client.15438 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:59.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/1250917053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f997f51ce0 con 0x55f9981ea0d0 2023-11-06T01:33:59.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f6dffb700 1 -- 172.21.15.53:0/1250917053 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f998165d90 con 0x55f9981ea0d0 2023-11-06T01:33:59.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f6dffb700 1 -- 172.21.15.53:0/1250917053 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f1f780991e0 con 0x55f9981ea0d0 2023-11-06T01:33:59.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f98c1ab80 10 client.15438 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:59.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f98c1ab80 10 client.15438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:33:59.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f98c1ab80 20 client.15438 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:59.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f998207770 0x55f998209b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f98c1ab80 4 client.15438 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:59.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.435+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/1250917053 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9981b8e90 con 0x55f998207770 2023-11-06T01:33:59.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.436+0000 7f1f85d9b700 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f998207770 0x55f998209b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.436+0000 7f1f6dffb700 10 client.15438 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:59.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.436+0000 7f1f85d9b700 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f998207770 0x55f998209b90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556c24af8809c773 server_cookie=cae248c3f0a8b541 in_seq=0 out_seq=0 2023-11-06T01:33:59.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f6dffb700 1 -- 172.21.15.53:0/1250917053 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f9981b8e90 con 0x55f998207770 2023-11-06T01:33:59.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f6dffb700 10 client.15438 handle_command_reply: tid=0 2023-11-06T01:33:59.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 1 client.15438 shutdown 2023-11-06T01:33:59.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/1250917053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 msgr2=0x55f9981fbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981fbf10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1f78096cf0 tx=0x7f1f7800ff40 comp rx=0 tx=0).stop 2023-11-06T01:33:59.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/1250917053 shutdown_connections 2023-11-06T01:33:59.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f998207770 0x55f998209b90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981f8040 0x55f998203490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981ea0d0 0x55f9981fbf10 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/1250917053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981eaa30 0x55f9981fc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/1250917053 >> 172.21.15.53:0/1250917053 conn(0x55f998149150 msgr2=0x55f9981e4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f6dffb700 0 client.15438 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:59.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/1250917053 shutdown_connections 2023-11-06T01:33:59.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/1250917053 wait complete. 2023-11-06T01:33:59.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.437+0000 7f1f98c1ab80 20 client.15438 trim_cache size 0 max 16384 2023-11-06T01:33:59.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f98c1ab80 1 Processor -- start 2023-11-06T01:33:59.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f98c1ab80 1 -- start start 2023-11-06T01:33:59.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f998149150 0x55f998201440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981ea0d0 0x55f9981fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f9981fc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f998165d90 con 0x55f9981eaa30 2023-11-06T01:33:59.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9981df4a0 con 0x55f998149150 2023-11-06T01:33:59.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9981fb3c0 con 0x55f9981ea0d0 2023-11-06T01:33:59.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f86d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f9981fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f86d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f9981fc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39752/0 (socket says 172.21.15.53:39752) 2023-11-06T01:33:59.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f86d9d700 1 -- 172.21.15.53:0/152210541 learned_addr learned my addr 172.21.15.53:0/152210541 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:59.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f85d9b700 1 --2- 172.21.15.53:0/152210541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981ea0d0 0x55f9981fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.455+0000 7f1f8659c700 1 --2- 172.21.15.53:0/152210541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f998149150 0x55f998201440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.456+0000 7f1f86d9d700 1 -- 172.21.15.53:0/152210541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981ea0d0 msgr2=0x55f9981fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.456+0000 7f1f86d9d700 1 --2- 172.21.15.53:0/152210541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981ea0d0 0x55f9981fbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.456+0000 7f1f86d9d700 1 -- 172.21.15.53:0/152210541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f998149150 msgr2=0x55f998201440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.456+0000 7f1f86d9d700 1 --2- 172.21.15.53:0/152210541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f998149150 0x55f998201440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.456+0000 7f1f86d9d700 1 -- 172.21.15.53:0/152210541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9981e1070 con 0x55f9981eaa30 2023-11-06T01:33:59.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.456+0000 7f1f8659c700 1 --2- 172.21.15.53:0/152210541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f998149150 0x55f998201440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:33:59.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.456+0000 7f1f86d9d700 1 --2- 172.21.15.53:0/152210541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f9981fc200 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009c10 tx=0x7f1f80009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8bbb42f738cede server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.456+0000 7f1f757fa700 1 -- 172.21.15.53:0/152210541 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80007a20 con 0x55f9981eaa30 2023-11-06T01:33:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.456+0000 7f1f757fa700 1 -- 172.21.15.53:0/152210541 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f80007b80 con 0x55f9981eaa30 2023-11-06T01:33:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.456+0000 7f1f757fa700 1 -- 172.21.15.53:0/152210541 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800155c0 con 0x55f9981eaa30 2023-11-06T01:33:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.457+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/152210541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 msgr2=0x55f9981fc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.457+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/152210541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f9981fc200 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009c10 tx=0x7f1f80009040 comp rx=0 tx=0).stop 2023-11-06T01:33:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.457+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/152210541 shutdown_connections 2023-11-06T01:33:59.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.457+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/152210541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f998149150 0x55f998201440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.457+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/152210541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f9981fc200 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.457+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/152210541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9981ea0d0 0x55f9981fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.457+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/152210541 >> 172.21.15.53:0/152210541 conn(0x55f9981e4d00 msgr2=0x55f998209550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.457+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/152210541 shutdown_connections 2023-11-06T01:33:59.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.457+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/152210541 wait complete. 2023-11-06T01:33:59.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.457+0000 7f1f98c1ab80 1 -- start start 2023-11-06T01:33:59.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f98c1ab80 1 Processor -- start 2023-11-06T01:33:59.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f998149150 0x55f998251a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981ea0d0 0x55f998251f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f98c1ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f998258fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9982594f0 con 0x55f9981eaa30 2023-11-06T01:33:59.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f998259630 con 0x55f9981ea0d0 2023-11-06T01:33:59.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f98c1ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9982597c0 con 0x55f998149150 2023-11-06T01:33:59.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f98c1ab80 10 client.0 _ll_register_callbacks cb 0x7f1f7c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:59.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f86d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f998258fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f86d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f998258fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39758/0 (socket says 172.21.15.53:39758) 2023-11-06T01:33:59.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f86d9d700 1 -- 172.21.15.53:0/253789605 learned_addr learned my addr 172.21.15.53:0/253789605 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:59.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f8659c700 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f998149150 0x55f998251a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f85d9b700 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981ea0d0 0x55f998251f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f86d9d700 1 -- 172.21.15.53:0/253789605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f998149150 msgr2=0x55f998251a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f86d9d700 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f998149150 0x55f998251a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f86d9d700 1 -- 172.21.15.53:0/253789605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981ea0d0 msgr2=0x55f998251f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f86d9d700 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981ea0d0 0x55f998251f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f86d9d700 1 -- 172.21.15.53:0/253789605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f80011040 con 0x55f9981eaa30 2023-11-06T01:33:59.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.458+0000 7f1f85d9b700 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981ea0d0 0x55f998251f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:33:59.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f86d9d700 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f998258fb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000f80 tx=0x7f1f80016610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f17ec69dae5483f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f6dffb700 1 -- 172.21.15.53:0/253789605 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80004cd0 con 0x55f9981eaa30 2023-11-06T01:33:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f6dffb700 1 -- 172.21.15.53:0/253789605 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1f80004e30 con 0x55f9981eaa30 2023-11-06T01:33:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/253789605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f78094e90 con 0x55f9981eaa30 2023-11-06T01:33:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f6dffb700 1 -- 172.21.15.53:0/253789605 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f80016b50 con 0x55f9981eaa30 2023-11-06T01:33:59.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f98c1ab80 10 client.15444 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:59.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/253789605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f997f8f2d0 con 0x55f9981eaa30 2023-11-06T01:33:59.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f6dffb700 1 -- 172.21.15.53:0/253789605 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f78094e90 con 0x55f9981eaa30 2023-11-06T01:33:59.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f6dffb700 1 -- 172.21.15.53:0/253789605 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f1f800053e0 con 0x55f9981eaa30 2023-11-06T01:33:59.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f98c1ab80 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:59.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f98c1ab80 10 client.15444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:33:59.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f98c1ab80 20 client.15444 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:59.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f9981f8040 0x55f99825d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f98c1ab80 4 client.15444 mds_command: new command op to 24487 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-06T01:33:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.459+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/253789605 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f998133dc0 con 0x55f9981f8040 2023-11-06T01:33:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.460+0000 7f1f8659c700 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f9981f8040 0x55f99825d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.460+0000 7f1f8659c700 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f9981f8040 0x55f99825d1f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=677e8ca02c8c38a7 server_cookie=3b29674749d609f in_seq=0 out_seq=0 2023-11-06T01:33:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.460+0000 7f1f6dffb700 10 client.15444 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:59.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.461+0000 7f1f6dffb700 1 -- 172.21.15.53:0/253789605 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f998133dc0 con 0x55f9981f8040 2023-11-06T01:33:59.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.461+0000 7f1f6dffb700 10 client.15444 handle_command_reply: tid=0 2023-11-06T01:33:59.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.461+0000 7f1f98c1ab80 1 client.15444 shutdown 2023-11-06T01:33:59.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.461+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/253789605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 msgr2=0x55f998258fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.461+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f998258fb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000f80 tx=0x7f1f80016610 comp rx=0 tx=0).stop 2023-11-06T01:33:59.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.462+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/253789605 shutdown_connections 2023-11-06T01:33:59.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.462+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f9981f8040 0x55f99825d1f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.462+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9981eaa30 0x55f998258fb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.462+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f998149150 0x55f998251a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.462+0000 7f1f98c1ab80 1 --2- 172.21.15.53:0/253789605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9981ea0d0 0x55f998251f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.462+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/253789605 >> 172.21.15.53:0/253789605 conn(0x55f9981e4d00 msgr2=0x55f9981fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.462+0000 7f1f6dffb700 0 client.15444 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:59.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.462+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/253789605 shutdown_connections 2023-11-06T01:33:59.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.462+0000 7f1f98c1ab80 1 -- 172.21.15.53:0/253789605 wait complete. 2023-11-06T01:33:59.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.462+0000 7f1f98c1ab80 20 client.15444 trim_cache size 0 max 16384 2023-11-06T01:33:59.511 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:59.511 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2023-11-06T01:33:59.511 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "2c6d1131-3c26-4657-860b-faee98233fd7", 2023-11-06T01:33:59.512 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2023-11-06T01:33:59.512 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:59.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/307248152 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f700a0520 msgr2=0x7f1f700a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f700a0520 0x7f1f700a29e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009be0 tx=0x7f1f7c009260 comp rx=0 tx=0).stop 2023-11-06T01:33:59.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/307248152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f880ffc80 msgr2=0x7f1f88068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f880ffc80 0x7f1f88068ac0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009fd0 tx=0x7f1f7800f740 comp rx=0 tx=0).stop 2023-11-06T01:33:59.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/307248152 shutdown_connections 2023-11-06T01:33:59.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f1f700a0520 0x7f1f700a29e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f88101810 0x7f1f8819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f880ffc80 0x7f1f88068ac0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 --2- 172.21.15.53:0/307248152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f88100e70 0x7f1f8819b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/307248152 >> 172.21.15.53:0/307248152 conn(0x7f1f880fb1f0 msgr2=0x7f1f880fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/307248152 shutdown_connections 2023-11-06T01:33:59.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.463+0000 7f1f8ca8a700 1 -- 172.21.15.53:0/307248152 wait complete. 2023-11-06T01:33:59.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:33:59.520 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-11-06T01:33:59.580 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:59 smithi053 ceph-mon[99472]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 9 op/s 2023-11-06T01:33:59.581 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:59 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2025882919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:59.581 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:59 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2859091038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:59.581 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:33:59 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2721552728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:59.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-11-06T01:33:59.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:59 smithi062 ceph-mon[103924]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 9 op/s 2023-11-06T01:33:59.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:59 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2025882919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:59.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:59 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2859091038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:59.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:33:59 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2721552728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:33:59.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.686+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/274011107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b40ffca0 msgr2=0x7fe6b41000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.686+0000 7fe6bc0a8700 1 --2- 172.21.15.53:0/274011107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b40ffca0 0x7fe6b41000c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00b0d0 tx=0x7fe6ac00b3e0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.686+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/274011107 shutdown_connections 2023-11-06T01:33:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.686+0000 7fe6bc0a8700 1 --2- 172.21.15.53:0/274011107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b4101830 0x7fe6b41080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.686+0000 7fe6bc0a8700 1 --2- 172.21.15.53:0/274011107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b40ffca0 0x7fe6b41000c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.686+0000 7fe6bc0a8700 1 --2- 172.21.15.53:0/274011107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6b4100e90 0x7fe6b41012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.686+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/274011107 >> 172.21.15.53:0/274011107 conn(0x7fe6b40fb210 msgr2=0x7fe6b40fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.686+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/274011107 shutdown_connections 2023-11-06T01:33:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.686+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/274011107 wait complete. 2023-11-06T01:33:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6bc0a8700 1 Processor -- start 2023-11-06T01:33:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6bc0a8700 1 -- start start 2023-11-06T01:33:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6bc0a8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b40ffca0 0x7fe6b4069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6bc0a8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6b4100e90 0x7fe6b419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6bc0a8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b4101830 0x7fe6b419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6bc0a8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b419e700 con 0x7fe6b4101830 2023-11-06T01:33:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6bc0a8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b419e870 con 0x7fe6b4100e90 2023-11-06T01:33:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6bc0a8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b419ea30 con 0x7fe6b40ffca0 2023-11-06T01:33:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6ba645700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b4101830 0x7fe6b419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6ba645700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b4101830 0x7fe6b419df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39772/0 (socket says 172.21.15.53:39772) 2023-11-06T01:33:59.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6ba645700 1 -- 172.21.15.53:0/2697498868 learned_addr learned my addr 172.21.15.53:0/2697498868 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:59.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b40ffca0 0x7fe6b4069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.687+0000 7fe6b9643700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6b4100e90 0x7fe6b419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.688+0000 7fe6ba645700 1 -- 172.21.15.53:0/2697498868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b40ffca0 msgr2=0x7fe6b4069cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.688+0000 7fe6ba645700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b40ffca0 0x7fe6b4069cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.688+0000 7fe6ba645700 1 -- 172.21.15.53:0/2697498868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6b4100e90 msgr2=0x7fe6b419b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.688+0000 7fe6ba645700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6b4100e90 0x7fe6b419b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.688+0000 7fe6ba645700 1 -- 172.21.15.53:0/2697498868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac009d00 con 0x7fe6b4101830 2023-11-06T01:33:59.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.688+0000 7fe6ba645700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b4101830 0x7fe6b419df60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400e940 tx=0x7fe6a400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.688+0000 7fe6b2ffd700 1 -- 172.21.15.53:0/2697498868 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40097b0 con 0x7fe6b4101830 2023-11-06T01:33:59.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.688+0000 7fe6b2ffd700 1 -- 172.21.15.53:0/2697498868 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6a4005020 con 0x7fe6b4101830 2023-11-06T01:33:59.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.688+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/2697498868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b419ed10 con 0x7fe6b4101830 2023-11-06T01:33:59.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.688+0000 7fe6b2ffd700 1 -- 172.21.15.53:0/2697498868 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a400cc40 con 0x7fe6b4101830 2023-11-06T01:33:59.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.688+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/2697498868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b419f1d0 con 0x7fe6b4101830 2023-11-06T01:33:59.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.690+0000 7fe6b2ffd700 1 -- 172.21.15.53:0/2697498868 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe6a4005400 con 0x7fe6b4101830 2023-11-06T01:33:59.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.691+0000 7fe6b2ffd700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe69c0a0520 0x7fe69c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.691+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe69c0a0520 0x7fe69c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.691+0000 7fe6b2ffd700 1 -- 172.21.15.53:0/2697498868 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe6a4014070 con 0x7fe6b4101830 2023-11-06T01:33:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.692+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe69c0a0520 0x7fe69c0a29e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009fd0 tx=0x7fe6ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.692+0000 7fe6c8238b80 1 Processor -- start 2023-11-06T01:33:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.692+0000 7fe6c8238b80 1 -- start start 2023-11-06T01:33:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.692+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c5a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.692+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5ae80 0x5577c0c59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.692+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c59800 0x5577c0c59c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.692+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0baca10 con 0x5577c0c5a500 2023-11-06T01:33:59.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.692+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0baa3d0 con 0x5577c0c5ae80 2023-11-06T01:33:59.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.692+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0ba5cc0 con 0x5577c0c59800 2023-11-06T01:33:59.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe6b9e44700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c5a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe6b9e44700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c5a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39776/0 (socket says 172.21.15.53:39776) 2023-11-06T01:33:59.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe6b9e44700 1 -- 172.21.15.53:0/1762544173 learned_addr learned my addr 172.21.15.53:0/1762544173 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:59.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe6b9643700 1 --2- 172.21.15.53:0/1762544173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5ae80 0x5577c0c59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe6b9e44700 1 -- 172.21.15.53:0/1762544173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c59800 msgr2=0x5577c0c59c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:33:59.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/1762544173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c59800 0x5577c0c59c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe6b9e44700 1 -- 172.21.15.53:0/1762544173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5ae80 msgr2=0x5577c0c59270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/1762544173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5ae80 0x5577c0c59270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe6b9e44700 1 -- 172.21.15.53:0/1762544173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577c09d3de0 con 0x5577c0c5a500 2023-11-06T01:33:59.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/1762544173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c5a940 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac015000 tx=0x7fe6ac01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0398939a20f7eb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe69bfff700 1 -- 172.21.15.53:0/1762544173 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac00e040 con 0x5577c0c5a500 2023-11-06T01:33:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe69bfff700 1 -- 172.21.15.53:0/1762544173 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6ac009b80 con 0x5577c0c5a500 2023-11-06T01:33:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.693+0000 7fe69bfff700 1 -- 172.21.15.53:0/1762544173 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac01c410 con 0x5577c0c5a500 2023-11-06T01:33:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.694+0000 7fe6c8238b80 1 -- 172.21.15.53:0/1762544173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 msgr2=0x5577c0c5a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.694+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/1762544173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c5a940 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac015000 tx=0x7fe6ac01d000 comp rx=0 tx=0).stop 2023-11-06T01:33:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.694+0000 7fe6c8238b80 1 -- 172.21.15.53:0/1762544173 shutdown_connections 2023-11-06T01:33:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.694+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/1762544173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c59800 0x5577c0c59c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.694+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/1762544173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c5a940 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.694+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/1762544173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5ae80 0x5577c0c59270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.694+0000 7fe6c8238b80 1 -- 172.21.15.53:0/1762544173 >> 172.21.15.53:0/1762544173 conn(0x5577c0bb40d0 msgr2=0x5577c0c4f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.694+0000 7fe6c8238b80 1 -- 172.21.15.53:0/1762544173 shutdown_connections 2023-11-06T01:33:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.694+0000 7fe6c8238b80 1 -- 172.21.15.53:0/1762544173 wait complete. 2023-11-06T01:33:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.694+0000 7fe6c8238b80 1 -- start start 2023-11-06T01:33:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6c8238b80 1 Processor -- start 2023-11-06T01:33:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c59800 0x5577c0c62dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c63310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5ae80 0x5577c0c6a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0b42510 con 0x5577c0c5a500 2023-11-06T01:33:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0b421b0 con 0x5577c0c5ae80 2023-11-06T01:33:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0a29920 con 0x5577c0c59800 2023-11-06T01:33:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6c8238b80 10 client.0 _ll_register_callbacks cb 0x5577c0a8fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6b9643700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c63310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6b9643700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c63310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39784/0 (socket says 172.21.15.53:39784) 2023-11-06T01:33:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6b9643700 1 -- 172.21.15.53:0/3866741852 learned_addr learned my addr 172.21.15.53:0/3866741852 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6ba645700 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5ae80 0x5577c0c6a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c59800 0x5577c0c62dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6b9643700 1 -- 172.21.15.53:0/3866741852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c59800 msgr2=0x5577c0c62dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6b9643700 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c59800 0x5577c0c62dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6b9643700 1 -- 172.21.15.53:0/3866741852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5ae80 msgr2=0x5577c0c6a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6b9643700 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5ae80 0x5577c0c6a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.695+0000 7fe6b9643700 1 -- 172.21.15.53:0/3866741852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b4108a30 con 0x5577c0c5a500 2023-11-06T01:33:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6b9643700 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c63310 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80127f0 tx=0x7fe6a8012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdd81a64f627fa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6997fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6997fa700 1 -- 172.21.15.53:0/3866741852 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8017070 con 0x5577c0c5a500 2023-11-06T01:33:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6997fa700 1 -- 172.21.15.53:0/3866741852 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe6a800f4e0 con 0x5577c0c5a500 2023-11-06T01:33:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6c8238b80 1 -- 172.21.15.53:0/3866741852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577c0c23e10 con 0x5577c0c5a500 2023-11-06T01:33:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6997fa700 1 -- 172.21.15.53:0/3866741852 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6a800f6a0 con 0x5577c0c5a500 2023-11-06T01:33:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6c8238b80 10 client.15456 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6c8238b80 1 -- 172.21.15.53:0/3866741852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5577c09bcc60 con 0x5577c0c5a500 2023-11-06T01:33:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6997fa700 1 -- 172.21.15.53:0/3866741852 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577c0c23e10 con 0x5577c0c5a500 2023-11-06T01:33:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6997fa700 1 -- 172.21.15.53:0/3866741852 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe6a8024020 con 0x5577c0c5a500 2023-11-06T01:33:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6c8238b80 10 client.15456 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6c8238b80 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:33:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6c8238b80 20 client.15456 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5577c0c6ec50 0x5577c0c71070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6c8238b80 4 client.15456 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:33:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.696+0000 7fe6c8238b80 1 -- 172.21.15.53:0/3866741852 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577c0b9ed40 con 0x5577c0c6ec50 2023-11-06T01:33:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.697+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5577c0c6ec50 0x5577c0c71070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.697+0000 7fe6997fa700 10 client.15456 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.697+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5577c0c6ec50 0x5577c0c71070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6937d817fdba731c server_cookie=fe664557df1132d in_seq=0 out_seq=0 2023-11-06T01:33:59.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6997fa700 1 -- 172.21.15.53:0/3866741852 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5577c0b9ed40 con 0x5577c0c6ec50 2023-11-06T01:33:59.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6997fa700 10 client.15456 handle_command_reply: tid=0 2023-11-06T01:33:59.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 1 client.15456 shutdown 2023-11-06T01:33:59.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 1 -- 172.21.15.53:0/3866741852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 msgr2=0x5577c0c63310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c63310 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80127f0 tx=0x7fe6a8012b00 comp rx=0 tx=0).stop 2023-11-06T01:33:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 1 -- 172.21.15.53:0/3866741852 shutdown_connections 2023-11-06T01:33:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5ae80 0x5577c0c6a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5577c0c6ec50 0x5577c0c71070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c59800 0x5577c0c62dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6997fa700 0 client.15456 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/3866741852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c5a500 0x5577c0c63310 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 1 -- 172.21.15.53:0/3866741852 >> 172.21.15.53:0/3866741852 conn(0x5577c0bb40d0 msgr2=0x5577c0c4f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 1 -- 172.21.15.53:0/3866741852 shutdown_connections 2023-11-06T01:33:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 1 -- 172.21.15.53:0/3866741852 wait complete. 2023-11-06T01:33:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.698+0000 7fe6c8238b80 20 client.15456 trim_cache size 0 max 16384 2023-11-06T01:33:59.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6c8238b80 1 Processor -- start 2023-11-06T01:33:59.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6c8238b80 1 -- start start 2023-11-06T01:33:59.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0c59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5a500 0x5577c0c68300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c5ae80 0x5577c0c5d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0c6c040 con 0x5577c0c59800 2023-11-06T01:33:59.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0c63a50 con 0x5577c0c5a500 2023-11-06T01:33:59.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0c60a60 con 0x5577c0c5ae80 2023-11-06T01:33:59.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6b9e44700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0c59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6b9e44700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0c59270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39790/0 (socket says 172.21.15.53:39790) 2023-11-06T01:33:59.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6b9e44700 1 -- 172.21.15.53:0/1318170437 learned_addr learned my addr 172.21.15.53:0/1318170437 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:59.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.715+0000 7fe6b9643700 1 --2- 172.21.15.53:0/1318170437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5a500 0x5577c0c68300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.716+0000 7fe6ba645700 1 --2- 172.21.15.53:0/1318170437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c5ae80 0x5577c0c5d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.716+0000 7fe6b9e44700 1 -- 172.21.15.53:0/1318170437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c5ae80 msgr2=0x5577c0c5d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.716+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/1318170437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c5ae80 0x5577c0c5d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.716+0000 7fe6b9e44700 1 -- 172.21.15.53:0/1318170437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5a500 msgr2=0x5577c0c68300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.716+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/1318170437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5a500 0x5577c0c68300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.716+0000 7fe6b9e44700 1 -- 172.21.15.53:0/1318170437 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577c09fa250 con 0x5577c0c59800 2023-11-06T01:33:59.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.716+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/1318170437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0c59270 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac007a30 tx=0x7fe6ac034040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a23a417cb482097 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.716+0000 7fe6b0ff9700 1 -- 172.21.15.53:0/1318170437 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac00e040 con 0x5577c0c59800 2023-11-06T01:33:59.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.716+0000 7fe6b0ff9700 1 -- 172.21.15.53:0/1318170437 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6ac00be60 con 0x5577c0c59800 2023-11-06T01:33:59.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.716+0000 7fe6b0ff9700 1 -- 172.21.15.53:0/1318170437 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac03bdd0 con 0x5577c0c59800 2023-11-06T01:33:59.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.717+0000 7fe6c8238b80 1 -- 172.21.15.53:0/1318170437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 msgr2=0x5577c0c59270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.717+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/1318170437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0c59270 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac007a30 tx=0x7fe6ac034040 comp rx=0 tx=0).stop 2023-11-06T01:33:59.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.717+0000 7fe6c8238b80 1 -- 172.21.15.53:0/1318170437 shutdown_connections 2023-11-06T01:33:59.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.717+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/1318170437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c5ae80 0x5577c0c5d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.717+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/1318170437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0c59270 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.717+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/1318170437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5a500 0x5577c0c68300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.717+0000 7fe6c8238b80 1 -- 172.21.15.53:0/1318170437 >> 172.21.15.53:0/1318170437 conn(0x5577c0bb40d0 msgr2=0x5577c0c61a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.717+0000 7fe6c8238b80 1 -- 172.21.15.53:0/1318170437 shutdown_connections 2023-11-06T01:33:59.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.717+0000 7fe6c8238b80 1 -- 172.21.15.53:0/1318170437 wait complete. 2023-11-06T01:33:59.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.717+0000 7fe6c8238b80 1 -- start start 2023-11-06T01:33:59.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.717+0000 7fe6c8238b80 1 Processor -- start 2023-11-06T01:33:59.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0cba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5a500 0x5577c0cbcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6c8238b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c5ae80 0x5577c0cbf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0cbfa00 con 0x5577c0c59800 2023-11-06T01:33:59.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0cbfb70 con 0x5577c0c5a500 2023-11-06T01:33:59.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6c8238b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577c0cbfd30 con 0x5577c0c5ae80 2023-11-06T01:33:59.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6c8238b80 10 client.0 _ll_register_callbacks cb 0x7fe6a8009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:33:59.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6b9e44700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0cba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6b9e44700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0cba730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39796/0 (socket says 172.21.15.53:39796) 2023-11-06T01:33:59.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6b9e44700 1 -- 172.21.15.53:0/528586145 learned_addr learned my addr 172.21.15.53:0/528586145 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:33:59.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6ba645700 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c5ae80 0x5577c0cbf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6b9643700 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5a500 0x5577c0cbcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6b9e44700 1 -- 172.21.15.53:0/528586145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c5ae80 msgr2=0x5577c0cbf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c5ae80 0x5577c0cbf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6b9e44700 1 -- 172.21.15.53:0/528586145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5a500 msgr2=0x5577c0cbcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:33:59.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5a500 0x5577c0cbcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6b9e44700 1 -- 172.21.15.53:0/528586145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577c09d3de0 con 0x5577c0c59800 2023-11-06T01:33:59.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6b9e44700 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0cba730 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00b0d0 tx=0x7fe6ac03b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcad835fd4748e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:33:59.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6997fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:33:59.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.718+0000 7fe6997fa700 1 -- 172.21.15.53:0/528586145 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac03b420 con 0x5577c0c59800 2023-11-06T01:33:59.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6997fa700 1 -- 172.21.15.53:0/528586145 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe6ac037070 con 0x5577c0c59800 2023-11-06T01:33:59.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6c8238b80 1 -- 172.21.15.53:0/528586145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577c0bd0d10 con 0x5577c0c59800 2023-11-06T01:33:59.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6997fa700 1 -- 172.21.15.53:0/528586145 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6ac028430 con 0x5577c0c59800 2023-11-06T01:33:59.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6c8238b80 10 client.15462 fetch_fsmap learned FSMap version 27 2023-11-06T01:33:59.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6c8238b80 1 -- 172.21.15.53:0/528586145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5577c09d22b0 con 0x5577c0c59800 2023-11-06T01:33:59.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6997fa700 1 -- 172.21.15.53:0/528586145 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577c0bd0d10 con 0x5577c0c59800 2023-11-06T01:33:59.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6997fa700 1 -- 172.21.15.53:0/528586145 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe6ac042070 con 0x5577c0c59800 2023-11-06T01:33:59.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6c8238b80 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T01:33:59.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6c8238b80 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:33:59.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6c8238b80 20 client.15462 populate_metadata read hostname 'smithi053' 2023-11-06T01:33:59.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5577c0cc1260 0x5577c0cc3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:33:59.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6c8238b80 4 client.15462 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:33:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6c8238b80 1 -- 172.21.15.53:0/528586145 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577c0b9ea60 con 0x5577c0cc1260 2023-11-06T01:33:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.719+0000 7fe6b9643700 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5577c0cc1260 0x5577c0cc3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:33:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6997fa700 10 client.15462 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6b9643700 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5577c0cc1260 0x5577c0cc3650 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1a4a0bc77f0301 server_cookie=cb3270fc0c68fdc3 in_seq=0 out_seq=0 2023-11-06T01:33:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6997fa700 1 -- 172.21.15.53:0/528586145 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5577c0b9ea60 con 0x5577c0cc1260 2023-11-06T01:33:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6997fa700 10 client.15462 handle_command_reply: tid=0 2023-11-06T01:33:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6c8238b80 1 client.15462 shutdown 2023-11-06T01:33:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6c8238b80 1 -- 172.21.15.53:0/528586145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 msgr2=0x5577c0cba730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0cba730 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00b0d0 tx=0x7fe6ac03b770 comp rx=0 tx=0).stop 2023-11-06T01:33:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6c8238b80 1 -- 172.21.15.53:0/528586145 shutdown_connections 2023-11-06T01:33:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5577c0cc1260 0x5577c0cc3650 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577c0c5ae80 0x5577c0cbf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6997fa700 0 client.15462 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:33:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577c0c59800 0x5577c0cba730 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.720+0000 7fe6c8238b80 1 --2- 172.21.15.53:0/528586145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577c0c5a500 0x5577c0cbcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6c8238b80 1 -- 172.21.15.53:0/528586145 >> 172.21.15.53:0/528586145 conn(0x5577c0bb40d0 msgr2=0x5577c0c5d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6c8238b80 1 -- 172.21.15.53:0/528586145 shutdown_connections 2023-11-06T01:33:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6c8238b80 1 -- 172.21.15.53:0/528586145 wait complete. 2023-11-06T01:33:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6c8238b80 20 client.15462 trim_cache size 0 max 16384 2023-11-06T01:33:59.743 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:33:59.743 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:33:59.743 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:33:59.743 INFO:teuthology.orchestra.run.smithi053.stdout: "2c6d1131-3c26-4657-860b-faee98233fd7": { 2023-11-06T01:33:59.743 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:33:59.744 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "2c6d1131-3c26-4657-860b-faee98233fd7", 2023-11-06T01:33:59.744 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:33:59.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:33:59.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:33:59.744 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:33:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/2697498868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe69c0a0520 msgr2=0x7fe69c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe69c0a0520 0x7fe69c0a29e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009fd0 tx=0x7fe6ac00b040 comp rx=0 tx=0).stop 2023-11-06T01:33:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/2697498868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b4101830 msgr2=0x7fe6b419df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:33:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b4101830 0x7fe6b419df60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400e940 tx=0x7fe6a400ec50 comp rx=0 tx=0).stop 2023-11-06T01:33:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/2697498868 shutdown_connections 2023-11-06T01:33:59.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe69c0a0520 0x7fe69c0a29e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b4101830 0x7fe6b419df60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b40ffca0 0x7fe6b4069cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 --2- 172.21.15.53:0/2697498868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6b4100e90 0x7fe6b419b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:33:59.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/2697498868 >> 172.21.15.53:0/2697498868 conn(0x7fe6b40fb210 msgr2=0x7fe6b40fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:33:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/2697498868 shutdown_connections 2023-11-06T01:33:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:33:59.721+0000 7fe6bc0a8700 1 -- 172.21.15.53:0/2697498868 wait complete. 2023-11-06T01:33:59.748 INFO:tasks.cephfs.filesystem:scrub status for tag:2c6d1131-3c26-4657-860b-faee98233fd7 - {'path': '/', 'tag': '2c6d1131-3c26-4657-860b-faee98233fd7', 'options': 'recursive,force'} 2023-11-06T01:33:59.749 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:59 smithi163 ceph-mon[104416]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 9 op/s 2023-11-06T01:33:59.749 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:59 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2025882919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:33:59.749 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:59 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2859091038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:33:59.749 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:33:59 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2721552728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:34:00.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:00 smithi053 ceph-mon[99472]: scrub queued for path: / 2023-11-06T01:34:00.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:00 smithi053 ceph-mon[99472]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:00.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:00 smithi053 ceph-mon[99472]: scrub summary: active paths [/] 2023-11-06T01:34:00.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:00 smithi062 ceph-mon[103924]: scrub queued for path: / 2023-11-06T01:34:00.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:00 smithi062 ceph-mon[103924]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:00.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:00 smithi062 ceph-mon[103924]: scrub summary: active paths [/] 2023-11-06T01:34:00.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:00 smithi163 ceph-mon[104416]: scrub queued for path: / 2023-11-06T01:34:00.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:00 smithi163 ceph-mon[104416]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:00.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:00 smithi163 ceph-mon[104416]: scrub summary: active paths [/] 2023-11-06T01:34:01.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:01 smithi053 ceph-mon[99472]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:01.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:01 smithi053 ceph-mon[99472]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2023-11-06T01:34:01.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:01 smithi062 ceph-mon[103924]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:01.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:01 smithi062 ceph-mon[103924]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2023-11-06T01:34:01.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:01 smithi163 ceph-mon[104416]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:01.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:01 smithi163 ceph-mon[104416]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2023-11-06T01:34:02.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-11-06T01:34:02.219 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-11-06T01:34:02.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-11-06T01:34:02.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-11-06T01:34:02.347 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/: 2023-11-06T01:34:02.347 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:02.348 INFO:teuthology.orchestra.run.smithi053.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:33 . 2023-11-06T01:34:02.348 INFO:teuthology.orchestra.run.smithi053.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:33 .. 2023-11-06T01:34:02.348 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:33 data 2023-11-06T01:34:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:02.348 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data: 2023-11-06T01:34:02.349 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:02.349 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:33 . 2023-11-06T01:34:02.349 INFO:teuthology.orchestra.run.smithi053.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:33 .. 2023-11-06T01:34:02.350 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-11-06T01:34:02.437 INFO:teuthology.orchestra.run.smithi053.stdout:[Service] 2023-11-06T01:34:02.438 INFO:teuthology.orchestra.run.smithi053.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-11-06T01:34:02.438 INFO:teuthology.orchestra.run.smithi053.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-11-06T01:34:02.440 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-11-06T01:34:02.537 INFO:teuthology.orchestra.run.smithi053.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-11-06T01:34:03.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:03 smithi053 ceph-mon[99472]: pgmap v302: 97 pgs: 97 active+clean; 651 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-11-06T01:34:03.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:03 smithi062 ceph-mon[103924]: pgmap v302: 97 pgs: 97 active+clean; 651 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-11-06T01:34:03.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:03 smithi163 ceph-mon[104416]: pgmap v302: 97 pgs: 97 active+clean; 651 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-11-06T01:34:05.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:05 smithi053 ceph-mon[99472]: pgmap v303: 97 pgs: 97 active+clean; 651 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2023-11-06T01:34:05.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:05 smithi053 ceph-mon[99472]: scrub summary: idle 2023-11-06T01:34:05.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:05 smithi062 ceph-mon[103924]: pgmap v303: 97 pgs: 97 active+clean; 651 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2023-11-06T01:34:05.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:05 smithi062 ceph-mon[103924]: scrub summary: idle 2023-11-06T01:34:05.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:05 smithi163 ceph-mon[104416]: pgmap v303: 97 pgs: 97 active+clean; 651 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2023-11-06T01:34:05.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:05 smithi163 ceph-mon[104416]: scrub summary: idle 2023-11-06T01:34:07.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:07 smithi053 ceph-mon[99472]: pgmap v304: 97 pgs: 97 active+clean; 10 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 838 KiB/s wr, 31 op/s 2023-11-06T01:34:07.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:07 smithi062 ceph-mon[103924]: pgmap v304: 97 pgs: 97 active+clean; 10 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 838 KiB/s wr, 31 op/s 2023-11-06T01:34:07.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:07 smithi163 ceph-mon[104416]: pgmap v304: 97 pgs: 97 active+clean; 10 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 838 KiB/s wr, 31 op/s 2023-11-06T01:34:09.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:09 smithi053 ceph-mon[99472]: pgmap v305: 97 pgs: 97 active+clean; 28 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.3 MiB/s wr, 41 op/s 2023-11-06T01:34:09.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:09 smithi062 ceph-mon[103924]: pgmap v305: 97 pgs: 97 active+clean; 28 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.3 MiB/s wr, 41 op/s 2023-11-06T01:34:09.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:09 smithi163 ceph-mon[104416]: pgmap v305: 97 pgs: 97 active+clean; 28 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.3 MiB/s wr, 41 op/s 2023-11-06T01:34:10.005 INFO:teuthology.orchestra.run.smithi053.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-11-06T01:34:10.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-11-06T01:34:10.055 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/: 2023-11-06T01:34:10.055 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.056 INFO:teuthology.orchestra.run.smithi053.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:33 . 2023-11-06T01:34:10.056 INFO:teuthology.orchestra.run.smithi053.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:33 .. 2023-11-06T01:34:10.056 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 data 2023-11-06T01:34:10.056 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.057 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data: 2023-11-06T01:34:10.057 INFO:teuthology.orchestra.run.smithi053.stdout:total 30 2023-11-06T01:34:10.057 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 . 2023-11-06T01:34:10.057 INFO:teuthology.orchestra.run.smithi053.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:33 .. 2023-11-06T01:34:10.057 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 base 2023-11-06T01:34:10.058 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Nov 6 01:34 global 2023-11-06T01:34:10.058 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 log 2023-11-06T01:34:10.058 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_commit_ts 2023-11-06T01:34:10.058 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_dynshmem 2023-11-06T01:34:10.058 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Nov 6 01:34 pg_hba.conf 2023-11-06T01:34:10.059 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Nov 6 01:34 pg_ident.conf 2023-11-06T01:34:10.059 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 pg_logical 2023-11-06T01:34:10.059 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:34 pg_multixact 2023-11-06T01:34:10.059 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 pg_notify 2023-11-06T01:34:10.059 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_replslot 2023-11-06T01:34:10.060 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_serial 2023-11-06T01:34:10.060 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_snapshots 2023-11-06T01:34:10.060 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_stat 2023-11-06T01:34:10.060 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_stat_tmp 2023-11-06T01:34:10.060 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 pg_subtrans 2023-11-06T01:34:10.061 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_tblspc 2023-11-06T01:34:10.061 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_twophase 2023-11-06T01:34:10.061 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 PG_VERSION 2023-11-06T01:34:10.061 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:34 pg_wal 2023-11-06T01:34:10.061 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 pg_xact 2023-11-06T01:34:10.062 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Nov 6 01:34 postgresql.auto.conf 2023-11-06T01:34:10.062 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Nov 6 01:34 postgresql.conf 2023-11-06T01:34:10.062 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.062 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/base: 2023-11-06T01:34:10.062 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.063 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 . 2023-11-06T01:34:10.063 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.063 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Nov 6 01:34 1 2023-11-06T01:34:10.063 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Nov 6 01:34 12346 2023-11-06T01:34:10.063 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Nov 6 01:34 12347 2023-11-06T01:34:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.064 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/base/1: 2023-11-06T01:34:10.064 INFO:teuthology.orchestra.run.smithi053.stdout:total 7193 2023-11-06T01:34:10.064 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Nov 6 01:34 . 2023-11-06T01:34:10.064 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 .. 2023-11-06T01:34:10.065 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 112 2023-11-06T01:34:10.065 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 113 2023-11-06T01:34:10.065 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Nov 6 01:34 12182 2023-11-06T01:34:10.065 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12182_fsm 2023-11-06T01:34:10.065 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12182_vm 2023-11-06T01:34:10.066 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12184 2023-11-06T01:34:10.066 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12186 2023-11-06T01:34:10.066 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187 2023-11-06T01:34:10.066 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12187_fsm 2023-11-06T01:34:10.066 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187_vm 2023-11-06T01:34:10.067 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12189 2023-11-06T01:34:10.067 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12191 2023-11-06T01:34:10.067 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192 2023-11-06T01:34:10.067 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12192_fsm 2023-11-06T01:34:10.067 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192_vm 2023-11-06T01:34:10.067 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12194 2023-11-06T01:34:10.068 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12196 2023-11-06T01:34:10.068 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197 2023-11-06T01:34:10.068 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12197_fsm 2023-11-06T01:34:10.068 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197_vm 2023-11-06T01:34:10.068 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12199 2023-11-06T01:34:10.069 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12201 2023-11-06T01:34:10.069 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202 2023-11-06T01:34:10.069 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12202_fsm 2023-11-06T01:34:10.069 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202_vm 2023-11-06T01:34:10.069 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12204 2023-11-06T01:34:10.070 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12206 2023-11-06T01:34:10.070 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207 2023-11-06T01:34:10.070 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12207_fsm 2023-11-06T01:34:10.070 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207_vm 2023-11-06T01:34:10.070 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12209 2023-11-06T01:34:10.071 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12211 2023-11-06T01:34:10.071 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12212 2023-11-06T01:34:10.071 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12214 2023-11-06T01:34:10.071 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12216 2023-11-06T01:34:10.071 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Nov 6 01:34 1247 2023-11-06T01:34:10.072 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1247_fsm 2023-11-06T01:34:10.072 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1247_vm 2023-11-06T01:34:10.072 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Nov 6 01:34 1249 2023-11-06T01:34:10.072 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1249_fsm 2023-11-06T01:34:10.072 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1249_vm 2023-11-06T01:34:10.073 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Nov 6 01:34 1255 2023-11-06T01:34:10.073 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1255_fsm 2023-11-06T01:34:10.073 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1255_vm 2023-11-06T01:34:10.073 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Nov 6 01:34 1259 2023-11-06T01:34:10.073 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1259_fsm 2023-11-06T01:34:10.074 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1259_vm 2023-11-06T01:34:10.074 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417 2023-11-06T01:34:10.074 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417_vm 2023-11-06T01:34:10.074 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418 2023-11-06T01:34:10.074 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418_vm 2023-11-06T01:34:10.075 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 174 2023-11-06T01:34:10.075 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 175 2023-11-06T01:34:10.075 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2187 2023-11-06T01:34:10.075 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224 2023-11-06T01:34:10.075 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224_vm 2023-11-06T01:34:10.076 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328 2023-11-06T01:34:10.076 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328_vm 2023-11-06T01:34:10.076 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336 2023-11-06T01:34:10.076 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336_vm 2023-11-06T01:34:10.076 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2337 2023-11-06T01:34:10.076 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2600 2023-11-06T01:34:10.077 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2600_fsm 2023-11-06T01:34:10.077 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2600_vm 2023-11-06T01:34:10.077 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601 2023-11-06T01:34:10.077 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2601_fsm 2023-11-06T01:34:10.077 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601_vm 2023-11-06T01:34:10.078 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2602 2023-11-06T01:34:10.078 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2602_fsm 2023-11-06T01:34:10.078 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2602_vm 2023-11-06T01:34:10.078 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2603 2023-11-06T01:34:10.078 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2603_fsm 2023-11-06T01:34:10.079 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2603_vm 2023-11-06T01:34:10.079 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604 2023-11-06T01:34:10.079 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604_vm 2023-11-06T01:34:10.079 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2605 2023-11-06T01:34:10.079 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2605_fsm 2023-11-06T01:34:10.080 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2605_vm 2023-11-06T01:34:10.080 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606 2023-11-06T01:34:10.080 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2606_fsm 2023-11-06T01:34:10.080 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606_vm 2023-11-06T01:34:10.080 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607 2023-11-06T01:34:10.081 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607_fsm 2023-11-06T01:34:10.081 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2607_vm 2023-11-06T01:34:10.081 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Nov 6 01:34 2608 2023-11-06T01:34:10.081 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2608_fsm 2023-11-06T01:34:10.081 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2608_vm 2023-11-06T01:34:10.081 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Nov 6 01:34 2609 2023-11-06T01:34:10.082 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2609_fsm 2023-11-06T01:34:10.082 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2609_vm 2023-11-06T01:34:10.082 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2610 2023-11-06T01:34:10.082 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2610_fsm 2023-11-06T01:34:10.082 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2610_vm 2023-11-06T01:34:10.083 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611 2023-11-06T01:34:10.083 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611_vm 2023-11-06T01:34:10.083 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612 2023-11-06T01:34:10.083 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2612_fsm 2023-11-06T01:34:10.083 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612_vm 2023-11-06T01:34:10.084 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613 2023-11-06T01:34:10.084 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613_vm 2023-11-06T01:34:10.084 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615 2023-11-06T01:34:10.084 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2615_fsm 2023-11-06T01:34:10.084 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615_vm 2023-11-06T01:34:10.085 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616 2023-11-06T01:34:10.085 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616_fsm 2023-11-06T01:34:10.085 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2616_vm 2023-11-06T01:34:10.085 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Nov 6 01:34 2617 2023-11-06T01:34:10.085 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2617_fsm 2023-11-06T01:34:10.086 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2617_vm 2023-11-06T01:34:10.086 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Nov 6 01:34 2618 2023-11-06T01:34:10.086 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2618_fsm 2023-11-06T01:34:10.086 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2618_vm 2023-11-06T01:34:10.086 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Nov 6 01:34 2619 2023-11-06T01:34:10.086 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2619_fsm 2023-11-06T01:34:10.087 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2619_vm 2023-11-06T01:34:10.087 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620 2023-11-06T01:34:10.087 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620_vm 2023-11-06T01:34:10.087 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2650 2023-11-06T01:34:10.087 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2651 2023-11-06T01:34:10.088 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2652 2023-11-06T01:34:10.088 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2653 2023-11-06T01:34:10.088 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2654 2023-11-06T01:34:10.088 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2655 2023-11-06T01:34:10.088 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2656 2023-11-06T01:34:10.089 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2657 2023-11-06T01:34:10.089 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Nov 6 01:34 2658 2023-11-06T01:34:10.089 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2659 2023-11-06T01:34:10.089 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2660 2023-11-06T01:34:10.089 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2661 2023-11-06T01:34:10.090 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2662 2023-11-06T01:34:10.090 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2663 2023-11-06T01:34:10.090 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2664 2023-11-06T01:34:10.090 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2665 2023-11-06T01:34:10.090 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2666 2023-11-06T01:34:10.091 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2667 2023-11-06T01:34:10.091 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2668 2023-11-06T01:34:10.091 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2669 2023-11-06T01:34:10.091 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2670 2023-11-06T01:34:10.092 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Nov 6 01:34 2673 2023-11-06T01:34:10.092 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Nov 6 01:34 2674 2023-11-06T01:34:10.092 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Nov 6 01:34 2675 2023-11-06T01:34:10.092 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2678 2023-11-06T01:34:10.092 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2679 2023-11-06T01:34:10.093 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2680 2023-11-06T01:34:10.093 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2681 2023-11-06T01:34:10.093 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2682 2023-11-06T01:34:10.093 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2683 2023-11-06T01:34:10.093 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2684 2023-11-06T01:34:10.094 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2685 2023-11-06T01:34:10.094 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2686 2023-11-06T01:34:10.094 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2687 2023-11-06T01:34:10.094 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2688 2023-11-06T01:34:10.094 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2689 2023-11-06T01:34:10.095 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2690 2023-11-06T01:34:10.095 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Nov 6 01:34 2691 2023-11-06T01:34:10.095 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2692 2023-11-06T01:34:10.095 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2693 2023-11-06T01:34:10.095 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2696 2023-11-06T01:34:10.096 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2699 2023-11-06T01:34:10.096 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2701 2023-11-06T01:34:10.096 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2702 2023-11-06T01:34:10.096 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2703 2023-11-06T01:34:10.096 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2704 2023-11-06T01:34:10.097 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2753 2023-11-06T01:34:10.097 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2753_fsm 2023-11-06T01:34:10.097 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2753_vm 2023-11-06T01:34:10.097 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2754 2023-11-06T01:34:10.097 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2755 2023-11-06T01:34:10.098 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2756 2023-11-06T01:34:10.098 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2757 2023-11-06T01:34:10.098 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830 2023-11-06T01:34:10.098 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830_vm 2023-11-06T01:34:10.098 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2831 2023-11-06T01:34:10.099 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832 2023-11-06T01:34:10.099 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832_vm 2023-11-06T01:34:10.099 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2833 2023-11-06T01:34:10.099 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834 2023-11-06T01:34:10.099 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834_vm 2023-11-06T01:34:10.100 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2835 2023-11-06T01:34:10.100 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836 2023-11-06T01:34:10.100 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836_vm 2023-11-06T01:34:10.100 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2837 2023-11-06T01:34:10.100 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Nov 6 01:34 2838 2023-11-06T01:34:10.101 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2838_fsm 2023-11-06T01:34:10.101 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2838_vm 2023-11-06T01:34:10.101 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2839 2023-11-06T01:34:10.101 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840 2023-11-06T01:34:10.101 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840_fsm 2023-11-06T01:34:10.102 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2840_vm 2023-11-06T01:34:10.102 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2841 2023-11-06T01:34:10.102 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995 2023-11-06T01:34:10.102 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995_vm 2023-11-06T01:34:10.102 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2996 2023-11-06T01:34:10.102 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079 2023-11-06T01:34:10.103 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3079_fsm 2023-11-06T01:34:10.103 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079_vm 2023-11-06T01:34:10.103 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3080 2023-11-06T01:34:10.103 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3081 2023-11-06T01:34:10.103 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3085 2023-11-06T01:34:10.104 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118 2023-11-06T01:34:10.104 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118_vm 2023-11-06T01:34:10.104 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3119 2023-11-06T01:34:10.104 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3164 2023-11-06T01:34:10.104 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256 2023-11-06T01:34:10.105 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256_vm 2023-11-06T01:34:10.105 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3257 2023-11-06T01:34:10.105 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3258 2023-11-06T01:34:10.105 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350 2023-11-06T01:34:10.105 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350_vm 2023-11-06T01:34:10.106 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3351 2023-11-06T01:34:10.106 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3379 2023-11-06T01:34:10.106 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3380 2023-11-06T01:34:10.106 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381 2023-11-06T01:34:10.106 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381_vm 2023-11-06T01:34:10.107 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3394 2023-11-06T01:34:10.107 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3394_fsm 2023-11-06T01:34:10.107 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3394_vm 2023-11-06T01:34:10.107 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3395 2023-11-06T01:34:10.107 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439 2023-11-06T01:34:10.108 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439_vm 2023-11-06T01:34:10.108 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3440 2023-11-06T01:34:10.108 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 3455 2023-11-06T01:34:10.108 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3456 2023-11-06T01:34:10.108 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3456_fsm 2023-11-06T01:34:10.109 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3456_vm 2023-11-06T01:34:10.109 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466 2023-11-06T01:34:10.109 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466_vm 2023-11-06T01:34:10.109 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3467 2023-11-06T01:34:10.109 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3468 2023-11-06T01:34:10.110 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501 2023-11-06T01:34:10.110 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501_vm 2023-11-06T01:34:10.110 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3502 2023-11-06T01:34:10.110 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3503 2023-11-06T01:34:10.110 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3534 2023-11-06T01:34:10.111 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541 2023-11-06T01:34:10.111 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3541_fsm 2023-11-06T01:34:10.111 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541_vm 2023-11-06T01:34:10.111 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3542 2023-11-06T01:34:10.111 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3574 2023-11-06T01:34:10.111 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3575 2023-11-06T01:34:10.112 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576 2023-11-06T01:34:10.112 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576_vm 2023-11-06T01:34:10.112 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596 2023-11-06T01:34:10.112 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596_vm 2023-11-06T01:34:10.112 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3597 2023-11-06T01:34:10.113 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598 2023-11-06T01:34:10.113 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598_vm 2023-11-06T01:34:10.113 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3599 2023-11-06T01:34:10.113 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600 2023-11-06T01:34:10.113 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3600_fsm 2023-11-06T01:34:10.114 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600_vm 2023-11-06T01:34:10.114 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601 2023-11-06T01:34:10.114 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3601_fsm 2023-11-06T01:34:10.114 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601_vm 2023-11-06T01:34:10.114 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602 2023-11-06T01:34:10.115 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3602_fsm 2023-11-06T01:34:10.115 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602_vm 2023-11-06T01:34:10.115 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3603 2023-11-06T01:34:10.115 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3603_fsm 2023-11-06T01:34:10.115 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3603_vm 2023-11-06T01:34:10.116 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3604 2023-11-06T01:34:10.116 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3605 2023-11-06T01:34:10.116 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3606 2023-11-06T01:34:10.119 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3607 2023-11-06T01:34:10.119 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3608 2023-11-06T01:34:10.120 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 3609 2023-11-06T01:34:10.120 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3712 2023-11-06T01:34:10.120 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764 2023-11-06T01:34:10.120 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3764_fsm 2023-11-06T01:34:10.120 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764_vm 2023-11-06T01:34:10.121 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3766 2023-11-06T01:34:10.121 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3767 2023-11-06T01:34:10.121 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3997 2023-11-06T01:34:10.121 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 5002 2023-11-06T01:34:10.121 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 548 2023-11-06T01:34:10.122 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 549 2023-11-06T01:34:10.122 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102 2023-11-06T01:34:10.122 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102_vm 2023-11-06T01:34:10.122 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104 2023-11-06T01:34:10.122 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104_vm 2023-11-06T01:34:10.123 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106 2023-11-06T01:34:10.123 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106_vm 2023-11-06T01:34:10.123 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6110 2023-11-06T01:34:10.123 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6111 2023-11-06T01:34:10.123 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6112 2023-11-06T01:34:10.123 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6113 2023-11-06T01:34:10.124 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6117 2023-11-06T01:34:10.124 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826 2023-11-06T01:34:10.124 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826_vm 2023-11-06T01:34:10.124 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 827 2023-11-06T01:34:10.124 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 828 2023-11-06T01:34:10.125 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Nov 6 01:34 pg_filenode.map 2023-11-06T01:34:10.125 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 PG_VERSION 2023-11-06T01:34:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.125 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-11-06T01:34:10.125 INFO:teuthology.orchestra.run.smithi053.stdout:total 7193 2023-11-06T01:34:10.126 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Nov 6 01:34 . 2023-11-06T01:34:10.126 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 .. 2023-11-06T01:34:10.126 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 112 2023-11-06T01:34:10.126 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 113 2023-11-06T01:34:10.126 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Nov 6 01:34 12182 2023-11-06T01:34:10.127 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12182_fsm 2023-11-06T01:34:10.127 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12182_vm 2023-11-06T01:34:10.127 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12184 2023-11-06T01:34:10.127 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12186 2023-11-06T01:34:10.127 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187 2023-11-06T01:34:10.128 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12187_fsm 2023-11-06T01:34:10.128 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187_vm 2023-11-06T01:34:10.128 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12189 2023-11-06T01:34:10.128 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12191 2023-11-06T01:34:10.128 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192 2023-11-06T01:34:10.128 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12192_fsm 2023-11-06T01:34:10.129 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192_vm 2023-11-06T01:34:10.129 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12194 2023-11-06T01:34:10.129 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12196 2023-11-06T01:34:10.129 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197 2023-11-06T01:34:10.129 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12197_fsm 2023-11-06T01:34:10.130 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197_vm 2023-11-06T01:34:10.130 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12199 2023-11-06T01:34:10.130 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12201 2023-11-06T01:34:10.130 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202 2023-11-06T01:34:10.130 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12202_fsm 2023-11-06T01:34:10.131 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202_vm 2023-11-06T01:34:10.131 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12204 2023-11-06T01:34:10.131 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12206 2023-11-06T01:34:10.131 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207 2023-11-06T01:34:10.131 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12207_fsm 2023-11-06T01:34:10.132 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207_vm 2023-11-06T01:34:10.132 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12209 2023-11-06T01:34:10.132 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12211 2023-11-06T01:34:10.132 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12212 2023-11-06T01:34:10.132 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12214 2023-11-06T01:34:10.133 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12216 2023-11-06T01:34:10.133 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Nov 6 01:34 1247 2023-11-06T01:34:10.133 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1247_fsm 2023-11-06T01:34:10.133 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1247_vm 2023-11-06T01:34:10.133 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Nov 6 01:34 1249 2023-11-06T01:34:10.133 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1249_fsm 2023-11-06T01:34:10.134 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1249_vm 2023-11-06T01:34:10.134 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Nov 6 01:34 1255 2023-11-06T01:34:10.134 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1255_fsm 2023-11-06T01:34:10.134 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1255_vm 2023-11-06T01:34:10.134 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Nov 6 01:34 1259 2023-11-06T01:34:10.135 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1259_fsm 2023-11-06T01:34:10.135 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1259_vm 2023-11-06T01:34:10.135 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417 2023-11-06T01:34:10.135 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417_vm 2023-11-06T01:34:10.135 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418 2023-11-06T01:34:10.136 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418_vm 2023-11-06T01:34:10.136 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 174 2023-11-06T01:34:10.136 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 175 2023-11-06T01:34:10.136 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2187 2023-11-06T01:34:10.137 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224 2023-11-06T01:34:10.137 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224_vm 2023-11-06T01:34:10.137 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328 2023-11-06T01:34:10.137 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328_vm 2023-11-06T01:34:10.137 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336 2023-11-06T01:34:10.138 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336_vm 2023-11-06T01:34:10.138 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2337 2023-11-06T01:34:10.138 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2600 2023-11-06T01:34:10.138 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2600_fsm 2023-11-06T01:34:10.138 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2600_vm 2023-11-06T01:34:10.139 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601 2023-11-06T01:34:10.139 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2601_fsm 2023-11-06T01:34:10.139 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601_vm 2023-11-06T01:34:10.139 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2602 2023-11-06T01:34:10.139 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2602_fsm 2023-11-06T01:34:10.140 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2602_vm 2023-11-06T01:34:10.140 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2603 2023-11-06T01:34:10.140 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2603_fsm 2023-11-06T01:34:10.140 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2603_vm 2023-11-06T01:34:10.140 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604 2023-11-06T01:34:10.141 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604_vm 2023-11-06T01:34:10.141 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2605 2023-11-06T01:34:10.141 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2605_fsm 2023-11-06T01:34:10.141 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2605_vm 2023-11-06T01:34:10.141 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606 2023-11-06T01:34:10.141 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2606_fsm 2023-11-06T01:34:10.142 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606_vm 2023-11-06T01:34:10.142 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607 2023-11-06T01:34:10.142 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607_fsm 2023-11-06T01:34:10.142 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2607_vm 2023-11-06T01:34:10.142 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Nov 6 01:34 2608 2023-11-06T01:34:10.143 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2608_fsm 2023-11-06T01:34:10.143 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2608_vm 2023-11-06T01:34:10.143 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Nov 6 01:34 2609 2023-11-06T01:34:10.143 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2609_fsm 2023-11-06T01:34:10.143 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2609_vm 2023-11-06T01:34:10.144 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2610 2023-11-06T01:34:10.144 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2610_fsm 2023-11-06T01:34:10.144 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2610_vm 2023-11-06T01:34:10.144 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611 2023-11-06T01:34:10.144 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611_vm 2023-11-06T01:34:10.145 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612 2023-11-06T01:34:10.145 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2612_fsm 2023-11-06T01:34:10.145 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612_vm 2023-11-06T01:34:10.145 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613 2023-11-06T01:34:10.145 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613_vm 2023-11-06T01:34:10.146 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615 2023-11-06T01:34:10.146 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2615_fsm 2023-11-06T01:34:10.146 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615_vm 2023-11-06T01:34:10.146 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616 2023-11-06T01:34:10.146 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616_fsm 2023-11-06T01:34:10.147 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2616_vm 2023-11-06T01:34:10.147 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Nov 6 01:34 2617 2023-11-06T01:34:10.147 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2617_fsm 2023-11-06T01:34:10.147 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2617_vm 2023-11-06T01:34:10.147 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Nov 6 01:34 2618 2023-11-06T01:34:10.148 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2618_fsm 2023-11-06T01:34:10.148 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2618_vm 2023-11-06T01:34:10.148 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Nov 6 01:34 2619 2023-11-06T01:34:10.148 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2619_fsm 2023-11-06T01:34:10.148 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2619_vm 2023-11-06T01:34:10.148 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620 2023-11-06T01:34:10.149 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620_vm 2023-11-06T01:34:10.149 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2650 2023-11-06T01:34:10.149 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2651 2023-11-06T01:34:10.149 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2652 2023-11-06T01:34:10.149 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2653 2023-11-06T01:34:10.150 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2654 2023-11-06T01:34:10.150 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2655 2023-11-06T01:34:10.150 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2656 2023-11-06T01:34:10.150 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2657 2023-11-06T01:34:10.151 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Nov 6 01:34 2658 2023-11-06T01:34:10.151 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2659 2023-11-06T01:34:10.151 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2660 2023-11-06T01:34:10.151 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2661 2023-11-06T01:34:10.151 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2662 2023-11-06T01:34:10.151 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2663 2023-11-06T01:34:10.152 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2664 2023-11-06T01:34:10.152 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2665 2023-11-06T01:34:10.152 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2666 2023-11-06T01:34:10.152 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2667 2023-11-06T01:34:10.152 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2668 2023-11-06T01:34:10.153 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2669 2023-11-06T01:34:10.153 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2670 2023-11-06T01:34:10.153 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Nov 6 01:34 2673 2023-11-06T01:34:10.153 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Nov 6 01:34 2674 2023-11-06T01:34:10.153 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Nov 6 01:34 2675 2023-11-06T01:34:10.154 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2678 2023-11-06T01:34:10.154 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2679 2023-11-06T01:34:10.154 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2680 2023-11-06T01:34:10.154 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2681 2023-11-06T01:34:10.155 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2682 2023-11-06T01:34:10.155 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2683 2023-11-06T01:34:10.155 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2684 2023-11-06T01:34:10.155 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2685 2023-11-06T01:34:10.155 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2686 2023-11-06T01:34:10.156 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2687 2023-11-06T01:34:10.156 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2688 2023-11-06T01:34:10.156 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2689 2023-11-06T01:34:10.156 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2690 2023-11-06T01:34:10.156 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Nov 6 01:34 2691 2023-11-06T01:34:10.156 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2692 2023-11-06T01:34:10.157 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2693 2023-11-06T01:34:10.157 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2696 2023-11-06T01:34:10.157 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2699 2023-11-06T01:34:10.157 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2701 2023-11-06T01:34:10.157 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2702 2023-11-06T01:34:10.158 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2703 2023-11-06T01:34:10.158 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2704 2023-11-06T01:34:10.158 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2753 2023-11-06T01:34:10.158 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2753_fsm 2023-11-06T01:34:10.158 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2753_vm 2023-11-06T01:34:10.159 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2754 2023-11-06T01:34:10.159 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2755 2023-11-06T01:34:10.159 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2756 2023-11-06T01:34:10.159 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2757 2023-11-06T01:34:10.159 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830 2023-11-06T01:34:10.160 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830_vm 2023-11-06T01:34:10.160 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2831 2023-11-06T01:34:10.160 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832 2023-11-06T01:34:10.160 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832_vm 2023-11-06T01:34:10.160 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2833 2023-11-06T01:34:10.161 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834 2023-11-06T01:34:10.161 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834_vm 2023-11-06T01:34:10.161 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2835 2023-11-06T01:34:10.161 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836 2023-11-06T01:34:10.161 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836_vm 2023-11-06T01:34:10.162 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2837 2023-11-06T01:34:10.162 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Nov 6 01:34 2838 2023-11-06T01:34:10.162 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2838_fsm 2023-11-06T01:34:10.162 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2838_vm 2023-11-06T01:34:10.162 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2839 2023-11-06T01:34:10.163 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840 2023-11-06T01:34:10.163 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840_fsm 2023-11-06T01:34:10.163 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2840_vm 2023-11-06T01:34:10.163 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2841 2023-11-06T01:34:10.163 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995 2023-11-06T01:34:10.163 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995_vm 2023-11-06T01:34:10.164 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2996 2023-11-06T01:34:10.164 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079 2023-11-06T01:34:10.164 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3079_fsm 2023-11-06T01:34:10.164 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079_vm 2023-11-06T01:34:10.164 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3080 2023-11-06T01:34:10.165 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3081 2023-11-06T01:34:10.165 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3085 2023-11-06T01:34:10.165 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118 2023-11-06T01:34:10.165 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118_vm 2023-11-06T01:34:10.165 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3119 2023-11-06T01:34:10.166 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3164 2023-11-06T01:34:10.166 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256 2023-11-06T01:34:10.166 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256_vm 2023-11-06T01:34:10.166 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3257 2023-11-06T01:34:10.166 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3258 2023-11-06T01:34:10.167 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350 2023-11-06T01:34:10.167 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350_vm 2023-11-06T01:34:10.167 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3351 2023-11-06T01:34:10.167 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3379 2023-11-06T01:34:10.167 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3380 2023-11-06T01:34:10.168 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381 2023-11-06T01:34:10.168 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381_vm 2023-11-06T01:34:10.168 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3394 2023-11-06T01:34:10.168 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3394_fsm 2023-11-06T01:34:10.168 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3394_vm 2023-11-06T01:34:10.168 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3395 2023-11-06T01:34:10.169 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439 2023-11-06T01:34:10.169 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439_vm 2023-11-06T01:34:10.169 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3440 2023-11-06T01:34:10.169 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 3455 2023-11-06T01:34:10.169 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3456 2023-11-06T01:34:10.170 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3456_fsm 2023-11-06T01:34:10.170 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3456_vm 2023-11-06T01:34:10.170 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466 2023-11-06T01:34:10.170 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466_vm 2023-11-06T01:34:10.170 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3467 2023-11-06T01:34:10.171 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3468 2023-11-06T01:34:10.171 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501 2023-11-06T01:34:10.171 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501_vm 2023-11-06T01:34:10.171 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3502 2023-11-06T01:34:10.171 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3503 2023-11-06T01:34:10.172 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3534 2023-11-06T01:34:10.172 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541 2023-11-06T01:34:10.172 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3541_fsm 2023-11-06T01:34:10.172 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541_vm 2023-11-06T01:34:10.172 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3542 2023-11-06T01:34:10.173 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3574 2023-11-06T01:34:10.173 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3575 2023-11-06T01:34:10.173 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576 2023-11-06T01:34:10.173 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576_vm 2023-11-06T01:34:10.173 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596 2023-11-06T01:34:10.174 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596_vm 2023-11-06T01:34:10.174 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3597 2023-11-06T01:34:10.174 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598 2023-11-06T01:34:10.174 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598_vm 2023-11-06T01:34:10.174 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3599 2023-11-06T01:34:10.174 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600 2023-11-06T01:34:10.175 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3600_fsm 2023-11-06T01:34:10.175 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600_vm 2023-11-06T01:34:10.175 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601 2023-11-06T01:34:10.175 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3601_fsm 2023-11-06T01:34:10.175 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601_vm 2023-11-06T01:34:10.176 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602 2023-11-06T01:34:10.176 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3602_fsm 2023-11-06T01:34:10.176 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602_vm 2023-11-06T01:34:10.176 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3603 2023-11-06T01:34:10.176 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3603_fsm 2023-11-06T01:34:10.177 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3603_vm 2023-11-06T01:34:10.177 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3604 2023-11-06T01:34:10.177 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3605 2023-11-06T01:34:10.177 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3606 2023-11-06T01:34:10.177 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3607 2023-11-06T01:34:10.178 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3608 2023-11-06T01:34:10.178 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 3609 2023-11-06T01:34:10.178 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3712 2023-11-06T01:34:10.178 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764 2023-11-06T01:34:10.178 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3764_fsm 2023-11-06T01:34:10.179 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764_vm 2023-11-06T01:34:10.179 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3766 2023-11-06T01:34:10.179 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3767 2023-11-06T01:34:10.179 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3997 2023-11-06T01:34:10.179 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 5002 2023-11-06T01:34:10.179 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 548 2023-11-06T01:34:10.180 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 549 2023-11-06T01:34:10.180 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102 2023-11-06T01:34:10.180 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102_vm 2023-11-06T01:34:10.180 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104 2023-11-06T01:34:10.180 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104_vm 2023-11-06T01:34:10.181 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106 2023-11-06T01:34:10.181 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106_vm 2023-11-06T01:34:10.181 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6110 2023-11-06T01:34:10.181 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6111 2023-11-06T01:34:10.181 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6112 2023-11-06T01:34:10.181 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6113 2023-11-06T01:34:10.182 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6117 2023-11-06T01:34:10.182 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826 2023-11-06T01:34:10.182 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826_vm 2023-11-06T01:34:10.182 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 827 2023-11-06T01:34:10.183 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 828 2023-11-06T01:34:10.183 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Nov 6 01:34 pg_filenode.map 2023-11-06T01:34:10.183 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 PG_VERSION 2023-11-06T01:34:10.183 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.183 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-11-06T01:34:10.183 INFO:teuthology.orchestra.run.smithi053.stdout:total 7193 2023-11-06T01:34:10.184 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Nov 6 01:34 . 2023-11-06T01:34:10.184 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 .. 2023-11-06T01:34:10.184 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 112 2023-11-06T01:34:10.184 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 113 2023-11-06T01:34:10.184 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Nov 6 01:34 12182 2023-11-06T01:34:10.185 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12182_fsm 2023-11-06T01:34:10.185 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12182_vm 2023-11-06T01:34:10.185 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12184 2023-11-06T01:34:10.185 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12186 2023-11-06T01:34:10.185 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187 2023-11-06T01:34:10.186 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12187_fsm 2023-11-06T01:34:10.186 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187_vm 2023-11-06T01:34:10.186 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12189 2023-11-06T01:34:10.186 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12191 2023-11-06T01:34:10.186 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192 2023-11-06T01:34:10.187 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12192_fsm 2023-11-06T01:34:10.187 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192_vm 2023-11-06T01:34:10.189 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12194 2023-11-06T01:34:10.190 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12196 2023-11-06T01:34:10.190 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197 2023-11-06T01:34:10.190 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12197_fsm 2023-11-06T01:34:10.190 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197_vm 2023-11-06T01:34:10.191 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12199 2023-11-06T01:34:10.191 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12201 2023-11-06T01:34:10.191 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202 2023-11-06T01:34:10.191 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12202_fsm 2023-11-06T01:34:10.191 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202_vm 2023-11-06T01:34:10.191 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12204 2023-11-06T01:34:10.192 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12206 2023-11-06T01:34:10.192 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207 2023-11-06T01:34:10.192 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12207_fsm 2023-11-06T01:34:10.192 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207_vm 2023-11-06T01:34:10.192 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12209 2023-11-06T01:34:10.193 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12211 2023-11-06T01:34:10.193 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12212 2023-11-06T01:34:10.193 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12214 2023-11-06T01:34:10.193 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12216 2023-11-06T01:34:10.193 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Nov 6 01:34 1247 2023-11-06T01:34:10.194 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1247_fsm 2023-11-06T01:34:10.194 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1247_vm 2023-11-06T01:34:10.194 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Nov 6 01:34 1249 2023-11-06T01:34:10.194 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1249_fsm 2023-11-06T01:34:10.194 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1249_vm 2023-11-06T01:34:10.195 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Nov 6 01:34 1255 2023-11-06T01:34:10.195 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1255_fsm 2023-11-06T01:34:10.195 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1255_vm 2023-11-06T01:34:10.195 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Nov 6 01:34 1259 2023-11-06T01:34:10.195 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1259_fsm 2023-11-06T01:34:10.195 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1259_vm 2023-11-06T01:34:10.196 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417 2023-11-06T01:34:10.196 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417_vm 2023-11-06T01:34:10.196 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418 2023-11-06T01:34:10.196 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418_vm 2023-11-06T01:34:10.196 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 174 2023-11-06T01:34:10.197 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 175 2023-11-06T01:34:10.197 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2187 2023-11-06T01:34:10.197 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224 2023-11-06T01:34:10.197 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224_vm 2023-11-06T01:34:10.197 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328 2023-11-06T01:34:10.198 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328_vm 2023-11-06T01:34:10.198 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336 2023-11-06T01:34:10.198 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336_vm 2023-11-06T01:34:10.199 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2337 2023-11-06T01:34:10.199 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2600 2023-11-06T01:34:10.199 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2600_fsm 2023-11-06T01:34:10.200 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2600_vm 2023-11-06T01:34:10.200 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601 2023-11-06T01:34:10.201 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2601_fsm 2023-11-06T01:34:10.201 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601_vm 2023-11-06T01:34:10.201 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2602 2023-11-06T01:34:10.202 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2602_fsm 2023-11-06T01:34:10.202 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2602_vm 2023-11-06T01:34:10.202 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2603 2023-11-06T01:34:10.203 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2603_fsm 2023-11-06T01:34:10.203 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2603_vm 2023-11-06T01:34:10.203 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604 2023-11-06T01:34:10.204 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604_vm 2023-11-06T01:34:10.204 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2605 2023-11-06T01:34:10.205 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2605_fsm 2023-11-06T01:34:10.205 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2605_vm 2023-11-06T01:34:10.205 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606 2023-11-06T01:34:10.206 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2606_fsm 2023-11-06T01:34:10.206 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606_vm 2023-11-06T01:34:10.206 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607 2023-11-06T01:34:10.207 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607_fsm 2023-11-06T01:34:10.207 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2607_vm 2023-11-06T01:34:10.207 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Nov 6 01:34 2608 2023-11-06T01:34:10.208 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2608_fsm 2023-11-06T01:34:10.208 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2608_vm 2023-11-06T01:34:10.208 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Nov 6 01:34 2609 2023-11-06T01:34:10.209 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2609_fsm 2023-11-06T01:34:10.209 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2609_vm 2023-11-06T01:34:10.210 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2610 2023-11-06T01:34:10.210 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2610_fsm 2023-11-06T01:34:10.210 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2610_vm 2023-11-06T01:34:10.211 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611 2023-11-06T01:34:10.211 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611_vm 2023-11-06T01:34:10.211 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612 2023-11-06T01:34:10.212 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2612_fsm 2023-11-06T01:34:10.212 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612_vm 2023-11-06T01:34:10.212 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613 2023-11-06T01:34:10.213 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613_vm 2023-11-06T01:34:10.213 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615 2023-11-06T01:34:10.213 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2615_fsm 2023-11-06T01:34:10.214 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615_vm 2023-11-06T01:34:10.214 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616 2023-11-06T01:34:10.215 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616_fsm 2023-11-06T01:34:10.215 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2616_vm 2023-11-06T01:34:10.215 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Nov 6 01:34 2617 2023-11-06T01:34:10.216 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2617_fsm 2023-11-06T01:34:10.216 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2617_vm 2023-11-06T01:34:10.216 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Nov 6 01:34 2618 2023-11-06T01:34:10.217 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2618_fsm 2023-11-06T01:34:10.217 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2618_vm 2023-11-06T01:34:10.217 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Nov 6 01:34 2619 2023-11-06T01:34:10.218 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2619_fsm 2023-11-06T01:34:10.218 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2619_vm 2023-11-06T01:34:10.218 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620 2023-11-06T01:34:10.219 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620_vm 2023-11-06T01:34:10.219 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2650 2023-11-06T01:34:10.220 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2651 2023-11-06T01:34:10.220 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2652 2023-11-06T01:34:10.220 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2653 2023-11-06T01:34:10.221 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2654 2023-11-06T01:34:10.221 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2655 2023-11-06T01:34:10.221 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2656 2023-11-06T01:34:10.222 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2657 2023-11-06T01:34:10.222 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Nov 6 01:34 2658 2023-11-06T01:34:10.222 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2659 2023-11-06T01:34:10.223 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2660 2023-11-06T01:34:10.223 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2661 2023-11-06T01:34:10.224 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2662 2023-11-06T01:34:10.224 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2663 2023-11-06T01:34:10.224 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2664 2023-11-06T01:34:10.225 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2665 2023-11-06T01:34:10.225 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2666 2023-11-06T01:34:10.225 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2667 2023-11-06T01:34:10.226 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2668 2023-11-06T01:34:10.226 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2669 2023-11-06T01:34:10.226 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2670 2023-11-06T01:34:10.227 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Nov 6 01:34 2673 2023-11-06T01:34:10.227 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Nov 6 01:34 2674 2023-11-06T01:34:10.228 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Nov 6 01:34 2675 2023-11-06T01:34:10.228 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2678 2023-11-06T01:34:10.228 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2679 2023-11-06T01:34:10.229 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2680 2023-11-06T01:34:10.229 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2681 2023-11-06T01:34:10.229 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2682 2023-11-06T01:34:10.230 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2683 2023-11-06T01:34:10.230 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2684 2023-11-06T01:34:10.230 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2685 2023-11-06T01:34:10.231 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2686 2023-11-06T01:34:10.231 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2687 2023-11-06T01:34:10.232 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2688 2023-11-06T01:34:10.232 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2689 2023-11-06T01:34:10.232 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2690 2023-11-06T01:34:10.233 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Nov 6 01:34 2691 2023-11-06T01:34:10.233 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2692 2023-11-06T01:34:10.233 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2693 2023-11-06T01:34:10.234 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2696 2023-11-06T01:34:10.234 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2699 2023-11-06T01:34:10.234 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2701 2023-11-06T01:34:10.235 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2702 2023-11-06T01:34:10.235 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2703 2023-11-06T01:34:10.235 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2704 2023-11-06T01:34:10.236 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2753 2023-11-06T01:34:10.236 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2753_fsm 2023-11-06T01:34:10.236 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2753_vm 2023-11-06T01:34:10.237 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2754 2023-11-06T01:34:10.237 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2755 2023-11-06T01:34:10.237 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2756 2023-11-06T01:34:10.238 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2757 2023-11-06T01:34:10.238 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830 2023-11-06T01:34:10.239 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830_vm 2023-11-06T01:34:10.239 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2831 2023-11-06T01:34:10.239 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832 2023-11-06T01:34:10.240 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832_vm 2023-11-06T01:34:10.240 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2833 2023-11-06T01:34:10.240 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834 2023-11-06T01:34:10.241 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834_vm 2023-11-06T01:34:10.241 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2835 2023-11-06T01:34:10.241 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836 2023-11-06T01:34:10.242 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836_vm 2023-11-06T01:34:10.242 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2837 2023-11-06T01:34:10.242 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Nov 6 01:34 2838 2023-11-06T01:34:10.243 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2838_fsm 2023-11-06T01:34:10.243 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2838_vm 2023-11-06T01:34:10.243 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2839 2023-11-06T01:34:10.244 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840 2023-11-06T01:34:10.244 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840_fsm 2023-11-06T01:34:10.245 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2840_vm 2023-11-06T01:34:10.245 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2841 2023-11-06T01:34:10.245 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995 2023-11-06T01:34:10.245 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995_vm 2023-11-06T01:34:10.245 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2996 2023-11-06T01:34:10.246 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079 2023-11-06T01:34:10.246 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3079_fsm 2023-11-06T01:34:10.246 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079_vm 2023-11-06T01:34:10.246 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3080 2023-11-06T01:34:10.247 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3081 2023-11-06T01:34:10.247 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3085 2023-11-06T01:34:10.247 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118 2023-11-06T01:34:10.247 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118_vm 2023-11-06T01:34:10.247 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3119 2023-11-06T01:34:10.248 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3164 2023-11-06T01:34:10.248 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256 2023-11-06T01:34:10.248 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256_vm 2023-11-06T01:34:10.248 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3257 2023-11-06T01:34:10.248 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3258 2023-11-06T01:34:10.249 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350 2023-11-06T01:34:10.249 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350_vm 2023-11-06T01:34:10.249 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3351 2023-11-06T01:34:10.249 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3379 2023-11-06T01:34:10.249 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3380 2023-11-06T01:34:10.250 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381 2023-11-06T01:34:10.250 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381_vm 2023-11-06T01:34:10.250 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3394 2023-11-06T01:34:10.250 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3394_fsm 2023-11-06T01:34:10.251 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3394_vm 2023-11-06T01:34:10.251 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3395 2023-11-06T01:34:10.251 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439 2023-11-06T01:34:10.251 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439_vm 2023-11-06T01:34:10.251 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3440 2023-11-06T01:34:10.252 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 3455 2023-11-06T01:34:10.252 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3456 2023-11-06T01:34:10.252 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3456_fsm 2023-11-06T01:34:10.252 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3456_vm 2023-11-06T01:34:10.252 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466 2023-11-06T01:34:10.253 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466_vm 2023-11-06T01:34:10.253 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3467 2023-11-06T01:34:10.253 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3468 2023-11-06T01:34:10.253 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501 2023-11-06T01:34:10.253 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501_vm 2023-11-06T01:34:10.254 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3502 2023-11-06T01:34:10.254 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3503 2023-11-06T01:34:10.254 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3534 2023-11-06T01:34:10.254 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541 2023-11-06T01:34:10.255 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3541_fsm 2023-11-06T01:34:10.255 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541_vm 2023-11-06T01:34:10.255 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3542 2023-11-06T01:34:10.255 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3574 2023-11-06T01:34:10.256 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3575 2023-11-06T01:34:10.256 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576 2023-11-06T01:34:10.256 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576_vm 2023-11-06T01:34:10.256 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596 2023-11-06T01:34:10.256 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596_vm 2023-11-06T01:34:10.257 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3597 2023-11-06T01:34:10.257 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598 2023-11-06T01:34:10.257 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598_vm 2023-11-06T01:34:10.257 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3599 2023-11-06T01:34:10.258 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600 2023-11-06T01:34:10.258 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3600_fsm 2023-11-06T01:34:10.258 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600_vm 2023-11-06T01:34:10.258 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601 2023-11-06T01:34:10.258 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3601_fsm 2023-11-06T01:34:10.259 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601_vm 2023-11-06T01:34:10.259 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602 2023-11-06T01:34:10.259 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3602_fsm 2023-11-06T01:34:10.259 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602_vm 2023-11-06T01:34:10.259 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3603 2023-11-06T01:34:10.260 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3603_fsm 2023-11-06T01:34:10.260 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3603_vm 2023-11-06T01:34:10.260 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3604 2023-11-06T01:34:10.260 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3605 2023-11-06T01:34:10.260 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3606 2023-11-06T01:34:10.261 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3607 2023-11-06T01:34:10.261 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3608 2023-11-06T01:34:10.261 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 3609 2023-11-06T01:34:10.261 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3712 2023-11-06T01:34:10.262 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764 2023-11-06T01:34:10.262 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3764_fsm 2023-11-06T01:34:10.262 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764_vm 2023-11-06T01:34:10.262 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3766 2023-11-06T01:34:10.262 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3767 2023-11-06T01:34:10.263 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3997 2023-11-06T01:34:10.263 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 5002 2023-11-06T01:34:10.263 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 548 2023-11-06T01:34:10.263 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 549 2023-11-06T01:34:10.264 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102 2023-11-06T01:34:10.264 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102_vm 2023-11-06T01:34:10.264 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104 2023-11-06T01:34:10.264 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104_vm 2023-11-06T01:34:10.264 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106 2023-11-06T01:34:10.265 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106_vm 2023-11-06T01:34:10.265 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6110 2023-11-06T01:34:10.265 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6111 2023-11-06T01:34:10.265 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6112 2023-11-06T01:34:10.265 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6113 2023-11-06T01:34:10.266 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6117 2023-11-06T01:34:10.266 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826 2023-11-06T01:34:10.266 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826_vm 2023-11-06T01:34:10.266 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 827 2023-11-06T01:34:10.266 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 828 2023-11-06T01:34:10.267 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Nov 6 01:34 pg_filenode.map 2023-11-06T01:34:10.267 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 PG_VERSION 2023-11-06T01:34:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.267 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/global: 2023-11-06T01:34:10.268 INFO:teuthology.orchestra.run.smithi053.stdout:total 553 2023-11-06T01:34:10.268 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Nov 6 01:34 . 2023-11-06T01:34:10.268 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.268 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1136 2023-11-06T01:34:10.268 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1136_fsm 2023-11-06T01:34:10.269 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1136_vm 2023-11-06T01:34:10.269 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 1137 2023-11-06T01:34:10.269 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1213 2023-11-06T01:34:10.269 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1213_fsm 2023-11-06T01:34:10.269 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1213_vm 2023-11-06T01:34:10.270 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1214 2023-11-06T01:34:10.270 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1214_fsm 2023-11-06T01:34:10.270 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1214_vm 2023-11-06T01:34:10.270 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 1232 2023-11-06T01:34:10.270 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 1233 2023-11-06T01:34:10.271 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1260 2023-11-06T01:34:10.271 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1260_fsm 2023-11-06T01:34:10.271 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1260_vm 2023-11-06T01:34:10.271 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1261 2023-11-06T01:34:10.271 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1261_fsm 2023-11-06T01:34:10.272 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1261_vm 2023-11-06T01:34:10.272 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1262 2023-11-06T01:34:10.272 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1262_fsm 2023-11-06T01:34:10.272 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1262_vm 2023-11-06T01:34:10.273 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2396 2023-11-06T01:34:10.273 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2396_fsm 2023-11-06T01:34:10.273 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2396_vm 2023-11-06T01:34:10.273 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2397 2023-11-06T01:34:10.273 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2671 2023-11-06T01:34:10.274 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2672 2023-11-06T01:34:10.274 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2676 2023-11-06T01:34:10.274 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2677 2023-11-06T01:34:10.274 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2694 2023-11-06T01:34:10.274 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2695 2023-11-06T01:34:10.275 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2697 2023-11-06T01:34:10.275 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2698 2023-11-06T01:34:10.275 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2846 2023-11-06T01:34:10.275 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2846_vm 2023-11-06T01:34:10.275 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2847 2023-11-06T01:34:10.276 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2964 2023-11-06T01:34:10.276 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2964_vm 2023-11-06T01:34:10.276 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2965 2023-11-06T01:34:10.276 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2966 2023-11-06T01:34:10.276 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2966_vm 2023-11-06T01:34:10.277 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2967 2023-11-06T01:34:10.277 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3592 2023-11-06T01:34:10.277 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3592_vm 2023-11-06T01:34:10.277 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3593 2023-11-06T01:34:10.278 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 4060 2023-11-06T01:34:10.278 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 4060_vm 2023-11-06T01:34:10.278 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 4061 2023-11-06T01:34:10.278 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6000 2023-11-06T01:34:10.278 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6000_vm 2023-11-06T01:34:10.279 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6001 2023-11-06T01:34:10.279 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6002 2023-11-06T01:34:10.279 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6100 2023-11-06T01:34:10.279 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6100_vm 2023-11-06T01:34:10.279 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6114 2023-11-06T01:34:10.280 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6115 2023-11-06T01:34:10.280 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 pg_control 2023-11-06T01:34:10.280 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Nov 6 01:34 pg_filenode.map 2023-11-06T01:34:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.281 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/log: 2023-11-06T01:34:10.281 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.281 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.281 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.282 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-11-06T01:34:10.282 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.282 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.282 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.283 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-11-06T01:34:10.283 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.283 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.283 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.284 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-11-06T01:34:10.284 INFO:teuthology.orchestra.run.smithi053.stdout:total 1 2023-11-06T01:34:10.284 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 . 2023-11-06T01:34:10.285 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.285 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 mappings 2023-11-06T01:34:10.285 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Nov 6 01:34 replorigin_checkpoint 2023-11-06T01:34:10.285 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 snapshots 2023-11-06T01:34:10.285 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.286 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-11-06T01:34:10.286 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.286 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.286 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 .. 2023-11-06T01:34:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.287 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-11-06T01:34:10.287 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.287 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.287 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 .. 2023-11-06T01:34:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.288 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-11-06T01:34:10.288 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.288 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:34 . 2023-11-06T01:34:10.288 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.289 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 members 2023-11-06T01:34:10.289 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 offsets 2023-11-06T01:34:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.289 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-11-06T01:34:10.289 INFO:teuthology.orchestra.run.smithi053.stdout:total 8 2023-11-06T01:34:10.290 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 . 2023-11-06T01:34:10.290 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:34 .. 2023-11-06T01:34:10.290 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 0000 2023-11-06T01:34:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.290 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-11-06T01:34:10.291 INFO:teuthology.orchestra.run.smithi053.stdout:total 8 2023-11-06T01:34:10.291 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 . 2023-11-06T01:34:10.291 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:34 .. 2023-11-06T01:34:10.291 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 0000 2023-11-06T01:34:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.292 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-11-06T01:34:10.292 INFO:teuthology.orchestra.run.smithi053.stdout:total 8 2023-11-06T01:34:10.292 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 . 2023-11-06T01:34:10.292 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.293 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 0000 2023-11-06T01:34:10.293 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.293 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-11-06T01:34:10.293 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.293 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-11-06T01:34:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.295 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.295 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-11-06T01:34:10.295 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.296 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.296 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.296 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-11-06T01:34:10.296 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.297 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.297 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.297 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-11-06T01:34:10.297 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.298 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.298 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.298 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.298 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-11-06T01:34:10.299 INFO:teuthology.orchestra.run.smithi053.stdout:total 8 2023-11-06T01:34:10.299 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 . 2023-11-06T01:34:10.299 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.299 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 0000 2023-11-06T01:34:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.300 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-11-06T01:34:10.300 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.300 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.300 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.301 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-11-06T01:34:10.301 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.301 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.301 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.302 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-11-06T01:34:10.302 INFO:teuthology.orchestra.run.smithi053.stdout:total 16384 2023-11-06T01:34:10.302 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:34 . 2023-11-06T01:34:10.302 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.303 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:34 000000010000000000000001 2023-11-06T01:34:10.303 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 archive_status 2023-11-06T01:34:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.303 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-11-06T01:34:10.303 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:34:10.304 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:34:10.304 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:34 .. 2023-11-06T01:34:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:10.304 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-11-06T01:34:10.305 INFO:teuthology.orchestra.run.smithi053.stdout:total 8 2023-11-06T01:34:10.305 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 . 2023-11-06T01:34:10.305 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Nov 6 01:34 .. 2023-11-06T01:34:10.305 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 0000 2023-11-06T01:34:10.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-11-06T01:34:10.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2023-11-06T01:34:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-11-06T01:34:10.491 INFO:teuthology.orchestra.run.smithi053.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-11-06T01:34:10.494 INFO:teuthology.orchestra.run.smithi053.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-11-06T01:34:10.498 INFO:teuthology.orchestra.run.smithi053.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-11-06T01:34:10.518 INFO:teuthology.orchestra.run.smithi053.stderr:creating tables... 2023-11-06T01:34:10.595 INFO:teuthology.orchestra.run.smithi053.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.08 s, remaining 2.34 s) 2023-11-06T01:34:10.776 INFO:teuthology.orchestra.run.smithi053.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.26 s, remaining 3.85 s) 2023-11-06T01:34:10.871 INFO:teuthology.orchestra.run.smithi053.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.35 s, remaining 3.40 s) 2023-11-06T01:34:11.025 INFO:teuthology.orchestra.run.smithi053.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.51 s, remaining 3.54 s) 2023-11-06T01:34:11.199 INFO:teuthology.orchestra.run.smithi053.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.68 s, remaining 3.67 s) 2023-11-06T01:34:11.362 INFO:teuthology.orchestra.run.smithi053.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.84 s, remaining 3.65 s) 2023-11-06T01:34:11.444 INFO:teuthology.orchestra.run.smithi053.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.92 s, remaining 3.30 s) 2023-11-06T01:34:11.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:11 smithi053 ceph-mon[99472]: pgmap v306: 97 pgs: 97 active+clean; 32 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.6 MiB/s wr, 55 op/s 2023-11-06T01:34:11.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:11 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:11.619 INFO:teuthology.orchestra.run.smithi053.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.10 s, remaining 3.30 s) 2023-11-06T01:34:11.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:11 smithi062 ceph-mon[103924]: pgmap v306: 97 pgs: 97 active+clean; 32 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.6 MiB/s wr, 55 op/s 2023-11-06T01:34:11.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:11 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:11.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:11 smithi163 ceph-mon[104416]: pgmap v306: 97 pgs: 97 active+clean; 32 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.6 MiB/s wr, 55 op/s 2023-11-06T01:34:11.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:11 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:11.797 INFO:teuthology.orchestra.run.smithi053.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.28 s, remaining 3.27 s) 2023-11-06T01:34:11.861 INFO:teuthology.orchestra.run.smithi053.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.34 s, remaining 2.95 s) 2023-11-06T01:34:12.043 INFO:teuthology.orchestra.run.smithi053.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.52 s, remaining 2.91 s) 2023-11-06T01:34:12.109 INFO:teuthology.orchestra.run.smithi053.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.59 s, remaining 2.65 s) 2023-11-06T01:34:12.280 INFO:teuthology.orchestra.run.smithi053.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.76 s, remaining 2.57 s) 2023-11-06T01:34:12.440 INFO:teuthology.orchestra.run.smithi053.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.92 s, remaining 2.47 s) 2023-11-06T01:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.455+0000 7f22682cd700 1 -- 172.21.15.53:0/4082596342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2260106ad0 msgr2=0x7f2260108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.455+0000 7f22682cd700 1 --2- 172.21.15.53:0/4082596342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2260106ad0 0x7f2260108f00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f225800b0a0 tx=0x7f225800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.455+0000 7f22682cd700 1 -- 172.21.15.53:0/4082596342 shutdown_connections 2023-11-06T01:34:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.455+0000 7f22682cd700 1 --2- 172.21.15.53:0/4082596342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2260101870 0x7f2260103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.455+0000 7f22682cd700 1 --2- 172.21.15.53:0/4082596342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2260106ad0 0x7f2260108f00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.455+0000 7f22682cd700 1 --2- 172.21.15.53:0/4082596342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22601041a0 0x7f2260106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.455+0000 7f22682cd700 1 -- 172.21.15.53:0/4082596342 >> 172.21.15.53:0/4082596342 conn(0x7f22600fb220 msgr2=0x7f22600fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.456+0000 7f22682cd700 1 -- 172.21.15.53:0/4082596342 shutdown_connections 2023-11-06T01:34:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.456+0000 7f22682cd700 1 -- 172.21.15.53:0/4082596342 wait complete. 2023-11-06T01:34:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.456+0000 7f22682cd700 1 Processor -- start 2023-11-06T01:34:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.456+0000 7f22682cd700 1 -- start start 2023-11-06T01:34:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.456+0000 7f22682cd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2260101870 0x7f22600ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.456+0000 7f22682cd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22601041a0 0x7f22600ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.456+0000 7f22682cd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2260106ad0 0x7f22601002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.456+0000 7f22682cd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22601a0bc0 con 0x7f2260101870 2023-11-06T01:34:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.456+0000 7f22682cd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22601a0d00 con 0x7f2260106ad0 2023-11-06T01:34:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.456+0000 7f22682cd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22601a0e40 con 0x7f22601041a0 2023-11-06T01:34:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f2266069700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2260101870 0x7f22600ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f2266069700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2260101870 0x7f22600ff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39830/0 (socket says 172.21.15.53:39830) 2023-11-06T01:34:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f2266069700 1 -- 172.21.15.53:0/2302299631 learned_addr learned my addr 172.21.15.53:0/2302299631 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f226686a700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2260106ad0 0x7f22601002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f2265868700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22601041a0 0x7f22600ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f2266069700 1 -- 172.21.15.53:0/2302299631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22601041a0 msgr2=0x7f22600ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f2266069700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22601041a0 0x7f22600ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f2266069700 1 -- 172.21.15.53:0/2302299631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2260106ad0 msgr2=0x7f22601002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f2266069700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2260106ad0 0x7f22601002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f2266069700 1 -- 172.21.15.53:0/2302299631 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2258009d00 con 0x7f2260101870 2023-11-06T01:34:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f2266069700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2260101870 0x7f22600ff820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e9a0 tx=0x7f225c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f22577fe700 1 -- 172.21.15.53:0/2302299631 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c009790 con 0x7f2260101870 2023-11-06T01:34:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f22577fe700 1 -- 172.21.15.53:0/2302299631 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f225c010430 con 0x7f2260101870 2023-11-06T01:34:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f22682cd700 1 -- 172.21.15.53:0/2302299631 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22601a10d0 con 0x7f2260101870 2023-11-06T01:34:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f22577fe700 1 -- 172.21.15.53:0/2302299631 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c018680 con 0x7f2260101870 2023-11-06T01:34:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.457+0000 7f22682cd700 1 -- 172.21.15.53:0/2302299631 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22601a1630 con 0x7f2260101870 2023-11-06T01:34:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.460+0000 7f22577fe700 1 -- 172.21.15.53:0/2302299631 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f225c0105a0 con 0x7f2260101870 2023-11-06T01:34:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.461+0000 7f22577fe700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f224c0a05a0 0x7f224c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.461+0000 7f227445db80 1 Processor -- start 2023-11-06T01:34:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.461+0000 7f22577fe700 1 -- 172.21.15.53:0/2302299631 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f225c014070 con 0x7f2260101870 2023-11-06T01:34:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f227445db80 1 -- start start 2023-11-06T01:34:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f227445db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6680 0x55951ecc6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f227445db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecc78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f227445db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecc8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f227445db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ec1e9b0 con 0x55951ecc6fe0 2023-11-06T01:34:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f227445db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ec1c370 con 0x55951ecc6680 2023-11-06T01:34:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f227445db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ec17c60 con 0x55951ecc7ed0 2023-11-06T01:34:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f2265868700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecc78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f2266069700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6680 0x55951ecc6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f2266069700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6680 0x55951ecc6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51276/0 (socket says 172.21.15.53:51276) 2023-11-06T01:34:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f2266069700 1 -- 172.21.15.53:0/516979494 learned_addr learned my addr 172.21.15.53:0/516979494 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f226686a700 1 --2- 172.21.15.53:0/516979494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecc8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f2266069700 1 -- 172.21.15.53:0/516979494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 msgr2=0x55951ecc8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f2266069700 1 --2- 172.21.15.53:0/516979494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecc8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f2266069700 1 -- 172.21.15.53:0/516979494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 msgr2=0x55951ecc78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f2266069700 1 --2- 172.21.15.53:0/516979494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecc78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.462+0000 7f2266069700 1 -- 172.21.15.53:0/516979494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55951ea45d80 con 0x55951ecc6680 2023-11-06T01:34:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.463+0000 7f2265868700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f224c0a05a0 0x7f224c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.463+0000 7f2266069700 1 --2- 172.21.15.53:0/516979494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6680 0x55951ecc6aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f225c099230 tx=0x7f225c098cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd9089744257865 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.463+0000 7f2254daa700 1 -- 172.21.15.53:0/516979494 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c09d0d0 con 0x55951ecc6680 2023-11-06T01:34:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.463+0000 7f2254daa700 1 -- 172.21.15.53:0/516979494 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f225c0a1310 con 0x55951ecc6680 2023-11-06T01:34:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.463+0000 7f2254daa700 1 -- 172.21.15.53:0/516979494 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c0989f0 con 0x55951ecc6680 2023-11-06T01:34:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f227445db80 1 -- 172.21.15.53:0/516979494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6680 msgr2=0x55951ecc6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f227445db80 1 --2- 172.21.15.53:0/516979494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6680 0x55951ecc6aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f225c099230 tx=0x7f225c098cd0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f2265868700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f224c0a05a0 0x7f224c0a2a60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2250005950 tx=0x7f22500047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f227445db80 1 -- 172.21.15.53:0/516979494 shutdown_connections 2023-11-06T01:34:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f227445db80 1 --2- 172.21.15.53:0/516979494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6680 0x55951ecc6aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f227445db80 1 --2- 172.21.15.53:0/516979494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecc8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f227445db80 1 --2- 172.21.15.53:0/516979494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecc78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f227445db80 1 -- 172.21.15.53:0/516979494 >> 172.21.15.53:0/516979494 conn(0x55951ec26070 msgr2=0x55951ecc3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f227445db80 1 -- 172.21.15.53:0/516979494 shutdown_connections 2023-11-06T01:34:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f227445db80 1 -- 172.21.15.53:0/516979494 wait complete. 2023-11-06T01:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f227445db80 1 -- start start 2023-11-06T01:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.464+0000 7f227445db80 1 Processor -- start 2023-11-06T01:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f227445db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecd0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f227445db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecd0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f227445db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecd5cb0 0x55951ecd80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f227445db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ebb44b0 con 0x55951ecc6fe0 2023-11-06T01:34:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f227445db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ebb4150 con 0x55951ecd5cb0 2023-11-06T01:34:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f227445db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ea9b8c0 con 0x55951ecc7ed0 2023-11-06T01:34:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f227445db80 10 client.0 _ll_register_callbacks cb 0x55951eb01a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2266069700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecd0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2266069700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecd0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39844/0 (socket says 172.21.15.53:39844) 2023-11-06T01:34:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2266069700 1 -- 172.21.15.53:0/3336375128 learned_addr learned my addr 172.21.15.53:0/3336375128 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2265868700 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecd0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f226686a700 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecd5cb0 0x55951ecd80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2266069700 1 -- 172.21.15.53:0/3336375128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 msgr2=0x55951ecd0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2266069700 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecd0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2266069700 1 -- 172.21.15.53:0/3336375128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecd5cb0 msgr2=0x55951ecd80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2266069700 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecd5cb0 0x55951ecd80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2266069700 1 -- 172.21.15.53:0/3336375128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22601a1630 con 0x55951ecc6fe0 2023-11-06T01:34:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2266069700 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecd0740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f225c0c9590 tx=0x7f225c099170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9998e94f25998894 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2249ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:34:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.465+0000 7f2249ffb700 1 -- 172.21.15.53:0/3336375128 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c0a0e80 con 0x55951ecc6fe0 2023-11-06T01:34:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f227445db80 1 -- 172.21.15.53:0/3336375128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55951ec42cb0 con 0x55951ecc6fe0 2023-11-06T01:34:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f2249ffb700 1 -- 172.21.15.53:0/3336375128 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f225c0a0fe0 con 0x55951ecc6fe0 2023-11-06T01:34:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f2249ffb700 1 -- 172.21.15.53:0/3336375128 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f225c0b7080 con 0x55951ecc6fe0 2023-11-06T01:34:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f227445db80 10 client.15474 fetch_fsmap learned FSMap version 28 2023-11-06T01:34:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f227445db80 1 -- 172.21.15.53:0/3336375128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55951ea2ec00 con 0x55951ecc6fe0 2023-11-06T01:34:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f2249ffb700 1 -- 172.21.15.53:0/3336375128 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f225c07d020 con 0x55951ecc6fe0 2023-11-06T01:34:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f2249ffb700 1 -- 172.21.15.53:0/3336375128 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f225c09b030 con 0x55951ecc6fe0 2023-11-06T01:34:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f227445db80 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2023-11-06T01:34:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f227445db80 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f227445db80 20 client.15474 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f227445db80 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55951ecdc5a0 0x55951ecde9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f227445db80 4 client.15474 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f227445db80 1 -- 172.21.15.53:0/3336375128 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55951ec95db0 con 0x55951ecdc5a0 2023-11-06T01:34:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.466+0000 7f2265868700 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55951ecdc5a0 0x55951ecde9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.467+0000 7f2249ffb700 10 client.15474 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.467+0000 7f2265868700 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55951ecdc5a0 0x55951ecde9c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d357b9acfb8789b server_cookie=a2eedf17d538379c in_seq=0 out_seq=0 2023-11-06T01:34:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.467+0000 7f2249ffb700 1 -- 172.21.15.53:0/3336375128 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55951ec95db0 con 0x55951ecdc5a0 2023-11-06T01:34:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.467+0000 7f2249ffb700 10 client.15474 handle_command_reply: tid=0 2023-11-06T01:34:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.467+0000 7f227445db80 1 client.15474 shutdown 2023-11-06T01:34:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f227445db80 1 -- 172.21.15.53:0/3336375128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 msgr2=0x55951ecd0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f227445db80 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecd0740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f225c0c9590 tx=0x7f225c099170 comp rx=0 tx=0).stop 2023-11-06T01:34:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f227445db80 1 -- 172.21.15.53:0/3336375128 shutdown_connections 2023-11-06T01:34:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f227445db80 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecd5cb0 0x55951ecd80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f227445db80 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55951ecdc5a0 0x55951ecde9c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f227445db80 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecd0740 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f2249ffb700 0 client.15474 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f227445db80 1 --2- 172.21.15.53:0/3336375128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecd0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f227445db80 1 -- 172.21.15.53:0/3336375128 >> 172.21.15.53:0/3336375128 conn(0x55951ec26070 msgr2=0x55951ecc13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f227445db80 1 -- 172.21.15.53:0/3336375128 shutdown_connections 2023-11-06T01:34:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f227445db80 1 -- 172.21.15.53:0/3336375128 wait complete. 2023-11-06T01:34:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.468+0000 7f227445db80 20 client.15474 trim_cache size 0 max 16384 2023-11-06T01:34:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.482+0000 7f227445db80 1 Processor -- start 2023-11-06T01:34:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.482+0000 7f227445db80 1 -- start start 2023-11-06T01:34:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f227445db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecebea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f227445db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecd80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f227445db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecd5cb0 0x55951ecc82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f227445db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ecb99f0 con 0x55951ecc6fe0 2023-11-06T01:34:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f227445db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ecc3770 con 0x55951ecd5cb0 2023-11-06T01:34:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f227445db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ecc6280 con 0x55951ecc7ed0 2023-11-06T01:34:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f2266069700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecebea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f2266069700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecebea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39852/0 (socket says 172.21.15.53:39852) 2023-11-06T01:34:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f226686a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecd5cb0 0x55951ecc82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f2266069700 1 -- 172.21.15.53:0/2768362535 learned_addr learned my addr 172.21.15.53:0/2768362535 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f2265868700 1 --2- 172.21.15.53:0/2768362535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecd80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f226686a700 1 -- 172.21.15.53:0/2768362535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 msgr2=0x55951ecd80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f226686a700 1 --2- 172.21.15.53:0/2768362535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecd80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f226686a700 1 -- 172.21.15.53:0/2768362535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 msgr2=0x55951ecebea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f226686a700 1 --2- 172.21.15.53:0/2768362535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecebea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f226686a700 1 -- 172.21.15.53:0/2768362535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55951ea6c1f0 con 0x55951ecd5cb0 2023-11-06T01:34:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f2265868700 1 --2- 172.21.15.53:0/2768362535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecd80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:34:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f226686a700 1 --2- 172.21.15.53:0/2768362535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecd5cb0 0x55951ecc82b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2258009fd0 tx=0x7f2258009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea25b452fa5c353 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.483+0000 7f22555ab700 1 -- 172.21.15.53:0/2768362535 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225800e040 con 0x55951ecd5cb0 2023-11-06T01:34:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.484+0000 7f22555ab700 1 -- 172.21.15.53:0/2768362535 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2258009b80 con 0x55951ecd5cb0 2023-11-06T01:34:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.484+0000 7f22555ab700 1 -- 172.21.15.53:0/2768362535 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22580052a0 con 0x55951ecd5cb0 2023-11-06T01:34:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.484+0000 7f227445db80 1 -- 172.21.15.53:0/2768362535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecd5cb0 msgr2=0x55951ecc82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.484+0000 7f227445db80 1 --2- 172.21.15.53:0/2768362535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecd5cb0 0x55951ecc82b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2258009fd0 tx=0x7f2258009390 comp rx=0 tx=0).stop 2023-11-06T01:34:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.484+0000 7f227445db80 1 -- 172.21.15.53:0/2768362535 shutdown_connections 2023-11-06T01:34:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.484+0000 7f227445db80 1 --2- 172.21.15.53:0/2768362535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecd5cb0 0x55951ecc82b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.484+0000 7f227445db80 1 --2- 172.21.15.53:0/2768362535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecc6fe0 0x55951ecebea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.484+0000 7f227445db80 1 --2- 172.21.15.53:0/2768362535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ecd80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.484+0000 7f227445db80 1 -- 172.21.15.53:0/2768362535 >> 172.21.15.53:0/2768362535 conn(0x55951ec26070 msgr2=0x55951ecce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.484+0000 7f227445db80 1 -- 172.21.15.53:0/2768362535 shutdown_connections 2023-11-06T01:34:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.484+0000 7f227445db80 1 -- 172.21.15.53:0/2768362535 wait complete. 2023-11-06T01:34:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f227445db80 1 -- start start 2023-11-06T01:34:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f227445db80 1 Processor -- start 2023-11-06T01:34:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f227445db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6fe0 0x55951ed28640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f227445db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ed2ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f227445db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecd5cb0 0x55951ed2d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f227445db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ecec280 con 0x55951ecd5cb0 2023-11-06T01:34:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f227445db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ed2d650 con 0x55951ecc6fe0 2023-11-06T01:34:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f227445db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951ed2d810 con 0x55951ecc7ed0 2023-11-06T01:34:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f227445db80 10 client.0 _ll_register_callbacks cb 0x7f2250009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f2266069700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6fe0 0x55951ed28640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f2266069700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6fe0 0x55951ed28640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51294/0 (socket says 172.21.15.53:51294) 2023-11-06T01:34:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f2266069700 1 -- 172.21.15.53:0/3554688304 learned_addr learned my addr 172.21.15.53:0/3554688304 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f226686a700 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecd5cb0 0x55951ed2d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f2265868700 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ed2ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f2266069700 1 -- 172.21.15.53:0/3554688304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 msgr2=0x55951ed2ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f2266069700 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ed2ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f2266069700 1 -- 172.21.15.53:0/3554688304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecd5cb0 msgr2=0x55951ed2d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f2266069700 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecd5cb0 0x55951ed2d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.485+0000 7f2266069700 1 -- 172.21.15.53:0/3554688304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2260109af0 con 0x55951ecc6fe0 2023-11-06T01:34:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f2266069700 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6fe0 0x55951ed28640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f225c0957a0 tx=0x7f225c010c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fadb7ec0d579f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f2249ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f2249ffb700 1 -- 172.21.15.53:0/3554688304 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c082cf0 con 0x55951ecc6fe0 2023-11-06T01:34:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f2249ffb700 1 -- 172.21.15.53:0/3554688304 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f225c07e080 con 0x55951ecc6fe0 2023-11-06T01:34:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f227445db80 1 -- 172.21.15.53:0/3554688304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55951ec10ce0 con 0x55951ecc6fe0 2023-11-06T01:34:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f2249ffb700 1 -- 172.21.15.53:0/3554688304 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f225c0984c0 con 0x55951ecc6fe0 2023-11-06T01:34:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f227445db80 10 client.25049 fetch_fsmap learned FSMap version 28 2023-11-06T01:34:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f227445db80 1 -- 172.21.15.53:0/3554688304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55951ea44250 con 0x55951ecc6fe0 2023-11-06T01:34:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f2249ffb700 1 -- 172.21.15.53:0/3554688304 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55951ec10ce0 con 0x55951ecc6fe0 2023-11-06T01:34:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f2249ffb700 1 -- 172.21.15.53:0/3554688304 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f225c09b070 con 0x55951ecc6fe0 2023-11-06T01:34:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f227445db80 10 client.25049 fetch_fsmap finished waiting for FSMap version 28 2023-11-06T01:34:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f227445db80 10 client.25049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f227445db80 20 client.25049 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f227445db80 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55951ed2ec60 0x55951ed31050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f227445db80 4 client.25049 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:34:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.486+0000 7f227445db80 1 -- 172.21.15.53:0/3554688304 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55951ec10a00 con 0x55951ed2ec60 2023-11-06T01:34:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.487+0000 7f2265868700 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55951ed2ec60 0x55951ed31050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.487+0000 7f2265868700 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55951ed2ec60 0x55951ed31050 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f67b7acbef8bb1ef server_cookie=62270d2a5b07f910 in_seq=0 out_seq=0 2023-11-06T01:34:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.487+0000 7f2249ffb700 10 client.25049 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:12.536 INFO:teuthology.orchestra.run.smithi053.stderr:1500000 of 3200000 tuples (46%) done (elapsed 2.02 s, remaining 2.29 s) 2023-11-06T01:34:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f2249ffb700 1 -- 172.21.15.53:0/3554688304 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55951ec10a00 con 0x55951ed2ec60 2023-11-06T01:34:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f2249ffb700 10 client.25049 handle_command_reply: tid=0 2023-11-06T01:34:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f227445db80 1 client.25049 shutdown 2023-11-06T01:34:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f227445db80 1 -- 172.21.15.53:0/3554688304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6fe0 msgr2=0x55951ed28640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f227445db80 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6fe0 0x55951ed28640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f225c0957a0 tx=0x7f225c010c50 comp rx=0 tx=0).stop 2023-11-06T01:34:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f227445db80 1 -- 172.21.15.53:0/3554688304 shutdown_connections 2023-11-06T01:34:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f227445db80 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55951ecc6fe0 0x55951ed28640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f227445db80 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55951ed2ec60 0x55951ed31050 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f227445db80 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55951ecd5cb0 0x55951ed2d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f227445db80 1 --2- 172.21.15.53:0/3554688304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55951ecc7ed0 0x55951ed2ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f2249ffb700 0 client.25049 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f227445db80 1 -- 172.21.15.53:0/3554688304 >> 172.21.15.53:0/3554688304 conn(0x55951ec26070 msgr2=0x55951ecce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f227445db80 1 -- 172.21.15.53:0/3554688304 shutdown_connections 2023-11-06T01:34:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.650+0000 7f227445db80 1 -- 172.21.15.53:0/3554688304 wait complete. 2023-11-06T01:34:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f227445db80 20 client.25049 trim_cache size 0 max 16384 2023-11-06T01:34:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:34:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:34:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:34:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:34:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 -- 172.21.15.53:0/2302299631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f224c0a05a0 msgr2=0x7f224c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f224c0a05a0 0x7f224c0a2a60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2250005950 tx=0x7f22500047c0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 -- 172.21.15.53:0/2302299631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2260101870 msgr2=0x7f22600ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2260101870 0x7f22600ff820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e9a0 tx=0x7f225c00ecb0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 -- 172.21.15.53:0/2302299631 shutdown_connections 2023-11-06T01:34:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f224c0a05a0 0x7f224c0a2a60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2260106ad0 0x7f22601002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2260101870 0x7f22600ff820 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 --2- 172.21.15.53:0/2302299631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22601041a0 0x7f22600ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 -- 172.21.15.53:0/2302299631 >> 172.21.15.53:0/2302299631 conn(0x7f22600fb220 msgr2=0x7f22601024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 -- 172.21.15.53:0/2302299631 shutdown_connections 2023-11-06T01:34:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:12.651+0000 7f22682cd700 1 -- 172.21.15.53:0/2302299631 wait complete. 2023-11-06T01:34:12.775 INFO:teuthology.orchestra.run.smithi053.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.26 s, remaining 2.26 s) 2023-11-06T01:34:12.864 INFO:teuthology.orchestra.run.smithi053.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.34 s, remaining 2.07 s) 2023-11-06T01:34:13.038 INFO:teuthology.orchestra.run.smithi053.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.52 s, remaining 1.96 s) 2023-11-06T01:34:13.203 INFO:teuthology.orchestra.run.smithi053.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.68 s, remaining 1.84 s) 2023-11-06T01:34:13.369 INFO:teuthology.orchestra.run.smithi053.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.85 s, remaining 1.71 s) 2023-11-06T01:34:13.448 INFO:teuthology.orchestra.run.smithi053.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.93 s, remaining 1.53 s) 2023-11-06T01:34:13.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:13 smithi053 ceph-mon[99472]: pgmap v307: 97 pgs: 97 active+clean; 41 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 3.7 MiB/s wr, 97 op/s 2023-11-06T01:34:13.624 INFO:teuthology.orchestra.run.smithi053.stderr:2200000 of 3200000 tuples (68%) done (elapsed 3.10 s, remaining 1.41 s) 2023-11-06T01:34:13.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:13 smithi062 ceph-mon[103924]: pgmap v307: 97 pgs: 97 active+clean; 41 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 3.7 MiB/s wr, 97 op/s 2023-11-06T01:34:13.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:13 smithi163 ceph-mon[104416]: pgmap v307: 97 pgs: 97 active+clean; 41 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 3.7 MiB/s wr, 97 op/s 2023-11-06T01:34:13.799 INFO:teuthology.orchestra.run.smithi053.stderr:2300000 of 3200000 tuples (71%) done (elapsed 3.28 s, remaining 1.28 s) 2023-11-06T01:34:13.879 INFO:teuthology.orchestra.run.smithi053.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.36 s, remaining 1.12 s) 2023-11-06T01:34:14.062 INFO:teuthology.orchestra.run.smithi053.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.54 s, remaining 0.99 s) 2023-11-06T01:34:14.133 INFO:teuthology.orchestra.run.smithi053.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.61 s, remaining 0.83 s) 2023-11-06T01:34:14.315 INFO:teuthology.orchestra.run.smithi053.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.80 s, remaining 0.70 s) 2023-11-06T01:34:14.421 INFO:teuthology.orchestra.run.smithi053.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.90 s, remaining 0.56 s) 2023-11-06T01:34:14.586 INFO:teuthology.orchestra.run.smithi053.stderr:2900000 of 3200000 tuples (90%) done (elapsed 4.07 s, remaining 0.42 s) 2023-11-06T01:34:14.757 INFO:teuthology.orchestra.run.smithi053.stderr:3000000 of 3200000 tuples (93%) done (elapsed 4.24 s, remaining 0.28 s) 2023-11-06T01:34:14.856 INFO:teuthology.orchestra.run.smithi053.stderr:3100000 of 3200000 tuples (96%) done (elapsed 4.34 s, remaining 0.14 s) 2023-11-06T01:34:15.030 INFO:teuthology.orchestra.run.smithi053.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.51 s, remaining 0.00 s) 2023-11-06T01:34:15.066 INFO:teuthology.orchestra.run.smithi053.stderr:vacuum... 2023-11-06T01:34:15.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:15 smithi053 ceph-mon[99472]: pgmap v308: 97 pgs: 97 active+clean; 59 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 7.5 MiB/s wr, 111 op/s 2023-11-06T01:34:15.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:15 smithi062 ceph-mon[103924]: pgmap v308: 97 pgs: 97 active+clean; 59 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 7.5 MiB/s wr, 111 op/s 2023-11-06T01:34:15.731 INFO:teuthology.orchestra.run.smithi053.stderr:set primary keys... 2023-11-06T01:34:15.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:15 smithi163 ceph-mon[104416]: pgmap v308: 97 pgs: 97 active+clean; 59 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 7.5 MiB/s wr, 111 op/s 2023-11-06T01:34:16.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:16 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:16.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:16 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:16.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:16 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:17.104 INFO:teuthology.orchestra.run.smithi053.stderr:done. 2023-11-06T01:34:17.108 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-11-06T01:34:17.182 INFO:teuthology.orchestra.run.smithi053.stderr:starting vacuum...end. 2023-11-06T01:34:17.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:17 smithi053 ceph-mon[99472]: pgmap v309: 97 pgs: 97 active+clean; 243 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 38 MiB/s wr, 246 op/s 2023-11-06T01:34:17.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:17 smithi062 ceph-mon[103924]: pgmap v309: 97 pgs: 97 active+clean; 243 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 38 MiB/s wr, 246 op/s 2023-11-06T01:34:17.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:17 smithi163 ceph-mon[104416]: pgmap v309: 97 pgs: 97 active+clean; 243 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 38 MiB/s wr, 246 op/s 2023-11-06T01:34:19.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:19 smithi053 ceph-mon[99472]: pgmap v310: 97 pgs: 97 active+clean; 355 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 55 MiB/s wr, 287 op/s 2023-11-06T01:34:19.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:19 smithi062 ceph-mon[103924]: pgmap v310: 97 pgs: 97 active+clean; 355 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 55 MiB/s wr, 287 op/s 2023-11-06T01:34:19.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:19 smithi163 ceph-mon[104416]: pgmap v310: 97 pgs: 97 active+clean; 355 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 55 MiB/s wr, 287 op/s 2023-11-06T01:34:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:21 smithi053 ceph-mon[99472]: pgmap v311: 97 pgs: 97 active+clean; 520 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 67 MiB/s wr, 320 op/s 2023-11-06T01:34:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:21 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:21.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:21 smithi062 ceph-mon[103924]: pgmap v311: 97 pgs: 97 active+clean; 520 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 67 MiB/s wr, 320 op/s 2023-11-06T01:34:21.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:21 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:21.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:21 smithi163 ceph-mon[104416]: pgmap v311: 97 pgs: 97 active+clean; 520 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 67 MiB/s wr, 320 op/s 2023-11-06T01:34:21.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:21 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:22.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 5.0 s, 3729.6 tps, lat 26.335 ms stddev 32.205 2023-11-06T01:34:23.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:23 smithi062 ceph-mon[103924]: pgmap v312: 97 pgs: 97 active+clean; 882 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 87 MiB/s wr, 420 op/s 2023-11-06T01:34:23.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:23 smithi163 ceph-mon[104416]: pgmap v312: 97 pgs: 97 active+clean; 882 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 87 MiB/s wr, 420 op/s 2023-11-06T01:34:23.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:23 smithi053 ceph-mon[99472]: pgmap v312: 97 pgs: 97 active+clean; 882 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 87 MiB/s wr, 420 op/s 2023-11-06T01:34:25.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:25 smithi062 ceph-mon[103924]: pgmap v313: 97 pgs: 97 active+clean; 997 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 93 MiB/s wr, 451 op/s 2023-11-06T01:34:25.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:25 smithi163 ceph-mon[104416]: pgmap v313: 97 pgs: 97 active+clean; 997 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 93 MiB/s wr, 451 op/s 2023-11-06T01:34:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.818+0000 7f0fe3026700 1 -- 172.21.15.53:0/3965483774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc104b80 msgr2=0x7f0fdc10b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.818+0000 7f0fe3026700 1 --2- 172.21.15.53:0/3965483774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc104b80 0x7f0fdc10b430 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400b0d0 tx=0x7f0fd400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.819+0000 7f0fe3026700 1 -- 172.21.15.53:0/3965483774 shutdown_connections 2023-11-06T01:34:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.819+0000 7f0fe3026700 1 --2- 172.21.15.53:0/3965483774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fdc102ff0 0x7f0fdc103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.819+0000 7f0fe3026700 1 --2- 172.21.15.53:0/3965483774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc104b80 0x7f0fdc10b430 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.819+0000 7f0fe3026700 1 --2- 172.21.15.53:0/3965483774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fdc1041e0 0x7f0fdc104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.819+0000 7f0fe3026700 1 -- 172.21.15.53:0/3965483774 >> 172.21.15.53:0/3965483774 conn(0x7f0fdc0fe540 msgr2=0x7f0fdc1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.819+0000 7f0fe3026700 1 -- 172.21.15.53:0/3965483774 shutdown_connections 2023-11-06T01:34:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.819+0000 7f0fe3026700 1 -- 172.21.15.53:0/3965483774 wait complete. 2023-11-06T01:34:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.819+0000 7f0fe3026700 1 Processor -- start 2023-11-06T01:34:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.820+0000 7f0fe3026700 1 -- start start 2023-11-06T01:34:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe3026700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fdc102ff0 0x7f0fdc10fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe3026700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc1041e0 0x7f0fdc1100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe3026700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fdc104b80 0x7f0fdc110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe3026700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc111610 con 0x7f0fdc1041e0 2023-11-06T01:34:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe3026700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc111780 con 0x7f0fdc102ff0 2023-11-06T01:34:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe3026700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc111940 con 0x7f0fdc104b80 2023-11-06T01:34:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fd3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc1041e0 0x7f0fdc1100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe0dc2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fdc102ff0 0x7f0fdc10fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe0dc2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fdc102ff0 0x7f0fdc10fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51312/0 (socket says 172.21.15.53:51312) 2023-11-06T01:34:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fd3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc1041e0 0x7f0fdc1100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39874/0 (socket says 172.21.15.53:39874) 2023-11-06T01:34:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe0dc2700 1 -- 172.21.15.53:0/3629301138 learned_addr learned my addr 172.21.15.53:0/3629301138 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe15c3700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fdc104b80 0x7f0fdc110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe0dc2700 1 -- 172.21.15.53:0/3629301138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fdc104b80 msgr2=0x7f0fdc110fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe0dc2700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fdc104b80 0x7f0fdc110fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe0dc2700 1 -- 172.21.15.53:0/3629301138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc1041e0 msgr2=0x7f0fdc1100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe0dc2700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc1041e0 0x7f0fdc1100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.821+0000 7f0fe0dc2700 1 -- 172.21.15.53:0/3629301138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd4009d00 con 0x7f0fdc102ff0 2023-11-06T01:34:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.822+0000 7f0fe0dc2700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fdc102ff0 0x7f0fdc10fba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800e970 tx=0x7f0fd800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.822+0000 7f0fd1ffb700 1 -- 172.21.15.53:0/3629301138 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8009790 con 0x7f0fdc102ff0 2023-11-06T01:34:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.822+0000 7f0fd1ffb700 1 -- 172.21.15.53:0/3629301138 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fd800c7e0 con 0x7f0fdc102ff0 2023-11-06T01:34:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.822+0000 7f0fd1ffb700 1 -- 172.21.15.53:0/3629301138 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8003cb0 con 0x7f0fdc102ff0 2023-11-06T01:34:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.822+0000 7f0fe3026700 1 -- 172.21.15.53:0/3629301138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc111c20 con 0x7f0fdc102ff0 2023-11-06T01:34:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.822+0000 7f0fe3026700 1 -- 172.21.15.53:0/3629301138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc1143e0 con 0x7f0fdc102ff0 2023-11-06T01:34:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.825+0000 7f0fd1ffb700 1 -- 172.21.15.53:0/3629301138 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f0fd800c950 con 0x7f0fdc102ff0 2023-11-06T01:34:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.826+0000 7f0fd1ffb700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0fc40a0670 0x7f0fc40a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.826+0000 7f0fd1ffb700 1 -- 172.21.15.53:0/3629301138 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0fd8014070 con 0x7f0fdc102ff0 2023-11-06T01:34:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.827+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0fc40a0670 0x7f0fc40a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.828+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0fc40a0670 0x7f0fc40a2b30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc110bf0 tx=0x7f0fcc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.828+0000 7f0fef1b6b80 1 Processor -- start 2023-11-06T01:34:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.829+0000 7f0fef1b6b80 1 -- start start 2023-11-06T01:34:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.829+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6680 0x5597222a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.829+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x5597222a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.829+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 0x5597222a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.829+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597221fe9b0 con 0x5597222a7ed0 2023-11-06T01:34:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.829+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597221fc370 con 0x5597222a6fe0 2023-11-06T01:34:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.829+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597221f7c60 con 0x5597222a6680 2023-11-06T01:34:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.830+0000 7f0fe15c3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 0x5597222a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.830+0000 7f0fe0dc2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6680 0x5597222a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.830+0000 7f0fe15c3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 0x5597222a8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39882/0 (socket says 172.21.15.53:39882) 2023-11-06T01:34:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.830+0000 7f0fe15c3700 1 -- 172.21.15.53:0/3114136007 learned_addr learned my addr 172.21.15.53:0/3114136007 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.830+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/3114136007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x5597222a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.831+0000 7f0fd3fff700 1 -- 172.21.15.53:0/3114136007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6680 msgr2=0x5597222a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.831+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/3114136007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6680 0x5597222a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.831+0000 7f0fd3fff700 1 -- 172.21.15.53:0/3114136007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 msgr2=0x5597222a8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.831+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/3114136007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 0x5597222a8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.831+0000 7f0fd3fff700 1 -- 172.21.15.53:0/3114136007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559722025d80 con 0x5597222a6fe0 2023-11-06T01:34:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.831+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/3114136007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x5597222a78f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc005950 tx=0x7f0fcc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf9a96017d74f2a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.831+0000 7f0fc2ffd700 1 -- 172.21.15.53:0/3114136007 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc011bb0 con 0x5597222a6fe0 2023-11-06T01:34:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.831+0000 7f0fc2ffd700 1 -- 172.21.15.53:0/3114136007 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fcc015070 con 0x5597222a6fe0 2023-11-06T01:34:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.831+0000 7f0fc2ffd700 1 -- 172.21.15.53:0/3114136007 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc01c610 con 0x5597222a6fe0 2023-11-06T01:34:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.833+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3114136007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 msgr2=0x5597222a78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.833+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/3114136007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x5597222a78f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc005950 tx=0x7f0fcc013000 comp rx=0 tx=0).stop 2023-11-06T01:34:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.833+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3114136007 shutdown_connections 2023-11-06T01:34:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.833+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/3114136007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 0x5597222a8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.833+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/3114136007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6680 0x5597222a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.833+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/3114136007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x5597222a78f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.833+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3114136007 >> 172.21.15.53:0/3114136007 conn(0x559722206070 msgr2=0x5597222a3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.834+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3114136007 shutdown_connections 2023-11-06T01:34:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.834+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3114136007 wait complete. 2023-11-06T01:34:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.834+0000 7f0fef1b6b80 1 -- start start 2023-11-06T01:34:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.834+0000 7f0fef1b6b80 1 Processor -- start 2023-11-06T01:34:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6680 0x5597222b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6fe0 0x5597222b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 0x5597222b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597221944b0 con 0x5597222a7ed0 2023-11-06T01:34:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722194150 con 0x5597222a6680 2023-11-06T01:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972207b8c0 con 0x5597222a6fe0 2023-11-06T01:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fef1b6b80 10 client.0 _ll_register_callbacks cb 0x5597220e1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fd3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6fe0 0x5597222b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fe15c3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 0x5597222b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fd3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6fe0 0x5597222b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54130/0 (socket says 172.21.15.53:54130) 2023-11-06T01:34:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fe0dc2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6680 0x5597222b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fd3fff700 1 -- 172.21.15.53:0/317598739 learned_addr learned my addr 172.21.15.53:0/317598739 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fe15c3700 1 -- 172.21.15.53:0/317598739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6fe0 msgr2=0x5597222b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fe15c3700 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6fe0 0x5597222b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fe15c3700 1 -- 172.21.15.53:0/317598739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6680 msgr2=0x5597222b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fe15c3700 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6680 0x5597222b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fe15c3700 1 -- 172.21.15.53:0/317598739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc010c20 con 0x5597222a7ed0 2023-11-06T01:34:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6fe0 0x5597222b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.835+0000 7f0fe0dc2700 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6680 0x5597222b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.836+0000 7f0fe15c3700 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 0x5597222b7aa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400b770 tx=0x7f0fd4007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4526a91ac65682 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.836+0000 7f0fbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:34:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.837+0000 7f0fbbfff700 1 -- 172.21.15.53:0/317598739 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd400e040 con 0x5597222a7ed0 2023-11-06T01:34:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.837+0000 7f0fbbfff700 1 -- 172.21.15.53:0/317598739 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0fd4003e10 con 0x5597222a7ed0 2023-11-06T01:34:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.837+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/317598739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559722222cb0 con 0x5597222a7ed0 2023-11-06T01:34:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.838+0000 7f0fbbfff700 1 -- 172.21.15.53:0/317598739 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fd4005070 con 0x5597222a7ed0 2023-11-06T01:34:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.838+0000 7f0fef1b6b80 10 client.15492 fetch_fsmap learned FSMap version 31 2023-11-06T01:34:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.838+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/317598739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55972200ec00 con 0x5597222a7ed0 2023-11-06T01:34:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.838+0000 7f0fbbfff700 1 -- 172.21.15.53:0/317598739 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559722222cb0 con 0x5597222a7ed0 2023-11-06T01:34:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.838+0000 7f0fbbfff700 1 -- 172.21.15.53:0/317598739 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f0fd4050680 con 0x5597222a7ed0 2023-11-06T01:34:25.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.838+0000 7f0fef1b6b80 10 client.15492 fetch_fsmap finished waiting for FSMap version 31 2023-11-06T01:34:25.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.838+0000 7f0fef1b6b80 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:25.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.838+0000 7f0fef1b6b80 20 client.15492 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:25.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.838+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5597222bc3d0 0x5597222be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.838+0000 7f0fef1b6b80 4 client.15492 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.838+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/317598739 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559722275db0 con 0x5597222bc3d0 2023-11-06T01:34:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.839+0000 7f0fe0dc2700 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5597222bc3d0 0x5597222be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.839+0000 7f0fbbfff700 10 client.15492 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.839+0000 7f0fe0dc2700 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5597222bc3d0 0x5597222be7f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfcbd0673c920658 server_cookie=54edb2f72f4bbfc2 in_seq=0 out_seq=0 2023-11-06T01:34:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.840+0000 7f0fbbfff700 1 -- 172.21.15.53:0/317598739 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559722275db0 con 0x5597222bc3d0 2023-11-06T01:34:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.840+0000 7f0fbbfff700 10 client.15492 handle_command_reply: tid=0 2023-11-06T01:34:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.840+0000 7f0fef1b6b80 1 client.15492 shutdown 2023-11-06T01:34:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/317598739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 msgr2=0x5597222b7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 0x5597222b7aa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400b770 tx=0x7f0fd4007780 comp rx=0 tx=0).stop 2023-11-06T01:34:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/317598739 shutdown_connections 2023-11-06T01:34:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6680 0x5597222b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5597222bc3d0 0x5597222be7f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a7ed0 0x5597222b7aa0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fbbfff700 0 client.15492 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/317598739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a6fe0 0x5597222b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/317598739 >> 172.21.15.53:0/317598739 conn(0x559722206070 msgr2=0x5597222a13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/317598739 shutdown_connections 2023-11-06T01:34:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/317598739 wait complete. 2023-11-06T01:34:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.841+0000 7f0fef1b6b80 20 client.15492 trim_cache size 0 max 16384 2023-11-06T01:34:25.856 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:25 smithi053 ceph-mon[99472]: pgmap v313: 97 pgs: 97 active+clean; 997 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 93 MiB/s wr, 451 op/s 2023-11-06T01:34:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fef1b6b80 1 Processor -- start 2023-11-06T01:34:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fef1b6b80 1 -- start start 2023-11-06T01:34:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 0x5597222b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x5597222ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a7ed0 0x5597222a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597222a26e0 con 0x5597222a6680 2023-11-06T01:34:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597222997a0 con 0x5597222a6fe0 2023-11-06T01:34:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597222af9d0 con 0x5597222a7ed0 2023-11-06T01:34:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fe0dc2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 0x5597222b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fe0dc2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 0x5597222b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39896/0 (socket says 172.21.15.53:39896) 2023-11-06T01:34:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fe0dc2700 1 -- 172.21.15.53:0/2002220884 learned_addr learned my addr 172.21.15.53:0/2002220884 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/2002220884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x5597222ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.865+0000 7f0fe15c3700 1 --2- 172.21.15.53:0/2002220884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a7ed0 0x5597222a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.866+0000 7f0fd3fff700 1 -- 172.21.15.53:0/2002220884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a7ed0 msgr2=0x5597222a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.866+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/2002220884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a7ed0 0x5597222a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.866+0000 7f0fd3fff700 1 -- 172.21.15.53:0/2002220884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 msgr2=0x5597222b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.866+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/2002220884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 0x5597222b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.866+0000 7f0fd3fff700 1 -- 172.21.15.53:0/2002220884 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55972204c1f0 con 0x5597222a6fe0 2023-11-06T01:34:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.866+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/2002220884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x5597222ae2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc005d90 tx=0x7f0fcc00ba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4960a8f2ac663f76 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.866+0000 7f0fc37fe700 1 -- 172.21.15.53:0/2002220884 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc02b040 con 0x5597222a6fe0 2023-11-06T01:34:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.866+0000 7f0fc37fe700 1 -- 172.21.15.53:0/2002220884 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fcc0054f0 con 0x5597222a6fe0 2023-11-06T01:34:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.866+0000 7f0fc37fe700 1 -- 172.21.15.53:0/2002220884 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc004bd0 con 0x5597222a6fe0 2023-11-06T01:34:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.867+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/2002220884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 msgr2=0x5597222ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.867+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/2002220884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x5597222ae2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc005d90 tx=0x7f0fcc00ba80 comp rx=0 tx=0).stop 2023-11-06T01:34:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.867+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/2002220884 shutdown_connections 2023-11-06T01:34:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.867+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/2002220884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a7ed0 0x5597222a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.867+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/2002220884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 0x5597222b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.867+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/2002220884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x5597222ae2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.867+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/2002220884 >> 172.21.15.53:0/2002220884 conn(0x559722206070 msgr2=0x5597222af170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.868+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/2002220884 shutdown_connections 2023-11-06T01:34:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.868+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/2002220884 wait complete. 2023-11-06T01:34:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.868+0000 7f0fef1b6b80 1 -- start start 2023-11-06T01:34:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fef1b6b80 1 Processor -- start 2023-11-06T01:34:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 0x559722307eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x55972230a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fef1b6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a7ed0 0x55972230ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972230d180 con 0x5597222a6680 2023-11-06T01:34:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972230d2f0 con 0x5597222a6fe0 2023-11-06T01:34:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fef1b6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972230d4b0 con 0x5597222a7ed0 2023-11-06T01:34:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fef1b6b80 10 client.0 _ll_register_callbacks cb 0x7f0fd400bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fe0dc2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 0x559722307eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fe0dc2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 0x559722307eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39902/0 (socket says 172.21.15.53:39902) 2023-11-06T01:34:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fe0dc2700 1 -- 172.21.15.53:0/3016868624 learned_addr learned my addr 172.21.15.53:0/3016868624 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x55972230a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.869+0000 7f0fe15c3700 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a7ed0 0x55972230ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.870+0000 7f0fe0dc2700 1 -- 172.21.15.53:0/3016868624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a7ed0 msgr2=0x55972230ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.870+0000 7f0fe0dc2700 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a7ed0 0x55972230ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.870+0000 7f0fe0dc2700 1 -- 172.21.15.53:0/3016868624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 msgr2=0x55972230a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.870+0000 7f0fe0dc2700 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x55972230a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.870+0000 7f0fe0dc2700 1 -- 172.21.15.53:0/3016868624 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559722025d80 con 0x5597222a6680 2023-11-06T01:34:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.870+0000 7f0fe15c3700 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a7ed0 0x55972230ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:34:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.870+0000 7f0fe0dc2700 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 0x559722307eb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80992e0 tx=0x7f0fd8003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ef1c69d4a8c9b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.870+0000 7f0fbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:34:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.870+0000 7f0fbbfff700 1 -- 172.21.15.53:0/3016868624 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd80b7300 con 0x5597222a6680 2023-11-06T01:34:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.871+0000 7f0fbbfff700 1 -- 172.21.15.53:0/3016868624 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0fd80a13c0 con 0x5597222a6680 2023-11-06T01:34:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.871+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3016868624 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fcc015070 con 0x5597222a6680 2023-11-06T01:34:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.871+0000 7f0fbbfff700 1 -- 172.21.15.53:0/3016868624 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fd80c9700 con 0x5597222a6680 2023-11-06T01:34:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.871+0000 7f0fef1b6b80 10 client.15498 fetch_fsmap learned FSMap version 31 2023-11-06T01:34:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.871+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3016868624 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559722024250 con 0x5597222a6680 2023-11-06T01:34:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.871+0000 7f0fbbfff700 1 -- 172.21.15.53:0/3016868624 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fcc015070 con 0x5597222a6680 2023-11-06T01:34:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.872+0000 7f0fbbfff700 1 -- 172.21.15.53:0/3016868624 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f0fd80a0f30 con 0x5597222a6680 2023-11-06T01:34:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.872+0000 7f0fef1b6b80 10 client.15498 fetch_fsmap finished waiting for FSMap version 31 2023-11-06T01:34:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.872+0000 7f0fef1b6b80 10 client.15498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.872+0000 7f0fef1b6b80 20 client.15498 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.872+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55972230eeb0 0x5597223112d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.873+0000 7f0fef1b6b80 4 client.15498 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:34:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.873+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3016868624 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5597221f0ce0 con 0x55972230eeb0 2023-11-06T01:34:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.873+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55972230eeb0 0x5597223112d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.875+0000 7f0fbbfff700 10 client.15498 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.875+0000 7f0fd3fff700 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55972230eeb0 0x5597223112d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12acc73f4fe76baa server_cookie=7242fc3ace70d781 in_seq=0 out_seq=0 2023-11-06T01:34:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fbbfff700 1 -- 172.21.15.53:0/3016868624 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5597221f0ce0 con 0x55972230eeb0 2023-11-06T01:34:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fbbfff700 10 client.15498 handle_command_reply: tid=0 2023-11-06T01:34:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fef1b6b80 1 client.15498 shutdown 2023-11-06T01:34:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3016868624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 msgr2=0x559722307eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 0x559722307eb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80992e0 tx=0x7f0fd8003710 comp rx=0 tx=0).stop 2023-11-06T01:34:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3016868624 shutdown_connections 2023-11-06T01:34:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55972230eeb0 0x5597223112d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5597222a7ed0 0x55972230ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597222a6680 0x559722307eb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fbbfff700 0 client.15498 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fef1b6b80 1 --2- 172.21.15.53:0/3016868624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597222a6fe0 0x55972230a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.896+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3016868624 >> 172.21.15.53:0/3016868624 conn(0x559722206070 msgr2=0x5597222ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.897+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3016868624 shutdown_connections 2023-11-06T01:34:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.897+0000 7f0fef1b6b80 1 -- 172.21.15.53:0/3016868624 wait complete. 2023-11-06T01:34:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.897+0000 7f0fef1b6b80 20 client.15498 trim_cache size 0 max 16384 2023-11-06T01:34:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:34:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:34:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:34:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:34:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.898+0000 7f0fe3026700 1 -- 172.21.15.53:0/3629301138 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0fc40a0670 msgr2=0x7f0fc40a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.898+0000 7f0fe3026700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0fc40a0670 0x7f0fc40a2b30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc110bf0 tx=0x7f0fcc00b410 comp rx=0 tx=0).stop 2023-11-06T01:34:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.898+0000 7f0fe3026700 1 -- 172.21.15.53:0/3629301138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fdc102ff0 msgr2=0x7f0fdc10fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.898+0000 7f0fe3026700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fdc102ff0 0x7f0fdc10fba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800e970 tx=0x7f0fd800ec80 comp rx=0 tx=0).stop 2023-11-06T01:34:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.900+0000 7f0fe3026700 1 -- 172.21.15.53:0/3629301138 shutdown_connections 2023-11-06T01:34:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.900+0000 7f0fe3026700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0fc40a0670 0x7f0fc40a2b30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.900+0000 7f0fe3026700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fdc102ff0 0x7f0fdc10fba0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.900+0000 7f0fe3026700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fdc104b80 0x7f0fdc110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.900+0000 7f0fe3026700 1 --2- 172.21.15.53:0/3629301138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc1041e0 0x7f0fdc1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.900+0000 7f0fe3026700 1 -- 172.21.15.53:0/3629301138 >> 172.21.15.53:0/3629301138 conn(0x7f0fdc0fe540 msgr2=0x7f0fdc0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.900+0000 7f0fe3026700 1 -- 172.21.15.53:0/3629301138 shutdown_connections 2023-11-06T01:34:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:25.900+0000 7f0fe3026700 1 -- 172.21.15.53:0/3629301138 wait complete. 2023-11-06T01:34:26.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:26 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:26.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:26 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:26.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:26 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:27.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 10.0 s, 4150.2 tps, lat 24.142 ms stddev 35.241 2023-11-06T01:34:27.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:27 smithi062 ceph-mon[103924]: pgmap v314: 97 pgs: 97 active+clean; 1.1 GiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 130 MiB/s wr, 881 op/s 2023-11-06T01:34:27.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:27 smithi163 ceph-mon[104416]: pgmap v314: 97 pgs: 97 active+clean; 1.1 GiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 130 MiB/s wr, 881 op/s 2023-11-06T01:34:27.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:27 smithi053 ceph-mon[99472]: pgmap v314: 97 pgs: 97 active+clean; 1.1 GiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 130 MiB/s wr, 881 op/s 2023-11-06T01:34:29.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:29 smithi062 ceph-mon[103924]: pgmap v315: 97 pgs: 97 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 115 MiB/s wr, 935 op/s 2023-11-06T01:34:29.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:29 smithi163 ceph-mon[104416]: pgmap v315: 97 pgs: 97 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 115 MiB/s wr, 935 op/s 2023-11-06T01:34:29.749 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:34:29.790 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:29 smithi053 ceph-mon[99472]: pgmap v315: 97 pgs: 97 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 115 MiB/s wr, 935 op/s 2023-11-06T01:34:29.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.962+0000 7faebf28f700 1 -- 172.21.15.53:0/3245213143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb8107d90 msgr2=0x7faeb810a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:29.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.962+0000 7faebf28f700 1 --2- 172.21.15.53:0/3245213143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb8107d90 0x7faeb810a1c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faeb40098f0 tx=0x7faeb4009c00 comp rx=0 tx=0).stop 2023-11-06T01:34:29.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.963+0000 7faebf28f700 1 -- 172.21.15.53:0/3245213143 shutdown_connections 2023-11-06T01:34:29.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.963+0000 7faebf28f700 1 --2- 172.21.15.53:0/3245213143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb8107d90 0x7faeb810a1c0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faeb40098f0 tx=0x7faeb4009c00 comp rx=0 tx=0).stop 2023-11-06T01:34:29.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.963+0000 7faebf28f700 1 --2- 172.21.15.53:0/3245213143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb810d0d0 0x7faeb810d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.963+0000 7faebf28f700 1 --2- 172.21.15.53:0/3245213143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb810a700 0x7faeb810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.963+0000 7faebf28f700 1 -- 172.21.15.53:0/3245213143 >> 172.21.15.53:0/3245213143 conn(0x7faeb806db40 msgr2=0x7faeb806ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:29.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.963+0000 7faebf28f700 1 -- 172.21.15.53:0/3245213143 shutdown_connections 2023-11-06T01:34:29.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.963+0000 7faebf28f700 1 -- 172.21.15.53:0/3245213143 wait complete. 2023-11-06T01:34:29.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.964+0000 7faebf28f700 1 Processor -- start 2023-11-06T01:34:29.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebf28f700 1 -- start start 2023-11-06T01:34:29.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebf28f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb810a700 0x7faeb81175c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:29.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebf28f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb810d0d0 0x7faeb8117b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:29.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebf28f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8118b40 0x7faeb8118fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:29.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebf28f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8119500 con 0x7faeb8118b40 2023-11-06T01:34:29.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebf28f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8119670 con 0x7faeb810d0d0 2023-11-06T01:34:29.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebf28f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8119830 con 0x7faeb810a700 2023-11-06T01:34:29.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebd02b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb810a700 0x7faeb81175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:29.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebd02b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb810a700 0x7faeb81175c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54156/0 (socket says 172.21.15.53:54156) 2023-11-06T01:34:29.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebd02b700 1 -- 172.21.15.53:0/2044443562 learned_addr learned my addr 172.21.15.53:0/2044443562 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:29.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebd82c700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8118b40 0x7faeb8118fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:29.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebd02b700 1 -- 172.21.15.53:0/2044443562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb810d0d0 msgr2=0x7faeb8117b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-06T01:34:29.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.965+0000 7faebd02b700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb810d0d0 0x7faeb8117b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.966+0000 7faebd02b700 1 -- 172.21.15.53:0/2044443562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8118b40 msgr2=0x7faeb8118fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:29.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.966+0000 7faebd02b700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8118b40 0x7faeb8118fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.966+0000 7faebd02b700 1 -- 172.21.15.53:0/2044443562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb4009520 con 0x7faeb810a700 2023-11-06T01:34:29.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.966+0000 7faebd02b700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb810a700 0x7faeb81175c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009810 tx=0x7faeb400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:29.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.966+0000 7faeae7fc700 1 -- 172.21.15.53:0/2044443562 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4005240 con 0x7faeb810a700 2023-11-06T01:34:29.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.966+0000 7faeae7fc700 1 -- 172.21.15.53:0/2044443562 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faeb400bbc0 con 0x7faeb810a700 2023-11-06T01:34:29.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.966+0000 7faebf28f700 1 -- 172.21.15.53:0/2044443562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb8119ab0 con 0x7faeb810a700 2023-11-06T01:34:29.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.966+0000 7faebf28f700 1 -- 172.21.15.53:0/2044443562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb81b6c70 con 0x7faeb810a700 2023-11-06T01:34:29.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.967+0000 7faeae7fc700 1 -- 172.21.15.53:0/2044443562 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4021440 con 0x7faeb810a700 2023-11-06T01:34:29.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.972+0000 7faeae7fc700 1 -- 172.21.15.53:0/2044443562 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7faeb4003680 con 0x7faeb810a700 2023-11-06T01:34:29.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.973+0000 7faecb41fb80 1 Processor -- start 2023-11-06T01:34:29.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.973+0000 7faeae7fc700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faea40a06a0 0x7faea40a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:29.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.973+0000 7faeae7fc700 1 -- 172.21.15.53:0/2044443562 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faeb40cca50 con 0x7faeb810a700 2023-11-06T01:34:29.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.974+0000 7faecb41fb80 1 -- start start 2023-11-06T01:34:29.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.974+0000 7faebd82c700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faea40a06a0 0x7faea40a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:29.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.974+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f38830f0 0x5569f3883530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:29.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.974+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f3883a70 0x5569f3881e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:29.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.974+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f3882870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:29.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.974+0000 7faecb41fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f37d9a10 con 0x5569f38830f0 2023-11-06T01:34:29.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.974+0000 7faecb41fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f37d73d0 con 0x5569f3883a70 2023-11-06T01:34:29.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.974+0000 7faecb41fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f37d2cc0 con 0x5569f38823f0 2023-11-06T01:34:29.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.974+0000 7faebc82a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f3883a70 0x5569f3881e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:29.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.974+0000 7faebc82a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f3883a70 0x5569f3881e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51366/0 (socket says 172.21.15.53:51366) 2023-11-06T01:34:29.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.974+0000 7faebc82a700 1 -- 172.21.15.53:0/3754580187 learned_addr learned my addr 172.21.15.53:0/3754580187 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:29.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faebd02b700 1 --2- 172.21.15.53:0/3754580187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f38830f0 0x5569f3883530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:29.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faebd82c700 1 --2- 172.21.15.53:0/3754580187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f3882870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:29.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faebc82a700 1 -- 172.21.15.53:0/3754580187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 msgr2=0x5569f3882870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:29.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faebc82a700 1 --2- 172.21.15.53:0/3754580187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f3882870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faebc82a700 1 -- 172.21.15.53:0/3754580187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f38830f0 msgr2=0x5569f3883530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:29.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faebc82a700 1 --2- 172.21.15.53:0/3754580187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f38830f0 0x5569f3883530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faebc82a700 1 -- 172.21.15.53:0/3754580187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569f3600de0 con 0x5569f3883a70 2023-11-06T01:34:29.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faebd82c700 1 --2- 172.21.15.53:0/3754580187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f3882870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:34:29.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faebc82a700 1 --2- 172.21.15.53:0/3754580187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f3883a70 0x5569f3881e60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faea800bb10 tx=0x7faea800be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7445e35b5fc4ae9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:29.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faebd82c700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faea40a06a0 0x7faea40a2b60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faeb000f590 tx=0x7faeb00099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:29.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faea37fe700 1 -- 172.21.15.53:0/3754580187 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea800da90 con 0x5569f3883a70 2023-11-06T01:34:29.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faea37fe700 1 -- 172.21.15.53:0/3754580187 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faea8004cf0 con 0x5569f3883a70 2023-11-06T01:34:29.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.975+0000 7faea37fe700 1 -- 172.21.15.53:0/3754580187 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8005690 con 0x5569f3883a70 2023-11-06T01:34:29.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.976+0000 7faecb41fb80 1 -- 172.21.15.53:0/3754580187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f3883a70 msgr2=0x5569f3881e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:29.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.976+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3754580187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f3883a70 0x5569f3881e60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faea800bb10 tx=0x7faea800be20 comp rx=0 tx=0).stop 2023-11-06T01:34:29.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.976+0000 7faecb41fb80 1 -- 172.21.15.53:0/3754580187 shutdown_connections 2023-11-06T01:34:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.976+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3754580187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f3882870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.976+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3754580187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f38830f0 0x5569f3883530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.976+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3754580187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f3883a70 0x5569f3881e60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.976+0000 7faecb41fb80 1 -- 172.21.15.53:0/3754580187 >> 172.21.15.53:0/3754580187 conn(0x5569f37e10d0 msgr2=0x5569f387c940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.977+0000 7faecb41fb80 1 -- 172.21.15.53:0/3754580187 shutdown_connections 2023-11-06T01:34:29.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.977+0000 7faecb41fb80 1 -- 172.21.15.53:0/3754580187 wait complete. 2023-11-06T01:34:29.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.977+0000 7faecb41fb80 1 -- start start 2023-11-06T01:34:29.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.978+0000 7faecb41fb80 1 Processor -- start 2023-11-06T01:34:29.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.978+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f388b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:29.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.978+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38830f0 0x5569f388bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:29.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.978+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 0x5569f3892f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:29.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.978+0000 7faecb41fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f376f510 con 0x5569f3883a70 2023-11-06T01:34:29.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.978+0000 7faecb41fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f376f1b0 con 0x5569f38830f0 2023-11-06T01:34:29.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.978+0000 7faecb41fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f3656920 con 0x5569f38823f0 2023-11-06T01:34:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.978+0000 7faecb41fb80 10 client.0 _ll_register_callbacks cb 0x5569f36bca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faebd02b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f388b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faebd02b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f388b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54172/0 (socket says 172.21.15.53:54172) 2023-11-06T01:34:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faebd02b700 1 -- 172.21.15.53:0/3374004499 learned_addr learned my addr 172.21.15.53:0/3374004499 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faebd82c700 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 0x5569f3892f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faebc82a700 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38830f0 0x5569f388bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faebd02b700 1 -- 172.21.15.53:0/3374004499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38830f0 msgr2=0x5569f388bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faebd02b700 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38830f0 0x5569f388bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faebd02b700 1 -- 172.21.15.53:0/3374004499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 msgr2=0x5569f3892f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faebd02b700 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 0x5569f3892f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faebd02b700 1 -- 172.21.15.53:0/3374004499 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea800b740 con 0x5569f38823f0 2023-11-06T01:34:29.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faebd02b700 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f388b9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faeb4096820 tx=0x7faeb409aa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb70953b6b283a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:29.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.979+0000 7faea0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:34:29.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.980+0000 7faea0ff9700 1 -- 172.21.15.53:0/3374004499 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb409a780 con 0x5569f38823f0 2023-11-06T01:34:29.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.980+0000 7faecb41fb80 1 -- 172.21.15.53:0/3374004499 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569f37fdd10 con 0x5569f38823f0 2023-11-06T01:34:29.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.980+0000 7faea0ff9700 1 -- 172.21.15.53:0/3374004499 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faeb40a2300 con 0x5569f38823f0 2023-11-06T01:34:29.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.980+0000 7faea0ff9700 1 -- 172.21.15.53:0/3374004499 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7faeb40b8240 con 0x5569f38823f0 2023-11-06T01:34:29.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.980+0000 7faecb41fb80 10 client.25165 fetch_fsmap learned FSMap version 31 2023-11-06T01:34:29.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.980+0000 7faecb41fb80 1 -- 172.21.15.53:0/3374004499 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5569f35e9c60 con 0x5569f38823f0 2023-11-06T01:34:29.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.981+0000 7faea0ff9700 1 -- 172.21.15.53:0/3374004499 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5569f37fdd10 con 0x5569f38823f0 2023-11-06T01:34:29.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.981+0000 7faea0ff9700 1 -- 172.21.15.53:0/3374004499 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) v1 ==== 7474+0+0 (secure 0 0 0) 0x7faeb40053a0 con 0x5569f38823f0 2023-11-06T01:34:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.982+0000 7faecb41fb80 10 client.25165 fetch_fsmap finished waiting for FSMap version 31 2023-11-06T01:34:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.982+0000 7faecb41fb80 10 client.25165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.982+0000 7faecb41fb80 20 client.25165 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.982+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569f3897430 0x5569f3899850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.982+0000 7faecb41fb80 4 client.25165 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:29.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.982+0000 7faecb41fb80 1 -- 172.21.15.53:0/3374004499 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569f3850e10 con 0x5569f3897430 2023-11-06T01:34:29.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.982+0000 7faebc82a700 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569f3897430 0x5569f3899850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:29.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.983+0000 7faebc82a700 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569f3897430 0x5569f3899850 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=980ef5cc1c1cd87c server_cookie=128dd55ff130226f in_seq=0 out_seq=0 2023-11-06T01:34:29.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.983+0000 7faea0ff9700 10 client.25165 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:29.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faea0ff9700 1 -- 172.21.15.53:0/3374004499 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5569f3850e10 con 0x5569f3897430 2023-11-06T01:34:29.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faea0ff9700 10 client.25165 handle_command_reply: tid=0 2023-11-06T01:34:29.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 1 client.25165 shutdown 2023-11-06T01:34:29.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 1 -- 172.21.15.53:0/3374004499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 msgr2=0x5569f388b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:29.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f388b9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faeb4096820 tx=0x7faeb409aa20 comp rx=0 tx=0).stop 2023-11-06T01:34:29.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 1 -- 172.21.15.53:0/3374004499 shutdown_connections 2023-11-06T01:34:29.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569f3897430 0x5569f3899850 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 0x5569f3892f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faea0ff9700 0 client.25165 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:29.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f388b9a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3374004499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38830f0 0x5569f388bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:29.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 1 -- 172.21.15.53:0/3374004499 >> 172.21.15.53:0/3374004499 conn(0x5569f37e10d0 msgr2=0x5569f387c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:29.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 1 -- 172.21.15.53:0/3374004499 shutdown_connections 2023-11-06T01:34:29.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 1 -- 172.21.15.53:0/3374004499 wait complete. 2023-11-06T01:34:29.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:29.984+0000 7faecb41fb80 20 client.25165 trim_cache size 0 max 16384 2023-11-06T01:34:30.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.008+0000 7faecb41fb80 1 Processor -- start 2023-11-06T01:34:30.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.008+0000 7faecb41fb80 1 -- start start 2023-11-06T01:34:30.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.008+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f3881e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:30.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.008+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38830f0 0x5569f3890ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:30.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.008+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 0x5569f388b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:30.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.009+0000 7faecb41fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f388b560 con 0x5569f3883a70 2023-11-06T01:34:30.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.009+0000 7faecb41fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f3874b80 con 0x5569f38830f0 2023-11-06T01:34:30.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.009+0000 7faecb41fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f388cb50 con 0x5569f38823f0 2023-11-06T01:34:30.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.011+0000 7faebd02b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f3881e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:30.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.011+0000 7faebd02b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f3881e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54178/0 (socket says 172.21.15.53:54178) 2023-11-06T01:34:30.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.011+0000 7faebd02b700 1 -- 172.21.15.53:0/3145422550 learned_addr learned my addr 172.21.15.53:0/3145422550 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:30.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.014+0000 7faebc82a700 1 --2- 172.21.15.53:0/3145422550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38830f0 0x5569f3890ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:30.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.017+0000 7faebc82a700 1 -- 172.21.15.53:0/3145422550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 msgr2=0x5569f3881e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:30.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.017+0000 7faebc82a700 1 --2- 172.21.15.53:0/3145422550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f3881e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.017+0000 7faebc82a700 1 -- 172.21.15.53:0/3145422550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 msgr2=0x5569f388b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:30.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.017+0000 7faebc82a700 1 --2- 172.21.15.53:0/3145422550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 0x5569f388b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.017+0000 7faebc82a700 1 -- 172.21.15.53:0/3145422550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569f3627250 con 0x5569f38830f0 2023-11-06T01:34:30.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.017+0000 7faebc82a700 1 --2- 172.21.15.53:0/3145422550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38830f0 0x5569f3890ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faea800d5c0 tx=0x7faea8005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f2b277717adfddb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:30.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.020+0000 7faea3fff700 1 -- 172.21.15.53:0/3145422550 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8020040 con 0x5569f38830f0 2023-11-06T01:34:30.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.020+0000 7faea3fff700 1 -- 172.21.15.53:0/3145422550 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faea8014070 con 0x5569f38830f0 2023-11-06T01:34:30.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.020+0000 7faea3fff700 1 -- 172.21.15.53:0/3145422550 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea80245b0 con 0x5569f38830f0 2023-11-06T01:34:30.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.021+0000 7faecb41fb80 1 -- 172.21.15.53:0/3145422550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38830f0 msgr2=0x5569f3890ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:30.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.021+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3145422550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38830f0 0x5569f3890ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faea800d5c0 tx=0x7faea8005f00 comp rx=0 tx=0).stop 2023-11-06T01:34:30.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.021+0000 7faecb41fb80 1 -- 172.21.15.53:0/3145422550 shutdown_connections 2023-11-06T01:34:30.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.021+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3145422550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 0x5569f388b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.021+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3145422550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38823f0 0x5569f3881e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.021+0000 7faecb41fb80 1 --2- 172.21.15.53:0/3145422550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38830f0 0x5569f3890ed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.021+0000 7faecb41fb80 1 -- 172.21.15.53:0/3145422550 >> 172.21.15.53:0/3145422550 conn(0x5569f37e10d0 msgr2=0x5569f38862d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:30.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.023+0000 7faecb41fb80 1 -- 172.21.15.53:0/3145422550 shutdown_connections 2023-11-06T01:34:30.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.024+0000 7faecb41fb80 1 -- 172.21.15.53:0/3145422550 wait complete. 2023-11-06T01:34:30.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.024+0000 7faecb41fb80 1 -- start start 2023-11-06T01:34:30.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.024+0000 7faecb41fb80 1 Processor -- start 2023-11-06T01:34:30.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.029+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38823f0 0x5569f3890d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:30.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.029+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38830f0 0x5569f388f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:30.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.029+0000 7faecb41fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 0x5569f388f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:30.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.029+0000 7faecb41fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f388fe80 con 0x5569f3883a70 2023-11-06T01:34:30.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.029+0000 7faecb41fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f388fff0 con 0x5569f38823f0 2023-11-06T01:34:30.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.029+0000 7faecb41fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569f38901b0 con 0x5569f38830f0 2023-11-06T01:34:30.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.029+0000 7faecb41fb80 10 client.0 _ll_register_callbacks cb 0x5569f36bca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:30.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.029+0000 7faebd02b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38823f0 0x5569f3890d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:30.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.029+0000 7faebd02b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38823f0 0x5569f3890d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51382/0 (socket says 172.21.15.53:51382) 2023-11-06T01:34:30.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.029+0000 7faebd02b700 1 -- 172.21.15.53:0/569102818 learned_addr learned my addr 172.21.15.53:0/569102818 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:30.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.030+0000 7faebd82c700 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 0x5569f388f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:30.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.030+0000 7faebc82a700 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38830f0 0x5569f388f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:30.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.030+0000 7faebc82a700 1 -- 172.21.15.53:0/569102818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38823f0 msgr2=0x5569f3890d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:30.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.030+0000 7faebc82a700 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38823f0 0x5569f3890d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.030+0000 7faebc82a700 1 -- 172.21.15.53:0/569102818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 msgr2=0x5569f388f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:30.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.030+0000 7faebc82a700 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 0x5569f388f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.030+0000 7faebc82a700 1 -- 172.21.15.53:0/569102818 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569f3600de0 con 0x5569f38830f0 2023-11-06T01:34:30.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.031+0000 7faebc82a700 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38830f0 0x5569f388f320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5569f38823c0 tx=0x7faea8005170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0494d2c97b16a17 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:30.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.031+0000 7faea0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:34:30.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.031+0000 7faea0ff9700 1 -- 172.21.15.53:0/569102818 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8020070 con 0x5569f38830f0 2023-11-06T01:34:30.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.031+0000 7faea0ff9700 1 -- 172.21.15.53:0/569102818 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faea800dca0 con 0x5569f38830f0 2023-11-06T01:34:30.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.031+0000 7faecb41fb80 1 -- 172.21.15.53:0/569102818 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faea8004cf0 con 0x5569f38830f0 2023-11-06T01:34:30.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.031+0000 7faea0ff9700 1 -- 172.21.15.53:0/569102818 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7faea8005420 con 0x5569f38830f0 2023-11-06T01:34:30.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.031+0000 7faecb41fb80 10 client.25171 fetch_fsmap learned FSMap version 31 2023-11-06T01:34:30.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.031+0000 7faecb41fb80 1 -- 172.21.15.53:0/569102818 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5569f35ff2b0 con 0x5569f38830f0 2023-11-06T01:34:30.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.033+0000 7faea0ff9700 1 -- 172.21.15.53:0/569102818 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faea8004cf0 con 0x5569f38830f0 2023-11-06T01:34:30.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.033+0000 7faea0ff9700 1 -- 172.21.15.53:0/569102818 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) v1 ==== 7474+0+0 (secure 0 0 0) 0x7faea8023070 con 0x5569f38830f0 2023-11-06T01:34:30.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.033+0000 7faecb41fb80 10 client.25171 fetch_fsmap finished waiting for FSMap version 31 2023-11-06T01:34:30.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.033+0000 7faecb41fb80 10 client.25171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:30.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.033+0000 7faecb41fb80 20 client.25171 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:30.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.033+0000 7faecb41fb80 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569f38ea560 0x5569f38ec980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:30.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.033+0000 7faecb41fb80 4 client.25171 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:34:30.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.033+0000 7faecb41fb80 1 -- 172.21.15.53:0/569102818 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5569f37cbd40 con 0x5569f38ea560 2023-11-06T01:34:30.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.034+0000 7faebd02b700 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569f38ea560 0x5569f38ec980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:30.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.034+0000 7faea0ff9700 10 client.25171 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:30.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.034+0000 7faebd02b700 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569f38ea560 0x5569f38ec980 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b3e7c005984773e server_cookie=b9734a9b9a84309 in_seq=0 out_seq=0 2023-11-06T01:34:30.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faea0ff9700 1 -- 172.21.15.53:0/569102818 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5569f37cbd40 con 0x5569f38ea560 2023-11-06T01:34:30.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faea0ff9700 10 client.25171 handle_command_reply: tid=0 2023-11-06T01:34:30.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faecb41fb80 1 client.25171 shutdown 2023-11-06T01:34:30.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faecb41fb80 1 -- 172.21.15.53:0/569102818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38830f0 msgr2=0x5569f388f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:30.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faecb41fb80 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38830f0 0x5569f388f320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5569f38823c0 tx=0x7faea8005170 comp rx=0 tx=0).stop 2023-11-06T01:34:30.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faecb41fb80 1 -- 172.21.15.53:0/569102818 shutdown_connections 2023-11-06T01:34:30.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faecb41fb80 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569f38823f0 0x5569f3890d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faecb41fb80 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569f38ea560 0x5569f38ec980 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faecb41fb80 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569f3883a70 0x5569f388f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faecb41fb80 1 --2- 172.21.15.53:0/569102818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569f38830f0 0x5569f388f320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faecb41fb80 1 -- 172.21.15.53:0/569102818 >> 172.21.15.53:0/569102818 conn(0x5569f37e10d0 msgr2=0x5569f388b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:30.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faea0ff9700 0 client.25171 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:30.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faecb41fb80 1 -- 172.21.15.53:0/569102818 shutdown_connections 2023-11-06T01:34:30.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.035+0000 7faecb41fb80 1 -- 172.21.15.53:0/569102818 wait complete. 2023-11-06T01:34:30.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.036+0000 7faecb41fb80 20 client.25171 trim_cache size 0 max 16384 2023-11-06T01:34:30.044 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:34:30.044 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2023-11-06T01:34:30.045 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2023-11-06T01:34:30.045 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:34:30.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.037+0000 7faebf28f700 1 -- 172.21.15.53:0/2044443562 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faea40a06a0 msgr2=0x7faea40a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:30.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.037+0000 7faebf28f700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faea40a06a0 0x7faea40a2b60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faeb000f590 tx=0x7faeb00099e0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.037+0000 7faebf28f700 1 -- 172.21.15.53:0/2044443562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb810a700 msgr2=0x7faeb81175c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:30.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.037+0000 7faebf28f700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb810a700 0x7faeb81175c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009810 tx=0x7faeb400b790 comp rx=0 tx=0).stop 2023-11-06T01:34:30.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.038+0000 7faebf28f700 1 -- 172.21.15.53:0/2044443562 shutdown_connections 2023-11-06T01:34:30.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.038+0000 7faebf28f700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7faea40a06a0 0x7faea40a2b60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.038+0000 7faebf28f700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8118b40 0x7faeb8118fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.038+0000 7faebf28f700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb810a700 0x7faeb81175c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.038+0000 7faebf28f700 1 --2- 172.21.15.53:0/2044443562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb810d0d0 0x7faeb8117b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.038+0000 7faebf28f700 1 -- 172.21.15.53:0/2044443562 >> 172.21.15.53:0/2044443562 conn(0x7faeb806db40 msgr2=0x7faeb806ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:30.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.038+0000 7faebf28f700 1 -- 172.21.15.53:0/2044443562 shutdown_connections 2023-11-06T01:34:30.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.038+0000 7faebf28f700 1 -- 172.21.15.53:0/2044443562 wait complete. 2023-11-06T01:34:30.059 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T01:34:30.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:34:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.280+0000 7f1021d3e700 1 -- 172.21.15.53:0/4294818124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c108ea0 msgr2=0x7f101c10b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.280+0000 7f1021d3e700 1 --2- 172.21.15.53:0/4294818124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c108ea0 0x7f101c10b290 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1010009990 tx=0x7f1010009ca0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.281+0000 7f1021d3e700 1 -- 172.21.15.53:0/4294818124 shutdown_connections 2023-11-06T01:34:30.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.281+0000 7f1021d3e700 1 --2- 172.21.15.53:0/4294818124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c108ea0 0x7f101c10b290 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1010009990 tx=0x7f1010009ca0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.281+0000 7f1021d3e700 1 --2- 172.21.15.53:0/4294818124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f101c10e230 0x7f101c1106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.281+0000 7f1021d3e700 1 --2- 172.21.15.53:0/4294818124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c10b7d0 0x7f101c10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.281+0000 7f1021d3e700 1 -- 172.21.15.53:0/4294818124 >> 172.21.15.53:0/4294818124 conn(0x7f101c06db90 msgr2=0x7f101c06ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.281+0000 7f1021d3e700 1 -- 172.21.15.53:0/4294818124 shutdown_connections 2023-11-06T01:34:30.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.281+0000 7f1021d3e700 1 -- 172.21.15.53:0/4294818124 wait complete. 2023-11-06T01:34:30.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.282+0000 7f1021d3e700 1 Processor -- start 2023-11-06T01:34:30.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.283+0000 7f1021d3e700 1 -- start start 2023-11-06T01:34:30.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.283+0000 7f1021d3e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c10b7d0 0x7f101c1a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:30.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.283+0000 7f1021d3e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c10e230 0x7f101c1a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:30.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.283+0000 7f1021d3e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f101c1ab090 0x7f101c1ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:30.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.283+0000 7f1021d3e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c1ada50 con 0x7f101c10e230 2023-11-06T01:34:30.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.283+0000 7f1021d3e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c1adb90 con 0x7f101c10b7d0 2023-11-06T01:34:30.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.283+0000 7f1021d3e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c1add20 con 0x7f101c1ab090 2023-11-06T01:34:30.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f101b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c10b7d0 0x7f101c1a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:30.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f101b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c10b7d0 0x7f101c1a5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51400/0 (socket says 172.21.15.53:51400) 2023-11-06T01:34:30.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f101b7fe700 1 -- 172.21.15.53:0/139266274 learned_addr learned my addr 172.21.15.53:0/139266274 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:30.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f101bfff700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f101c1ab090 0x7f101c1ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:30.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f101affd700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c10e230 0x7f101c1a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:30.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f101b7fe700 1 -- 172.21.15.53:0/139266274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f101c1ab090 msgr2=0x7f101c1ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:30.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f101b7fe700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f101c1ab090 0x7f101c1ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f101b7fe700 1 -- 172.21.15.53:0/139266274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c10e230 msgr2=0x7f101c1a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:30.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f101b7fe700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c10e230 0x7f101c1a6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f101b7fe700 1 -- 172.21.15.53:0/139266274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10100095f0 con 0x7f101c10b7d0 2023-11-06T01:34:30.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f101b7fe700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c10b7d0 0x7f101c1a5b20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10100098e0 tx=0x7f101000ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:30.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.284+0000 7f1018ff9700 1 -- 172.21.15.53:0/139266274 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101001d070 con 0x7f101c10b7d0 2023-11-06T01:34:30.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.285+0000 7f1018ff9700 1 -- 172.21.15.53:0/139266274 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1010003d50 con 0x7f101c10b7d0 2023-11-06T01:34:30.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.285+0000 7f1021d3e700 1 -- 172.21.15.53:0/139266274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c1adfa0 con 0x7f101c10b7d0 2023-11-06T01:34:30.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.285+0000 7f1018ff9700 1 -- 172.21.15.53:0/139266274 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101000fd70 con 0x7f101c10b7d0 2023-11-06T01:34:30.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.285+0000 7f1021d3e700 1 -- 172.21.15.53:0/139266274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c1ae310 con 0x7f101c10b7d0 2023-11-06T01:34:30.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.286+0000 7f1021d3e700 1 -- 172.21.15.53:0/139266274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c111f50 con 0x7f101c10b7d0 2023-11-06T01:34:30.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.287+0000 7f1018ff9700 1 -- 172.21.15.53:0/139266274 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f101000f460 con 0x7f101c10b7d0 2023-11-06T01:34:30.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.288+0000 7f1018ff9700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10040a05f0 0x7f10040a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:30.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.288+0000 7f1018ff9700 1 -- 172.21.15.53:0/139266274 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f10100ccda0 con 0x7f101c10b7d0 2023-11-06T01:34:30.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.290+0000 7f101affd700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10040a05f0 0x7f10040a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:30.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.291+0000 7f101affd700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10040a05f0 0x7f10040a2ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f101400f540 tx=0x7f101400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:30.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.293+0000 7f1018ff9700 1 -- 172.21.15.53:0/139266274 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f10100969b0 con 0x7f101c10b7d0 2023-11-06T01:34:30.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:30 smithi163 ceph-mon[104416]: pgmap v316: 97 pgs: 97 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 105 MiB/s wr, 996 op/s 2023-11-06T01:34:30.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:30 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:30.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.763+0000 7f1021d3e700 1 -- 172.21.15.53:0/139266274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f101c1ae8c0 con 0x7f101c10b7d0 2023-11-06T01:34:30.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.764+0000 7f1018ff9700 1 -- 172.21.15.53:0/139266274 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f1010099fd0 con 0x7f101c10b7d0 2023-11-06T01:34:30.767 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:34:30.767 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-06T01:32:26.093194+0000","modified":"2023-11-06T01:34:30.371333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24472,"mds_2":24464,"mds_3":24422,"mds_4":24395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24395":{"gid":24395,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T01:34:30.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 -- 172.21.15.53:0/139266274 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10040a05f0 msgr2=0x7f10040a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:30.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10040a05f0 0x7f10040a2ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f101400f540 tx=0x7f101400b040 comp rx=0 tx=0).stop 2023-11-06T01:34:30.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 -- 172.21.15.53:0/139266274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c10b7d0 msgr2=0x7f101c1a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:30.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c10b7d0 0x7f101c1a5b20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10100098e0 tx=0x7f101000ba50 comp rx=0 tx=0).stop 2023-11-06T01:34:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 -- 172.21.15.53:0/139266274 shutdown_connections 2023-11-06T01:34:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10040a05f0 0x7f10040a2ab0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c10b7d0 0x7f101c1a5b20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f101c1ab090 0x7f101c1ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 --2- 172.21.15.53:0/139266274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c10e230 0x7f101c1a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 -- 172.21.15.53:0/139266274 >> 172.21.15.53:0/139266274 conn(0x7f101c06db90 msgr2=0x7f101c10aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:30.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 -- 172.21.15.53:0/139266274 shutdown_connections 2023-11-06T01:34:30.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.768+0000 7f1021d3e700 1 -- 172.21.15.53:0/139266274 wait complete. 2023-11-06T01:34:30.772 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 32 2023-11-06T01:34:30.790 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-11-06T01:34:30.791 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:30 smithi053 ceph-mon[99472]: pgmap v316: 97 pgs: 97 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 105 MiB/s wr, 996 op/s 2023-11-06T01:34:30.791 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:30 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:30.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:30 smithi062 ceph-mon[103924]: pgmap v316: 97 pgs: 97 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 105 MiB/s wr, 996 op/s 2023-11-06T01:34:30.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:30 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:30.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.994+0000 7f9045b87700 1 -- 172.21.15.53:0/1678214477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9040100e70 msgr2=0x7f90401012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:30.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.994+0000 7f9045b87700 1 --2- 172.21.15.53:0/1678214477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9040100e70 0x7f90401012d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f90280099c0 tx=0x7f9028009cd0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.994+0000 7f9045b87700 1 -- 172.21.15.53:0/1678214477 shutdown_connections 2023-11-06T01:34:30.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.994+0000 7f9045b87700 1 --2- 172.21.15.53:0/1678214477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9040101810 0x7f90401080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.994+0000 7f9045b87700 1 --2- 172.21.15.53:0/1678214477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90400ffc80 0x7f90401000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.994+0000 7f9045b87700 1 --2- 172.21.15.53:0/1678214477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9040100e70 0x7f90401012d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:30.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.994+0000 7f9045b87700 1 -- 172.21.15.53:0/1678214477 >> 172.21.15.53:0/1678214477 conn(0x7f90400fb1f0 msgr2=0x7f90400fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:30.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.994+0000 7f9045b87700 1 -- 172.21.15.53:0/1678214477 shutdown_connections 2023-11-06T01:34:30.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.995+0000 7f9045b87700 1 -- 172.21.15.53:0/1678214477 wait complete. 2023-11-06T01:34:30.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.995+0000 7f9045b87700 1 Processor -- start 2023-11-06T01:34:30.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f9045b87700 1 -- start start 2023-11-06T01:34:30.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f9045b87700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90400ffc80 0x7f9040072970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f9045b87700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9040100e70 0x7f9040070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f9045b87700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9040101810 0x7f90400714f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f9045b87700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90401a51d0 con 0x7f90400ffc80 2023-11-06T01:34:31.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f9045b87700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90401a5310 con 0x7f9040100e70 2023-11-06T01:34:31.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f9045b87700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9040071a30 con 0x7f9040101810 2023-11-06T01:34:31.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f903effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9040100e70 0x7f9040070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f903ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9040101810 0x7f90400714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f903effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9040100e70 0x7f9040070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51410/0 (socket says 172.21.15.53:51410) 2023-11-06T01:34:31.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f903effd700 1 -- 172.21.15.53:0/3276229821 learned_addr learned my addr 172.21.15.53:0/3276229821 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f903f7fe700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90400ffc80 0x7f9040072970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f903ffff700 1 -- 172.21.15.53:0/3276229821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9040100e70 msgr2=0x7f9040070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.996+0000 7f903ffff700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9040100e70 0x7f9040070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.997+0000 7f903ffff700 1 -- 172.21.15.53:0/3276229821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90400ffc80 msgr2=0x7f9040072970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.997+0000 7f903ffff700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90400ffc80 0x7f9040072970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.997+0000 7f903ffff700 1 -- 172.21.15.53:0/3276229821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90280095f0 con 0x7f9040101810 2023-11-06T01:34:31.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.997+0000 7f903effd700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9040100e70 0x7f9040070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:31.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.999+0000 7f903ffff700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9040101810 0x7f90400714f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9038007f10 tx=0x7f903800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.999+0000 7f903cff9700 1 -- 172.21.15.53:0/3276229821 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038004db0 con 0x7f9040101810 2023-11-06T01:34:31.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.999+0000 7f903cff9700 1 -- 172.21.15.53:0/3276229821 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f903800f040 con 0x7f9040101810 2023-11-06T01:34:31.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.999+0000 7f903cff9700 1 -- 172.21.15.53:0/3276229821 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038005990 con 0x7f9040101810 2023-11-06T01:34:31.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.999+0000 7f9045b87700 1 -- 172.21.15.53:0/3276229821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9040071d10 con 0x7f9040101810 2023-11-06T01:34:31.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:30.999+0000 7f9045b87700 1 -- 172.21.15.53:0/3276229821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90401adf40 con 0x7f9040101810 2023-11-06T01:34:31.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.002+0000 7f903cff9700 1 -- 172.21.15.53:0/3276229821 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f903800db20 con 0x7f9040101810 2023-11-06T01:34:31.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.004+0000 7f9051d17b80 1 Processor -- start 2023-11-06T01:34:31.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.005+0000 7f903cff9700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f902c0a0620 0x7f902c0a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.005+0000 7f903effd700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f902c0a0620 0x7f902c0a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.005+0000 7f903cff9700 1 -- 172.21.15.53:0/3276229821 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f90380cb1e0 con 0x7f9040101810 2023-11-06T01:34:31.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f9051d17b80 1 -- start start 2023-11-06T01:34:31.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f903effd700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f902c0a0620 0x7f902c0a2ae0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9028009fd0 tx=0x7f90280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc640f0 0x55920bc64530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 0x55920bc62e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc633f0 0x55920bc63870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f9051d17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920bbbaa10 con 0x55920bc64a70 2023-11-06T01:34:31.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f9051d17b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920bbb83d0 con 0x55920bc633f0 2023-11-06T01:34:31.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f9051d17b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920bbb3cc0 con 0x55920bc640f0 2023-11-06T01:34:31.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f903effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 0x55920bc62e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f903effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 0x55920bc62e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39980/0 (socket says 172.21.15.53:39980) 2023-11-06T01:34:31.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f903effd700 1 -- 172.21.15.53:0/1293221094 learned_addr learned my addr 172.21.15.53:0/1293221094 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f903ffff700 1 --2- 172.21.15.53:0/1293221094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc633f0 0x55920bc63870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.006+0000 7f903effd700 1 -- 172.21.15.53:0/1293221094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc640f0 msgr2=0x55920bc64530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.007+0000 7f903effd700 1 --2- 172.21.15.53:0/1293221094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc640f0 0x55920bc64530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.007+0000 7f903effd700 1 -- 172.21.15.53:0/1293221094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc633f0 msgr2=0x55920bc63870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.007+0000 7f903effd700 1 --2- 172.21.15.53:0/1293221094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc633f0 0x55920bc63870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.007+0000 7f903effd700 1 -- 172.21.15.53:0/1293221094 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55920b9e1de0 con 0x55920bc64a70 2023-11-06T01:34:31.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.007+0000 7f903effd700 1 --2- 172.21.15.53:0/1293221094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 0x55920bc62e60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9028005240 tx=0x7f902801a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb7758f2c30dc94 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.008+0000 7f9035ffb700 1 -- 172.21.15.53:0/1293221094 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9028003e40 con 0x55920bc64a70 2023-11-06T01:34:31.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.008+0000 7f9035ffb700 1 -- 172.21.15.53:0/1293221094 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f902801e040 con 0x55920bc64a70 2023-11-06T01:34:31.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.008+0000 7f9035ffb700 1 -- 172.21.15.53:0/1293221094 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90280106d0 con 0x55920bc64a70 2023-11-06T01:34:31.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.008+0000 7f9051d17b80 1 -- 172.21.15.53:0/1293221094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 msgr2=0x55920bc62e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.009+0000 7f9051d17b80 1 --2- 172.21.15.53:0/1293221094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 0x55920bc62e60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9028005240 tx=0x7f902801a070 comp rx=0 tx=0).stop 2023-11-06T01:34:31.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.009+0000 7f9051d17b80 1 -- 172.21.15.53:0/1293221094 shutdown_connections 2023-11-06T01:34:31.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.009+0000 7f9051d17b80 1 --2- 172.21.15.53:0/1293221094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc633f0 0x55920bc63870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.009+0000 7f9051d17b80 1 --2- 172.21.15.53:0/1293221094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc640f0 0x55920bc64530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.009+0000 7f9051d17b80 1 --2- 172.21.15.53:0/1293221094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 0x55920bc62e60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.009+0000 7f9051d17b80 1 -- 172.21.15.53:0/1293221094 >> 172.21.15.53:0/1293221094 conn(0x55920bbc20d0 msgr2=0x55920bc5d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.009+0000 7f9051d17b80 1 -- 172.21.15.53:0/1293221094 shutdown_connections 2023-11-06T01:34:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.010+0000 7f9051d17b80 1 -- 172.21.15.53:0/1293221094 wait complete. 2023-11-06T01:34:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.010+0000 7f9051d17b80 1 -- start start 2023-11-06T01:34:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.010+0000 7f9051d17b80 1 Processor -- start 2023-11-06T01:34:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc633f0 0x55920bc6c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc640f0 0x55920bc6ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 0x55920bc73ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f9051d17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920bb50510 con 0x55920bc633f0 2023-11-06T01:34:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f9051d17b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920bb501b0 con 0x55920bc64a70 2023-11-06T01:34:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f9051d17b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920ba37920 con 0x55920bc640f0 2023-11-06T01:34:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f9051d17b80 10 client.0 _ll_register_callbacks cb 0x55920ba9da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc633f0 0x55920bc6c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc640f0 0x55920bc6ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 0x55920bc73ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc640f0 0x55920bc6ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54226/0 (socket says 172.21.15.53:54226) 2023-11-06T01:34:31.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 0x55920bc73ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51426/0 (socket says 172.21.15.53:51426) 2023-11-06T01:34:31.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903effd700 1 -- 172.21.15.53:0/2761343650 learned_addr learned my addr 172.21.15.53:0/2761343650 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc633f0 0x55920bc6c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39984/0 (socket says 172.21.15.53:39984) 2023-11-06T01:34:31.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903ffff700 1 -- 172.21.15.53:0/2761343650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc640f0 msgr2=0x55920bc6ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903ffff700 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc640f0 0x55920bc6ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903ffff700 1 -- 172.21.15.53:0/2761343650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc633f0 msgr2=0x55920bc6c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903ffff700 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc633f0 0x55920bc6c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903ffff700 1 -- 172.21.15.53:0/2761343650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9040108a10 con 0x55920bc64a70 2023-11-06T01:34:31.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903effd700 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc640f0 0x55920bc6ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:31.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903f7fe700 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc633f0 0x55920bc6c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:31.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.011+0000 7f903ffff700 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 0x55920bc73ef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90380986c0 tx=0x7f9038098790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0366f53cab7f2d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.012+0000 7f901effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:31.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.013+0000 7f901effd700 1 -- 172.21.15.53:0/2761343650 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90380b0070 con 0x55920bc64a70 2023-11-06T01:34:31.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.013+0000 7f901effd700 1 -- 172.21.15.53:0/2761343650 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f90380990c0 con 0x55920bc64a70 2023-11-06T01:34:31.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.013+0000 7f9051d17b80 1 -- 172.21.15.53:0/2761343650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55920bbded10 con 0x55920bc64a70 2023-11-06T01:34:31.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.013+0000 7f901effd700 1 -- 172.21.15.53:0/2761343650 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9038098320 con 0x55920bc64a70 2023-11-06T01:34:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.013+0000 7f9051d17b80 10 client.25097 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.013+0000 7f9051d17b80 1 -- 172.21.15.53:0/2761343650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55920b9cac60 con 0x55920bc64a70 2023-11-06T01:34:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.014+0000 7f901effd700 1 -- 172.21.15.53:0/2761343650 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55920bbded10 con 0x55920bc64a70 2023-11-06T01:34:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.014+0000 7f901effd700 1 -- 172.21.15.53:0/2761343650 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9038097eb0 con 0x55920bc64a70 2023-11-06T01:34:31.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.014+0000 7f9051d17b80 10 client.25097 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.014+0000 7f9051d17b80 10 client.25097 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-11-06T01:34:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.014+0000 7f9051d17b80 20 client.25097 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.014+0000 7f9051d17b80 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x55920bc787d0 0x55920bc7abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.014+0000 7f9051d17b80 4 client.25097 mds_command: new command op to 24395 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:31.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.014+0000 7f9051d17b80 1 -- 172.21.15.53:0/2761343650 --> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55920bc31e10 con 0x55920bc787d0 2023-11-06T01:34:31.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.014+0000 7f903f7fe700 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x55920bc787d0 0x55920bc7abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.015+0000 7f903f7fe700 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x55920bc787d0 0x55920bc7abf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f46373ea09140c75 server_cookie=a2870a371545cccd in_seq=0 out_seq=0 2023-11-06T01:34:31.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.015+0000 7f901effd700 10 client.25097 ms_handle_connect on v2:172.21.15.62:6832/585581012 2023-11-06T01:34:31.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.016+0000 7f901effd700 1 -- 172.21.15.53:0/2761343650 <== mds.4 v2:172.21.15.62:6832/585581012 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55920bc31e10 con 0x55920bc787d0 2023-11-06T01:34:31.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.016+0000 7f901effd700 10 client.25097 handle_command_reply: tid=0 2023-11-06T01:34:31.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.022+0000 7f9051d17b80 1 client.25097 shutdown 2023-11-06T01:34:31.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.022+0000 7f9051d17b80 1 -- 172.21.15.53:0/2761343650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 msgr2=0x55920bc73ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.023+0000 7f9051d17b80 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 0x55920bc73ef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90380986c0 tx=0x7f9038098790 comp rx=0 tx=0).stop 2023-11-06T01:34:31.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.023+0000 7f9051d17b80 1 -- 172.21.15.53:0/2761343650 shutdown_connections 2023-11-06T01:34:31.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.023+0000 7f9051d17b80 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x55920bc787d0 0x55920bc7abf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.023+0000 7f9051d17b80 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 0x55920bc73ef0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.023+0000 7f9051d17b80 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc633f0 0x55920bc6c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.023+0000 7f9051d17b80 1 --2- 172.21.15.53:0/2761343650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc640f0 0x55920bc6ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.023+0000 7f9051d17b80 1 -- 172.21.15.53:0/2761343650 >> 172.21.15.53:0/2761343650 conn(0x55920bbc20d0 msgr2=0x55920bc5d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.023+0000 7f901effd700 0 client.25097 ms_handle_reset on v2:172.21.15.62:6832/585581012 2023-11-06T01:34:31.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.025+0000 7f9051d17b80 1 -- 172.21.15.53:0/2761343650 shutdown_connections 2023-11-06T01:34:31.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.025+0000 7f9051d17b80 1 -- 172.21.15.53:0/2761343650 wait complete. 2023-11-06T01:34:31.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.026+0000 7f9051d17b80 20 client.25097 trim_cache size 0 max 16384 2023-11-06T01:34:31.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f9051d17b80 1 Processor -- start 2023-11-06T01:34:31.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f9051d17b80 1 -- start start 2023-11-06T01:34:31.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 0x55920bc62e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc640f0 0x55920bc71ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 0x55920bc6c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f9051d17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920bc6c4e0 con 0x55920bc64a70 2023-11-06T01:34:31.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f9051d17b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920bc6b060 con 0x55920bc640f0 2023-11-06T01:34:31.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f9051d17b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920bc6d5f0 con 0x55920bc633f0 2023-11-06T01:34:31.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f903ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 0x55920bc6c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f903ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 0x55920bc6c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39994/0 (socket says 172.21.15.53:39994) 2023-11-06T01:34:31.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f903f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 0x55920bc62e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f903ffff700 1 -- 172.21.15.53:0/2228659647 learned_addr learned my addr 172.21.15.53:0/2228659647 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.045+0000 7f903effd700 1 --2- 172.21.15.53:0/2228659647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc640f0 0x55920bc71ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.046+0000 7f903f7fe700 1 -- 172.21.15.53:0/2228659647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc640f0 msgr2=0x55920bc71ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.046+0000 7f903f7fe700 1 --2- 172.21.15.53:0/2228659647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc640f0 0x55920bc71ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.046+0000 7f903f7fe700 1 -- 172.21.15.53:0/2228659647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 msgr2=0x55920bc6c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.046+0000 7f903f7fe700 1 --2- 172.21.15.53:0/2228659647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 0x55920bc6c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.046+0000 7f903f7fe700 1 -- 172.21.15.53:0/2228659647 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55920ba08250 con 0x55920bc633f0 2023-11-06T01:34:31.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.046+0000 7f903f7fe700 1 --2- 172.21.15.53:0/2228659647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 0x55920bc62e60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55920bc7b4b0 tx=0x7f9030004480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d8c5dd79e996adf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.046+0000 7f90367fc700 1 -- 172.21.15.53:0/2228659647 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030017040 con 0x55920bc633f0 2023-11-06T01:34:31.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.046+0000 7f90367fc700 1 -- 172.21.15.53:0/2228659647 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9030019040 con 0x55920bc633f0 2023-11-06T01:34:31.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.046+0000 7f90367fc700 1 -- 172.21.15.53:0/2228659647 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903001ed60 con 0x55920bc633f0 2023-11-06T01:34:31.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.047+0000 7f9051d17b80 1 -- 172.21.15.53:0/2228659647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 msgr2=0x55920bc62e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.047+0000 7f9051d17b80 1 --2- 172.21.15.53:0/2228659647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 0x55920bc62e60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55920bc7b4b0 tx=0x7f9030004480 comp rx=0 tx=0).stop 2023-11-06T01:34:31.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.047+0000 7f9051d17b80 1 -- 172.21.15.53:0/2228659647 shutdown_connections 2023-11-06T01:34:31.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.047+0000 7f9051d17b80 1 --2- 172.21.15.53:0/2228659647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc64a70 0x55920bc6c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.047+0000 7f9051d17b80 1 --2- 172.21.15.53:0/2228659647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 0x55920bc62e60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.047+0000 7f9051d17b80 1 --2- 172.21.15.53:0/2228659647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc640f0 0x55920bc71ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.047+0000 7f9051d17b80 1 -- 172.21.15.53:0/2228659647 >> 172.21.15.53:0/2228659647 conn(0x55920bbc20d0 msgr2=0x55920bc673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.047+0000 7f9051d17b80 1 -- 172.21.15.53:0/2228659647 shutdown_connections 2023-11-06T01:34:31.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.047+0000 7f9051d17b80 1 -- 172.21.15.53:0/2228659647 wait complete. 2023-11-06T01:34:31.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f9051d17b80 1 -- start start 2023-11-06T01:34:31.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f9051d17b80 1 Processor -- start 2023-11-06T01:34:31.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 0x55920bcd5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc640f0 0x55920bcd79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f9051d17b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 0x55920bcd9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f9051d17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920bcda720 con 0x55920bc640f0 2023-11-06T01:34:31.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f9051d17b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920bcda890 con 0x55920bc64a70 2023-11-06T01:34:31.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f9051d17b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920bcdaa50 con 0x55920bc633f0 2023-11-06T01:34:31.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f9051d17b80 10 client.0 _ll_register_callbacks cb 0x7f9038005af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:31.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f903effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc640f0 0x55920bcd79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f903ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 0x55920bcd9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f903effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc640f0 0x55920bcd79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39996/0 (socket says 172.21.15.53:39996) 2023-11-06T01:34:31.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f903effd700 1 -- 172.21.15.53:0/1429941799 learned_addr learned my addr 172.21.15.53:0/1429941799 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.048+0000 7f903f7fe700 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 0x55920bcd5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f903ffff700 1 -- 172.21.15.53:0/1429941799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 msgr2=0x55920bcd5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f903ffff700 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 0x55920bcd5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f903ffff700 1 -- 172.21.15.53:0/1429941799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc640f0 msgr2=0x55920bcd79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f903ffff700 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc640f0 0x55920bcd79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f903ffff700 1 -- 172.21.15.53:0/1429941799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9030009e70 con 0x55920bc64a70 2023-11-06T01:34:31.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f903f7fe700 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 0x55920bcd5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:34:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f903ffff700 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 0x55920bcd9fe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f90380a0500 tx=0x7f90380053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1bf7f777f985645 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f901effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f901effd700 1 -- 172.21.15.53:0/1429941799 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90380b0040 con 0x55920bc64a70 2023-11-06T01:34:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f901effd700 1 -- 172.21.15.53:0/1429941799 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f90380990c0 con 0x55920bc64a70 2023-11-06T01:34:31.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f9051d17b80 1 -- 172.21.15.53:0/1429941799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f902801e040 con 0x55920bc64a70 2023-11-06T01:34:31.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.049+0000 7f901effd700 1 -- 172.21.15.53:0/1429941799 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9038098320 con 0x55920bc64a70 2023-11-06T01:34:31.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.050+0000 7f9051d17b80 10 client.25103 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:31.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.050+0000 7f9051d17b80 1 -- 172.21.15.53:0/1429941799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55920b9e02b0 con 0x55920bc64a70 2023-11-06T01:34:31.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.051+0000 7f901effd700 1 -- 172.21.15.53:0/1429941799 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f902801e040 con 0x55920bc64a70 2023-11-06T01:34:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.051+0000 7f901effd700 1 -- 172.21.15.53:0/1429941799 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f90380947b0 con 0x55920bc64a70 2023-11-06T01:34:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.051+0000 7f9051d17b80 10 client.25103 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.051+0000 7f9051d17b80 10 client.25103 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-11-06T01:34:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.051+0000 7f9051d17b80 20 client.25103 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.051+0000 7f9051d17b80 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x55920bcdbeb0 0x55920bcde2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.051+0000 7f9051d17b80 4 client.25103 mds_command: new command op to 24395 tid=0[{"prefix": "damage ls"}] 2023-11-06T01:34:31.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.051+0000 7f9051d17b80 1 -- 172.21.15.53:0/1429941799 --> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55920bbacd40 con 0x55920bcdbeb0 2023-11-06T01:34:31.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.051+0000 7f903f7fe700 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x55920bcdbeb0 0x55920bcde2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.051+0000 7f901effd700 10 client.25103 ms_handle_connect on v2:172.21.15.62:6832/585581012 2023-11-06T01:34:31.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.051+0000 7f903f7fe700 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x55920bcdbeb0 0x55920bcde2d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1fa9f11e1d82b073 server_cookie=ea52d0329c7dc897 in_seq=0 out_seq=0 2023-11-06T01:34:31.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f901effd700 1 -- 172.21.15.53:0/1429941799 <== mds.4 v2:172.21.15.62:6832/585581012 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55920bbacd40 con 0x55920bcdbeb0 2023-11-06T01:34:31.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f901effd700 10 client.25103 handle_command_reply: tid=0 2023-11-06T01:34:31.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f9051d17b80 1 client.25103 shutdown 2023-11-06T01:34:31.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f9051d17b80 1 -- 172.21.15.53:0/1429941799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 msgr2=0x55920bcd9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f9051d17b80 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 0x55920bcd9fe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f90380a0500 tx=0x7f90380053b0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f9051d17b80 1 -- 172.21.15.53:0/1429941799 shutdown_connections 2023-11-06T01:34:31.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f9051d17b80 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.62:6832/585581012,v1:172.21.15.62:6833/585581012] conn(0x55920bcdbeb0 0x55920bcde2d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f9051d17b80 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55920bc64a70 0x55920bcd9fe0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f9051d17b80 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55920bc633f0 0x55920bcd5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f9051d17b80 1 --2- 172.21.15.53:0/1429941799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55920bc640f0 0x55920bcd79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f9051d17b80 1 -- 172.21.15.53:0/1429941799 >> 172.21.15.53:0/1429941799 conn(0x55920bbc20d0 msgr2=0x55920bc6c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f901effd700 0 client.25103 ms_handle_reset on v2:172.21.15.62:6832/585581012 2023-11-06T01:34:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.052+0000 7f9051d17b80 1 -- 172.21.15.53:0/1429941799 shutdown_connections 2023-11-06T01:34:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.053+0000 7f9051d17b80 1 -- 172.21.15.53:0/1429941799 wait complete. 2023-11-06T01:34:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.053+0000 7f9051d17b80 20 client.25103 trim_cache size 0 max 16384 2023-11-06T01:34:31.068 INFO:teuthology.orchestra.run.smithi053.stdout:[] 2023-11-06T01:34:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 -- 172.21.15.53:0/3276229821 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f902c0a0620 msgr2=0x7f902c0a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f902c0a0620 0x7f902c0a2ae0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9028009fd0 tx=0x7f90280058e0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 -- 172.21.15.53:0/3276229821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9040101810 msgr2=0x7f90400714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9040101810 0x7f90400714f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9038007f10 tx=0x7f903800d3b0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 -- 172.21.15.53:0/3276229821 shutdown_connections 2023-11-06T01:34:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f902c0a0620 0x7f902c0a2ae0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9040101810 0x7f90400714f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90400ffc80 0x7f9040072970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 --2- 172.21.15.53:0/3276229821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9040100e70 0x7f9040070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 -- 172.21.15.53:0/3276229821 >> 172.21.15.53:0/3276229821 conn(0x7f90400fb1f0 msgr2=0x7f9040067ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 -- 172.21.15.53:0/3276229821 shutdown_connections 2023-11-06T01:34:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.054+0000 7f9045b87700 1 -- 172.21.15.53:0/3276229821 wait complete. 2023-11-06T01:34:31.074 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-11-06T01:34:31.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.297+0000 7f82d51dd700 1 -- 172.21.15.53:0/1046880772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d00ffc80 msgr2=0x7f82d01000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.297+0000 7f82d51dd700 1 --2- 172.21.15.53:0/1046880772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d00ffc80 0x7f82d01000a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009920 tx=0x7f82c0009c30 comp rx=0 tx=0).stop 2023-11-06T01:34:31.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.297+0000 7f82d51dd700 1 -- 172.21.15.53:0/1046880772 shutdown_connections 2023-11-06T01:34:31.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.297+0000 7f82d51dd700 1 --2- 172.21.15.53:0/1046880772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d00ffc80 0x7f82d01000a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.297+0000 7f82d51dd700 1 --2- 172.21.15.53:0/1046880772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82d0101810 0x7f82d01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.297+0000 7f82d51dd700 1 --2- 172.21.15.53:0/1046880772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d0100e70 0x7f82d01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.297+0000 7f82d51dd700 1 -- 172.21.15.53:0/1046880772 >> 172.21.15.53:0/1046880772 conn(0x7f82d00fb1f0 msgr2=0x7f82d00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.297+0000 7f82d51dd700 1 -- 172.21.15.53:0/1046880772 shutdown_connections 2023-11-06T01:34:31.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.297+0000 7f82d51dd700 1 -- 172.21.15.53:0/1046880772 wait complete. 2023-11-06T01:34:31.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.298+0000 7f82d51dd700 1 Processor -- start 2023-11-06T01:34:31.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.298+0000 7f82d51dd700 1 -- start start 2023-11-06T01:34:31.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.298+0000 7f82d51dd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82d00ffc80 0x7f82d0196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.298+0000 7f82d51dd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d0100e70 0x7f82d0197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.298+0000 7f82d51dd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d0101810 0x7f82d019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.298+0000 7f82d51dd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d019e6c0 con 0x7f82d0101810 2023-11-06T01:34:31.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.298+0000 7f82d51dd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d019e800 con 0x7f82d0100e70 2023-11-06T01:34:31.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.298+0000 7f82d51dd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d019e990 con 0x7f82d00ffc80 2023-11-06T01:34:31.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82ced9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82d00ffc80 0x7f82d0196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82ced9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82d00ffc80 0x7f82d0196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54250/0 (socket says 172.21.15.53:54250) 2023-11-06T01:34:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d0101810 0x7f82d019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82ced9d700 1 -- 172.21.15.53:0/3683335126 learned_addr learned my addr 172.21.15.53:0/3683335126 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82ce59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d0100e70 0x7f82d0197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82ced9d700 1 -- 172.21.15.53:0/3683335126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d0100e70 msgr2=0x7f82d0197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82ced9d700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d0100e70 0x7f82d0197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82ced9d700 1 -- 172.21.15.53:0/3683335126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d0101810 msgr2=0x7f82d019e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82ced9d700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d0101810 0x7f82d019e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82ced9d700 1 -- 172.21.15.53:0/3683335126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c40095f0 con 0x7f82d00ffc80 2023-11-06T01:34:31.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82ce59c700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d0100e70 0x7f82d0197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:31.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82ced9d700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82d00ffc80 0x7f82d0196b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009230 tx=0x7f82c000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82b7fff700 1 -- 172.21.15.53:0/3683335126 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c001c070 con 0x7f82d00ffc80 2023-11-06T01:34:31.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82b7fff700 1 -- 172.21.15.53:0/3683335126 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c000f7d0 con 0x7f82d00ffc80 2023-11-06T01:34:31.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82d51dd700 1 -- 172.21.15.53:0/3683335126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c0009520 con 0x7f82d00ffc80 2023-11-06T01:34:31.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82b7fff700 1 -- 172.21.15.53:0/3683335126 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c00038a0 con 0x7f82d00ffc80 2023-11-06T01:34:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.299+0000 7f82d51dd700 1 -- 172.21.15.53:0/3683335126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d019ee40 con 0x7f82d00ffc80 2023-11-06T01:34:31.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.302+0000 7f82b7fff700 1 -- 172.21.15.53:0/3683335126 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f82c0020070 con 0x7f82d00ffc80 2023-11-06T01:34:31.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82e136db80 1 Processor -- start 2023-11-06T01:34:31.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82e136db80 1 -- start start 2023-11-06T01:34:31.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82b7fff700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f82b80a0520 0x7f82b80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82e136db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9240f0 0x5557dd924530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82e136db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 0x5557dd922e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82b7fff700 1 -- 172.21.15.53:0/3683335126 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f82c00cc4e0 con 0x7f82d00ffc80 2023-11-06T01:34:31.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82cf59e700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f82b80a0520 0x7f82b80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82e136db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9233f0 0x5557dd923870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82e136db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd87aa10 con 0x5557dd924a70 2023-11-06T01:34:31.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82e136db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd8783d0 con 0x5557dd9233f0 2023-11-06T01:34:31.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82e136db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd873cc0 con 0x5557dd9240f0 2023-11-06T01:34:31.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.303+0000 7f82ced9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9240f0 0x5557dd924530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9233f0 0x5557dd923870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82ced9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9240f0 0x5557dd924530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54258/0 (socket says 172.21.15.53:54258) 2023-11-06T01:34:31.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82ced9d700 1 -- 172.21.15.53:0/3686277265 learned_addr learned my addr 172.21.15.53:0/3686277265 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82ce59c700 1 --2- 172.21.15.53:0/3686277265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 0x5557dd922e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82ced9d700 1 -- 172.21.15.53:0/3686277265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9233f0 msgr2=0x5557dd923870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82ced9d700 1 --2- 172.21.15.53:0/3686277265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9233f0 0x5557dd923870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82ced9d700 1 -- 172.21.15.53:0/3686277265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 msgr2=0x5557dd922e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82ced9d700 1 --2- 172.21.15.53:0/3686277265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 0x5557dd922e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82ced9d700 1 -- 172.21.15.53:0/3686277265 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557dd6a1de0 con 0x5557dd9240f0 2023-11-06T01:34:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82cf59e700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f82b80a0520 0x7f82b80a29e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009ec0 tx=0x7f82c800fc40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82ced9d700 1 --2- 172.21.15.53:0/3686277265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9240f0 0x5557dd924530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f82c00967c0 tx=0x7f82c0009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a076cb732c755a07 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82b57fa700 1 -- 172.21.15.53:0/3686277265 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c00b7ea0 con 0x5557dd9240f0 2023-11-06T01:34:31.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82b57fa700 1 -- 172.21.15.53:0/3686277265 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c0099d50 con 0x5557dd9240f0 2023-11-06T01:34:31.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.304+0000 7f82b57fa700 1 -- 172.21.15.53:0/3686277265 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0099fb0 con 0x5557dd9240f0 2023-11-06T01:34:31.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.305+0000 7f82e136db80 1 -- 172.21.15.53:0/3686277265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9240f0 msgr2=0x5557dd924530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.305+0000 7f82e136db80 1 --2- 172.21.15.53:0/3686277265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9240f0 0x5557dd924530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f82c00967c0 tx=0x7f82c0009480 comp rx=0 tx=0).stop 2023-11-06T01:34:31.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.305+0000 7f82e136db80 1 -- 172.21.15.53:0/3686277265 shutdown_connections 2023-11-06T01:34:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.305+0000 7f82e136db80 1 --2- 172.21.15.53:0/3686277265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9233f0 0x5557dd923870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.305+0000 7f82e136db80 1 --2- 172.21.15.53:0/3686277265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9240f0 0x5557dd924530 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.305+0000 7f82e136db80 1 --2- 172.21.15.53:0/3686277265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 0x5557dd922e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.305+0000 7f82e136db80 1 -- 172.21.15.53:0/3686277265 >> 172.21.15.53:0/3686277265 conn(0x5557dd8820d0 msgr2=0x5557dd91d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.305+0000 7f82e136db80 1 -- 172.21.15.53:0/3686277265 shutdown_connections 2023-11-06T01:34:31.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.305+0000 7f82e136db80 1 -- 172.21.15.53:0/3686277265 wait complete. 2023-11-06T01:34:31.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.305+0000 7f82e136db80 1 -- start start 2023-11-06T01:34:31.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82e136db80 1 Processor -- start 2023-11-06T01:34:31.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82e136db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9233f0 0x5557dd92c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82e136db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9240f0 0x5557dd92cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82e136db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd924a70 0x5557dd933bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82e136db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd810510 con 0x5557dd9233f0 2023-11-06T01:34:31.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82e136db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd8101b0 con 0x5557dd9240f0 2023-11-06T01:34:31.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82e136db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd6f7920 con 0x5557dd924a70 2023-11-06T01:34:31.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82e136db80 10 client.0 _ll_register_callbacks cb 0x5557dd75da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:31.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82ced9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9233f0 0x5557dd92c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82ced9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9233f0 0x5557dd92c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40024/0 (socket says 172.21.15.53:40024) 2023-11-06T01:34:31.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82ced9d700 1 -- 172.21.15.53:0/3322584418 learned_addr learned my addr 172.21.15.53:0/3322584418 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82cf59e700 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd924a70 0x5557dd933bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82ce59c700 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9240f0 0x5557dd92cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82cf59e700 1 -- 172.21.15.53:0/3322584418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9240f0 msgr2=0x5557dd92cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82cf59e700 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9240f0 0x5557dd92cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82cf59e700 1 -- 172.21.15.53:0/3322584418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9233f0 msgr2=0x5557dd92c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.306+0000 7f82cf59e700 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9233f0 0x5557dd92c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82cf59e700 1 -- 172.21.15.53:0/3322584418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d019ee40 con 0x5557dd924a70 2023-11-06T01:34:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82ced9d700 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9233f0 0x5557dd92c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82ce59c700 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9240f0 0x5557dd92cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:34:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82cf59e700 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd924a70 0x5557dd933bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82c8018640 tx=0x7f82c8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dfe90fc8fc8d78f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:34:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82b2ff5700 1 -- 172.21.15.53:0/3322584418 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c800e040 con 0x5557dd924a70 2023-11-06T01:34:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82b2ff5700 1 -- 172.21.15.53:0/3322584418 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f82c8003e10 con 0x5557dd924a70 2023-11-06T01:34:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82e136db80 1 -- 172.21.15.53:0/3322584418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557dd89ed10 con 0x5557dd924a70 2023-11-06T01:34:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82b2ff5700 1 -- 172.21.15.53:0/3322584418 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c8004030 con 0x5557dd924a70 2023-11-06T01:34:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82e136db80 10 client.25204 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82e136db80 1 -- 172.21.15.53:0/3322584418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5557dd68ac60 con 0x5557dd924a70 2023-11-06T01:34:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82b2ff5700 1 -- 172.21.15.53:0/3322584418 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557dd89ed10 con 0x5557dd924a70 2023-11-06T01:34:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82b2ff5700 1 -- 172.21.15.53:0/3322584418 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f82c80088b0 con 0x5557dd924a70 2023-11-06T01:34:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82e136db80 10 client.25204 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.307+0000 7f82e136db80 10 client.25204 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-11-06T01:34:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.308+0000 7f82e136db80 20 client.25204 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.308+0000 7f82e136db80 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x5557dd938380 0x5557dd93a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.308+0000 7f82e136db80 4 client.25204 mds_command: new command op to 24422 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.308+0000 7f82e136db80 1 -- 172.21.15.53:0/3322584418 --> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557dd8f1e10 con 0x5557dd938380 2023-11-06T01:34:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.308+0000 7f82ced9d700 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x5557dd938380 0x5557dd93a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.308+0000 7f82ced9d700 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x5557dd938380 0x5557dd93a7a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=31fedf4f5ff2f210 server_cookie=5b00d0adb906c03e in_seq=0 out_seq=0 2023-11-06T01:34:31.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.308+0000 7f82b2ff5700 10 client.25204 ms_handle_connect on v2:172.21.15.62:6834/3901057483 2023-11-06T01:34:31.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82b2ff5700 1 -- 172.21.15.53:0/3322584418 <== mds.3 v2:172.21.15.62:6834/3901057483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5557dd8f1e10 con 0x5557dd938380 2023-11-06T01:34:31.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82b2ff5700 10 client.25204 handle_command_reply: tid=0 2023-11-06T01:34:31.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 1 client.25204 shutdown 2023-11-06T01:34:31.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 1 -- 172.21.15.53:0/3322584418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd924a70 msgr2=0x5557dd933bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd924a70 0x5557dd933bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82c8018640 tx=0x7f82c8004b40 comp rx=0 tx=0).stop 2023-11-06T01:34:31.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 1 -- 172.21.15.53:0/3322584418 shutdown_connections 2023-11-06T01:34:31.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x5557dd938380 0x5557dd93a7a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd924a70 0x5557dd933bd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9233f0 0x5557dd92c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 1 --2- 172.21.15.53:0/3322584418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9240f0 0x5557dd92cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 1 -- 172.21.15.53:0/3322584418 >> 172.21.15.53:0/3322584418 conn(0x5557dd8820d0 msgr2=0x5557dd91d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82b2ff5700 0 client.25204 ms_handle_reset on v2:172.21.15.62:6834/3901057483 2023-11-06T01:34:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 1 -- 172.21.15.53:0/3322584418 shutdown_connections 2023-11-06T01:34:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 1 -- 172.21.15.53:0/3322584418 wait complete. 2023-11-06T01:34:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.309+0000 7f82e136db80 20 client.25204 trim_cache size 0 max 16384 2023-11-06T01:34:31.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.324+0000 7f82e136db80 1 Processor -- start 2023-11-06T01:34:31.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.324+0000 7f82e136db80 1 -- start start 2023-11-06T01:34:31.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82e136db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9233f0 0x5557dd931b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82e136db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9240f0 0x5557dd927210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82e136db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 0x5557dd927750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82e136db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd915b50 con 0x5557dd924a70 2023-11-06T01:34:31.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82e136db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd8f1e10 con 0x5557dd9240f0 2023-11-06T01:34:31.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82e136db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd89ed10 con 0x5557dd9233f0 2023-11-06T01:34:31.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 0x5557dd927750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 0x5557dd927750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40036/0 (socket says 172.21.15.53:40036) 2023-11-06T01:34:31.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82cf59e700 1 -- 172.21.15.53:0/797636449 learned_addr learned my addr 172.21.15.53:0/797636449 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82ced9d700 1 --2- 172.21.15.53:0/797636449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9233f0 0x5557dd931b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82cf59e700 1 -- 172.21.15.53:0/797636449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9233f0 msgr2=0x5557dd931b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82cf59e700 1 --2- 172.21.15.53:0/797636449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9233f0 0x5557dd931b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82cf59e700 1 -- 172.21.15.53:0/797636449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9240f0 msgr2=0x5557dd927210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82cf59e700 1 --2- 172.21.15.53:0/797636449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9240f0 0x5557dd927210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82cf59e700 1 -- 172.21.15.53:0/797636449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557dd923aa0 con 0x5557dd924a70 2023-11-06T01:34:31.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.325+0000 7f82ced9d700 1 --2- 172.21.15.53:0/797636449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9233f0 0x5557dd931b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:34:31.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.326+0000 7f82cf59e700 1 --2- 172.21.15.53:0/797636449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 0x5557dd927750 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009e90 tx=0x7f82c800f550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186a00df4d16f9cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.326+0000 7f82b5ffb700 1 -- 172.21.15.53:0/797636449 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c800e040 con 0x5557dd924a70 2023-11-06T01:34:31.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.326+0000 7f82b5ffb700 1 -- 172.21.15.53:0/797636449 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c800f040 con 0x5557dd924a70 2023-11-06T01:34:31.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.326+0000 7f82b5ffb700 1 -- 172.21.15.53:0/797636449 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c8004bb0 con 0x5557dd924a70 2023-11-06T01:34:31.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.327+0000 7f82e136db80 1 -- 172.21.15.53:0/797636449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 msgr2=0x5557dd927750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.327+0000 7f82e136db80 1 --2- 172.21.15.53:0/797636449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 0x5557dd927750 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009e90 tx=0x7f82c800f550 comp rx=0 tx=0).stop 2023-11-06T01:34:31.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.327+0000 7f82e136db80 1 -- 172.21.15.53:0/797636449 shutdown_connections 2023-11-06T01:34:31.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.327+0000 7f82e136db80 1 --2- 172.21.15.53:0/797636449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd924a70 0x5557dd927750 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.327+0000 7f82e136db80 1 --2- 172.21.15.53:0/797636449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd9233f0 0x5557dd931b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.327+0000 7f82e136db80 1 --2- 172.21.15.53:0/797636449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9240f0 0x5557dd927210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.327+0000 7f82e136db80 1 -- 172.21.15.53:0/797636449 >> 172.21.15.53:0/797636449 conn(0x5557dd8820d0 msgr2=0x5557dd9344c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.327+0000 7f82e136db80 1 -- 172.21.15.53:0/797636449 shutdown_connections 2023-11-06T01:34:31.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.327+0000 7f82e136db80 1 -- 172.21.15.53:0/797636449 wait complete. 2023-11-06T01:34:31.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.327+0000 7f82e136db80 1 -- start start 2023-11-06T01:34:31.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82e136db80 1 Processor -- start 2023-11-06T01:34:31.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82e136db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9233f0 0x5557dd92f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82e136db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9240f0 0x5557dd92fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82e136db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd924a70 0x5557dd930c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82e136db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd9311a0 con 0x5557dd9240f0 2023-11-06T01:34:31.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82e136db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd931310 con 0x5557dd9233f0 2023-11-06T01:34:31.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82e136db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd9314d0 con 0x5557dd924a70 2023-11-06T01:34:31.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82e136db80 10 client.0 _ll_register_callbacks cb 0x7f82c800fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:31.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82ce59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9240f0 0x5557dd92fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82ce59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9240f0 0x5557dd92fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40040/0 (socket says 172.21.15.53:40040) 2023-11-06T01:34:31.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82ce59c700 1 -- 172.21.15.53:0/2712530791 learned_addr learned my addr 172.21.15.53:0/2712530791 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82ced9d700 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9233f0 0x5557dd92f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82ce59c700 1 -- 172.21.15.53:0/2712530791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd924a70 msgr2=0x5557dd930c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82ce59c700 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd924a70 0x5557dd930c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82ce59c700 1 -- 172.21.15.53:0/2712530791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9233f0 msgr2=0x5557dd92f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82ce59c700 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9233f0 0x5557dd92f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82ce59c700 1 -- 172.21.15.53:0/2712530791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557dd68ac60 con 0x5557dd9240f0 2023-11-06T01:34:31.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.328+0000 7f82ced9d700 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9233f0 0x5557dd92f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:34:31.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.329+0000 7f82ce59c700 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9240f0 0x5557dd92fc50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f82c4010d00 tx=0x7f82c400f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8644ca70037c29a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.329+0000 7f82b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:34:31.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.329+0000 7f82b2ff5700 1 -- 172.21.15.53:0/2712530791 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4004050 con 0x5557dd9240f0 2023-11-06T01:34:31.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.329+0000 7f82b2ff5700 1 -- 172.21.15.53:0/2712530791 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f82c40041b0 con 0x5557dd9240f0 2023-11-06T01:34:31.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.329+0000 7f82e136db80 1 -- 172.21.15.53:0/2712530791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82c0099d50 con 0x5557dd9240f0 2023-11-06T01:34:31.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.329+0000 7f82b2ff5700 1 -- 172.21.15.53:0/2712530791 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c4004370 con 0x5557dd9240f0 2023-11-06T01:34:31.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.329+0000 7f82e136db80 10 client.15555 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:31.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.329+0000 7f82e136db80 1 -- 172.21.15.53:0/2712530791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5557dd6c8250 con 0x5557dd9240f0 2023-11-06T01:34:31.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.329+0000 7f82b2ff5700 1 -- 172.21.15.53:0/2712530791 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c0099d50 con 0x5557dd9240f0 2023-11-06T01:34:31.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.330+0000 7f82b2ff5700 1 -- 172.21.15.53:0/2712530791 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f82c4025020 con 0x5557dd9240f0 2023-11-06T01:34:31.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.330+0000 7f82e136db80 10 client.15555 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:31.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.330+0000 7f82e136db80 10 client.15555 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-11-06T01:34:31.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.330+0000 7f82e136db80 20 client.15555 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:31.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.330+0000 7f82e136db80 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x5557dd98ad30 0x5557dd98d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.330+0000 7f82e136db80 4 client.15555 mds_command: new command op to 24422 tid=0[{"prefix": "damage ls"}] 2023-11-06T01:34:31.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.330+0000 7f82e136db80 1 -- 172.21.15.53:0/2712530791 --> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5557dd86cd40 con 0x5557dd98ad30 2023-11-06T01:34:31.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.330+0000 7f82ced9d700 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x5557dd98ad30 0x5557dd98d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.331+0000 7f82b2ff5700 10 client.15555 ms_handle_connect on v2:172.21.15.62:6834/3901057483 2023-11-06T01:34:31.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.331+0000 7f82ced9d700 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x5557dd98ad30 0x5557dd98d150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1767c69edea3209f server_cookie=a27a57bd1245c77c in_seq=0 out_seq=0 2023-11-06T01:34:31.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82b2ff5700 1 -- 172.21.15.53:0/2712530791 <== mds.3 v2:172.21.15.62:6834/3901057483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5557dd86cd40 con 0x5557dd98ad30 2023-11-06T01:34:31.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82b2ff5700 10 client.15555 handle_command_reply: tid=0 2023-11-06T01:34:31.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 1 client.15555 shutdown 2023-11-06T01:34:31.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 1 -- 172.21.15.53:0/2712530791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9240f0 msgr2=0x5557dd92fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9240f0 0x5557dd92fc50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f82c4010d00 tx=0x7f82c400f6a0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 1 -- 172.21.15.53:0/2712530791 shutdown_connections 2023-11-06T01:34:31.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.62:6834/3901057483,v1:172.21.15.62:6835/3901057483] conn(0x5557dd98ad30 0x5557dd98d150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd9233f0 0x5557dd92f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82b2ff5700 0 client.15555 ms_handle_reset on v2:172.21.15.62:6834/3901057483 2023-11-06T01:34:31.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557dd924a70 0x5557dd930c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 1 --2- 172.21.15.53:0/2712530791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557dd9240f0 0x5557dd92fc50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 1 -- 172.21.15.53:0/2712530791 >> 172.21.15.53:0/2712530791 conn(0x5557dd8820d0 msgr2=0x5557dd927210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 1 -- 172.21.15.53:0/2712530791 shutdown_connections 2023-11-06T01:34:31.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 1 -- 172.21.15.53:0/2712530791 wait complete. 2023-11-06T01:34:31.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.332+0000 7f82e136db80 20 client.15555 trim_cache size 0 max 16384 2023-11-06T01:34:31.355 INFO:teuthology.orchestra.run.smithi053.stdout:[] 2023-11-06T01:34:31.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 -- 172.21.15.53:0/3683335126 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f82b80a0520 msgr2=0x7f82b80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f82b80a0520 0x7f82b80a29e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009ec0 tx=0x7f82c800fc40 comp rx=0 tx=0).stop 2023-11-06T01:34:31.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 -- 172.21.15.53:0/3683335126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82d00ffc80 msgr2=0x7f82d0196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82d00ffc80 0x7f82d0196b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009230 tx=0x7f82c000b790 comp rx=0 tx=0).stop 2023-11-06T01:34:31.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 -- 172.21.15.53:0/3683335126 shutdown_connections 2023-11-06T01:34:31.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f82b80a0520 0x7f82b80a29e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d0101810 0x7f82d019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82d00ffc80 0x7f82d0196b40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 --2- 172.21.15.53:0/3683335126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d0100e70 0x7f82d0197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 -- 172.21.15.53:0/3683335126 >> 172.21.15.53:0/3683335126 conn(0x7f82d00fb1f0 msgr2=0x7f82d00fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 -- 172.21.15.53:0/3683335126 shutdown_connections 2023-11-06T01:34:31.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.333+0000 7f82d51dd700 1 -- 172.21.15.53:0/3683335126 wait complete. 2023-11-06T01:34:31.360 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-06T01:34:31.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.582+0000 7f835001a700 1 -- 172.21.15.53:0/551146295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83481073b0 msgr2=0x7f83481077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.582+0000 7f835001a700 1 --2- 172.21.15.53:0/551146295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83481073b0 0x7f83481077d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f834000b0d0 tx=0x7f834000b3e0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.582+0000 7f835001a700 1 -- 172.21.15.53:0/551146295 shutdown_connections 2023-11-06T01:34:31.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.582+0000 7f835001a700 1 --2- 172.21.15.53:0/551146295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83481073b0 0x7f83481077d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.582+0000 7f835001a700 1 --2- 172.21.15.53:0/551146295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8348108f40 0x7f834810f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.582+0000 7f835001a700 1 --2- 172.21.15.53:0/551146295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83481085a0 0x7f8348108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.582+0000 7f835001a700 1 -- 172.21.15.53:0/551146295 >> 172.21.15.53:0/551146295 conn(0x7f8348076330 msgr2=0x7f8348078790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.582+0000 7f835001a700 1 -- 172.21.15.53:0/551146295 shutdown_connections 2023-11-06T01:34:31.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.583+0000 7f835001a700 1 -- 172.21.15.53:0/551146295 wait complete. 2023-11-06T01:34:31.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.584+0000 7f835001a700 1 Processor -- start 2023-11-06T01:34:31.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.584+0000 7f835001a700 1 -- start start 2023-11-06T01:34:31.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.584+0000 7f835001a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83481073b0 0x7f834819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.584+0000 7f835001a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83481085a0 0x7f834819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.584+0000 7f835001a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8348108f40 0x7f83481a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.584+0000 7f835001a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83481a4d70 con 0x7f8348108f40 2023-11-06T01:34:31.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.584+0000 7f835001a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83481a4eb0 con 0x7f83481085a0 2023-11-06T01:34:31.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.584+0000 7f835001a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83481a5040 con 0x7f83481073b0 2023-11-06T01:34:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.584+0000 7f834e5b7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8348108f40 0x7f83481a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.584+0000 7f834e5b7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8348108f40 0x7f83481a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40052/0 (socket says 172.21.15.53:40052) 2023-11-06T01:34:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.584+0000 7f834e5b7700 1 -- 172.21.15.53:0/116970233 learned_addr learned my addr 172.21.15.53:0/116970233 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.585+0000 7f834e5b7700 1 -- 172.21.15.53:0/116970233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83481073b0 msgr2=0x7f834819d1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-06T01:34:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.585+0000 7f834e5b7700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83481073b0 0x7f834819d1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.585+0000 7f834e5b7700 1 -- 172.21.15.53:0/116970233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83481085a0 msgr2=0x7f834819d730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-06T01:34:31.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.585+0000 7f834e5b7700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83481085a0 0x7f834819d730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.585+0000 7f834e5b7700 1 -- 172.21.15.53:0/116970233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8340009d00 con 0x7f8348108f40 2023-11-06T01:34:31.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.585+0000 7f834e5b7700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8348108f40 0x7f83481a47a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f834400c900 tx=0x7f834400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.586+0000 7f833effd700 1 -- 172.21.15.53:0/116970233 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344007b00 con 0x7f8348108f40 2023-11-06T01:34:31.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.586+0000 7f833effd700 1 -- 172.21.15.53:0/116970233 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8344007c60 con 0x7f8348108f40 2023-11-06T01:34:31.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.586+0000 7f833effd700 1 -- 172.21.15.53:0/116970233 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344018660 con 0x7f8348108f40 2023-11-06T01:34:31.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.586+0000 7f835001a700 1 -- 172.21.15.53:0/116970233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83481a5320 con 0x7f8348108f40 2023-11-06T01:34:31.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.586+0000 7f835001a700 1 -- 172.21.15.53:0/116970233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83481a5860 con 0x7f8348108f40 2023-11-06T01:34:31.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.591+0000 7f833effd700 1 -- 172.21.15.53:0/116970233 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f834400f450 con 0x7f8348108f40 2023-11-06T01:34:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.591+0000 7f835c1aab80 1 Processor -- start 2023-11-06T01:34:31.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f835c1aab80 1 -- start start 2023-11-06T01:34:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b0667a0f0 0x557b0667a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667aa70 0x557b06678e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 0x557b06679870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f835c1aab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b065d0a10 con 0x557b0667aa70 2023-11-06T01:34:31.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f835c1aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b065ce3d0 con 0x557b066793f0 2023-11-06T01:34:31.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f835c1aab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b065c9cc0 con 0x557b0667a0f0 2023-11-06T01:34:31.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f833effd700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83340a06f0 0x7f83340a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f834ddb6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667aa70 0x557b06678e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f834ddb6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667aa70 0x557b06678e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40058/0 (socket says 172.21.15.53:40058) 2023-11-06T01:34:31.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f834ddb6700 1 -- 172.21.15.53:0/944097516 learned_addr learned my addr 172.21.15.53:0/944097516 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f834e5b7700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83340a06f0 0x7f83340a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.592+0000 7f834d5b5700 1 --2- 172.21.15.53:0/944097516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 0x557b06679870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f834d5b5700 1 --2- 172.21.15.53:0/944097516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b0667a0f0 0x557b0667a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f834ddb6700 1 -- 172.21.15.53:0/944097516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b0667a0f0 msgr2=0x557b0667a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f834ddb6700 1 --2- 172.21.15.53:0/944097516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b0667a0f0 0x557b0667a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f834ddb6700 1 -- 172.21.15.53:0/944097516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 msgr2=0x557b06679870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f834ddb6700 1 --2- 172.21.15.53:0/944097516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 0x557b06679870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f834ddb6700 1 -- 172.21.15.53:0/944097516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b063f7de0 con 0x557b0667aa70 2023-11-06T01:34:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f833effd700 1 -- 172.21.15.53:0/116970233 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f83440cb8d0 con 0x7f8348108f40 2023-11-06T01:34:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f834ddb6700 1 --2- 172.21.15.53:0/944097516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667aa70 0x557b06678e60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8340000f80 tx=0x7f8340009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27ad40f720905d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f834e5b7700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83340a06f0 0x7f83340a2bb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8344098db0 tx=0x7f8344093b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f8333fff700 1 -- 172.21.15.53:0/944097516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834000e040 con 0x557b0667aa70 2023-11-06T01:34:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f8333fff700 1 -- 172.21.15.53:0/944097516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83400074a0 con 0x557b0667aa70 2023-11-06T01:34:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.593+0000 7f8333fff700 1 -- 172.21.15.53:0/944097516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340005330 con 0x557b0667aa70 2023-11-06T01:34:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.594+0000 7f835c1aab80 1 -- 172.21.15.53:0/944097516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667aa70 msgr2=0x557b06678e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.594+0000 7f835c1aab80 1 --2- 172.21.15.53:0/944097516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667aa70 0x557b06678e60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8340000f80 tx=0x7f8340009c40 comp rx=0 tx=0).stop 2023-11-06T01:34:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.595+0000 7f835c1aab80 1 -- 172.21.15.53:0/944097516 shutdown_connections 2023-11-06T01:34:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.595+0000 7f835c1aab80 1 --2- 172.21.15.53:0/944097516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 0x557b06679870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.595+0000 7f835c1aab80 1 --2- 172.21.15.53:0/944097516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b0667a0f0 0x557b0667a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.595+0000 7f835c1aab80 1 --2- 172.21.15.53:0/944097516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667aa70 0x557b06678e60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.595+0000 7f835c1aab80 1 -- 172.21.15.53:0/944097516 >> 172.21.15.53:0/944097516 conn(0x557b065d80d0 msgr2=0x557b06673940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.595+0000 7f835c1aab80 1 -- 172.21.15.53:0/944097516 shutdown_connections 2023-11-06T01:34:31.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.595+0000 7f835c1aab80 1 -- 172.21.15.53:0/944097516 wait complete. 2023-11-06T01:34:31.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.596+0000 7f835c1aab80 1 -- start start 2023-11-06T01:34:31.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.596+0000 7f835c1aab80 1 Processor -- start 2023-11-06T01:34:31.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.596+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 0x557b06682920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.596+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 0x557b06682e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b0667aa70 0x557b06689ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f835c1aab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b06566510 con 0x557b0667a0f0 2023-11-06T01:34:31.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f835c1aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b065661b0 con 0x557b066793f0 2023-11-06T01:34:31.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f835c1aab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0644d920 con 0x557b0667aa70 2023-11-06T01:34:31.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f835c1aab80 10 client.0 _ll_register_callbacks cb 0x557b064b3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:31.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f834d5b5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 0x557b06682920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f834d5b5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b0667aa70 0x557b06689ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f834d5b5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 0x557b06682920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51504/0 (socket says 172.21.15.53:51504) 2023-11-06T01:34:31.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f834d5b5700 1 -- 172.21.15.53:0/1681855602 learned_addr learned my addr 172.21.15.53:0/1681855602 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f834ddb6700 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 0x557b06682e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f834d5b5700 1 -- 172.21.15.53:0/1681855602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b0667aa70 msgr2=0x557b06689ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f834d5b5700 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b0667aa70 0x557b06689ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f834d5b5700 1 -- 172.21.15.53:0/1681855602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 msgr2=0x557b06682e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f834d5b5700 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 0x557b06682e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f834d5b5700 1 -- 172.21.15.53:0/1681855602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8348110140 con 0x557b066793f0 2023-11-06T01:34:31.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.597+0000 7f834ddb6700 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 0x557b06682e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:34:31.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.598+0000 7f834d5b5700 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 0x557b06682920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f833801ae30 tx=0x7f8338007e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acbf7417e7c29553 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.598+0000 7f83317fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:31.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.598+0000 7f83317fa700 1 -- 172.21.15.53:0/1681855602 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338018dd0 con 0x557b066793f0 2023-11-06T01:34:31.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.598+0000 7f83317fa700 1 -- 172.21.15.53:0/1681855602 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f833801b410 con 0x557b066793f0 2023-11-06T01:34:31.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.598+0000 7f835c1aab80 1 -- 172.21.15.53:0/1681855602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b065f4d10 con 0x557b066793f0 2023-11-06T01:34:31.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f83317fa700 1 -- 172.21.15.53:0/1681855602 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83380014a0 con 0x557b066793f0 2023-11-06T01:34:31.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f835c1aab80 10 client.25124 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:31.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f835c1aab80 1 -- 172.21.15.53:0/1681855602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557b063e0c60 con 0x557b066793f0 2023-11-06T01:34:31.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f83317fa700 1 -- 172.21.15.53:0/1681855602 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f833802a020 con 0x557b066793f0 2023-11-06T01:34:31.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f83317fa700 1 -- 172.21.15.53:0/1681855602 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8338024490 con 0x557b066793f0 2023-11-06T01:34:31.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f835c1aab80 10 client.25124 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:31.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f835c1aab80 10 client.25124 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-11-06T01:34:31.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f835c1aab80 20 client.25124 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:31.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x557b0668e6b0 0x557b06690ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f835c1aab80 4 client.25124 mds_command: new command op to 24464 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:31.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f835c1aab80 1 -- 172.21.15.53:0/1681855602 --> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b06647e10 con 0x557b0668e6b0 2023-11-06T01:34:31.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.599+0000 7f834ddb6700 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x557b0668e6b0 0x557b06690ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.600+0000 7f834ddb6700 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x557b0668e6b0 0x557b06690ad0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42d6f3e750f300ee server_cookie=4d4eadd39bb2cf88 in_seq=0 out_seq=0 2023-11-06T01:34:31.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.601+0000 7f83317fa700 10 client.25124 ms_handle_connect on v2:172.21.15.62:6838/1997633670 2023-11-06T01:34:31.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.601+0000 7f83317fa700 1 -- 172.21.15.53:0/1681855602 <== mds.2 v2:172.21.15.62:6838/1997633670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557b06647e10 con 0x557b0668e6b0 2023-11-06T01:34:31.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.601+0000 7f83317fa700 10 client.25124 handle_command_reply: tid=0 2023-11-06T01:34:31.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.601+0000 7f835c1aab80 1 client.25124 shutdown 2023-11-06T01:34:31.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.601+0000 7f835c1aab80 1 -- 172.21.15.53:0/1681855602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 msgr2=0x557b06682920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.601+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 0x557b06682920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f833801ae30 tx=0x7f8338007e10 comp rx=0 tx=0).stop 2023-11-06T01:34:31.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.602+0000 7f835c1aab80 1 -- 172.21.15.53:0/1681855602 shutdown_connections 2023-11-06T01:34:31.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.602+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x557b0668e6b0 0x557b06690ad0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.602+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b066793f0 0x557b06682920 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.602+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b0667aa70 0x557b06689ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.602+0000 7f83317fa700 0 client.25124 ms_handle_reset on v2:172.21.15.62:6838/1997633670 2023-11-06T01:34:31.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.602+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1681855602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 0x557b06682e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.602+0000 7f835c1aab80 1 -- 172.21.15.53:0/1681855602 >> 172.21.15.53:0/1681855602 conn(0x557b065d80d0 msgr2=0x557b066730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.602+0000 7f835c1aab80 1 -- 172.21.15.53:0/1681855602 shutdown_connections 2023-11-06T01:34:31.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.602+0000 7f835c1aab80 1 -- 172.21.15.53:0/1681855602 wait complete. 2023-11-06T01:34:31.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.603+0000 7f835c1aab80 20 client.25124 trim_cache size 0 max 16384 2023-11-06T01:34:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.628+0000 7f835c1aab80 1 Processor -- start 2023-11-06T01:34:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.630+0000 7f835c1aab80 1 -- start start 2023-11-06T01:34:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.630+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b066793f0 0x557b06687e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.630+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 0x557b06681650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.630+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b0667aa70 0x557b0667d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.630+0000 7f835c1aab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b06647e10 con 0x557b0667a0f0 2023-11-06T01:34:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.630+0000 7f835c1aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b06683b10 con 0x557b0667aa70 2023-11-06T01:34:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.630+0000 7f835c1aab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b066745f0 con 0x557b066793f0 2023-11-06T01:34:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.634+0000 7f834ddb6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 0x557b06681650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.634+0000 7f834ddb6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 0x557b06681650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40078/0 (socket says 172.21.15.53:40078) 2023-11-06T01:34:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.634+0000 7f834ddb6700 1 -- 172.21.15.53:0/1076103289 learned_addr learned my addr 172.21.15.53:0/1076103289 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.635+0000 7f834ddb6700 1 -- 172.21.15.53:0/1076103289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b066793f0 msgr2=0x557b06687e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.635+0000 7f834ddb6700 1 --2- 172.21.15.53:0/1076103289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b066793f0 0x557b06687e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.635+0000 7f834ddb6700 1 -- 172.21.15.53:0/1076103289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b0667aa70 msgr2=0x557b0667d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.635+0000 7f834ddb6700 1 --2- 172.21.15.53:0/1076103289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b0667aa70 0x557b0667d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.635+0000 7f834ddb6700 1 -- 172.21.15.53:0/1076103289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b0641e250 con 0x557b0667a0f0 2023-11-06T01:34:31.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.635+0000 7f834ddb6700 1 --2- 172.21.15.53:0/1076103289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 0x557b06681650 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8340009cd0 tx=0x7f834000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bb91b2a3c7c339 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.636+0000 7f833cff9700 1 -- 172.21.15.53:0/1076103289 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834000e040 con 0x557b0667a0f0 2023-11-06T01:34:31.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.636+0000 7f833cff9700 1 -- 172.21.15.53:0/1076103289 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8340007610 con 0x557b0667a0f0 2023-11-06T01:34:31.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.636+0000 7f833cff9700 1 -- 172.21.15.53:0/1076103289 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83400056a0 con 0x557b0667a0f0 2023-11-06T01:34:31.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.637+0000 7f835c1aab80 1 -- 172.21.15.53:0/1076103289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 msgr2=0x557b06681650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.637+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1076103289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 0x557b06681650 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8340009cd0 tx=0x7f834000b7e0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.637+0000 7f835c1aab80 1 -- 172.21.15.53:0/1076103289 shutdown_connections 2023-11-06T01:34:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.637+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1076103289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b0667aa70 0x557b0667d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.638+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1076103289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b066793f0 0x557b06687e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.638+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1076103289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b0667a0f0 0x557b06681650 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.638+0000 7f835c1aab80 1 -- 172.21.15.53:0/1076103289 >> 172.21.15.53:0/1076103289 conn(0x557b065d80d0 msgr2=0x557b06673910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.641+0000 7f835c1aab80 1 -- 172.21.15.53:0/1076103289 shutdown_connections 2023-11-06T01:34:31.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.644+0000 7f835c1aab80 1 -- 172.21.15.53:0/1076103289 wait complete. 2023-11-06T01:34:31.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.644+0000 7f835c1aab80 1 -- start start 2023-11-06T01:34:31.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.644+0000 7f835c1aab80 1 Processor -- start 2023-11-06T01:34:31.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.645+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b0667a0f0 0x557b066e9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.645+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b066e9940 0x557b066ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.645+0000 7f835c1aab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b066e9e80 0x557b066f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.645+0000 7f835c1aab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b066f13b0 con 0x557b066e9e80 2023-11-06T01:34:31.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.645+0000 7f835c1aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b066f1520 con 0x557b0667a0f0 2023-11-06T01:34:31.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.645+0000 7f835c1aab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b067026f0 con 0x557b066e9940 2023-11-06T01:34:31.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.645+0000 7f835c1aab80 10 client.0 _ll_register_callbacks cb 0x7f833801a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:31.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.646+0000 7f834ddb6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b066e9940 0x557b066ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.646+0000 7f834ddb6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b066e9940 0x557b066ee6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54320/0 (socket says 172.21.15.53:54320) 2023-11-06T01:34:31.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.646+0000 7f834ddb6700 1 -- 172.21.15.53:0/1764169511 learned_addr learned my addr 172.21.15.53:0/1764169511 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.646+0000 7f834ddb6700 1 -- 172.21.15.53:0/1764169511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b0667a0f0 msgr2=0x557b066e9400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:34:31.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.646+0000 7f834ddb6700 1 --2- 172.21.15.53:0/1764169511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b0667a0f0 0x557b066e9400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.646+0000 7f834ddb6700 1 -- 172.21.15.53:0/1764169511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b066e9e80 msgr2=0x557b066f0ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:34:31.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.646+0000 7f834ddb6700 1 --2- 172.21.15.53:0/1764169511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b066e9e80 0x557b066f0ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.646+0000 7f834ddb6700 1 -- 172.21.15.53:0/1764169511 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b063f7de0 con 0x557b066e9940 2023-11-06T01:34:31.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.646+0000 7f834ddb6700 1 --2- 172.21.15.53:0/1764169511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b066e9940 0x557b066ee6d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f834000afe0 tx=0x7f8340003ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68016168454012fd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.646+0000 7f83317fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:34:31.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.647+0000 7f83317fa700 1 -- 172.21.15.53:0/1764169511 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834000e030 con 0x557b066e9940 2023-11-06T01:34:31.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.647+0000 7f83317fa700 1 -- 172.21.15.53:0/1764169511 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8340003cd0 con 0x557b066e9940 2023-11-06T01:34:31.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.647+0000 7f835c1aab80 1 -- 172.21.15.53:0/1764169511 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83400074a0 con 0x557b066e9940 2023-11-06T01:34:31.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.647+0000 7f83317fa700 1 -- 172.21.15.53:0/1764169511 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8340005400 con 0x557b066e9940 2023-11-06T01:34:31.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.647+0000 7f835c1aab80 10 client.25216 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:31.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.647+0000 7f835c1aab80 1 -- 172.21.15.53:0/1764169511 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557b063f62b0 con 0x557b066e9940 2023-11-06T01:34:31.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.648+0000 7f83317fa700 1 -- 172.21.15.53:0/1764169511 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83400074a0 con 0x557b066e9940 2023-11-06T01:34:31.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.648+0000 7f83317fa700 1 -- 172.21.15.53:0/1764169511 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f834000b500 con 0x557b066e9940 2023-11-06T01:34:31.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.648+0000 7f835c1aab80 10 client.25216 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:31.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.648+0000 7f835c1aab80 10 client.25216 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-11-06T01:34:31.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.648+0000 7f835c1aab80 20 client.25216 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:31.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.648+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1764169511 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x557b06703b70 0x557b06705f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.648+0000 7f835c1aab80 4 client.25216 mds_command: new command op to 24464 tid=0[{"prefix": "damage ls"}] 2023-11-06T01:34:31.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.648+0000 7f835c1aab80 1 -- 172.21.15.53:0/1764169511 --> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557b065c2d40 con 0x557b06703b70 2023-11-06T01:34:31.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.648+0000 7f834d5b5700 1 --2- 172.21.15.53:0/1764169511 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x557b06703b70 0x557b06705f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.649+0000 7f83317fa700 10 client.25216 ms_handle_connect on v2:172.21.15.62:6838/1997633670 2023-11-06T01:34:31.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.649+0000 7f834d5b5700 1 --2- 172.21.15.53:0/1764169511 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x557b06703b70 0x557b06705f90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2d7b5c954641182 server_cookie=72853d37c8d917e7 in_seq=0 out_seq=0 2023-11-06T01:34:31.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.649+0000 7f83317fa700 1 -- 172.21.15.53:0/1764169511 <== mds.2 v2:172.21.15.62:6838/1997633670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557b065c2d40 con 0x557b06703b70 2023-11-06T01:34:31.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.649+0000 7f83317fa700 10 client.25216 handle_command_reply: tid=0 2023-11-06T01:34:31.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.649+0000 7f835c1aab80 1 client.25216 shutdown 2023-11-06T01:34:31.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.650+0000 7f835c1aab80 1 -- 172.21.15.53:0/1764169511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b066e9940 msgr2=0x557b066ee6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.650+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1764169511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b066e9940 0x557b066ee6d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f834000afe0 tx=0x7f8340003ab0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.651+0000 7f835c1aab80 1 -- 172.21.15.53:0/1764169511 shutdown_connections 2023-11-06T01:34:31.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.651+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1764169511 >> [v2:172.21.15.62:6838/1997633670,v1:172.21.15.62:6839/1997633670] conn(0x557b06703b70 0x557b06705f90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.651+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1764169511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b0667a0f0 0x557b066e9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.651+0000 7f83317fa700 0 client.25216 ms_handle_reset on v2:172.21.15.62:6838/1997633670 2023-11-06T01:34:31.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.651+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1764169511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557b066e9e80 0x557b066f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.651+0000 7f835c1aab80 1 --2- 172.21.15.53:0/1764169511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b066e9940 0x557b066ee6d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.651+0000 7f835c1aab80 1 -- 172.21.15.53:0/1764169511 >> 172.21.15.53:0/1764169511 conn(0x557b065d80d0 msgr2=0x557b06673880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.651+0000 7f835c1aab80 1 -- 172.21.15.53:0/1764169511 shutdown_connections 2023-11-06T01:34:31.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.651+0000 7f835c1aab80 1 -- 172.21.15.53:0/1764169511 wait complete. 2023-11-06T01:34:31.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.651+0000 7f835c1aab80 20 client.25216 trim_cache size 0 max 16384 2023-11-06T01:34:31.663 INFO:teuthology.orchestra.run.smithi053.stdout:[] 2023-11-06T01:34:31.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.654+0000 7f835001a700 1 -- 172.21.15.53:0/116970233 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83340a06f0 msgr2=0x7f83340a2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.654+0000 7f835001a700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83340a06f0 0x7f83340a2bb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8344098db0 tx=0x7f8344093b10 comp rx=0 tx=0).stop 2023-11-06T01:34:31.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.654+0000 7f835001a700 1 -- 172.21.15.53:0/116970233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8348108f40 msgr2=0x7f83481a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.654+0000 7f835001a700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8348108f40 0x7f83481a47a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f834400c900 tx=0x7f834400ccc0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.655+0000 7f835001a700 1 -- 172.21.15.53:0/116970233 shutdown_connections 2023-11-06T01:34:31.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.655+0000 7f835001a700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83340a06f0 0x7f83340a2bb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.655+0000 7f835001a700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8348108f40 0x7f83481a47a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.655+0000 7f835001a700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83481073b0 0x7f834819d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.655+0000 7f835001a700 1 --2- 172.21.15.53:0/116970233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83481085a0 0x7f834819d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.655+0000 7f835001a700 1 -- 172.21.15.53:0/116970233 >> 172.21.15.53:0/116970233 conn(0x7f8348076330 msgr2=0x7f8348078500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.655+0000 7f835001a700 1 -- 172.21.15.53:0/116970233 shutdown_connections 2023-11-06T01:34:31.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.656+0000 7f835001a700 1 -- 172.21.15.53:0/116970233 wait complete. 2023-11-06T01:34:31.674 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:31 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/139266274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:34:31.675 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-06T01:34:31.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:31 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/139266274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:34:31.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.876+0000 7f143fc29700 1 -- 172.21.15.53:0/1217809545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f143810a9c0 msgr2=0x7f143810ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.876+0000 7f143fc29700 1 --2- 172.21.15.53:0/1217809545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f143810a9c0 0x7f143810ce50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1428009990 tx=0x7f1428009ca0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.877+0000 7f143fc29700 1 -- 172.21.15.53:0/1217809545 shutdown_connections 2023-11-06T01:34:31.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.877+0000 7f143fc29700 1 --2- 172.21.15.53:0/1217809545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f143810d4e0 0x7f143810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.877+0000 7f143fc29700 1 --2- 172.21.15.53:0/1217809545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1438108000 0x7f143810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.877+0000 7f143fc29700 1 --2- 172.21.15.53:0/1217809545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f143810a9c0 0x7f143810ce50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.877+0000 7f143fc29700 1 -- 172.21.15.53:0/1217809545 >> 172.21.15.53:0/1217809545 conn(0x7f143806df90 msgr2=0x7f14380703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.878+0000 7f143fc29700 1 -- 172.21.15.53:0/1217809545 shutdown_connections 2023-11-06T01:34:31.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.878+0000 7f143fc29700 1 -- 172.21.15.53:0/1217809545 wait complete. 2023-11-06T01:34:31.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.878+0000 7f143fc29700 1 Processor -- start 2023-11-06T01:34:31.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.878+0000 7f143fc29700 1 -- start start 2023-11-06T01:34:31.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143fc29700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1438108000 0x7f14381a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143fc29700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f143810a9c0 0x7f14381a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143fc29700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f143810d4e0 0x7f14381ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143fc29700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14381ad450 con 0x7f1438108000 2023-11-06T01:34:31.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143fc29700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14381ad590 con 0x7f143810a9c0 2023-11-06T01:34:31.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143fc29700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14381ad720 con 0x7f143810d4e0 2023-11-06T01:34:31.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143d9c5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1438108000 0x7f14381a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143d9c5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1438108000 0x7f14381a58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40088/0 (socket says 172.21.15.53:40088) 2023-11-06T01:34:31.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143d9c5700 1 -- 172.21.15.53:0/666263829 learned_addr learned my addr 172.21.15.53:0/666263829 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143e1c6700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f143810d4e0 0x7f14381ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143e1c6700 1 -- 172.21.15.53:0/666263829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f143810a9c0 msgr2=0x7f14381a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143e1c6700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f143810a9c0 0x7f14381a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143e1c6700 1 -- 172.21.15.53:0/666263829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1438108000 msgr2=0x7f14381a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.879+0000 7f143e1c6700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1438108000 0x7f14381a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.880+0000 7f143e1c6700 1 -- 172.21.15.53:0/666263829 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14280095f0 con 0x7f143810d4e0 2023-11-06T01:34:31.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.880+0000 7f143e1c6700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f143810d4e0 0x7f14381ace80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f143000efc0 tx=0x7f143000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.880+0000 7f142effd700 1 -- 172.21.15.53:0/666263829 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430018070 con 0x7f143810d4e0 2023-11-06T01:34:31.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.881+0000 7f142effd700 1 -- 172.21.15.53:0/666263829 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f143000f040 con 0x7f143810d4e0 2023-11-06T01:34:31.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.881+0000 7f142effd700 1 -- 172.21.15.53:0/666263829 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430003e30 con 0x7f143810d4e0 2023-11-06T01:34:31.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.881+0000 7f143fc29700 1 -- 172.21.15.53:0/666263829 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14381ad9a0 con 0x7f143810d4e0 2023-11-06T01:34:31.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.881+0000 7f143fc29700 1 -- 172.21.15.53:0/666263829 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14381ade60 con 0x7f143810d4e0 2023-11-06T01:34:31.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.884+0000 7f142effd700 1 -- 172.21.15.53:0/666263829 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f1430004ad0 con 0x7f143810d4e0 2023-11-06T01:34:31.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.885+0000 7f142effd700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14240a0650 0x7f14240a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.885+0000 7f142effd700 1 -- 172.21.15.53:0/666263829 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f14300cb5b0 con 0x7f143810d4e0 2023-11-06T01:34:31.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.885+0000 7f143d9c5700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14240a0650 0x7f14240a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.885+0000 7f144bdb9b80 1 Processor -- start 2023-11-06T01:34:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f144bdb9b80 1 -- start start 2023-11-06T01:34:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 0x55ce31d09540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 0x55ce31d07e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31d08880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f143d9c5700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14240a0650 0x7f14240a2b10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1434005950 tx=0x7f1434009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31c4ea10 con 0x55ce31d08400 2023-11-06T01:34:31.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31c4c3d0 con 0x55ce31d09100 2023-11-06T01:34:31.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31c47cc0 con 0x55ce31d09a80 2023-11-06T01:34:31.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f143d9c5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 0x55ce31d09540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f143d9c5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 0x55ce31d09540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51532/0 (socket says 172.21.15.53:51532) 2023-11-06T01:34:31.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f143d9c5700 1 -- 172.21.15.53:0/2152115712 learned_addr learned my addr 172.21.15.53:0/2152115712 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.886+0000 7f143d1c4700 1 --2- 172.21.15.53:0/2152115712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 0x55ce31d07e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.887+0000 7f143e1c6700 1 --2- 172.21.15.53:0/2152115712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31d08880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.887+0000 7f143d9c5700 1 -- 172.21.15.53:0/2152115712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 msgr2=0x55ce31d07e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.887+0000 7f143d9c5700 1 --2- 172.21.15.53:0/2152115712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 0x55ce31d07e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.887+0000 7f143d9c5700 1 -- 172.21.15.53:0/2152115712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 msgr2=0x55ce31d08880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.887+0000 7f143d9c5700 1 --2- 172.21.15.53:0/2152115712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31d08880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.887+0000 7f143d9c5700 1 -- 172.21.15.53:0/2152115712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce31a75de0 con 0x55ce31d09100 2023-11-06T01:34:31.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.887+0000 7f143e1c6700 1 --2- 172.21.15.53:0/2152115712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31d08880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:34:31.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.887+0000 7f143d9c5700 1 --2- 172.21.15.53:0/2152115712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 0x55ce31d09540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1434011ed0 tx=0x7f1434014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb25205a9858c5f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.888+0000 7f1423fff700 1 -- 172.21.15.53:0/2152115712 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434012dc0 con 0x55ce31d09100 2023-11-06T01:34:31.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.888+0000 7f1423fff700 1 -- 172.21.15.53:0/2152115712 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1434016070 con 0x55ce31d09100 2023-11-06T01:34:31.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.888+0000 7f1423fff700 1 -- 172.21.15.53:0/2152115712 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143403f620 con 0x55ce31d09100 2023-11-06T01:34:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.889+0000 7f144bdb9b80 1 -- 172.21.15.53:0/2152115712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 msgr2=0x55ce31d09540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.889+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/2152115712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 0x55ce31d09540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1434011ed0 tx=0x7f1434014000 comp rx=0 tx=0).stop 2023-11-06T01:34:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.889+0000 7f144bdb9b80 1 -- 172.21.15.53:0/2152115712 shutdown_connections 2023-11-06T01:34:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.889+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/2152115712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 0x55ce31d09540 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.889+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/2152115712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31d08880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.889+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/2152115712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 0x55ce31d07e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.889+0000 7f144bdb9b80 1 -- 172.21.15.53:0/2152115712 >> 172.21.15.53:0/2152115712 conn(0x55ce31c560d0 msgr2=0x55ce31cf1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.889+0000 7f144bdb9b80 1 -- 172.21.15.53:0/2152115712 shutdown_connections 2023-11-06T01:34:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.889+0000 7f144bdb9b80 1 -- 172.21.15.53:0/2152115712 wait complete. 2023-11-06T01:34:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.889+0000 7f144bdb9b80 1 -- start start 2023-11-06T01:34:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f144bdb9b80 1 Processor -- start 2023-11-06T01:34:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d08400 0x55ce31cf8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09100 0x55ce31cf8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d09a80 0x55ce31d00010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31be4510 con 0x55ce31d09a80 2023-11-06T01:34:31.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31be41b0 con 0x55ce31d08400 2023-11-06T01:34:31.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31acb920 con 0x55ce31d09100 2023-11-06T01:34:31.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f144bdb9b80 10 client.0 _ll_register_callbacks cb 0x55ce31b31a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:31.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f143d9c5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d08400 0x55ce31cf8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f143d1c4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09100 0x55ce31cf8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f143d9c5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d08400 0x55ce31cf8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51538/0 (socket says 172.21.15.53:51538) 2023-11-06T01:34:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f143e1c6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d09a80 0x55ce31d00010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f143d9c5700 1 -- 172.21.15.53:0/930121144 learned_addr learned my addr 172.21.15.53:0/930121144 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f143d9c5700 1 -- 172.21.15.53:0/930121144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09100 msgr2=0x55ce31cf8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f143d9c5700 1 --2- 172.21.15.53:0/930121144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09100 0x55ce31cf8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f143d9c5700 1 -- 172.21.15.53:0/930121144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d09a80 msgr2=0x55ce31d00010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f143d9c5700 1 --2- 172.21.15.53:0/930121144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d09a80 0x55ce31d00010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.890+0000 7f143d9c5700 1 -- 172.21.15.53:0/930121144 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1434011c60 con 0x55ce31d08400 2023-11-06T01:34:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.891+0000 7f143d9c5700 1 --2- 172.21.15.53:0/930121144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d08400 0x55ce31cf8a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f14340416b0 tx=0x7f14340416e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b4676a3f1149509 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.891+0000 7f14217fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:31.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.891+0000 7f14217fa700 1 -- 172.21.15.53:0/930121144 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143404f070 con 0x55ce31d08400 2023-11-06T01:34:31.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.891+0000 7f14217fa700 1 -- 172.21.15.53:0/930121144 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1434018070 con 0x55ce31d08400 2023-11-06T01:34:31.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.891+0000 7f144bdb9b80 1 -- 172.21.15.53:0/930121144 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce31c72d10 con 0x55ce31d08400 2023-11-06T01:34:31.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f14217fa700 1 -- 172.21.15.53:0/930121144 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1434005ce0 con 0x55ce31d08400 2023-11-06T01:34:31.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f144bdb9b80 10 client.25133 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:31.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f144bdb9b80 1 -- 172.21.15.53:0/930121144 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ce31a5ec60 con 0x55ce31d08400 2023-11-06T01:34:31.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f14217fa700 1 -- 172.21.15.53:0/930121144 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1434063080 con 0x55ce31d08400 2023-11-06T01:34:31.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f14217fa700 1 -- 172.21.15.53:0/930121144 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f143405d030 con 0x55ce31d08400 2023-11-06T01:34:31.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f144bdb9b80 10 client.25133 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:31.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f144bdb9b80 10 client.25133 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2023-11-06T01:34:31.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f144bdb9b80 20 client.25133 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:31.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/930121144 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x55ce31d046e0 0x55ce31d06b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f144bdb9b80 4 client.25133 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:31.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f144bdb9b80 1 -- 172.21.15.53:0/930121144 --> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce31cc5e10 con 0x55ce31d046e0 2023-11-06T01:34:31.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.892+0000 7f143d1c4700 1 --2- 172.21.15.53:0/930121144 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x55ce31d046e0 0x55ce31d06b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.893+0000 7f143d1c4700 1 --2- 172.21.15.53:0/930121144 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x55ce31d046e0 0x55ce31d06b00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7bebdf698583585a server_cookie=48416de35e263d54 in_seq=0 out_seq=0 2023-11-06T01:34:31.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.893+0000 7f14217fa700 10 client.25133 ms_handle_connect on v2:172.21.15.163:6836/1759271265 2023-11-06T01:34:31.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.894+0000 7f14217fa700 1 -- 172.21.15.53:0/930121144 <== mds.1 v2:172.21.15.163:6836/1759271265 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ce31cc5e10 con 0x55ce31d046e0 2023-11-06T01:34:31.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.894+0000 7f14217fa700 10 client.25133 handle_command_reply: tid=0 2023-11-06T01:34:31.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f144bdb9b80 1 client.25133 shutdown 2023-11-06T01:34:31.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f144bdb9b80 1 -- 172.21.15.53:0/930121144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d08400 msgr2=0x55ce31cf8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/930121144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d08400 0x55ce31cf8a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f14340416b0 tx=0x7f14340416e0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f144bdb9b80 1 -- 172.21.15.53:0/930121144 shutdown_connections 2023-11-06T01:34:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/930121144 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x55ce31d046e0 0x55ce31d06b00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/930121144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d08400 0x55ce31cf8a90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/930121144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d09a80 0x55ce31d00010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f14217fa700 0 client.25133 ms_handle_reset on v2:172.21.15.163:6836/1759271265 2023-11-06T01:34:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/930121144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09100 0x55ce31cf8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f144bdb9b80 1 -- 172.21.15.53:0/930121144 >> 172.21.15.53:0/930121144 conn(0x55ce31c560d0 msgr2=0x55ce31cf10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f144bdb9b80 1 -- 172.21.15.53:0/930121144 shutdown_connections 2023-11-06T01:34:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.895+0000 7f144bdb9b80 1 -- 172.21.15.53:0/930121144 wait complete. 2023-11-06T01:34:31.909 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:31 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/139266274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:34:31.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.896+0000 7f144bdb9b80 20 client.25133 trim_cache size 0 max 16384 2023-11-06T01:34:31.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.913+0000 7f144bdb9b80 1 Processor -- start 2023-11-06T01:34:31.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.914+0000 7f144bdb9b80 1 -- start start 2023-11-06T01:34:31.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.914+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31cfdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.914+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 0x55ce31cf8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.914+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 0x55ce31cf2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.914+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31cf7060 con 0x55ce31d08400 2023-11-06T01:34:31.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.914+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31cec660 con 0x55ce31d09100 2023-11-06T01:34:31.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.914+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31cf7f40 con 0x55ce31d09a80 2023-11-06T01:34:31.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.915+0000 7f143e1c6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 0x55ce31cf2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.915+0000 7f143e1c6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 0x55ce31cf2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54348/0 (socket says 172.21.15.53:54348) 2023-11-06T01:34:31.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.915+0000 7f143e1c6700 1 -- 172.21.15.53:0/1867588601 learned_addr learned my addr 172.21.15.53:0/1867588601 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.915+0000 7f143d1c4700 1 --2- 172.21.15.53:0/1867588601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 0x55ce31cf8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.915+0000 7f143e1c6700 1 -- 172.21.15.53:0/1867588601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 msgr2=0x55ce31cf8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.915+0000 7f143e1c6700 1 --2- 172.21.15.53:0/1867588601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 0x55ce31cf8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.915+0000 7f143e1c6700 1 -- 172.21.15.53:0/1867588601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 msgr2=0x55ce31cfdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.915+0000 7f143e1c6700 1 --2- 172.21.15.53:0/1867588601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31cfdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.915+0000 7f143e1c6700 1 -- 172.21.15.53:0/1867588601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14381202b0 con 0x55ce31d09a80 2023-11-06T01:34:31.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.916+0000 7f143e1c6700 1 --2- 172.21.15.53:0/1867588601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 0x55ce31cf2240 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14300958f0 tx=0x7f1430098860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9437855a414c3f3d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.916+0000 7f142cdaa700 1 -- 172.21.15.53:0/1867588601 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430098e80 con 0x55ce31d09a80 2023-11-06T01:34:31.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.916+0000 7f142cdaa700 1 -- 172.21.15.53:0/1867588601 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1430099e90 con 0x55ce31d09a80 2023-11-06T01:34:31.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.916+0000 7f142cdaa700 1 -- 172.21.15.53:0/1867588601 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430098370 con 0x55ce31d09a80 2023-11-06T01:34:31.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.917+0000 7f144bdb9b80 1 -- 172.21.15.53:0/1867588601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 msgr2=0x55ce31cf2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.917+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/1867588601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 0x55ce31cf2240 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14300958f0 tx=0x7f1430098860 comp rx=0 tx=0).stop 2023-11-06T01:34:31.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.917+0000 7f144bdb9b80 1 -- 172.21.15.53:0/1867588601 shutdown_connections 2023-11-06T01:34:31.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.917+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/1867588601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09a80 0x55ce31cf2240 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.917+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/1867588601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31cfdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.917+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/1867588601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09100 0x55ce31cf8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.917+0000 7f144bdb9b80 1 -- 172.21.15.53:0/1867588601 >> 172.21.15.53:0/1867588601 conn(0x55ce31c560d0 msgr2=0x55ce31cedd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.917+0000 7f144bdb9b80 1 -- 172.21.15.53:0/1867588601 shutdown_connections 2023-11-06T01:34:31.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.917+0000 7f144bdb9b80 1 -- 172.21.15.53:0/1867588601 wait complete. 2023-11-06T01:34:31.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.918+0000 7f144bdb9b80 1 -- start start 2023-11-06T01:34:31.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.918+0000 7f144bdb9b80 1 Processor -- start 2023-11-06T01:34:31.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.918+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31cfd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.918+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09100 0x55ce31cfd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.918+0000 7f144bdb9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09a80 0x55ce31d5d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.918+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31d5d820 con 0x55ce31d08400 2023-11-06T01:34:31.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.918+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31d5d960 con 0x55ce31d09a80 2023-11-06T01:34:31.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.918+0000 7f144bdb9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce31d5daf0 con 0x55ce31d09100 2023-11-06T01:34:31.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f144bdb9b80 10 client.0 _ll_register_callbacks cb 0x7f1434005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:31.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143d9c5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31cfd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143d1c4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09100 0x55ce31cfd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143e1c6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09a80 0x55ce31d5d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143d1c4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09100 0x55ce31cfd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54354/0 (socket says 172.21.15.53:54354) 2023-11-06T01:34:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143d1c4700 1 -- 172.21.15.53:0/65118444 learned_addr learned my addr 172.21.15.53:0/65118444 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143e1c6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09a80 0x55ce31d5d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51556/0 (socket says 172.21.15.53:51556) 2023-11-06T01:34:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143e1c6700 1 -- 172.21.15.53:0/65118444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09100 msgr2=0x55ce31cfd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143e1c6700 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09100 0x55ce31cfd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143e1c6700 1 -- 172.21.15.53:0/65118444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 msgr2=0x55ce31cfd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143e1c6700 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31cfd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143e1c6700 1 -- 172.21.15.53:0/65118444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14381ade60 con 0x55ce31d09a80 2023-11-06T01:34:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143d9c5700 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31cfd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f143e1c6700 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09a80 0x55ce31d5d2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14300c90e0 tx=0x7f1430009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b9a528c72da2014 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f14217fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f14217fa700 1 -- 172.21.15.53:0/65118444 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14300b5070 con 0x55ce31d09a80 2023-11-06T01:34:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f14217fa700 1 -- 172.21.15.53:0/65118444 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1430098e80 con 0x55ce31d09a80 2023-11-06T01:34:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.919+0000 7f144bdb9b80 1 -- 172.21.15.53:0/65118444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1434016070 con 0x55ce31d09a80 2023-11-06T01:34:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.920+0000 7f14217fa700 1 -- 172.21.15.53:0/65118444 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14300a0e60 con 0x55ce31d09a80 2023-11-06T01:34:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.920+0000 7f144bdb9b80 10 client.25139 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.920+0000 7f144bdb9b80 1 -- 172.21.15.53:0/65118444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ce31a9c250 con 0x55ce31d09a80 2023-11-06T01:34:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.922+0000 7f14217fa700 1 -- 172.21.15.53:0/65118444 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1434016070 con 0x55ce31d09a80 2023-11-06T01:34:31.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.922+0000 7f14217fa700 1 -- 172.21.15.53:0/65118444 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f14300cb220 con 0x55ce31d09a80 2023-11-06T01:34:31.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.922+0000 7f144bdb9b80 10 client.25139 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:31.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.922+0000 7f144bdb9b80 10 client.25139 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2023-11-06T01:34:31.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.922+0000 7f144bdb9b80 20 client.25139 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:31.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.922+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x55ce31d5f030 0x55ce31d61420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:31.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.922+0000 7f144bdb9b80 4 client.25139 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-06T01:34:31.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.922+0000 7f144bdb9b80 1 -- 172.21.15.53:0/65118444 --> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ce31c40d40 con 0x55ce31d5f030 2023-11-06T01:34:31.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.922+0000 7f143d9c5700 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x55ce31d5f030 0x55ce31d61420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:31.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f143d9c5700 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x55ce31d5f030 0x55ce31d61420 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbe8a55d1de21797 server_cookie=5a97c4a3d98026ee in_seq=0 out_seq=0 2023-11-06T01:34:31.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f14217fa700 10 client.25139 ms_handle_connect on v2:172.21.15.163:6836/1759271265 2023-11-06T01:34:31.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f14217fa700 1 -- 172.21.15.53:0/65118444 <== mds.1 v2:172.21.15.163:6836/1759271265 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ce31c40d40 con 0x55ce31d5f030 2023-11-06T01:34:31.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f14217fa700 10 client.25139 handle_command_reply: tid=0 2023-11-06T01:34:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f144bdb9b80 1 client.25139 shutdown 2023-11-06T01:34:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f144bdb9b80 1 -- 172.21.15.53:0/65118444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09a80 msgr2=0x55ce31d5d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09a80 0x55ce31d5d2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14300c90e0 tx=0x7f1430009450 comp rx=0 tx=0).stop 2023-11-06T01:34:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f144bdb9b80 1 -- 172.21.15.53:0/65118444 shutdown_connections 2023-11-06T01:34:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.163:6836/1759271265,v1:172.21.15.163:6837/1759271265] conn(0x55ce31d5f030 0x55ce31d61420 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce31d09a80 0x55ce31d5d2e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce31d08400 0x55ce31cfd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f14217fa700 0 client.25139 ms_handle_reset on v2:172.21.15.163:6836/1759271265 2023-11-06T01:34:31.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f144bdb9b80 1 --2- 172.21.15.53:0/65118444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ce31d09100 0x55ce31cfd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.924+0000 7f144bdb9b80 1 -- 172.21.15.53:0/65118444 >> 172.21.15.53:0/65118444 conn(0x55ce31c560d0 msgr2=0x55ce31cf2240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.925+0000 7f144bdb9b80 1 -- 172.21.15.53:0/65118444 shutdown_connections 2023-11-06T01:34:31.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.926+0000 7f144bdb9b80 1 -- 172.21.15.53:0/65118444 wait complete. 2023-11-06T01:34:31.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.926+0000 7f144bdb9b80 20 client.25139 trim_cache size 0 max 16384 2023-11-06T01:34:31.937 INFO:teuthology.orchestra.run.smithi053.stdout:[] 2023-11-06T01:34:31.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.927+0000 7f143fc29700 1 -- 172.21.15.53:0/666263829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14240a0650 msgr2=0x7f14240a2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.927+0000 7f143fc29700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14240a0650 0x7f14240a2b10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1434005950 tx=0x7f1434009260 comp rx=0 tx=0).stop 2023-11-06T01:34:31.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.927+0000 7f143fc29700 1 -- 172.21.15.53:0/666263829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f143810d4e0 msgr2=0x7f14381ace80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:31.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.927+0000 7f143fc29700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f143810d4e0 0x7f14381ace80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f143000efc0 tx=0x7f143000c3a0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.928+0000 7f143fc29700 1 -- 172.21.15.53:0/666263829 shutdown_connections 2023-11-06T01:34:31.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.928+0000 7f143fc29700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f14240a0650 0x7f14240a2b10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.928+0000 7f143fc29700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f143810d4e0 0x7f14381ace80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.928+0000 7f143fc29700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1438108000 0x7f14381a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.928+0000 7f143fc29700 1 --2- 172.21.15.53:0/666263829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f143810a9c0 0x7f14381a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:31.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.928+0000 7f143fc29700 1 -- 172.21.15.53:0/666263829 >> 172.21.15.53:0/666263829 conn(0x7f143806df90 msgr2=0x7f143810b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:31.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.928+0000 7f143fc29700 1 -- 172.21.15.53:0/666263829 shutdown_connections 2023-11-06T01:34:31.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:31.928+0000 7f143fc29700 1 -- 172.21.15.53:0/666263829 wait complete. 2023-11-06T01:34:31.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-06T01:34:32.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.167+0000 7f26c921e700 1 -- 172.21.15.53:0/3404384537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c4100f50 msgr2=0x7f26c4107870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:32.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.167+0000 7f26c921e700 1 --2- 172.21.15.53:0/3404384537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c4100f50 0x7f26c4107870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009990 tx=0x7f26bc009ca0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.167+0000 7f26c921e700 1 -- 172.21.15.53:0/3404384537 shutdown_connections 2023-11-06T01:34:32.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.167+0000 7f26c921e700 1 --2- 172.21.15.53:0/3404384537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c4100f50 0x7f26c4107870 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.167+0000 7f26c921e700 1 --2- 172.21.15.53:0/3404384537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26c40ff5e0 0x7f26c40ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.167+0000 7f26c921e700 1 --2- 172.21.15.53:0/3404384537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c4100590 0x7f26c4100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.167+0000 7f26c921e700 1 -- 172.21.15.53:0/3404384537 >> 172.21.15.53:0/3404384537 conn(0x7f26c40fa520 msgr2=0x7f26c40fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:32.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.168+0000 7f26c921e700 1 -- 172.21.15.53:0/3404384537 shutdown_connections 2023-11-06T01:34:32.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.168+0000 7f26c921e700 1 -- 172.21.15.53:0/3404384537 wait complete. 2023-11-06T01:34:32.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26c921e700 1 Processor -- start 2023-11-06T01:34:32.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26c921e700 1 -- start start 2023-11-06T01:34:32.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26c921e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26c40ff5e0 0x7f26c410fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26c921e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c4100590 0x7f26c41100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26c921e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c4100f50 0x7f26c4117110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26c921e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c41176e0 con 0x7f26c4100590 2023-11-06T01:34:32.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26c921e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c4117850 con 0x7f26c4100f50 2023-11-06T01:34:32.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26c921e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c4117a10 con 0x7f26c40ff5e0 2023-11-06T01:34:32.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26c2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26c40ff5e0 0x7f26c410fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26ba59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c4100590 0x7f26c41100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26c2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26c40ff5e0 0x7f26c410fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54368/0 (socket says 172.21.15.53:54368) 2023-11-06T01:34:32.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.169+0000 7f26c2d9d700 1 -- 172.21.15.53:0/2722783952 learned_addr learned my addr 172.21.15.53:0/2722783952 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:32.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.170+0000 7f26c359e700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c4100f50 0x7f26c4117110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.170+0000 7f26c2d9d700 1 -- 172.21.15.53:0/2722783952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c4100f50 msgr2=0x7f26c4117110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:32.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.170+0000 7f26c2d9d700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c4100f50 0x7f26c4117110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.170+0000 7f26c2d9d700 1 -- 172.21.15.53:0/2722783952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c4100590 msgr2=0x7f26c41100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:32.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.170+0000 7f26c2d9d700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c4100590 0x7f26c41100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.170+0000 7f26c2d9d700 1 -- 172.21.15.53:0/2722783952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc0095f0 con 0x7f26c40ff5e0 2023-11-06T01:34:32.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.170+0000 7f26c359e700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c4100f50 0x7f26c4117110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:32.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.170+0000 7f26ba59c700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c4100590 0x7f26c41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:32.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.170+0000 7f26c2d9d700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26c40ff5e0 0x7f26c410fb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f26b400ba50 tx=0x7f26b400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:32.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.170+0000 7f26c0d99700 1 -- 172.21.15.53:0/2722783952 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b400c930 con 0x7f26c40ff5e0 2023-11-06T01:34:32.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.170+0000 7f26c921e700 1 -- 172.21.15.53:0/2722783952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c4117c90 con 0x7f26c40ff5e0 2023-11-06T01:34:32.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.171+0000 7f26c921e700 1 -- 172.21.15.53:0/2722783952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c41181f0 con 0x7f26c40ff5e0 2023-11-06T01:34:32.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.173+0000 7f26c0d99700 1 -- 172.21.15.53:0/2722783952 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b4014440 con 0x7f26c40ff5e0 2023-11-06T01:34:32.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.173+0000 7f26c0d99700 1 -- 172.21.15.53:0/2722783952 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b40075d0 con 0x7f26c40ff5e0 2023-11-06T01:34:32.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.176+0000 7f26c0d99700 1 -- 172.21.15.53:0/2722783952 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f26b40077f0 con 0x7f26c40ff5e0 2023-11-06T01:34:32.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.176+0000 7f26d53aeb80 1 Processor -- start 2023-11-06T01:34:32.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.176+0000 7f26d53aeb80 1 -- start start 2023-11-06T01:34:32.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.177+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.177+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 0x5650171b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.177+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.177+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017109a10 con 0x5650171b23f0 2023-11-06T01:34:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.177+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650171073d0 con 0x5650171b3a70 2023-11-06T01:34:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.177+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017102cc0 con 0x5650171b30f0 2023-11-06T01:34:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.177+0000 7f26c2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.177+0000 7f26c2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54376/0 (socket says 172.21.15.53:54376) 2023-11-06T01:34:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.177+0000 7f26c2d9d700 1 -- 172.21.15.53:0/4213591350 learned_addr learned my addr 172.21.15.53:0/4213591350 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.177+0000 7f26ba59c700 1 --2- 172.21.15.53:0/4213591350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 0x5650171b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.177+0000 7f26c359e700 1 --2- 172.21.15.53:0/4213591350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.178+0000 7f26c0d99700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f26b00a05a0 0x7f26b00a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.178+0000 7f26c0d99700 1 -- 172.21.15.53:0/2722783952 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f26b4006e40 con 0x7f26c40ff5e0 2023-11-06T01:34:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.178+0000 7f26c2d9d700 1 -- 172.21.15.53:0/4213591350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 msgr2=0x5650171b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:32.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.178+0000 7f26c2d9d700 1 --2- 172.21.15.53:0/4213591350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 0x5650171b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.178+0000 7f26c2d9d700 1 -- 172.21.15.53:0/4213591350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 msgr2=0x5650171b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:32.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.178+0000 7f26c2d9d700 1 --2- 172.21.15.53:0/4213591350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.178+0000 7f26c2d9d700 1 -- 172.21.15.53:0/4213591350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565016f30de0 con 0x5650171b30f0 2023-11-06T01:34:32.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.178+0000 7f26ba59c700 1 --2- 172.21.15.53:0/4213591350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 0x5650171b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:34:32.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.178+0000 7f26c359e700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f26b00a05a0 0x7f26b00a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.178+0000 7f26c2d9d700 1 --2- 172.21.15.53:0/4213591350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171b3530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26b4012040 tx=0x7f26b4004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec01b55b184fc186 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:32.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.179+0000 7f26b959a700 1 -- 172.21.15.53:0/4213591350 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4099410 con 0x5650171b30f0 2023-11-06T01:34:32.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.179+0000 7f26b959a700 1 -- 172.21.15.53:0/4213591350 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b4099230 con 0x5650171b30f0 2023-11-06T01:34:32.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.179+0000 7f26b959a700 1 -- 172.21.15.53:0/4213591350 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4097c70 con 0x5650171b30f0 2023-11-06T01:34:32.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.180+0000 7f26d53aeb80 1 -- 172.21.15.53:0/4213591350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 msgr2=0x5650171b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:32.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.180+0000 7f26c359e700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f26b00a05a0 0x7f26b00a2a60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00b5c0 tx=0x7f26bc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:32.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.180+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/4213591350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171b3530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26b4012040 tx=0x7f26b4004be0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.181+0000 7f26d53aeb80 1 -- 172.21.15.53:0/4213591350 shutdown_connections 2023-11-06T01:34:32.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.181+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/4213591350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.181+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/4213591350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171b3530 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.181+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/4213591350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 0x5650171b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.181+0000 7f26d53aeb80 1 -- 172.21.15.53:0/4213591350 >> 172.21.15.53:0/4213591350 conn(0x5650171110d0 msgr2=0x5650171ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:32.187 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 15.0 s, 4881.2 tps, lat 20.521 ms stddev 23.740 2023-11-06T01:34:32.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.181+0000 7f26d53aeb80 1 -- 172.21.15.53:0/4213591350 shutdown_connections 2023-11-06T01:34:32.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.181+0000 7f26d53aeb80 1 -- 172.21.15.53:0/4213591350 wait complete. 2023-11-06T01:34:32.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.181+0000 7f26d53aeb80 1 -- start start 2023-11-06T01:34:32.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.182+0000 7f26d53aeb80 1 Processor -- start 2023-11-06T01:34:32.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.182+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b23f0 0x5650171bba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.182+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.182+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b3a70 0x5650171c3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.182+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501709f510 con 0x5650171b3a70 2023-11-06T01:34:32.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.182+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501709f1b0 con 0x5650171b23f0 2023-11-06T01:34:32.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.182+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x565016f86920 con 0x5650171b30f0 2023-11-06T01:34:32.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.182+0000 7f26d53aeb80 10 client.0 _ll_register_callbacks cb 0x565016feca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:32.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.183+0000 7f26c2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b23f0 0x5650171bba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.183+0000 7f26c2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b23f0 0x5650171bba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51582/0 (socket says 172.21.15.53:51582) 2023-11-06T01:34:32.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.183+0000 7f26c359e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b3a70 0x5650171c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.183+0000 7f26c2d9d700 1 -- 172.21.15.53:0/747622933 learned_addr learned my addr 172.21.15.53:0/747622933 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:32.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.183+0000 7f26c359e700 1 -- 172.21.15.53:0/747622933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 msgr2=0x5650171bbfc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:34:32.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.183+0000 7f26c359e700 1 --2- 172.21.15.53:0/747622933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171bbfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.183+0000 7f26c359e700 1 -- 172.21.15.53:0/747622933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b23f0 msgr2=0x5650171bba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:32.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.183+0000 7f26c359e700 1 --2- 172.21.15.53:0/747622933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b23f0 0x5650171bba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.183+0000 7f26c359e700 1 -- 172.21.15.53:0/747622933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c41181f0 con 0x5650171b3a70 2023-11-06T01:34:32.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.184+0000 7f26c359e700 1 --2- 172.21.15.53:0/747622933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b3a70 0x5650171c3000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f26bc006010 tx=0x7f26bc005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819eb3a8add178b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:32.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.184+0000 7f26aa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:34:32.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.185+0000 7f26aa7fc700 1 -- 172.21.15.53:0/747622933 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc01d040 con 0x5650171b3a70 2023-11-06T01:34:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.185+0000 7f26aa7fc700 1 -- 172.21.15.53:0/747622933 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f26bc02d040 con 0x5650171b3a70 2023-11-06T01:34:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.185+0000 7f26d53aeb80 1 -- 172.21.15.53:0/747622933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56501712dd10 con 0x5650171b3a70 2023-11-06T01:34:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.185+0000 7f26aa7fc700 1 -- 172.21.15.53:0/747622933 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26bc004ea0 con 0x5650171b3a70 2023-11-06T01:34:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.185+0000 7f26d53aeb80 10 client.15591 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.185+0000 7f26d53aeb80 1 -- 172.21.15.53:0/747622933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x565016f19c60 con 0x5650171b3a70 2023-11-06T01:34:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.186+0000 7f26aa7fc700 1 -- 172.21.15.53:0/747622933 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56501712dd10 con 0x5650171b3a70 2023-11-06T01:34:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.186+0000 7f26aa7fc700 1 -- 172.21.15.53:0/747622933 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f26bc039070 con 0x5650171b3a70 2023-11-06T01:34:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.186+0000 7f26d53aeb80 10 client.15591 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.186+0000 7f26d53aeb80 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.186+0000 7f26d53aeb80 20 client.15591 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.186+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/747622933 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5650171c78b0 0x5650171c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.186+0000 7f26d53aeb80 4 client.15591 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.186+0000 7f26d53aeb80 1 -- 172.21.15.53:0/747622933 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565017180e10 con 0x5650171c78b0 2023-11-06T01:34:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.186+0000 7f26c2d9d700 1 --2- 172.21.15.53:0/747622933 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5650171c78b0 0x5650171c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.187+0000 7f26c2d9d700 1 --2- 172.21.15.53:0/747622933 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5650171c78b0 0x5650171c9cd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b9e382fb2b3aa3 server_cookie=58d5e56aa23b73a7 in_seq=0 out_seq=0 2023-11-06T01:34:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.187+0000 7f26aa7fc700 10 client.15591 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.189+0000 7f26aa7fc700 1 -- 172.21.15.53:0/747622933 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x565017180e10 con 0x5650171c78b0 2023-11-06T01:34:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.189+0000 7f26aa7fc700 10 client.15591 handle_command_reply: tid=0 2023-11-06T01:34:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.189+0000 7f26d53aeb80 1 client.15591 shutdown 2023-11-06T01:34:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.190+0000 7f26d53aeb80 1 -- 172.21.15.53:0/747622933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b3a70 msgr2=0x5650171c3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.190+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/747622933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b3a70 0x5650171c3000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f26bc006010 tx=0x7f26bc005960 comp rx=0 tx=0).stop 2023-11-06T01:34:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.190+0000 7f26d53aeb80 1 -- 172.21.15.53:0/747622933 shutdown_connections 2023-11-06T01:34:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.190+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/747622933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b23f0 0x5650171bba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.190+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/747622933 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5650171c78b0 0x5650171c9cd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.190+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/747622933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b3a70 0x5650171c3000 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.190+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/747622933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171bbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.190+0000 7f26d53aeb80 1 -- 172.21.15.53:0/747622933 >> 172.21.15.53:0/747622933 conn(0x5650171110d0 msgr2=0x5650171ac0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.190+0000 7f26aa7fc700 0 client.15591 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.190+0000 7f26d53aeb80 1 -- 172.21.15.53:0/747622933 shutdown_connections 2023-11-06T01:34:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.190+0000 7f26d53aeb80 1 -- 172.21.15.53:0/747622933 wait complete. 2023-11-06T01:34:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.191+0000 7f26d53aeb80 20 client.15591 trim_cache size 0 max 16384 2023-11-06T01:34:32.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.215+0000 7f26d53aeb80 1 Processor -- start 2023-11-06T01:34:32.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.215+0000 7f26d53aeb80 1 -- start start 2023-11-06T01:34:32.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b30f0 0x5650171c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b3a70 0x5650171b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501712dd10 con 0x5650171b23f0 2023-11-06T01:34:32.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650171a4c50 con 0x5650171b30f0 2023-11-06T01:34:32.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650171ae3c0 con 0x5650171b3a70 2023-11-06T01:34:32.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26c2d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26c2d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171b63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40148/0 (socket says 172.21.15.53:40148) 2023-11-06T01:34:32.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26c2d9d700 1 -- 172.21.15.53:0/3990458839 learned_addr learned my addr 172.21.15.53:0/3990458839 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:32.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26ba59c700 1 --2- 172.21.15.53:0/3990458839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b30f0 0x5650171c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26c359e700 1 --2- 172.21.15.53:0/3990458839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b3a70 0x5650171b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26ba59c700 1 -- 172.21.15.53:0/3990458839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b3a70 msgr2=0x5650171b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:32.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26ba59c700 1 --2- 172.21.15.53:0/3990458839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b3a70 0x5650171b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26ba59c700 1 -- 172.21.15.53:0/3990458839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 msgr2=0x5650171b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:32.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26ba59c700 1 --2- 172.21.15.53:0/3990458839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26ba59c700 1 -- 172.21.15.53:0/3990458839 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565016f57250 con 0x5650171b30f0 2023-11-06T01:34:32.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.216+0000 7f26ba59c700 1 --2- 172.21.15.53:0/3990458839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b30f0 0x5650171c2fc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5650171b3e50 tx=0x7f26ac012730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ed26515129309d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:32.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.217+0000 7f26b9d9b700 1 -- 172.21.15.53:0/3990458839 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac010660 con 0x5650171b30f0 2023-11-06T01:34:32.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.217+0000 7f26b9d9b700 1 -- 172.21.15.53:0/3990458839 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26ac010c80 con 0x5650171b30f0 2023-11-06T01:34:32.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.217+0000 7f26b9d9b700 1 -- 172.21.15.53:0/3990458839 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac01ad20 con 0x5650171b30f0 2023-11-06T01:34:32.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.218+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3990458839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b30f0 msgr2=0x5650171c2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:32.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.218+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/3990458839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b30f0 0x5650171c2fc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5650171b3e50 tx=0x7f26ac012730 comp rx=0 tx=0).stop 2023-11-06T01:34:32.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.219+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3990458839 shutdown_connections 2023-11-06T01:34:32.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.219+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/3990458839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b3a70 0x5650171b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.219+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/3990458839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.219+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/3990458839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b30f0 0x5650171c2fc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.219+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3990458839 >> 172.21.15.53:0/3990458839 conn(0x5650171110d0 msgr2=0x5650171c38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:32.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.219+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3990458839 shutdown_connections 2023-11-06T01:34:32.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.219+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3990458839 wait complete. 2023-11-06T01:34:32.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.219+0000 7f26d53aeb80 1 -- start start 2023-11-06T01:34:32.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.219+0000 7f26d53aeb80 1 Processor -- start 2023-11-06T01:34:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.220+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.220+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171bf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.220+0000 7f26d53aeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 0x5650171bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.220+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650171c0260 con 0x5650171b23f0 2023-11-06T01:34:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.220+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650171c03d0 con 0x5650171b3a70 2023-11-06T01:34:32.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.220+0000 7f26d53aeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501722a3e0 con 0x5650171b30f0 2023-11-06T01:34:32.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.220+0000 7f26d53aeb80 10 client.0 _ll_register_callbacks cb 0x7f26bc009f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:32.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.220+0000 7f26c359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 0x5650171bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.220+0000 7f26c359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 0x5650171bfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51592/0 (socket says 172.21.15.53:51592) 2023-11-06T01:34:32.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.220+0000 7f26c359e700 1 -- 172.21.15.53:0/3610670256 learned_addr learned my addr 172.21.15.53:0/3610670256 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:32.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.220+0000 7f26ba59c700 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171bf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.221+0000 7f26c2d9d700 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.221+0000 7f26c359e700 1 -- 172.21.15.53:0/3610670256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 msgr2=0x5650171bf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:32.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.221+0000 7f26c359e700 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171bf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.221+0000 7f26c359e700 1 -- 172.21.15.53:0/3610670256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 msgr2=0x5650171c2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:32.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.221+0000 7f26c359e700 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171c2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.221+0000 7f26c359e700 1 -- 172.21.15.53:0/3610670256 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac00bc70 con 0x5650171b3a70 2023-11-06T01:34:32.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.222+0000 7f26c359e700 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 0x5650171bfce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0059d0 tx=0x7f26bc0051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3502aae4b0394edf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:32.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.222+0000 7f26aa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:32.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.222+0000 7f26aa7fc700 1 -- 172.21.15.53:0/3610670256 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc01d040 con 0x5650171b3a70 2023-11-06T01:34:32.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.222+0000 7f26aa7fc700 1 -- 172.21.15.53:0/3610670256 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f26bc00f990 con 0x5650171b3a70 2023-11-06T01:34:32.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.222+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3610670256 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26b4099230 con 0x5650171b3a70 2023-11-06T01:34:32.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26aa7fc700 1 -- 172.21.15.53:0/3610670256 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26bc03a3a0 con 0x5650171b3a70 2023-11-06T01:34:32.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26d53aeb80 10 client.25157 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:32.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3610670256 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x565016f2f2b0 con 0x5650171b3a70 2023-11-06T01:34:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26aa7fc700 1 -- 172.21.15.53:0/3610670256 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b4099230 con 0x5650171b3a70 2023-11-06T01:34:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26aa7fc700 1 -- 172.21.15.53:0/3610670256 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f26bc021070 con 0x5650171b3a70 2023-11-06T01:34:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26d53aeb80 10 client.25157 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26d53aeb80 10 client.25157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26d53aeb80 20 client.25157 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56501722baf0 0x56501722df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26d53aeb80 4 client.25157 mds_command: new command op to 24487 tid=0[{"prefix": "damage ls"}] 2023-11-06T01:34:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3610670256 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5650170fbd40 con 0x56501722baf0 2023-11-06T01:34:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.223+0000 7f26c2d9d700 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56501722baf0 0x56501722df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26aa7fc700 10 client.25157 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26c2d9d700 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56501722baf0 0x56501722df10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=930f4400d2388dae server_cookie=d0d08f7395b4f891 in_seq=0 out_seq=0 2023-11-06T01:34:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26aa7fc700 1 -- 172.21.15.53:0/3610670256 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5650170fbd40 con 0x56501722baf0 2023-11-06T01:34:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26aa7fc700 10 client.25157 handle_command_reply: tid=0 2023-11-06T01:34:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26d53aeb80 1 client.25157 shutdown 2023-11-06T01:34:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3610670256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 msgr2=0x5650171bfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 0x5650171bfce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0059d0 tx=0x7f26bc0051f0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3610670256 shutdown_connections 2023-11-06T01:34:32.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650171b3a70 0x5650171bfce0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56501722baf0 0x56501722df10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650171b23f0 0x5650171c2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26aa7fc700 0 client.25157 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:32.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26d53aeb80 1 --2- 172.21.15.53:0/3610670256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650171b30f0 0x5650171bf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.224+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3610670256 >> 172.21.15.53:0/3610670256 conn(0x5650171110d0 msgr2=0x5650171b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:32.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.234+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3610670256 shutdown_connections 2023-11-06T01:34:32.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.234+0000 7f26d53aeb80 1 -- 172.21.15.53:0/3610670256 wait complete. 2023-11-06T01:34:32.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.234+0000 7f26d53aeb80 20 client.25157 trim_cache size 0 max 16384 2023-11-06T01:34:32.239 INFO:teuthology.orchestra.run.smithi053.stdout:[] 2023-11-06T01:34:32.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.235+0000 7f26c921e700 1 -- 172.21.15.53:0/2722783952 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f26b00a05a0 msgr2=0x7f26b00a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:32.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.235+0000 7f26c921e700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f26b00a05a0 0x7f26b00a2a60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00b5c0 tx=0x7f26bc009310 comp rx=0 tx=0).stop 2023-11-06T01:34:32.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.235+0000 7f26c921e700 1 -- 172.21.15.53:0/2722783952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26c40ff5e0 msgr2=0x7f26c410fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:32.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.235+0000 7f26c921e700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26c40ff5e0 0x7f26c410fb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f26b400ba50 tx=0x7f26b400bd60 comp rx=0 tx=0).stop 2023-11-06T01:34:32.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.235+0000 7f26c921e700 1 -- 172.21.15.53:0/2722783952 shutdown_connections 2023-11-06T01:34:32.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.235+0000 7f26c921e700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f26b00a05a0 0x7f26b00a2a60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.235+0000 7f26c921e700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c4100f50 0x7f26c4117110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.235+0000 7f26c921e700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26c40ff5e0 0x7f26c410fb60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.235+0000 7f26c921e700 1 --2- 172.21.15.53:0/2722783952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c4100590 0x7f26c41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:32.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.235+0000 7f26c921e700 1 -- 172.21.15.53:0/2722783952 >> 172.21.15.53:0/2722783952 conn(0x7f26c40fa520 msgr2=0x7f26c40fb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:32.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.236+0000 7f26c921e700 1 -- 172.21.15.53:0/2722783952 shutdown_connections 2023-11-06T01:34:32.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:32.237+0000 7f26c921e700 1 -- 172.21.15.53:0/2722783952 wait complete. 2023-11-06T01:34:32.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:32 smithi053 ceph-mon[99472]: pgmap v317: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 104 MiB/s wr, 1.19k op/s 2023-11-06T01:34:32.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:32 smithi163 ceph-mon[104416]: pgmap v317: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 104 MiB/s wr, 1.19k op/s 2023-11-06T01:34:32.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:32 smithi062 ceph-mon[103924]: pgmap v317: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 104 MiB/s wr, 1.19k op/s 2023-11-06T01:34:33.264 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-06T01:34:33.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-06T01:34:33.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.465+0000 7f6f90d77700 1 -- 172.21.15.53:0/70364301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f8c1073d0 msgr2=0x7f6f8c1077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.465+0000 7f6f90d77700 1 --2- 172.21.15.53:0/70364301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f8c1073d0 0x7f6f8c1077f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400b0d0 tx=0x7f6f8400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.466+0000 7f6f90d77700 1 -- 172.21.15.53:0/70364301 shutdown_connections 2023-11-06T01:34:33.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.466+0000 7f6f90d77700 1 --2- 172.21.15.53:0/70364301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c108f60 0x7f6f8c10f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.466+0000 7f6f90d77700 1 --2- 172.21.15.53:0/70364301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f8c1073d0 0x7f6f8c1077f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.466+0000 7f6f90d77700 1 --2- 172.21.15.53:0/70364301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f8c1085c0 0x7f6f8c108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.466+0000 7f6f90d77700 1 -- 172.21.15.53:0/70364301 >> 172.21.15.53:0/70364301 conn(0x7f6f8c076340 msgr2=0x7f6f8c0787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.466+0000 7f6f90d77700 1 -- 172.21.15.53:0/70364301 shutdown_connections 2023-11-06T01:34:33.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.466+0000 7f6f90d77700 1 -- 172.21.15.53:0/70364301 wait complete. 2023-11-06T01:34:33.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.467+0000 7f6f90d77700 1 Processor -- start 2023-11-06T01:34:33.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.469+0000 7f6f90d77700 1 -- start start 2023-11-06T01:34:33.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.469+0000 7f6f90d77700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c1073d0 0x7f6f8c19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.469+0000 7f6f90d77700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f8c1085c0 0x7f6f8c19d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.469+0000 7f6f8a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c1073d0 0x7f6f8c19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.469+0000 7f6f8a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c1073d0 0x7f6f8c19d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54408/0 (socket says 172.21.15.53:54408) 2023-11-06T01:34:33.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.469+0000 7f6f90d77700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f8c108f60 0x7f6f8c1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.469+0000 7f6f90d77700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1a4d90 con 0x7f6f8c108f60 2023-11-06T01:34:33.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.469+0000 7f6f90d77700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1a4ed0 con 0x7f6f8c1085c0 2023-11-06T01:34:33.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.469+0000 7f6f90d77700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1a5060 con 0x7f6f8c1073d0 2023-11-06T01:34:33.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.469+0000 7f6f8a59c700 1 -- 172.21.15.53:0/3376978118 learned_addr learned my addr 172.21.15.53:0/3376978118 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:33.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f89d9b700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f8c1085c0 0x7f6f8c19d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f8a59c700 1 -- 172.21.15.53:0/3376978118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f8c1085c0 msgr2=0x7f6f8c19d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f8c1085c0 0x7f6f8c19d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f8a59c700 1 -- 172.21.15.53:0/3376978118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f8c108f60 msgr2=0x7f6f8c1a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f8c108f60 0x7f6f8c1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f8a59c700 1 -- 172.21.15.53:0/3376978118 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f84009d00 con 0x7f6f8c1073d0 2023-11-06T01:34:33.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c1073d0 0x7f6f8c19d210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6f84000f80 tx=0x7f6f84008ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f7b7fe700 1 -- 172.21.15.53:0/3376978118 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8400e040 con 0x7f6f8c1073d0 2023-11-06T01:34:33.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f7b7fe700 1 -- 172.21.15.53:0/3376978118 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f8400be30 con 0x7f6f8c1073d0 2023-11-06T01:34:33.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f90d77700 1 -- 172.21.15.53:0/3376978118 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c1a52e0 con 0x7f6f8c1073d0 2023-11-06T01:34:33.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f7b7fe700 1 -- 172.21.15.53:0/3376978118 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8401bac0 con 0x7f6f8c1073d0 2023-11-06T01:34:33.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.470+0000 7f6f90d77700 1 -- 172.21.15.53:0/3376978118 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c1a5760 con 0x7f6f8c1073d0 2023-11-06T01:34:33.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.473+0000 7f6f7b7fe700 1 -- 172.21.15.53:0/3376978118 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f6f8401d670 con 0x7f6f8c1073d0 2023-11-06T01:34:33.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.474+0000 7f6f7b7fe700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6f740a05a0 0x7f6f740a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.474+0000 7f6f9cf07b80 1 Processor -- start 2023-11-06T01:34:33.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.474+0000 7f6f7b7fe700 1 -- 172.21.15.53:0/3376978118 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6f840ccf50 con 0x7f6f8c1073d0 2023-11-06T01:34:33.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.475+0000 7f6f9cf07b80 1 -- start start 2023-11-06T01:34:33.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.475+0000 7f6f89d9b700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6f740a05a0 0x7f6f740a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.475+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 0x555a780575b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.475+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057af0 0x555a78055ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.475+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 0x555a780568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.475+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a77fada90 con 0x555a78056470 2023-11-06T01:34:33.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.475+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a77fab450 con 0x555a78057170 2023-11-06T01:34:33.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.475+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a77fa6d40 con 0x555a78057af0 2023-11-06T01:34:33.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.476+0000 7f6f89d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057af0 0x555a78055ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.476+0000 7f6f89d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057af0 0x555a78055ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54416/0 (socket says 172.21.15.53:54416) 2023-11-06T01:34:33.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.476+0000 7f6f89d9b700 1 -- 172.21.15.53:0/1999040064 learned_addr learned my addr 172.21.15.53:0/1999040064 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:33.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.476+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/1999040064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 0x555a780575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.477+0000 7f6f8ad9d700 1 --2- 172.21.15.53:0/1999040064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 0x555a780568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.477+0000 7f6f8a59c700 1 -- 172.21.15.53:0/1999040064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057af0 msgr2=0x555a78055ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.477+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/1999040064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057af0 0x555a78055ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.477+0000 7f6f8a59c700 1 -- 172.21.15.53:0/1999040064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 msgr2=0x555a780568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.477+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/1999040064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 0x555a780568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.477+0000 7f6f8a59c700 1 -- 172.21.15.53:0/1999040064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a77dd4e60 con 0x555a78057170 2023-11-06T01:34:33.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.477+0000 7f6f89d9b700 1 --2- 172.21.15.53:0/1999040064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057af0 0x555a78055ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:34:33.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.477+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/1999040064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 0x555a780575b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f84008fe0 tx=0x7f6f840099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=937cd79c846dcb91 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.478+0000 7f6f78ff9700 1 -- 172.21.15.53:0/1999040064 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8409aec0 con 0x555a78057170 2023-11-06T01:34:33.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.478+0000 7f6f89d9b700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6f740a05a0 0x7f6f740a2a60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009fd0 tx=0x7f6f7c009930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.478+0000 7f6f78ff9700 1 -- 172.21.15.53:0/1999040064 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f840cae90 con 0x555a78057170 2023-11-06T01:34:33.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.478+0000 7f6f78ff9700 1 -- 172.21.15.53:0/1999040064 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8401dd30 con 0x555a78057170 2023-11-06T01:34:33.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.479+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/1999040064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 msgr2=0x555a780575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.479+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/1999040064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 0x555a780575b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f84008fe0 tx=0x7f6f840099e0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.479+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/1999040064 shutdown_connections 2023-11-06T01:34:33.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.479+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/1999040064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 0x555a780575b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.479+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/1999040064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 0x555a780568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.479+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/1999040064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057af0 0x555a78055ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.479+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/1999040064 >> 172.21.15.53:0/1999040064 conn(0x555a77fb5150 msgr2=0x555a780509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.479+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/1999040064 shutdown_connections 2023-11-06T01:34:33.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.480+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/1999040064 wait complete. 2023-11-06T01:34:33.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.480+0000 7f6f9cf07b80 1 -- start start 2023-11-06T01:34:33.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.480+0000 7f6f9cf07b80 1 Processor -- start 2023-11-06T01:34:33.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 0x555a78070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057170 0x555a78070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057af0 0x555a78077e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a77f43590 con 0x555a78056470 2023-11-06T01:34:33.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a77f43230 con 0x555a78057af0 2023-11-06T01:34:33.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a77e2a9a0 con 0x555a78057170 2023-11-06T01:34:33.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f9cf07b80 10 client.0 _ll_register_callbacks cb 0x555a77e90b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:33.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f89d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057170 0x555a78070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f8a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 0x555a78070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f8a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 0x555a78070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40182/0 (socket says 172.21.15.53:40182) 2023-11-06T01:34:33.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f89d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057170 0x555a78070dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54422/0 (socket says 172.21.15.53:54422) 2023-11-06T01:34:33.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.481+0000 7f6f8a59c700 1 -- 172.21.15.53:0/3949333982 learned_addr learned my addr 172.21.15.53:0/3949333982 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:33.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.482+0000 7f6f8a59c700 1 -- 172.21.15.53:0/3949333982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057170 msgr2=0x555a78070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.482+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/3949333982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057170 0x555a78070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.482+0000 7f6f8a59c700 1 -- 172.21.15.53:0/3949333982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057af0 msgr2=0x555a78077e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.482+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/3949333982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057af0 0x555a78077e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.482+0000 7f6f8a59c700 1 -- 172.21.15.53:0/3949333982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c1a5760 con 0x555a78056470 2023-11-06T01:34:33.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.483+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/3949333982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 0x555a78070880 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6f8401b4d0 tx=0x7f6f84096440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb25a75addc6087 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.483+0000 7f6f71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:34:33.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.484+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3949333982 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840b8030 con 0x555a78056470 2023-11-06T01:34:33.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.484+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3949333982 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6f8409e5f0 con 0x555a78056470 2023-11-06T01:34:33.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.484+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3949333982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a77fd1d90 con 0x555a78056470 2023-11-06T01:34:33.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.484+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3949333982 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f8401d920 con 0x555a78056470 2023-11-06T01:34:33.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.484+0000 7f6f9cf07b80 10 client.15606 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:33.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.484+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3949333982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555a77dbdce0 con 0x555a78056470 2023-11-06T01:34:33.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.485+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3949333982 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a77fd1d90 con 0x555a78056470 2023-11-06T01:34:33.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.485+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3949333982 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6f8409aa20 con 0x555a78056470 2023-11-06T01:34:33.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.485+0000 7f6f9cf07b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:33.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.485+0000 7f6f9cf07b80 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:33.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.485+0000 7f6f9cf07b80 20 client.15606 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:33.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.485+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3949333982 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555a7808d1f0 0x555a7808f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.485+0000 7f6f9cf07b80 4 client.15606 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:33.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.485+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3949333982 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a78024e90 con 0x555a7808d1f0 2023-11-06T01:34:33.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.486+0000 7f6f89d9b700 1 --2- 172.21.15.53:0/3949333982 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555a7808d1f0 0x555a7808f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.487+0000 7f6f89d9b700 1 --2- 172.21.15.53:0/3949333982 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555a7808d1f0 0x555a7808f610 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f2734413126d07 server_cookie=fe1a22dfd36a8b88 in_seq=0 out_seq=0 2023-11-06T01:34:33.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.487+0000 7f6f71ffb700 10 client.15606 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:33.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.488+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3949333982 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555a78024e90 con 0x555a7808d1f0 2023-11-06T01:34:33.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.488+0000 7f6f71ffb700 10 client.15606 handle_command_reply: tid=0 2023-11-06T01:34:33.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.488+0000 7f6f9cf07b80 1 client.15606 shutdown 2023-11-06T01:34:33.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.489+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3949333982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 msgr2=0x555a78070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.489+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3949333982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 0x555a78070880 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6f8401b4d0 tx=0x7f6f84096440 comp rx=0 tx=0).stop 2023-11-06T01:34:33.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.489+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3949333982 shutdown_connections 2023-11-06T01:34:33.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.489+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3949333982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057af0 0x555a78077e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.489+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3949333982 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555a7808d1f0 0x555a7808f610 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.490+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3949333982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78056470 0x555a78070880 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.490+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3949333982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78057170 0x555a78070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.490+0000 7f6f71ffb700 0 client.15606 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:33.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.490+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3949333982 >> 172.21.15.53:0/3949333982 conn(0x555a77fb5150 msgr2=0x555a78050130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.491+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3949333982 shutdown_connections 2023-11-06T01:34:33.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.491+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3949333982 wait complete. 2023-11-06T01:34:33.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.492+0000 7f6f9cf07b80 20 client.15606 trim_cache size 0 max 16384 2023-11-06T01:34:33.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.521+0000 7f6f9cf07b80 1 Processor -- start 2023-11-06T01:34:33.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.521+0000 7f6f9cf07b80 1 -- start start 2023-11-06T01:34:33.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.522+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78056470 0x555a78055ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.522+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78057170 0x555a78077d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.522+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057af0 0x555a78070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.522+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a780524f0 con 0x555a78057170 2023-11-06T01:34:33.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.522+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a78024e90 con 0x555a78057af0 2023-11-06T01:34:33.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.522+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a77fd1d90 con 0x555a78056470 2023-11-06T01:34:33.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.522+0000 7f6f89d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78057170 0x555a78077d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.522+0000 7f6f89d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78057170 0x555a78077d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40192/0 (socket says 172.21.15.53:40192) 2023-11-06T01:34:33.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.522+0000 7f6f89d9b700 1 -- 172.21.15.53:0/1889916565 learned_addr learned my addr 172.21.15.53:0/1889916565 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:33.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.522+0000 7f6f89d9b700 1 -- 172.21.15.53:0/1889916565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78056470 msgr2=0x555a78055ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:34:33.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.523+0000 7f6f89d9b700 1 --2- 172.21.15.53:0/1889916565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78056470 0x555a78055ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.523+0000 7f6f89d9b700 1 -- 172.21.15.53:0/1889916565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057af0 msgr2=0x555a78070630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:34:33.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.523+0000 7f6f89d9b700 1 --2- 172.21.15.53:0/1889916565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057af0 0x555a78070630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.523+0000 7f6f89d9b700 1 -- 172.21.15.53:0/1889916565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a7804d070 con 0x555a78057170 2023-11-06T01:34:33.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.523+0000 7f6f89d9b700 1 --2- 172.21.15.53:0/1889916565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78057170 0x555a78077d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x555a78057ed0 tx=0x7f6f7c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a28fd11a98e1a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.523+0000 7f6f797fa700 1 -- 172.21.15.53:0/1889916565 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c025040 con 0x555a78057170 2023-11-06T01:34:33.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.523+0000 7f6f797fa700 1 -- 172.21.15.53:0/1889916565 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f7c02bd70 con 0x555a78057170 2023-11-06T01:34:33.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.523+0000 7f6f797fa700 1 -- 172.21.15.53:0/1889916565 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c018040 con 0x555a78057170 2023-11-06T01:34:33.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.524+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/1889916565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78057170 msgr2=0x555a78077d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.524+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/1889916565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78057170 0x555a78077d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x555a78057ed0 tx=0x7f6f7c009260 comp rx=0 tx=0).stop 2023-11-06T01:34:33.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.524+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/1889916565 shutdown_connections 2023-11-06T01:34:33.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.524+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/1889916565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057af0 0x555a78070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.524+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/1889916565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78056470 0x555a78055ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.524+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/1889916565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78057170 0x555a78077d90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.524+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/1889916565 >> 172.21.15.53:0/1889916565 conn(0x555a77fb5150 msgr2=0x555a7805a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.524+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/1889916565 shutdown_connections 2023-11-06T01:34:33.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.524+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/1889916565 wait complete. 2023-11-06T01:34:33.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.525+0000 7f6f9cf07b80 1 -- start start 2023-11-06T01:34:33.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.525+0000 7f6f9cf07b80 1 Processor -- start 2023-11-06T01:34:33.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.525+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78056470 0x555a78074910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.525+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 0x555a78074e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.525+0000 7f6f9cf07b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78057af0 0x555a780decc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.525+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a78077e70 con 0x555a78057af0 2023-11-06T01:34:33.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.525+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a78077fe0 con 0x555a78057170 2023-11-06T01:34:33.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.525+0000 7f6f9cf07b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a780781a0 con 0x555a78056470 2023-11-06T01:34:33.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.525+0000 7f6f9cf07b80 10 client.0 _ll_register_callbacks cb 0x7f6f80009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:33.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.526+0000 7f6f89d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 0x555a78074e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.526+0000 7f6f89d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 0x555a78074e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51634/0 (socket says 172.21.15.53:51634) 2023-11-06T01:34:33.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.526+0000 7f6f89d9b700 1 -- 172.21.15.53:0/3188848991 learned_addr learned my addr 172.21.15.53:0/3188848991 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:33.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.526+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78056470 0x555a78074910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.526+0000 7f6f8a59c700 1 -- 172.21.15.53:0/3188848991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 msgr2=0x555a78074e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.526+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 0x555a78074e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.526+0000 7f6f8a59c700 1 -- 172.21.15.53:0/3188848991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78057af0 msgr2=0x555a780decc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.526+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78057af0 0x555a780decc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.526+0000 7f6f8a59c700 1 -- 172.21.15.53:0/3188848991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7c013250 con 0x555a78056470 2023-11-06T01:34:33.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.527+0000 7f6f8a59c700 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78056470 0x555a78074910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6f8409a4c0 tx=0x7f6f84098d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9332fbdca3a105c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.527+0000 7f6f71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:34:33.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.527+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3188848991 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840b8050 con 0x555a78056470 2023-11-06T01:34:33.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.527+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3188848991 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6f8401db30 con 0x555a78056470 2023-11-06T01:34:33.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.527+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3188848991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f840cae90 con 0x555a78056470 2023-11-06T01:34:33.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.527+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3188848991 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f84099dd0 con 0x555a78056470 2023-11-06T01:34:33.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.529+0000 7f6f9cf07b80 10 client.25261 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:33.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.529+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3188848991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555a77dfb2d0 con 0x555a78056470 2023-11-06T01:34:33.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.529+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3188848991 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f840cae90 con 0x555a78056470 2023-11-06T01:34:33.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.529+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3188848991 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6f840a2310 con 0x555a78056470 2023-11-06T01:34:33.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.530+0000 7f6f9cf07b80 10 client.25261 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:33.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.530+0000 7f6f9cf07b80 10 client.25261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:33.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.530+0000 7f6f9cf07b80 20 client.25261 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:33.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.530+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555a780e04b0 0x555a780e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.530+0000 7f6f9cf07b80 4 client.25261 mds_command: new command op to 24487 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-06T01:34:33.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.530+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3188848991 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555a77f9fdc0 con 0x555a780e04b0 2023-11-06T01:34:33.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.530+0000 7f6f89d9b700 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555a780e04b0 0x555a780e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.531+0000 7f6f89d9b700 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555a780e04b0 0x555a780e28a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ffecdc482f5678c server_cookie=35a881353cd4744e in_seq=0 out_seq=0 2023-11-06T01:34:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.531+0000 7f6f71ffb700 10 client.25261 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f71ffb700 1 -- 172.21.15.53:0/3188848991 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555a77f9fdc0 con 0x555a780e04b0 2023-11-06T01:34:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f71ffb700 10 client.25261 handle_command_reply: tid=0 2023-11-06T01:34:33.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 1 client.25261 shutdown 2023-11-06T01:34:33.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3188848991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78056470 msgr2=0x555a78074910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78056470 0x555a78074910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6f8409a4c0 tx=0x7f6f84098d70 comp rx=0 tx=0).stop 2023-11-06T01:34:33.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3188848991 shutdown_connections 2023-11-06T01:34:33.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555a780e04b0 0x555a780e28a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a78057af0 0x555a780decc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f71ffb700 0 client.25261 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:33.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555a78056470 0x555a78074910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 1 --2- 172.21.15.53:0/3188848991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a78057170 0x555a78074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3188848991 >> 172.21.15.53:0/3188848991 conn(0x555a77fb5150 msgr2=0x555a78070630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3188848991 shutdown_connections 2023-11-06T01:34:33.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 1 -- 172.21.15.53:0/3188848991 wait complete. 2023-11-06T01:34:33.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.532+0000 7f6f9cf07b80 20 client.25261 trim_cache size 0 max 16384 2023-11-06T01:34:33.543 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:34:33.544 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2023-11-06T01:34:33.544 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:34:33.544 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2023-11-06T01:34:33.544 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:34:33.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.534+0000 7f6f90d77700 1 -- 172.21.15.53:0/3376978118 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6f740a05a0 msgr2=0x7f6f740a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.534+0000 7f6f90d77700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6f740a05a0 0x7f6f740a2a60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009fd0 tx=0x7f6f7c009930 comp rx=0 tx=0).stop 2023-11-06T01:34:33.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.534+0000 7f6f90d77700 1 -- 172.21.15.53:0/3376978118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c1073d0 msgr2=0x7f6f8c19d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.534+0000 7f6f90d77700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c1073d0 0x7f6f8c19d210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6f84000f80 tx=0x7f6f84008ec0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.534+0000 7f6f90d77700 1 -- 172.21.15.53:0/3376978118 shutdown_connections 2023-11-06T01:34:33.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.534+0000 7f6f90d77700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6f740a05a0 0x7f6f740a2a60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.534+0000 7f6f90d77700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f8c108f60 0x7f6f8c1a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.534+0000 7f6f90d77700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c1073d0 0x7f6f8c19d210 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6f84000f80 tx=0x7f6f84008ec0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.534+0000 7f6f90d77700 1 --2- 172.21.15.53:0/3376978118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f8c1085c0 0x7f6f8c19d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.534+0000 7f6f90d77700 1 -- 172.21.15.53:0/3376978118 >> 172.21.15.53:0/3376978118 conn(0x7f6f8c076340 msgr2=0x7f6f8c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.535+0000 7f6f90d77700 1 -- 172.21.15.53:0/3376978118 shutdown_connections 2023-11-06T01:34:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.535+0000 7f6f90d77700 1 -- 172.21.15.53:0/3376978118 wait complete. 2023-11-06T01:34:33.562 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:33 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:34:33.564 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:34:33.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:33 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:34:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.803+0000 7f86d74c1700 1 -- 172.21.15.53:0/1668913895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d0100e70 msgr2=0x7f86d01012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.803+0000 7f86d74c1700 1 --2- 172.21.15.53:0/1668913895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d0100e70 0x7f86d01012d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f86c800b0d0 tx=0x7f86c800b3e0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.803+0000 7f86d74c1700 1 -- 172.21.15.53:0/1668913895 shutdown_connections 2023-11-06T01:34:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.803+0000 7f86d74c1700 1 --2- 172.21.15.53:0/1668913895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86d0101810 0x7f86d01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.803+0000 7f86d74c1700 1 --2- 172.21.15.53:0/1668913895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86d00ffc80 0x7f86d01000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.803+0000 7f86d74c1700 1 --2- 172.21.15.53:0/1668913895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d0100e70 0x7f86d01012d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.803+0000 7f86d74c1700 1 -- 172.21.15.53:0/1668913895 >> 172.21.15.53:0/1668913895 conn(0x7f86d00fb1f0 msgr2=0x7f86d00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.804+0000 7f86d74c1700 1 -- 172.21.15.53:0/1668913895 shutdown_connections 2023-11-06T01:34:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.804+0000 7f86d74c1700 1 -- 172.21.15.53:0/1668913895 wait complete. 2023-11-06T01:34:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d74c1700 1 Processor -- start 2023-11-06T01:34:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d74c1700 1 -- start start 2023-11-06T01:34:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d74c1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86d00ffc80 0x7f86d010fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d74c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86d0100e70 0x7f86d0110280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d74c1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d0101810 0x7f86d01172f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d74c1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01178c0 con 0x7f86d00ffc80 2023-11-06T01:34:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d74c1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01ad7b0 con 0x7f86d0101810 2023-11-06T01:34:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d74c1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01ad8f0 con 0x7f86d0100e70 2023-11-06T01:34:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d5a5e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d0101810 0x7f86d01172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d525d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86d00ffc80 0x7f86d010fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d4a5c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86d0100e70 0x7f86d0110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d5a5e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d0101810 0x7f86d01172f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51650/0 (socket says 172.21.15.53:51650) 2023-11-06T01:34:33.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d4a5c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86d0100e70 0x7f86d0110280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54450/0 (socket says 172.21.15.53:54450) 2023-11-06T01:34:33.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d5a5e700 1 -- 172.21.15.53:0/1650917453 learned_addr learned my addr 172.21.15.53:0/1650917453 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:33.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d5a5e700 1 -- 172.21.15.53:0/1650917453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86d0100e70 msgr2=0x7f86d0110280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d5a5e700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86d0100e70 0x7f86d0110280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d5a5e700 1 -- 172.21.15.53:0/1650917453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86d00ffc80 msgr2=0x7f86d010fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d5a5e700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86d00ffc80 0x7f86d010fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d5a5e700 1 -- 172.21.15.53:0/1650917453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c8009d00 con 0x7f86d0101810 2023-11-06T01:34:33.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.805+0000 7f86d4a5c700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86d0100e70 0x7f86d0110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:33.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.806+0000 7f86d5a5e700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d0101810 0x7f86d01172f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00d980 tx=0x7f86cc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.806+0000 7f86c67fc700 1 -- 172.21.15.53:0/1650917453 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc004a30 con 0x7f86d0101810 2023-11-06T01:34:33.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.806+0000 7f86c67fc700 1 -- 172.21.15.53:0/1650917453 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86cc0054e0 con 0x7f86d0101810 2023-11-06T01:34:33.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.806+0000 7f86d74c1700 1 -- 172.21.15.53:0/1650917453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d01adb40 con 0x7f86d0101810 2023-11-06T01:34:33.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.806+0000 7f86c67fc700 1 -- 172.21.15.53:0/1650917453 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc00fbc0 con 0x7f86d0101810 2023-11-06T01:34:33.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.806+0000 7f86d74c1700 1 -- 172.21.15.53:0/1650917453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d01adff0 con 0x7f86d0101810 2023-11-06T01:34:33.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.809+0000 7f86c67fc700 1 -- 172.21.15.53:0/1650917453 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f86cc004c20 con 0x7f86d0101810 2023-11-06T01:34:33.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.810+0000 7f86c67fc700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f86bc0a0620 0x7f86bc0a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.810+0000 7f86c67fc700 1 -- 172.21.15.53:0/1650917453 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f86cc0cb7a0 con 0x7f86d0101810 2023-11-06T01:34:33.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.811+0000 7f86e3651b80 1 Processor -- start 2023-11-06T01:34:33.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.811+0000 7f86d525d700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f86bc0a0620 0x7f86bc0a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.811+0000 7f86e3651b80 1 -- start start 2023-11-06T01:34:33.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.812+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 0x5592c1680530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.812+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c1680a70 0x5592c167ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.812+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c167f3f0 0x5592c167f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.812+0000 7f86e3651b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c15d6a10 con 0x5592c167f3f0 2023-11-06T01:34:33.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.812+0000 7f86e3651b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c15d43d0 con 0x5592c16800f0 2023-11-06T01:34:33.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.812+0000 7f86e3651b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c15cfcc0 con 0x5592c1680a70 2023-11-06T01:34:33.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.812+0000 7f86d4a5c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c1680a70 0x5592c167ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.812+0000 7f86d4a5c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c1680a70 0x5592c167ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54456/0 (socket says 172.21.15.53:54456) 2023-11-06T01:34:33.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.812+0000 7f86d4a5c700 1 -- 172.21.15.53:0/4288650045 learned_addr learned my addr 172.21.15.53:0/4288650045 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:33.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.812+0000 7f86d525d700 1 --2- 172.21.15.53:0/4288650045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 0x5592c1680530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.812+0000 7f86d4a5c700 1 -- 172.21.15.53:0/4288650045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 msgr2=0x5592c1680530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.813+0000 7f86d4a5c700 1 --2- 172.21.15.53:0/4288650045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 0x5592c1680530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.813+0000 7f86d4a5c700 1 -- 172.21.15.53:0/4288650045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c167f3f0 msgr2=0x5592c167f870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:34:33.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.813+0000 7f86d4a5c700 1 --2- 172.21.15.53:0/4288650045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c167f3f0 0x5592c167f870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.813+0000 7f86d4a5c700 1 -- 172.21.15.53:0/4288650045 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592c13fdde0 con 0x5592c1680a70 2023-11-06T01:34:33.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.813+0000 7f86d525d700 1 --2- 172.21.15.53:0/4288650045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 0x5592c1680530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:34:33.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.813+0000 7f86d525d700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f86bc0a0620 0x7f86bc0a2ae0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f86d0100cd0 tx=0x7f86c0011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.813+0000 7f86d4a5c700 1 --2- 172.21.15.53:0/4288650045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c1680a70 0x5592c167ee60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f86c800b0d0 tx=0x7f86c80092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73728352784eec03 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.813+0000 7f86bb7fe700 1 -- 172.21.15.53:0/4288650045 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c800e040 con 0x5592c1680a70 2023-11-06T01:34:33.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.814+0000 7f86bb7fe700 1 -- 172.21.15.53:0/4288650045 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c800bca0 con 0x5592c1680a70 2023-11-06T01:34:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.814+0000 7f86bb7fe700 1 -- 172.21.15.53:0/4288650045 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8005720 con 0x5592c1680a70 2023-11-06T01:34:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.815+0000 7f86e3651b80 1 -- 172.21.15.53:0/4288650045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c1680a70 msgr2=0x5592c167ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.815+0000 7f86e3651b80 1 --2- 172.21.15.53:0/4288650045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c1680a70 0x5592c167ee60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f86c800b0d0 tx=0x7f86c80092e0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.815+0000 7f86e3651b80 1 -- 172.21.15.53:0/4288650045 shutdown_connections 2023-11-06T01:34:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.815+0000 7f86e3651b80 1 --2- 172.21.15.53:0/4288650045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 0x5592c1680530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.815+0000 7f86e3651b80 1 --2- 172.21.15.53:0/4288650045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c167f3f0 0x5592c167f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.815+0000 7f86e3651b80 1 --2- 172.21.15.53:0/4288650045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c1680a70 0x5592c167ee60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.815+0000 7f86e3651b80 1 -- 172.21.15.53:0/4288650045 >> 172.21.15.53:0/4288650045 conn(0x5592c15de0d0 msgr2=0x5592c1679940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.815+0000 7f86e3651b80 1 -- 172.21.15.53:0/4288650045 shutdown_connections 2023-11-06T01:34:33.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.816+0000 7f86e3651b80 1 -- 172.21.15.53:0/4288650045 wait complete. 2023-11-06T01:34:33.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.817+0000 7f86e3651b80 1 -- start start 2023-11-06T01:34:33.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.817+0000 7f86e3651b80 1 Processor -- start 2023-11-06T01:34:33.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.817+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c167f3f0 0x5592c1688890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.817+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16800f0 0x5592c1688dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.817+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c1680a70 0x5592c168fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.817+0000 7f86e3651b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c156c510 con 0x5592c167f3f0 2023-11-06T01:34:33.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.817+0000 7f86e3651b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c156c1b0 con 0x5592c1680a70 2023-11-06T01:34:33.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.817+0000 7f86e3651b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c1453920 con 0x5592c16800f0 2023-11-06T01:34:33.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.817+0000 7f86e3651b80 10 client.0 _ll_register_callbacks cb 0x5592c14b9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:33.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d5a5e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c1680a70 0x5592c168fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d525d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c167f3f0 0x5592c1688890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d5a5e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c1680a70 0x5592c168fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51658/0 (socket says 172.21.15.53:51658) 2023-11-06T01:34:33.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d5a5e700 1 -- 172.21.15.53:0/1647448068 learned_addr learned my addr 172.21.15.53:0/1647448068 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:33.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d4a5c700 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16800f0 0x5592c1688dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d5a5e700 1 -- 172.21.15.53:0/1647448068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16800f0 msgr2=0x5592c1688dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d5a5e700 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16800f0 0x5592c1688dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d5a5e700 1 -- 172.21.15.53:0/1647448068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c167f3f0 msgr2=0x5592c1688890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d5a5e700 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c167f3f0 0x5592c1688890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d5a5e700 1 -- 172.21.15.53:0/1647448068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d0108a10 con 0x5592c1680a70 2023-11-06T01:34:33.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d4a5c700 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16800f0 0x5592c1688dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:33.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d525d700 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c167f3f0 0x5592c1688890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:33.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86d5a5e700 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c1680a70 0x5592c168fe10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f86cc095480 tx=0x7f86cc098f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1deac4e3cc0fdc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.818+0000 7f86b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:33.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.820+0000 7f86b8ff9700 1 -- 172.21.15.53:0/1647448068 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0a0ff0 con 0x5592c1680a70 2023-11-06T01:34:33.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.820+0000 7f86b8ff9700 1 -- 172.21.15.53:0/1647448068 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f86cc022300 con 0x5592c1680a70 2023-11-06T01:34:33.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.820+0000 7f86e3651b80 1 -- 172.21.15.53:0/1647448068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592c15fad10 con 0x5592c1680a70 2023-11-06T01:34:33.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.822+0000 7f86b8ff9700 1 -- 172.21.15.53:0/1647448068 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86cc0224c0 con 0x5592c1680a70 2023-11-06T01:34:33.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.822+0000 7f86e3651b80 10 client.25178 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:33.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.822+0000 7f86e3651b80 1 -- 172.21.15.53:0/1647448068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5592c13e6c60 con 0x5592c1680a70 2023-11-06T01:34:33.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.822+0000 7f86b8ff9700 1 -- 172.21.15.53:0/1647448068 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592c15fad10 con 0x5592c1680a70 2023-11-06T01:34:33.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.822+0000 7f86b8ff9700 1 -- 172.21.15.53:0/1647448068 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f86cc0a0b60 con 0x5592c1680a70 2023-11-06T01:34:33.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.822+0000 7f86e3651b80 10 client.25178 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:33.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.822+0000 7f86e3651b80 10 client.25178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:33.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.822+0000 7f86e3651b80 20 client.25178 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:33.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.822+0000 7f86e3651b80 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5592c1694520 0x5592c1696940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.822+0000 7f86e3651b80 4 client.25178 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:33.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.822+0000 7f86e3651b80 1 -- 172.21.15.53:0/1647448068 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592c164de10 con 0x5592c1694520 2023-11-06T01:34:33.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.823+0000 7f86d525d700 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5592c1694520 0x5592c1696940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.823+0000 7f86d525d700 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5592c1694520 0x5592c1696940 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d8bec7f848cf22 server_cookie=a92705745d14ebe0 in_seq=0 out_seq=0 2023-11-06T01:34:33.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.824+0000 7f86b8ff9700 10 client.25178 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:33.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.824+0000 7f86b8ff9700 1 -- 172.21.15.53:0/1647448068 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5592c164de10 con 0x5592c1694520 2023-11-06T01:34:33.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.824+0000 7f86b8ff9700 10 client.25178 handle_command_reply: tid=0 2023-11-06T01:34:33.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.824+0000 7f86e3651b80 1 client.25178 shutdown 2023-11-06T01:34:33.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.824+0000 7f86e3651b80 1 -- 172.21.15.53:0/1647448068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c1680a70 msgr2=0x5592c168fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.824+0000 7f86e3651b80 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c1680a70 0x5592c168fe10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f86cc095480 tx=0x7f86cc098f80 comp rx=0 tx=0).stop 2023-11-06T01:34:33.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.826+0000 7f86e3651b80 1 -- 172.21.15.53:0/1647448068 shutdown_connections 2023-11-06T01:34:33.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.826+0000 7f86e3651b80 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c1680a70 0x5592c168fe10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.826+0000 7f86e3651b80 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5592c1694520 0x5592c1696940 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.826+0000 7f86e3651b80 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c167f3f0 0x5592c1688890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.826+0000 7f86e3651b80 1 --2- 172.21.15.53:0/1647448068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16800f0 0x5592c1688dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.826+0000 7f86e3651b80 1 -- 172.21.15.53:0/1647448068 >> 172.21.15.53:0/1647448068 conn(0x5592c15de0d0 msgr2=0x5592c16790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.827+0000 7f86b8ff9700 0 client.25178 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:33.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.827+0000 7f86e3651b80 1 -- 172.21.15.53:0/1647448068 shutdown_connections 2023-11-06T01:34:33.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.827+0000 7f86e3651b80 1 -- 172.21.15.53:0/1647448068 wait complete. 2023-11-06T01:34:33.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.828+0000 7f86e3651b80 20 client.25178 trim_cache size 0 max 16384 2023-11-06T01:34:33.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.858+0000 7f86e3651b80 1 Processor -- start 2023-11-06T01:34:33.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.858+0000 7f86e3651b80 1 -- start start 2023-11-06T01:34:33.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.858+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c167f3f0 0x5592c167ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.858+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16800f0 0x5592c168dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.858+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 0x5592c1688850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.858+0000 7f86e3651b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c15fad10 con 0x5592c1675d20 2023-11-06T01:34:33.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.858+0000 7f86e3651b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c1671c50 con 0x5592c167f3f0 2023-11-06T01:34:33.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.858+0000 7f86e3651b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c1687d40 con 0x5592c16800f0 2023-11-06T01:34:33.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.859+0000 7f86d525d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c167f3f0 0x5592c167ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.859+0000 7f86d525d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c167f3f0 0x5592c167ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51662/0 (socket says 172.21.15.53:51662) 2023-11-06T01:34:33.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.859+0000 7f86d525d700 1 -- 172.21.15.53:0/3241914717 learned_addr learned my addr 172.21.15.53:0/3241914717 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:33.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.859+0000 7f86d4a5c700 1 --2- 172.21.15.53:0/3241914717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16800f0 0x5592c168dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.859+0000 7f86d5a5e700 1 --2- 172.21.15.53:0/3241914717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 0x5592c1688850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.859+0000 7f86d525d700 1 -- 172.21.15.53:0/3241914717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16800f0 msgr2=0x5592c168dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.859+0000 7f86d525d700 1 --2- 172.21.15.53:0/3241914717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16800f0 0x5592c168dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.859+0000 7f86d525d700 1 -- 172.21.15.53:0/3241914717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 msgr2=0x5592c1688850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.859+0000 7f86d525d700 1 --2- 172.21.15.53:0/3241914717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 0x5592c1688850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.859+0000 7f86d525d700 1 -- 172.21.15.53:0/3241914717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592c167faa0 con 0x5592c167f3f0 2023-11-06T01:34:33.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.859+0000 7f86d5a5e700 1 --2- 172.21.15.53:0/3241914717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 0x5592c1688850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:34:33.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.860+0000 7f86d525d700 1 --2- 172.21.15.53:0/3241914717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c167f3f0 0x5592c167ee60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005bf0 tx=0x7f86c0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7c62376237296a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.860+0000 7f86bbfff700 1 -- 172.21.15.53:0/3241914717 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0025040 con 0x5592c167f3f0 2023-11-06T01:34:33.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.860+0000 7f86bbfff700 1 -- 172.21.15.53:0/3241914717 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c001ec20 con 0x5592c167f3f0 2023-11-06T01:34:33.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.860+0000 7f86bbfff700 1 -- 172.21.15.53:0/3241914717 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0027de0 con 0x5592c167f3f0 2023-11-06T01:34:33.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.861+0000 7f86e3651b80 1 -- 172.21.15.53:0/3241914717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c167f3f0 msgr2=0x5592c167ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.861+0000 7f86e3651b80 1 --2- 172.21.15.53:0/3241914717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c167f3f0 0x5592c167ee60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005bf0 tx=0x7f86c0009260 comp rx=0 tx=0).stop 2023-11-06T01:34:33.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.861+0000 7f86e3651b80 1 -- 172.21.15.53:0/3241914717 shutdown_connections 2023-11-06T01:34:33.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.861+0000 7f86e3651b80 1 --2- 172.21.15.53:0/3241914717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c167f3f0 0x5592c167ee60 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005bf0 tx=0x7f86c0009260 comp rx=0 tx=0).stop 2023-11-06T01:34:33.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.861+0000 7f86e3651b80 1 --2- 172.21.15.53:0/3241914717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 0x5592c1688850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.861+0000 7f86e3651b80 1 --2- 172.21.15.53:0/3241914717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16800f0 0x5592c168dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.861+0000 7f86e3651b80 1 -- 172.21.15.53:0/3241914717 >> 172.21.15.53:0/3241914717 conn(0x5592c15de0d0 msgr2=0x5592c16868c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.861+0000 7f86e3651b80 1 -- 172.21.15.53:0/3241914717 shutdown_connections 2023-11-06T01:34:33.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.862+0000 7f86e3651b80 1 -- 172.21.15.53:0/3241914717 wait complete. 2023-11-06T01:34:33.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.862+0000 7f86e3651b80 1 -- start start 2023-11-06T01:34:33.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.862+0000 7f86e3651b80 1 Processor -- start 2023-11-06T01:34:33.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 0x5592c168cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 0x5592c168d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86e3651b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16e3080 0x5592c16e5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86e3651b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c168d740 con 0x5592c1675d20 2023-11-06T01:34:33.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86e3651b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c16e59b0 con 0x5592c16800f0 2023-11-06T01:34:33.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86e3651b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c16e5af0 con 0x5592c16e3080 2023-11-06T01:34:33.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86e3651b80 10 client.0 _ll_register_callbacks cb 0x7f86cc0988e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:33.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86d4a5c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 0x5592c168d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86d5a5e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16e3080 0x5592c16e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86d525d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 0x5592c168cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86d4a5c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 0x5592c168d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51670/0 (socket says 172.21.15.53:51670) 2023-11-06T01:34:33.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86d4a5c700 1 -- 172.21.15.53:0/3723678743 learned_addr learned my addr 172.21.15.53:0/3723678743 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:33.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86d525d700 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 0x5592c168cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40230/0 (socket says 172.21.15.53:40230) 2023-11-06T01:34:33.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86d4a5c700 1 -- 172.21.15.53:0/3723678743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16e3080 msgr2=0x5592c16e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86d4a5c700 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16e3080 0x5592c16e5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86d4a5c700 1 -- 172.21.15.53:0/3723678743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 msgr2=0x5592c168cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:33.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86d4a5c700 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 0x5592c168cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.863+0000 7f86d4a5c700 1 -- 172.21.15.53:0/3723678743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592c1424250 con 0x5592c16800f0 2023-11-06T01:34:33.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.865+0000 7f86d4a5c700 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 0x5592c168d0f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5592c1680e50 tx=0x7f86c80097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c330407c1cf5708c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:33.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.865+0000 7f86b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:33.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.865+0000 7f86b8ff9700 1 -- 172.21.15.53:0/3723678743 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8029070 con 0x5592c16800f0 2023-11-06T01:34:33.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.865+0000 7f86e3651b80 1 -- 172.21.15.53:0/3723678743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86c800bca0 con 0x5592c16800f0 2023-11-06T01:34:33.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.865+0000 7f86b8ff9700 1 -- 172.21.15.53:0/3723678743 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f86c8006000 con 0x5592c16800f0 2023-11-06T01:34:33.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.865+0000 7f86b8ff9700 1 -- 172.21.15.53:0/3723678743 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c800e040 con 0x5592c16800f0 2023-11-06T01:34:33.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.865+0000 7f86e3651b80 10 client.25184 fetch_fsmap learned FSMap version 32 2023-11-06T01:34:33.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.865+0000 7f86e3651b80 1 -- 172.21.15.53:0/3723678743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5592c13fc2b0 con 0x5592c16800f0 2023-11-06T01:34:33.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.865+0000 7f86b8ff9700 1 -- 172.21.15.53:0/3723678743 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c800bca0 con 0x5592c16800f0 2023-11-06T01:34:33.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.866+0000 7f86b8ff9700 1 -- 172.21.15.53:0/3723678743 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f86c8022020 con 0x5592c16800f0 2023-11-06T01:34:33.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.866+0000 7f86e3651b80 10 client.25184 fetch_fsmap finished waiting for FSMap version 32 2023-11-06T01:34:33.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.866+0000 7f86e3651b80 10 client.25184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:33.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.866+0000 7f86e3651b80 20 client.25184 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:33.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.866+0000 7f86e3651b80 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5592c16e7010 0x5592c16e9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:33.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.866+0000 7f86e3651b80 4 client.25184 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:34:33.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.866+0000 7f86e3651b80 1 -- 172.21.15.53:0/3723678743 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5592c15c8d40 con 0x5592c16e7010 2023-11-06T01:34:33.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.866+0000 7f86d525d700 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5592c16e7010 0x5592c16e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:33.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.868+0000 7f86b8ff9700 10 client.25184 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:33.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.868+0000 7f86d525d700 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5592c16e7010 0x5592c16e9400 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a05d27bc06456207 server_cookie=32b41773500f9932 in_seq=0 out_seq=0 2023-11-06T01:34:33.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.868+0000 7f86b8ff9700 1 -- 172.21.15.53:0/3723678743 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5592c15c8d40 con 0x5592c16e7010 2023-11-06T01:34:33.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.868+0000 7f86b8ff9700 10 client.25184 handle_command_reply: tid=0 2023-11-06T01:34:33.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.868+0000 7f86e3651b80 1 client.25184 shutdown 2023-11-06T01:34:33.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.868+0000 7f86e3651b80 1 -- 172.21.15.53:0/3723678743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 msgr2=0x5592c168d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.868+0000 7f86e3651b80 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 0x5592c168d0f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5592c1680e50 tx=0x7f86c80097a0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.868+0000 7f86e3651b80 1 -- 172.21.15.53:0/3723678743 shutdown_connections 2023-11-06T01:34:33.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.869+0000 7f86e3651b80 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5592c16e7010 0x5592c16e9400 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.869+0000 7f86e3651b80 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5592c16e3080 0x5592c16e5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.869+0000 7f86e3651b80 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592c1675d20 0x5592c168cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.869+0000 7f86e3651b80 1 --2- 172.21.15.53:0/3723678743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592c16800f0 0x5592c168d0f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.869+0000 7f86e3651b80 1 -- 172.21.15.53:0/3723678743 >> 172.21.15.53:0/3723678743 conn(0x5592c15de0d0 msgr2=0x5592c16833f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.869+0000 7f86b8ff9700 0 client.25184 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:33.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.870+0000 7f86e3651b80 1 -- 172.21.15.53:0/3723678743 shutdown_connections 2023-11-06T01:34:33.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.870+0000 7f86e3651b80 1 -- 172.21.15.53:0/3723678743 wait complete. 2023-11-06T01:34:33.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.870+0000 7f86e3651b80 20 client.25184 trim_cache size 0 max 16384 2023-11-06T01:34:33.881 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:34:33.881 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:34:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:34:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:34:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:34:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:34:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:34:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:34:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:34:33.883 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:34:33.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.872+0000 7f86d74c1700 1 -- 172.21.15.53:0/1650917453 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f86bc0a0620 msgr2=0x7f86bc0a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.872+0000 7f86d74c1700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f86bc0a0620 0x7f86bc0a2ae0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f86d0100cd0 tx=0x7f86c0011720 comp rx=0 tx=0).stop 2023-11-06T01:34:33.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.872+0000 7f86d74c1700 1 -- 172.21.15.53:0/1650917453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d0101810 msgr2=0x7f86d01172f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:33.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.872+0000 7f86d74c1700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d0101810 0x7f86d01172f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00d980 tx=0x7f86cc00dc90 comp rx=0 tx=0).stop 2023-11-06T01:34:33.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.872+0000 7f86d74c1700 1 -- 172.21.15.53:0/1650917453 shutdown_connections 2023-11-06T01:34:33.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.872+0000 7f86d74c1700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f86bc0a0620 0x7f86bc0a2ae0 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f86d0100cd0 tx=0x7f86c0011720 comp rx=0 tx=0).stop 2023-11-06T01:34:33.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.872+0000 7f86d74c1700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d0101810 0x7f86d01172f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.872+0000 7f86d74c1700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86d00ffc80 0x7f86d010fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.872+0000 7f86d74c1700 1 --2- 172.21.15.53:0/1650917453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86d0100e70 0x7f86d0110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:33.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.872+0000 7f86d74c1700 1 -- 172.21.15.53:0/1650917453 >> 172.21.15.53:0/1650917453 conn(0x7f86d00fb1f0 msgr2=0x7f86d00fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:33.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.872+0000 7f86d74c1700 1 -- 172.21.15.53:0/1650917453 shutdown_connections 2023-11-06T01:34:33.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:34:33.873+0000 7f86d74c1700 1 -- 172.21.15.53:0/1650917453 wait complete. 2023-11-06T01:34:33.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:33 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:34:33.897 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:34:34.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:34 smithi163 ceph-mon[104416]: pgmap v318: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 86 MiB/s wr, 1.14k op/s 2023-11-06T01:34:34.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:34 smithi163 ceph-mon[104416]: scrub summary: active paths [/] 2023-11-06T01:34:34.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:34 smithi163 ceph-mon[104416]: scrub summary: active paths [/] 2023-11-06T01:34:34.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:34 smithi163 ceph-mon[104416]: scrub summary: active paths [/] 2023-11-06T01:34:34.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:34 smithi163 ceph-mon[104416]: scrub summary: active paths [/] 2023-11-06T01:34:34.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:34 smithi163 ceph-mon[104416]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:34.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:34 smithi163 ceph-mon[104416]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:34.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:34 smithi163 ceph-mon[104416]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:34.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:34 smithi163 ceph-mon[104416]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:34.851 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:34 smithi053 ceph-mon[99472]: pgmap v318: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 86 MiB/s wr, 1.14k op/s 2023-11-06T01:34:34.851 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:34 smithi053 ceph-mon[99472]: scrub summary: active paths [/] 2023-11-06T01:34:34.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:34 smithi053 ceph-mon[99472]: scrub summary: active paths [/] 2023-11-06T01:34:34.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:34 smithi053 ceph-mon[99472]: scrub summary: active paths [/] 2023-11-06T01:34:34.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:34 smithi053 ceph-mon[99472]: scrub summary: active paths [/] 2023-11-06T01:34:34.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:34 smithi053 ceph-mon[99472]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:34.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:34 smithi053 ceph-mon[99472]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:34.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:34 smithi053 ceph-mon[99472]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:34.853 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:34 smithi053 ceph-mon[99472]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:34.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:34 smithi062 ceph-mon[103924]: pgmap v318: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 86 MiB/s wr, 1.14k op/s 2023-11-06T01:34:34.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:34 smithi062 ceph-mon[103924]: scrub summary: active paths [/] 2023-11-06T01:34:34.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:34 smithi062 ceph-mon[103924]: scrub summary: active paths [/] 2023-11-06T01:34:34.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:34 smithi062 ceph-mon[103924]: scrub summary: active paths [/] 2023-11-06T01:34:34.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:34 smithi062 ceph-mon[103924]: scrub summary: active paths [/] 2023-11-06T01:34:34.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:34 smithi062 ceph-mon[103924]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:34.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:34 smithi062 ceph-mon[103924]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:34.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:34 smithi062 ceph-mon[103924]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:34.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:34 smithi062 ceph-mon[103924]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:35.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:35 smithi163 ceph-mon[104416]: scrub queued for path: / 2023-11-06T01:34:35.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:35 smithi163 ceph-mon[104416]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:35.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:35 smithi163 ceph-mon[104416]: scrub summary: active paths [/] 2023-11-06T01:34:35.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:35 smithi053 ceph-mon[99472]: scrub queued for path: / 2023-11-06T01:34:35.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:35 smithi053 ceph-mon[99472]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:35.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:35 smithi053 ceph-mon[99472]: scrub summary: active paths [/] 2023-11-06T01:34:35.902 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:35 smithi062 ceph-mon[103924]: scrub queued for path: / 2023-11-06T01:34:35.903 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:35 smithi062 ceph-mon[103924]: scrub summary: idle+waiting paths [/] 2023-11-06T01:34:35.903 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:35 smithi062 ceph-mon[103924]: scrub summary: active paths [/] 2023-11-06T01:34:36.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:36 smithi163 ceph-mon[104416]: pgmap v319: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 88 MiB/s wr, 1.39k op/s 2023-11-06T01:34:36.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:36.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:36.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:36.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:36.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:36.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:36 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:36.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:36 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:37.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:36 smithi053 ceph-mon[99472]: pgmap v319: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 88 MiB/s wr, 1.39k op/s 2023-11-06T01:34:37.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:36 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.098 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:36 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:37.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:36 smithi062 ceph-mon[103924]: pgmap v319: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 88 MiB/s wr, 1.39k op/s 2023-11-06T01:34:37.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:36 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:37.146 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:36 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:34:37.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 20.0 s, 3335.8 tps, lat 29.809 ms stddev 34.847 2023-11-06T01:34:37.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:37 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:34:37.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:37 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:34:37.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:37 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:38.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:34:38.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:34:38.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:37 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:38.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:37 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:34:38.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:37 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:34:38.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:37 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:34:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.070+0000 7fbbf54de700 1 -- 172.21.15.53:0/1390226951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf0101820 msgr2=0x7fbbf01080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.070+0000 7fbbf54de700 1 --2- 172.21.15.53:0/1390226951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf0101820 0x7fbbf01080d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009f90 tx=0x7fbbe800b330 comp rx=0 tx=0).stop 2023-11-06T01:34:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.070+0000 7fbbf54de700 1 -- 172.21.15.53:0/1390226951 shutdown_connections 2023-11-06T01:34:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.071+0000 7fbbf54de700 1 --2- 172.21.15.53:0/1390226951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf0101820 0x7fbbf01080d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.071+0000 7fbbf54de700 1 --2- 172.21.15.53:0/1390226951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf00ffc90 0x7fbbf01000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.071+0000 7fbbf54de700 1 --2- 172.21.15.53:0/1390226951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf0100e80 0x7fbbf01012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.071+0000 7fbbf54de700 1 -- 172.21.15.53:0/1390226951 >> 172.21.15.53:0/1390226951 conn(0x7fbbf00fb1e0 msgr2=0x7fbbf00fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.071+0000 7fbbf54de700 1 -- 172.21.15.53:0/1390226951 shutdown_connections 2023-11-06T01:34:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.071+0000 7fbbf54de700 1 -- 172.21.15.53:0/1390226951 wait complete. 2023-11-06T01:34:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.072+0000 7fbbf54de700 1 Processor -- start 2023-11-06T01:34:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.072+0000 7fbbf54de700 1 -- start start 2023-11-06T01:34:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbf54de700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf00ffc90 0x7fbbf0071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbf54de700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf0100e80 0x7fbbf00723c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbf54de700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf0101820 0x7fbbf00705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbf54de700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0070ae0 con 0x7fbbf00ffc90 2023-11-06T01:34:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbf54de700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0070c50 con 0x7fbbf0100e80 2023-11-06T01:34:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbf54de700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0070e10 con 0x7fbbf0101820 2023-11-06T01:34:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbee7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf0100e80 0x7fbbf00723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbef7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf0101820 0x7fbbf00705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbeeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf00ffc90 0x7fbbf0071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbef7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf0101820 0x7fbbf00705a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54488/0 (socket says 172.21.15.53:54488) 2023-11-06T01:34:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbef7fe700 1 -- 172.21.15.53:0/1505260921 learned_addr learned my addr 172.21.15.53:0/1505260921 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.073+0000 7fbbeeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf00ffc90 0x7fbbf0071e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40246/0 (socket says 172.21.15.53:40246) 2023-11-06T01:34:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.074+0000 7fbbeeffd700 1 -- 172.21.15.53:0/1505260921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf0101820 msgr2=0x7fbbf00705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.074+0000 7fbbeeffd700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf0101820 0x7fbbf00705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.074+0000 7fbbeeffd700 1 -- 172.21.15.53:0/1505260921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf0100e80 msgr2=0x7fbbf00723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.074+0000 7fbbeeffd700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf0100e80 0x7fbbf00723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.074+0000 7fbbeeffd700 1 -- 172.21.15.53:0/1505260921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00095f0 con 0x7fbbf00ffc90 2023-11-06T01:34:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.074+0000 7fbbeeffd700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf00ffc90 0x7fbbf0071e80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0005fd0 tx=0x7fbbe000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.074+0000 7fbbd7fff700 1 -- 172.21.15.53:0/1505260921 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000cbd0 con 0x7fbbf00ffc90 2023-11-06T01:34:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.074+0000 7fbbf54de700 1 -- 172.21.15.53:0/1505260921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe8009c40 con 0x7fbbf00ffc90 2023-11-06T01:34:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.074+0000 7fbbd7fff700 1 -- 172.21.15.53:0/1505260921 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe000cd30 con 0x7fbbf00ffc90 2023-11-06T01:34:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.074+0000 7fbbd7fff700 1 -- 172.21.15.53:0/1505260921 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0018710 con 0x7fbbf00ffc90 2023-11-06T01:34:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.074+0000 7fbbf54de700 1 -- 172.21.15.53:0/1505260921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf0068180 con 0x7fbbf00ffc90 2023-11-06T01:34:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.077+0000 7fbbd7fff700 1 -- 172.21.15.53:0/1505260921 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fbbe0018870 con 0x7fbbf00ffc90 2023-11-06T01:34:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.078+0000 7fbbd7fff700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbd80a0520 0x7fbbd80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.078+0000 7fbbd7fff700 1 -- 172.21.15.53:0/1505260921 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbbe0014070 con 0x7fbbf00ffc90 2023-11-06T01:34:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.078+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbd80a0520 0x7fbbd80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.079+0000 7fbc0166eb80 1 Processor -- start 2023-11-06T01:34:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbd80a0520 0x7fbbd80a29e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0072f90 tx=0x7fbbe4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbc0166eb80 1 -- start start 2023-11-06T01:34:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24affe0 0x5647d24b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24b0ed0 0x5647d24b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d24079b0 con 0x5647d24af680 2023-11-06T01:34:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d2405370 con 0x5647d24b0ed0 2023-11-06T01:34:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d2400c60 con 0x5647d24affe0 2023-11-06T01:34:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbbee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24affe0 0x5647d24b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbbeeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbbee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24affe0 0x5647d24b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54494/0 (socket says 172.21.15.53:54494) 2023-11-06T01:34:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbbee7fc700 1 -- 172.21.15.53:0/904579440 learned_addr learned my addr 172.21.15.53:0/904579440 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.080+0000 7fbbef7fe700 1 --2- 172.21.15.53:0/904579440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24b0ed0 0x5647d24b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.081+0000 7fbbee7fc700 1 -- 172.21.15.53:0/904579440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24b0ed0 msgr2=0x5647d24b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.081+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/904579440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24b0ed0 0x5647d24b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.081+0000 7fbbee7fc700 1 -- 172.21.15.53:0/904579440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 msgr2=0x5647d24afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.081+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/904579440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.081+0000 7fbbee7fc700 1 -- 172.21.15.53:0/904579440 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647d222ed80 con 0x5647d24affe0 2023-11-06T01:34:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.081+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/904579440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24affe0 0x5647d24b08f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4011d50 tx=0x7fbbe4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15822ddfd5504fc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.081+0000 7fbbd57fa700 1 -- 172.21.15.53:0/904579440 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe400eb20 con 0x5647d24affe0 2023-11-06T01:34:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.081+0000 7fbbd57fa700 1 -- 172.21.15.53:0/904579440 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe4021070 con 0x5647d24affe0 2023-11-06T01:34:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.081+0000 7fbbd57fa700 1 -- 172.21.15.53:0/904579440 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401d670 con 0x5647d24affe0 2023-11-06T01:34:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.082+0000 7fbc0166eb80 1 -- 172.21.15.53:0/904579440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24affe0 msgr2=0x5647d24b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.082+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/904579440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24affe0 0x5647d24b08f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4011d50 tx=0x7fbbe4014000 comp rx=0 tx=0).stop 2023-11-06T01:34:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.082+0000 7fbc0166eb80 1 -- 172.21.15.53:0/904579440 shutdown_connections 2023-11-06T01:34:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.082+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/904579440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24b0ed0 0x5647d24b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.082+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/904579440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.082+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/904579440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24affe0 0x5647d24b08f0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4011d50 tx=0x7fbbe4014000 comp rx=0 tx=0).stop 2023-11-06T01:34:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.082+0000 7fbc0166eb80 1 -- 172.21.15.53:0/904579440 >> 172.21.15.53:0/904579440 conn(0x5647d240f070 msgr2=0x5647d24ac040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.083+0000 7fbc0166eb80 1 -- 172.21.15.53:0/904579440 shutdown_connections 2023-11-06T01:34:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.083+0000 7fbc0166eb80 1 -- 172.21.15.53:0/904579440 wait complete. 2023-11-06T01:34:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbc0166eb80 1 -- start start 2023-11-06T01:34:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbc0166eb80 1 Processor -- start 2023-11-06T01:34:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b0ed0 0x5647d24b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d24c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d239d4b0 con 0x5647d24af680 2023-11-06T01:34:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d239d150 con 0x5647d24becd0 2023-11-06T01:34:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d22848c0 con 0x5647d24b0ed0 2023-11-06T01:34:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbc0166eb80 10 client.0 _ll_register_callbacks cb 0x5647d22eaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbbeeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbbee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b0ed0 0x5647d24b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbbeeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40258/0 (socket says 172.21.15.53:40258) 2023-11-06T01:34:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbbee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b0ed0 0x5647d24b9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54500/0 (socket says 172.21.15.53:54500) 2023-11-06T01:34:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbbeeffd700 1 -- 172.21.15.53:0/3652852693 learned_addr learned my addr 172.21.15.53:0/3652852693 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.084+0000 7fbbef7fe700 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d24c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbbee7fc700 1 -- 172.21.15.53:0/3652852693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 msgr2=0x5647d24c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d24c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbbee7fc700 1 -- 172.21.15.53:0/3652852693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 msgr2=0x5647d24b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbbee7fc700 1 -- 172.21.15.53:0/3652852693 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe4017070 con 0x5647d24b0ed0 2023-11-06T01:34:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbbeeffd700 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b0ed0 0x5647d24b9ca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4020000 tx=0x7fbbe4011ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a383190759cfa78 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbbd2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:34:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/3652852693 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe400ed30 con 0x5647d24b0ed0 2023-11-06T01:34:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/3652852693 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbbe401c410 con 0x5647d24b0ed0 2023-11-06T01:34:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbc0166eb80 1 -- 172.21.15.53:0/3652852693 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647d242bcb0 con 0x5647d24b0ed0 2023-11-06T01:34:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/3652852693 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbe400ee90 con 0x5647d24b0ed0 2023-11-06T01:34:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.085+0000 7fbc0166eb80 10 client.25291 fetch_fsmap learned FSMap version 33 2023-11-06T01:34:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.086+0000 7fbc0166eb80 1 -- 172.21.15.53:0/3652852693 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5647d2217c00 con 0x5647d24b0ed0 2023-11-06T01:34:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.086+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/3652852693 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647d242bcb0 con 0x5647d24b0ed0 2023-11-06T01:34:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.087+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/3652852693 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbbe4037020 con 0x5647d24b0ed0 2023-11-06T01:34:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.087+0000 7fbc0166eb80 10 client.25291 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:34:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.087+0000 7fbc0166eb80 10 client.25291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.087+0000 7fbc0166eb80 20 client.25291 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.087+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5647d24c5840 0x5647d24c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.087+0000 7fbc0166eb80 4 client.25291 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.087+0000 7fbc0166eb80 1 -- 172.21.15.53:0/3652852693 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647d247edb0 con 0x5647d24c5840 2023-11-06T01:34:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.088+0000 7fbbeeffd700 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5647d24c5840 0x5647d24c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.089+0000 7fbbd2ff5700 10 client.25291 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.089+0000 7fbbeeffd700 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5647d24c5840 0x5647d24c7c60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3909e1fd8b80c995 server_cookie=d5017b42d4845559 in_seq=0 out_seq=0 2023-11-06T01:34:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.090+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/3652852693 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5647d247edb0 con 0x5647d24c5840 2023-11-06T01:34:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.090+0000 7fbbd2ff5700 10 client.25291 handle_command_reply: tid=0 2023-11-06T01:34:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.090+0000 7fbc0166eb80 1 client.25291 shutdown 2023-11-06T01:34:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.096+0000 7fbc0166eb80 1 -- 172.21.15.53:0/3652852693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b0ed0 msgr2=0x5647d24b9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.096+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b0ed0 0x5647d24b9ca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4020000 tx=0x7fbbe4011ee0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.096+0000 7fbc0166eb80 1 -- 172.21.15.53:0/3652852693 shutdown_connections 2023-11-06T01:34:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.096+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d24c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.096+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5647d24c5840 0x5647d24c7c60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.096+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.096+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/3652852693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b0ed0 0x5647d24b9ca0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4020000 tx=0x7fbbe4011ee0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.096+0000 7fbc0166eb80 1 -- 172.21.15.53:0/3652852693 >> 172.21.15.53:0/3652852693 conn(0x5647d240f070 msgr2=0x5647d24aa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.096+0000 7fbbd2ff5700 0 client.25291 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.098+0000 7fbc0166eb80 1 -- 172.21.15.53:0/3652852693 shutdown_connections 2023-11-06T01:34:39.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.098+0000 7fbc0166eb80 1 -- 172.21.15.53:0/3652852693 wait complete. 2023-11-06T01:34:39.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.105+0000 7fbc0166eb80 20 client.25291 trim_cache size 0 max 16384 2023-11-06T01:34:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.132+0000 7fbc0166eb80 1 Processor -- start 2023-11-06T01:34:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.132+0000 7fbc0166eb80 1 -- start start 2023-11-06T01:34:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.132+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24af680 0x5647d24d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.132+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d24b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.132+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24b45e0 0x5647d24b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.132+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d24a2ed0 con 0x5647d24b45e0 2023-11-06T01:34:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.132+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d24c1590 con 0x5647d24becd0 2023-11-06T01:34:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.132+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d24a27a0 con 0x5647d24af680 2023-11-06T01:34:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.133+0000 7fbbeeffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24af680 0x5647d24d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.133+0000 7fbbee7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d24b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.133+0000 7fbbef7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24b45e0 0x5647d24b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.133+0000 7fbbee7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d24b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51706/0 (socket says 172.21.15.53:51706) 2023-11-06T01:34:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.133+0000 7fbbeeffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24af680 0x5647d24d5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54506/0 (socket says 172.21.15.53:54506) 2023-11-06T01:34:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.133+0000 7fbbee7fc700 1 -- 172.21.15.53:0/1532447023 learned_addr learned my addr 172.21.15.53:0/1532447023 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.133+0000 7fbbee7fc700 1 -- 172.21.15.53:0/1532447023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24af680 msgr2=0x5647d24d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.133+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/1532447023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24af680 0x5647d24d5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.133+0000 7fbbee7fc700 1 -- 172.21.15.53:0/1532447023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24b45e0 msgr2=0x5647d24b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.133+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/1532447023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24b45e0 0x5647d24b9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.133+0000 7fbbee7fc700 1 -- 172.21.15.53:0/1532447023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647d22551f0 con 0x5647d24becd0 2023-11-06T01:34:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.134+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/1532447023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d24b40a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4016040 tx=0x7fbbe4003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad1e3db4847aab6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.134+0000 7fbbd5ffb700 1 -- 172.21.15.53:0/1532447023 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe40092d0 con 0x5647d24becd0 2023-11-06T01:34:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.134+0000 7fbbd5ffb700 1 -- 172.21.15.53:0/1532447023 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe400ed30 con 0x5647d24becd0 2023-11-06T01:34:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.134+0000 7fbbd5ffb700 1 -- 172.21.15.53:0/1532447023 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401c410 con 0x5647d24becd0 2023-11-06T01:34:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.135+0000 7fbc0166eb80 1 -- 172.21.15.53:0/1532447023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 msgr2=0x5647d24b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.135+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/1532447023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d24b40a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4016040 tx=0x7fbbe4003820 comp rx=0 tx=0).stop 2023-11-06T01:34:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.135+0000 7fbc0166eb80 1 -- 172.21.15.53:0/1532447023 shutdown_connections 2023-11-06T01:34:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.135+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/1532447023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24b45e0 0x5647d24b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.135+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/1532447023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24af680 0x5647d24d5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.135+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/1532447023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d24b40a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.135+0000 7fbc0166eb80 1 -- 172.21.15.53:0/1532447023 >> 172.21.15.53:0/1532447023 conn(0x5647d240f070 msgr2=0x5647d24b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.135+0000 7fbc0166eb80 1 -- 172.21.15.53:0/1532447023 shutdown_connections 2023-11-06T01:34:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.135+0000 7fbc0166eb80 1 -- 172.21.15.53:0/1532447023 wait complete. 2023-11-06T01:34:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.136+0000 7fbc0166eb80 1 -- start start 2023-11-06T01:34:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.136+0000 7fbc0166eb80 1 Processor -- start 2023-11-06T01:34:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.136+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24d4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.136+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b45e0 0x5647d24d48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.136+0000 7fbc0166eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d251e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.136+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d24d4f20 con 0x5647d24af680 2023-11-06T01:34:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.136+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d251ea00 con 0x5647d24becd0 2023-11-06T01:34:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.136+0000 7fbc0166eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647d2513ca0 con 0x5647d24b45e0 2023-11-06T01:34:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.136+0000 7fbc0166eb80 10 client.0 _ll_register_callbacks cb 0x7fbbe00091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbeeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24d4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbef7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d251e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbef7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d251e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51712/0 (socket says 172.21.15.53:51712) 2023-11-06T01:34:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbef7fe700 1 -- 172.21.15.53:0/58202317 learned_addr learned my addr 172.21.15.53:0/58202317 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbeeffd700 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24d4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40272/0 (socket says 172.21.15.53:40272) 2023-11-06T01:34:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbee7fc700 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b45e0 0x5647d24d48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbef7fe700 1 -- 172.21.15.53:0/58202317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b45e0 msgr2=0x5647d24d48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbef7fe700 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b45e0 0x5647d24d48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbef7fe700 1 -- 172.21.15.53:0/58202317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 msgr2=0x5647d24d4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbef7fe700 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24d4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbef7fe700 1 -- 172.21.15.53:0/58202317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647d2217c00 con 0x5647d24becd0 2023-11-06T01:34:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbef7fe700 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d251e4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009790 tx=0x7fbbe8007f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5109c3096bd628f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.137+0000 7fbbd2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/58202317 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8020070 con 0x5647d24becd0 2023-11-06T01:34:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbc0166eb80 1 -- 172.21.15.53:0/58202317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbe4021070 con 0x5647d24becd0 2023-11-06T01:34:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/58202317 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbbe8003dd0 con 0x5647d24becd0 2023-11-06T01:34:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/58202317 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbe800e040 con 0x5647d24becd0 2023-11-06T01:34:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbc0166eb80 10 client.25202 fetch_fsmap learned FSMap version 33 2023-11-06T01:34:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbc0166eb80 1 -- 172.21.15.53:0/58202317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5647d222d250 con 0x5647d24becd0 2023-11-06T01:34:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/58202317 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe4021070 con 0x5647d24becd0 2023-11-06T01:34:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/58202317 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbbe800b050 con 0x5647d24becd0 2023-11-06T01:34:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbc0166eb80 10 client.25202 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:34:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbc0166eb80 10 client.25202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbc0166eb80 20 client.25202 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5647d2515790 0x5647d2517bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbc0166eb80 4 client.25202 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:34:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbc0166eb80 1 -- 172.21.15.53:0/58202317 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5647d23f9ce0 con 0x5647d2515790 2023-11-06T01:34:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.138+0000 7fbbeeffd700 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5647d2515790 0x5647d2517bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.139+0000 7fbbd2ff5700 10 client.25202 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.139+0000 7fbbeeffd700 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5647d2515790 0x5647d2517bb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=744c254e951bb3ba server_cookie=e80a440053c8b07 in_seq=0 out_seq=0 2023-11-06T01:34:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.157+0000 7fbbd2ff5700 1 -- 172.21.15.53:0/58202317 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5647d23f9ce0 con 0x5647d2515790 2023-11-06T01:34:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.157+0000 7fbbd2ff5700 10 client.25202 handle_command_reply: tid=0 2023-11-06T01:34:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbc0166eb80 1 client.25202 shutdown 2023-11-06T01:34:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbc0166eb80 1 -- 172.21.15.53:0/58202317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 msgr2=0x5647d251e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d251e4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009790 tx=0x7fbbe8007f40 comp rx=0 tx=0).stop 2023-11-06T01:34:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbc0166eb80 1 -- 172.21.15.53:0/58202317 shutdown_connections 2023-11-06T01:34:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647d24becd0 0x5647d251e4c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5647d2515790 0x5647d2517bb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d24af680 0x5647d24d4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbc0166eb80 1 --2- 172.21.15.53:0/58202317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5647d24b45e0 0x5647d24d48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbbd2ff5700 0 client.25202 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbc0166eb80 1 -- 172.21.15.53:0/58202317 >> 172.21.15.53:0/58202317 conn(0x5647d240f070 msgr2=0x5647d24b9700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbc0166eb80 1 -- 172.21.15.53:0/58202317 shutdown_connections 2023-11-06T01:34:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.158+0000 7fbc0166eb80 1 -- 172.21.15.53:0/58202317 wait complete. 2023-11-06T01:34:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.160+0000 7fbc0166eb80 20 client.25202 trim_cache size 0 max 16384 2023-11-06T01:34:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:34:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:34:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:34:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:34:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 -- 172.21.15.53:0/1505260921 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbd80a0520 msgr2=0x7fbbd80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbd80a0520 0x7fbbd80a29e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0072f90 tx=0x7fbbe4009260 comp rx=0 tx=0).stop 2023-11-06T01:34:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 -- 172.21.15.53:0/1505260921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf00ffc90 msgr2=0x7fbbf0071e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf00ffc90 0x7fbbf0071e80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0005fd0 tx=0x7fbbe000ee70 comp rx=0 tx=0).stop 2023-11-06T01:34:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 -- 172.21.15.53:0/1505260921 shutdown_connections 2023-11-06T01:34:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbd80a0520 0x7fbbd80a29e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf0101820 0x7fbbf00705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf00ffc90 0x7fbbf0071e80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 --2- 172.21.15.53:0/1505260921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf0100e80 0x7fbbf00723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 -- 172.21.15.53:0/1505260921 >> 172.21.15.53:0/1505260921 conn(0x7fbbf00fb1e0 msgr2=0x7fbbf0104330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 -- 172.21.15.53:0/1505260921 shutdown_connections 2023-11-06T01:34:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:39.161+0000 7fbbf54de700 1 -- 172.21.15.53:0/1505260921 wait complete. 2023-11-06T01:34:39.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:39 smithi163 ceph-mon[104416]: pgmap v320: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 56 MiB/s wr, 1.12k op/s 2023-11-06T01:34:39.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:39 smithi053 ceph-mon[99472]: pgmap v320: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 56 MiB/s wr, 1.12k op/s 2023-11-06T01:34:39.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:39 smithi062 ceph-mon[103924]: pgmap v320: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 56 MiB/s wr, 1.12k op/s 2023-11-06T01:34:41.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:41 smithi163 ceph-mon[104416]: pgmap v321: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 47 MiB/s wr, 1.10k op/s 2023-11-06T01:34:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:41 smithi053 ceph-mon[99472]: pgmap v321: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 47 MiB/s wr, 1.10k op/s 2023-11-06T01:34:41.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:41 smithi062 ceph-mon[103924]: pgmap v321: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 47 MiB/s wr, 1.10k op/s 2023-11-06T01:34:42.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 25.0 s, 3518.2 tps, lat 28.020 ms stddev 34.436 2023-11-06T01:34:43.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:43 smithi163 ceph-mon[104416]: pgmap v322: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 54 MiB/s wr, 1.32k op/s 2023-11-06T01:34:43.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:43 smithi053 ceph-mon[99472]: pgmap v322: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 54 MiB/s wr, 1.32k op/s 2023-11-06T01:34:43.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:43 smithi062 ceph-mon[103924]: pgmap v322: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 54 MiB/s wr, 1.32k op/s 2023-11-06T01:34:45.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:45 smithi163 ceph-mon[104416]: pgmap v323: 97 pgs: 97 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 48 MiB/s wr, 1.29k op/s 2023-11-06T01:34:45.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:45 smithi053 ceph-mon[99472]: pgmap v323: 97 pgs: 97 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 48 MiB/s wr, 1.29k op/s 2023-11-06T01:34:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:45 smithi062 ceph-mon[103924]: pgmap v323: 97 pgs: 97 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 48 MiB/s wr, 1.29k op/s 2023-11-06T01:34:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 30.0 s, 4130.0 tps, lat 24.672 ms stddev 44.221 2023-11-06T01:34:47.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:47 smithi163 ceph-mon[104416]: pgmap v324: 97 pgs: 97 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 72 MiB/s wr, 2.30k op/s 2023-11-06T01:34:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:47 smithi053 ceph-mon[99472]: pgmap v324: 97 pgs: 97 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 72 MiB/s wr, 2.30k op/s 2023-11-06T01:34:47.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:47 smithi062 ceph-mon[103924]: pgmap v324: 97 pgs: 97 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 72 MiB/s wr, 2.30k op/s 2023-11-06T01:34:49.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:49 smithi163 ceph-mon[104416]: pgmap v325: 97 pgs: 97 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 72 MiB/s wr, 2.45k op/s 2023-11-06T01:34:49.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:49 smithi053 ceph-mon[99472]: pgmap v325: 97 pgs: 97 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 72 MiB/s wr, 2.45k op/s 2023-11-06T01:34:49.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:49 smithi062 ceph-mon[103924]: pgmap v325: 97 pgs: 97 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 72 MiB/s wr, 2.45k op/s 2023-11-06T01:34:51.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:51 smithi163 ceph-mon[104416]: pgmap v326: 97 pgs: 97 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 68 MiB/s wr, 2.49k op/s 2023-11-06T01:34:51.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:51 smithi053 ceph-mon[99472]: pgmap v326: 97 pgs: 97 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 68 MiB/s wr, 2.49k op/s 2023-11-06T01:34:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:51 smithi062 ceph-mon[103924]: pgmap v326: 97 pgs: 97 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 68 MiB/s wr, 2.49k op/s 2023-11-06T01:34:52.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 35.0 s, 4718.1 tps, lat 21.190 ms stddev 25.304 2023-11-06T01:34:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.347+0000 7fe9b167f700 1 -- 172.21.15.53:0/3860642716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9ac100fa0 msgr2=0x7fe9ac107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.347+0000 7fe9b167f700 1 --2- 172.21.15.53:0/3860642716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9ac100fa0 0x7fe9ac107850 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00aad0 tx=0x7fe99c00ade0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.349+0000 7fe9b167f700 1 -- 172.21.15.53:0/3860642716 shutdown_connections 2023-11-06T01:34:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.349+0000 7fe9b167f700 1 --2- 172.21.15.53:0/3860642716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac0ff410 0x7fe9ac0ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.349+0000 7fe9b167f700 1 --2- 172.21.15.53:0/3860642716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9ac100fa0 0x7fe9ac107850 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.349+0000 7fe9b167f700 1 --2- 172.21.15.53:0/3860642716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9ac100600 0x7fe9ac100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.349+0000 7fe9b167f700 1 -- 172.21.15.53:0/3860642716 >> 172.21.15.53:0/3860642716 conn(0x7fe9ac0fa960 msgr2=0x7fe9ac0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.350+0000 7fe9b167f700 1 -- 172.21.15.53:0/3860642716 shutdown_connections 2023-11-06T01:34:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.351+0000 7fe9b167f700 1 -- 172.21.15.53:0/3860642716 wait complete. 2023-11-06T01:34:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9b167f700 1 Processor -- start 2023-11-06T01:34:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9b167f700 1 -- start start 2023-11-06T01:34:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9b167f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac0ff410 0x7fe9ac19d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9b167f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9ac100600 0x7fe9ac19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9b167f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9ac100fa0 0x7fe9ac19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9b167f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac19eb90 con 0x7fe9ac100600 2023-11-06T01:34:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9b167f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac19ed00 con 0x7fe9ac0ff410 2023-11-06T01:34:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9b167f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac19eec0 con 0x7fe9ac100fa0 2023-11-06T01:34:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac0ff410 0x7fe9ac19d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9aa7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9ac100600 0x7fe9ac19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac0ff410 0x7fe9ac19d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51722/0 (socket says 172.21.15.53:51722) 2023-11-06T01:34:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9aaffd700 1 -- 172.21.15.53:0/2605763116 learned_addr learned my addr 172.21.15.53:0/2605763116 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.352+0000 7fe9ab7fe700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9ac100fa0 0x7fe9ac19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.353+0000 7fe9aaffd700 1 -- 172.21.15.53:0/2605763116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9ac100fa0 msgr2=0x7fe9ac19e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.353+0000 7fe9aaffd700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9ac100fa0 0x7fe9ac19e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.353+0000 7fe9aaffd700 1 -- 172.21.15.53:0/2605763116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9ac100600 msgr2=0x7fe9ac19d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.353+0000 7fe9aaffd700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9ac100600 0x7fe9ac19d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.353+0000 7fe9aaffd700 1 -- 172.21.15.53:0/2605763116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c00a700 con 0x7fe9ac0ff410 2023-11-06T01:34:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.353+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9ac100600 0x7fe9ac19d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:34:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.353+0000 7fe9aaffd700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac0ff410 0x7fe9ac19d170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400efc0 tx=0x7fe9a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.354+0000 7fe993fff700 1 -- 172.21.15.53:0/2605763116 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4018070 con 0x7fe9ac0ff410 2023-11-06T01:34:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.354+0000 7fe993fff700 1 -- 172.21.15.53:0/2605763116 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9a400f040 con 0x7fe9ac0ff410 2023-11-06T01:34:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.354+0000 7fe993fff700 1 -- 172.21.15.53:0/2605763116 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4004280 con 0x7fe9ac0ff410 2023-11-06T01:34:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.354+0000 7fe9b167f700 1 -- 172.21.15.53:0/2605763116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ac19f140 con 0x7fe9ac0ff410 2023-11-06T01:34:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.354+0000 7fe9b167f700 1 -- 172.21.15.53:0/2605763116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ac1a5960 con 0x7fe9ac0ff410 2023-11-06T01:34:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.358+0000 7fe993fff700 1 -- 172.21.15.53:0/2605763116 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe9a400ca70 con 0x7fe9ac0ff410 2023-11-06T01:34:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.360+0000 7fe993fff700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe9940a06f0 0x7fe9940a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.361+0000 7fe993fff700 1 -- 172.21.15.53:0/2605763116 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe9a40cb820 con 0x7fe9ac0ff410 2023-11-06T01:34:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.361+0000 7fe9bd80fb80 1 Processor -- start 2023-11-06T01:34:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.361+0000 7fe9bd80fb80 1 -- start start 2023-11-06T01:34:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.361+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e27413aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.361+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e27413b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.361+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 0x55e27413c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.361+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2740929b0 con 0x55e27413bed0 2023-11-06T01:34:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.361+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e274090370 con 0x55e27413afe0 2023-11-06T01:34:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.361+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e27408bc60 con 0x55e27413a680 2023-11-06T01:34:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.362+0000 7fe9aaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e27413aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.362+0000 7fe9aaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e27413aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54532/0 (socket says 172.21.15.53:54532) 2023-11-06T01:34:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.362+0000 7fe9aaffd700 1 -- 172.21.15.53:0/3999235477 learned_addr learned my addr 172.21.15.53:0/3999235477 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.362+0000 7fe9ab7fe700 1 --2- 172.21.15.53:0/3999235477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 0x55e27413c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.362+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/3999235477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e27413b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.362+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe9940a06f0 0x7fe9940a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.363+0000 7fe9aa7fc700 1 -- 172.21.15.53:0/3999235477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 msgr2=0x55e27413aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.363+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/3999235477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e27413aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.363+0000 7fe9aa7fc700 1 -- 172.21.15.53:0/3999235477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 msgr2=0x55e27413c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.363+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/3999235477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 0x55e27413c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.363+0000 7fe9aa7fc700 1 -- 172.21.15.53:0/3999235477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e273eb9d80 con 0x55e27413afe0 2023-11-06T01:34:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.363+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/3999235477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e27413b8f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00137b0 tx=0x7fe9a0013ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=768d982cb9cf079 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.363+0000 7fe9ab7fe700 1 --2- 172.21.15.53:0/3999235477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 0x55e27413c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:34:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.363+0000 7fe9917fa700 1 -- 172.21.15.53:0/3999235477 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0005a20 con 0x55e27413afe0 2023-11-06T01:34:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.363+0000 7fe9917fa700 1 -- 172.21.15.53:0/3999235477 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9a0021930 con 0x55e27413afe0 2023-11-06T01:34:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.363+0000 7fe9917fa700 1 -- 172.21.15.53:0/3999235477 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0016670 con 0x55e27413afe0 2023-11-06T01:34:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.364+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/3999235477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 msgr2=0x55e27413b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.364+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe9940a06f0 0x7fe9940a2bb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0020000 tx=0x7fe9a001b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.364+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/3999235477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e27413b8f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00137b0 tx=0x7fe9a0013ac0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.364+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/3999235477 shutdown_connections 2023-11-06T01:34:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.364+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/3999235477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 0x55e27413c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.364+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/3999235477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e27413aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.364+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/3999235477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e27413b8f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.364+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/3999235477 >> 172.21.15.53:0/3999235477 conn(0x55e27409a070 msgr2=0x55e274137040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.365+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/3999235477 shutdown_connections 2023-11-06T01:34:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.365+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/3999235477 wait complete. 2023-11-06T01:34:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.365+0000 7fe9bd80fb80 1 -- start start 2023-11-06T01:34:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.365+0000 7fe9bd80fb80 1 Processor -- start 2023-11-06T01:34:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e274155440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413afe0 0x55e274155980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413bed0 0x55e27415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2740284b0 con 0x55e27413afe0 2023-11-06T01:34:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e274028150 con 0x55e27413bed0 2023-11-06T01:34:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e273f0f8c0 con 0x55e27413a680 2023-11-06T01:34:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9bd80fb80 10 client.0 _ll_register_callbacks cb 0x55e273f75a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9ab7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413bed0 0x55e27415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9ab7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413bed0 0x55e27415c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51736/0 (socket says 172.21.15.53:51736) 2023-11-06T01:34:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9ab7fe700 1 -- 172.21.15.53:0/2335495463 learned_addr learned my addr 172.21.15.53:0/2335495463 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9aaffd700 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e274155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9ab7fe700 1 -- 172.21.15.53:0/2335495463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 msgr2=0x55e274155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9ab7fe700 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e274155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9ab7fe700 1 -- 172.21.15.53:0/2335495463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413afe0 msgr2=0x55e274155980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:34:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9ab7fe700 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413afe0 0x55e274155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.366+0000 7fe9ab7fe700 1 -- 172.21.15.53:0/2335495463 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a0013410 con 0x55e27413bed0 2023-11-06T01:34:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.367+0000 7fe9ab7fe700 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413bed0 0x55e27415c9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00aad0 tx=0x7fe99c005050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee86056b87875a60 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.367+0000 7fe98eff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.367+0000 7fe98eff5700 1 -- 172.21.15.53:0/2335495463 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c0054f0 con 0x55e27413bed0 2023-11-06T01:34:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.367+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2335495463 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2740b6cb0 con 0x55e27413bed0 2023-11-06T01:34:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.367+0000 7fe98eff5700 1 -- 172.21.15.53:0/2335495463 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe99c012070 con 0x55e27413bed0 2023-11-06T01:34:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.367+0000 7fe98eff5700 1 -- 172.21.15.53:0/2335495463 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe99c005650 con 0x55e27413bed0 2023-11-06T01:34:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.370+0000 7fe9bd80fb80 10 client.25214 fetch_fsmap learned FSMap version 33 2023-11-06T01:34:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.370+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2335495463 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e273ea2c00 con 0x55e27413bed0 2023-11-06T01:34:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.370+0000 7fe98eff5700 1 -- 172.21.15.53:0/2335495463 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e2740b6cb0 con 0x55e27413bed0 2023-11-06T01:34:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.370+0000 7fe98eff5700 1 -- 172.21.15.53:0/2335495463 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe99c016540 con 0x55e27413bed0 2023-11-06T01:34:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.370+0000 7fe9bd80fb80 10 client.25214 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:34:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.370+0000 7fe9bd80fb80 10 client.25214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.370+0000 7fe9bd80fb80 20 client.25214 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.370+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e274161380 0x55e2741637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.370+0000 7fe9bd80fb80 4 client.25214 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:34:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.370+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2335495463 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e274109db0 con 0x55e274161380 2023-11-06T01:34:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.371+0000 7fe9aaffd700 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e274161380 0x55e2741637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.372+0000 7fe9aaffd700 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e274161380 0x55e2741637a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9a892084da117e server_cookie=9e775b5862411f2c in_seq=0 out_seq=0 2023-11-06T01:34:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.372+0000 7fe98eff5700 10 client.25214 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe98eff5700 1 -- 172.21.15.53:0/2335495463 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e274109db0 con 0x55e274161380 2023-11-06T01:34:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe98eff5700 10 client.25214 handle_command_reply: tid=0 2023-11-06T01:34:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe9bd80fb80 1 client.25214 shutdown 2023-11-06T01:34:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2335495463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413bed0 msgr2=0x55e27415c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413bed0 0x55e27415c9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00aad0 tx=0x7fe99c005050 comp rx=0 tx=0).stop 2023-11-06T01:34:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2335495463 shutdown_connections 2023-11-06T01:34:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413bed0 0x55e27415c9c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e274161380 0x55e2741637a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e274155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2335495463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413afe0 0x55e274155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe98eff5700 0 client.25214 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2335495463 >> 172.21.15.53:0/2335495463 conn(0x55e27409a070 msgr2=0x55e2741353c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2335495463 shutdown_connections 2023-11-06T01:34:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.373+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2335495463 wait complete. 2023-11-06T01:34:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.374+0000 7fe9bd80fb80 20 client.25214 trim_cache size 0 max 16384 2023-11-06T01:34:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.395+0000 7fe9bd80fb80 1 Processor -- start 2023-11-06T01:34:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.395+0000 7fe9bd80fb80 1 -- start start 2023-11-06T01:34:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.396+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e27415a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.396+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e27413c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.396+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 0x55e274134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.396+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e274154fb0 con 0x55e27413bed0 2023-11-06T01:34:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.396+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e27415e630 con 0x55e27413afe0 2023-11-06T01:34:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.396+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e274141ff0 con 0x55e27413a680 2023-11-06T01:34:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.396+0000 7fe9aa7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e27413c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.396+0000 7fe9ab7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 0x55e274134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.396+0000 7fe9ab7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 0x55e274134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40308/0 (socket says 172.21.15.53:40308) 2023-11-06T01:34:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.396+0000 7fe9aa7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e27413c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51744/0 (socket says 172.21.15.53:51744) 2023-11-06T01:34:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.396+0000 7fe9ab7fe700 1 -- 172.21.15.53:0/2634272661 learned_addr learned my addr 172.21.15.53:0/2634272661 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.397+0000 7fe9aa7fc700 1 -- 172.21.15.53:0/2634272661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 msgr2=0x55e27415a970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:34:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.397+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/2634272661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e27415a970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.397+0000 7fe9aa7fc700 1 -- 172.21.15.53:0/2634272661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 msgr2=0x55e274134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.397+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/2634272661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 0x55e274134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.397+0000 7fe9aa7fc700 1 -- 172.21.15.53:0/2634272661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e273ee01f0 con 0x55e27413afe0 2023-11-06T01:34:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.397+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/2634272661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e27413c2b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0013e50 tx=0x7fe9a00047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c8712e55defb07 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.397+0000 7fe991ffb700 1 -- 172.21.15.53:0/2634272661 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0016430 con 0x55e27413afe0 2023-11-06T01:34:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.397+0000 7fe991ffb700 1 -- 172.21.15.53:0/2634272661 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9a0016590 con 0x55e27413afe0 2023-11-06T01:34:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.397+0000 7fe991ffb700 1 -- 172.21.15.53:0/2634272661 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0005de0 con 0x55e27413afe0 2023-11-06T01:34:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.398+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2634272661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 msgr2=0x55e27413c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.398+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2634272661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e27413c2b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0013e50 tx=0x7fe9a00047d0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.398+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2634272661 shutdown_connections 2023-11-06T01:34:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.398+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2634272661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413bed0 0x55e274134660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.398+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2634272661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413a680 0x55e27415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.398+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2634272661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e27413c2b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.398+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2634272661 >> 172.21.15.53:0/2634272661 conn(0x55e27409a070 msgr2=0x55e274142240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.399+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2634272661 shutdown_connections 2023-11-06T01:34:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.399+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2634272661 wait complete. 2023-11-06T01:34:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.399+0000 7fe9bd80fb80 1 -- start start 2023-11-06T01:34:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.399+0000 7fe9bd80fb80 1 Processor -- start 2023-11-06T01:34:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.399+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413a680 0x55e274156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.399+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e274157100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.399+0000 7fe9bd80fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413bed0 0x55e27415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.399+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e27415a610 con 0x55e27413a680 2023-11-06T01:34:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.399+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e27415a780 con 0x55e27413afe0 2023-11-06T01:34:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.399+0000 7fe9bd80fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e27415a940 con 0x55e27413bed0 2023-11-06T01:34:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.400+0000 7fe9bd80fb80 10 client.0 _ll_register_callbacks cb 0x7fe9a40097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:34:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.400+0000 7fe9ab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413bed0 0x55e27415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.400+0000 7fe9aa7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e274157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.400+0000 7fe9ab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413bed0 0x55e27415a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54550/0 (socket says 172.21.15.53:54550) 2023-11-06T01:34:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.400+0000 7fe9aa7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e274157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51750/0 (socket says 172.21.15.53:51750) 2023-11-06T01:34:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.400+0000 7fe9ab7fe700 1 -- 172.21.15.53:0/2546911382 learned_addr learned my addr 172.21.15.53:0/2546911382 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:34:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.400+0000 7fe9aa7fc700 1 -- 172.21.15.53:0/2546911382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413bed0 msgr2=0x55e27415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:34:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.400+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/2546911382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413bed0 0x55e27415a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.400+0000 7fe9aa7fc700 1 -- 172.21.15.53:0/2546911382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413a680 msgr2=0x55e274156bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:34:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.400+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/2546911382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413a680 0x55e274156bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.400+0000 7fe9aa7fc700 1 -- 172.21.15.53:0/2546911382 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a0025070 con 0x55e27413afe0 2023-11-06T01:34:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.401+0000 7fe9aa7fc700 1 --2- 172.21.15.53:0/2546911382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e274157100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9a002c040 tx=0x7fe9a0006f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa33b4e8c88a1e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:34:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.401+0000 7fe98eff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:34:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.401+0000 7fe98eff5700 1 -- 172.21.15.53:0/2546911382 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0021670 con 0x55e27413afe0 2023-11-06T01:34:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.401+0000 7fe98eff5700 1 -- 172.21.15.53:0/2546911382 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe9a0005b20 con 0x55e27413afe0 2023-11-06T01:34:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.401+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2546911382 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9a0021930 con 0x55e27413afe0 2023-11-06T01:34:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.402+0000 7fe98eff5700 1 -- 172.21.15.53:0/2546911382 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9a00217d0 con 0x55e27413afe0 2023-11-06T01:34:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.402+0000 7fe9bd80fb80 10 client.25220 fetch_fsmap learned FSMap version 33 2023-11-06T01:34:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.402+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2546911382 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e273eb8250 con 0x55e27413afe0 2023-11-06T01:34:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.402+0000 7fe98eff5700 1 -- 172.21.15.53:0/2546911382 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9a0021930 con 0x55e27413afe0 2023-11-06T01:34:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.403+0000 7fe98eff5700 1 -- 172.21.15.53:0/2546911382 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe9a003c070 con 0x55e27413afe0 2023-11-06T01:34:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.403+0000 7fe9bd80fb80 10 client.25220 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:34:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.403+0000 7fe9bd80fb80 10 client.25220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:34:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.403+0000 7fe9bd80fb80 20 client.25220 populate_metadata read hostname 'smithi053' 2023-11-06T01:34:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.403+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2546911382 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e2741c5030 0x55e2741c7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:34:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.403+0000 7fe9bd80fb80 4 client.25220 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:34:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.403+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2546911382 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e274084ce0 con 0x55e2741c5030 2023-11-06T01:34:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.403+0000 7fe9aaffd700 1 --2- 172.21.15.53:0/2546911382 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e2741c5030 0x55e2741c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:34:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.404+0000 7fe98eff5700 10 client.25220 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.404+0000 7fe9aaffd700 1 --2- 172.21.15.53:0/2546911382 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e2741c5030 0x55e2741c7450 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe9cf8dbececd28 server_cookie=cb0f2a1cf4f26429 in_seq=0 out_seq=0 2023-11-06T01:34:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.425+0000 7fe98eff5700 1 -- 172.21.15.53:0/2546911382 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e274084ce0 con 0x55e2741c5030 2023-11-06T01:34:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.425+0000 7fe98eff5700 10 client.25220 handle_command_reply: tid=0 2023-11-06T01:34:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.425+0000 7fe9bd80fb80 1 client.25220 shutdown 2023-11-06T01:34:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.426+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2546911382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 msgr2=0x55e274157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.426+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2546911382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e274157100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9a002c040 tx=0x7fe9a0006f10 comp rx=0 tx=0).stop 2023-11-06T01:34:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.426+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2546911382 shutdown_connections 2023-11-06T01:34:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.426+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2546911382 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e2741c5030 0x55e2741c7450 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.426+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2546911382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e27413bed0 0x55e27415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.426+0000 7fe98eff5700 0 client.25220 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:34:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.426+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2546911382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e27413a680 0x55e274156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.426+0000 7fe9bd80fb80 1 --2- 172.21.15.53:0/2546911382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e27413afe0 0x55e274157100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.426+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2546911382 >> 172.21.15.53:0/2546911382 conn(0x55e27409a070 msgr2=0x55e274134660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.426+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2546911382 shutdown_connections 2023-11-06T01:34:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:34:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:34:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:34:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:34:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.426+0000 7fe9bd80fb80 1 -- 172.21.15.53:0/2546911382 wait complete. 2023-11-06T01:34:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.427+0000 7fe9bd80fb80 20 client.25220 trim_cache size 0 max 16384 2023-11-06T01:34:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 -- 172.21.15.53:0/2605763116 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe9940a06f0 msgr2=0x7fe9940a2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe9940a06f0 0x7fe9940a2bb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0020000 tx=0x7fe9a001b000 comp rx=0 tx=0).stop 2023-11-06T01:34:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 -- 172.21.15.53:0/2605763116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac0ff410 msgr2=0x7fe9ac19d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:34:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac0ff410 0x7fe9ac19d170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400efc0 tx=0x7fe9a400c3a0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 -- 172.21.15.53:0/2605763116 shutdown_connections 2023-11-06T01:34:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe9940a06f0 0x7fe9940a2bb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac0ff410 0x7fe9ac19d170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9ac100fa0 0x7fe9ac19e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 --2- 172.21.15.53:0/2605763116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9ac100600 0x7fe9ac19d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:34:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 -- 172.21.15.53:0/2605763116 >> 172.21.15.53:0/2605763116 conn(0x7fe9ac0fa960 msgr2=0x7fe9ac10d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:34:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 -- 172.21.15.53:0/2605763116 shutdown_connections 2023-11-06T01:34:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:34:52.428+0000 7fe9b167f700 1 -- 172.21.15.53:0/2605763116 wait complete. 2023-11-06T01:34:53.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:53 smithi053 ceph-mon[99472]: pgmap v327: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 73 MiB/s wr, 2.72k op/s 2023-11-06T01:34:53.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:53 smithi062 ceph-mon[103924]: pgmap v327: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 73 MiB/s wr, 2.72k op/s 2023-11-06T01:34:53.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:53 smithi163 ceph-mon[104416]: pgmap v327: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 73 MiB/s wr, 2.72k op/s 2023-11-06T01:34:55.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:55 smithi053 ceph-mon[99472]: pgmap v328: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 59 MiB/s wr, 2.46k op/s 2023-11-06T01:34:55.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:55 smithi062 ceph-mon[103924]: pgmap v328: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 59 MiB/s wr, 2.46k op/s 2023-11-06T01:34:55.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:55 smithi163 ceph-mon[104416]: pgmap v328: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 59 MiB/s wr, 2.46k op/s 2023-11-06T01:34:57.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 40.0 s, 3988.5 tps, lat 25.048 ms stddev 27.606 2023-11-06T01:34:57.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:57 smithi053 ceph-mon[99472]: pgmap v329: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 64 MiB/s wr, 2.65k op/s 2023-11-06T01:34:57.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:57 smithi062 ceph-mon[103924]: pgmap v329: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 64 MiB/s wr, 2.65k op/s 2023-11-06T01:34:57.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:57 smithi163 ceph-mon[104416]: pgmap v329: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 64 MiB/s wr, 2.65k op/s 2023-11-06T01:34:59.598 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:34:59 smithi053 ceph-mon[99472]: pgmap v330: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 42 MiB/s wr, 1.73k op/s 2023-11-06T01:34:59.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:34:59 smithi062 ceph-mon[103924]: pgmap v330: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 42 MiB/s wr, 1.73k op/s 2023-11-06T01:34:59.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:34:59 smithi163 ceph-mon[104416]: pgmap v330: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 42 MiB/s wr, 1.73k op/s 2023-11-06T01:35:01.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:01 smithi053 ceph-mon[99472]: pgmap v331: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 MiB/s wr, 1.41k op/s 2023-11-06T01:35:01.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:01 smithi062 ceph-mon[103924]: pgmap v331: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 MiB/s wr, 1.41k op/s 2023-11-06T01:35:01.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:01 smithi163 ceph-mon[104416]: pgmap v331: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 MiB/s wr, 1.41k op/s 2023-11-06T01:35:02.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 45.0 s, 3648.0 tps, lat 27.474 ms stddev 47.276 2023-11-06T01:35:03.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:03 smithi053 ceph-mon[99472]: pgmap v332: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 40 MiB/s wr, 1.49k op/s 2023-11-06T01:35:03.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:03 smithi062 ceph-mon[103924]: pgmap v332: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 40 MiB/s wr, 1.49k op/s 2023-11-06T01:35:03.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:03 smithi163 ceph-mon[104416]: pgmap v332: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 40 MiB/s wr, 1.49k op/s 2023-11-06T01:35:03.897 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:35:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.131+0000 7f3f4f418700 1 -- 172.21.15.53:0/1974226104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f481073b0 msgr2=0x7f3f481077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:04.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.131+0000 7f3f4f418700 1 --2- 172.21.15.53:0/1974226104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f481073b0 0x7f3f481077d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3f440098f0 tx=0x7f3f44009c00 comp rx=0 tx=0).stop 2023-11-06T01:35:04.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.131+0000 7f3f4f418700 1 -- 172.21.15.53:0/1974226104 shutdown_connections 2023-11-06T01:35:04.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.131+0000 7f3f4f418700 1 --2- 172.21.15.53:0/1974226104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f48108f40 0x7f3f4810f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.131+0000 7f3f4f418700 1 --2- 172.21.15.53:0/1974226104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f481073b0 0x7f3f481077d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.131+0000 7f3f4f418700 1 --2- 172.21.15.53:0/1974226104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f481085a0 0x7f3f48108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.131+0000 7f3f4f418700 1 -- 172.21.15.53:0/1974226104 >> 172.21.15.53:0/1974226104 conn(0x7f3f48076370 msgr2=0x7f3f48078790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.131+0000 7f3f4f418700 1 -- 172.21.15.53:0/1974226104 shutdown_connections 2023-11-06T01:35:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.131+0000 7f3f4f418700 1 -- 172.21.15.53:0/1974226104 wait complete. 2023-11-06T01:35:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.132+0000 7f3f4f418700 1 Processor -- start 2023-11-06T01:35:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.132+0000 7f3f4f418700 1 -- start start 2023-11-06T01:35:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.132+0000 7f3f4f418700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f481073b0 0x7f3f4819d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.132+0000 7f3f4f418700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f481085a0 0x7f3f4819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.132+0000 7f3f4f418700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f48108f40 0x7f3f481a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.132+0000 7f3f4f418700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f481a4cb0 con 0x7f3f481073b0 2023-11-06T01:35:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.132+0000 7f3f4f418700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4806a7b0 con 0x7f3f481085a0 2023-11-06T01:35:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.132+0000 7f3f4f418700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4806a940 con 0x7f3f48108f40 2023-11-06T01:35:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.133+0000 7f3f4c9b3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f481085a0 0x7f3f4819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.133+0000 7f3f4c9b3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f481085a0 0x7f3f4819d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51762/0 (socket says 172.21.15.53:51762) 2023-11-06T01:35:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.133+0000 7f3f4c9b3700 1 -- 172.21.15.53:0/2702536150 learned_addr learned my addr 172.21.15.53:0/2702536150 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.133+0000 7f3f4c9b3700 1 -- 172.21.15.53:0/2702536150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f48108f40 msgr2=0x7f3f481a46e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:35:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.133+0000 7f3f4c9b3700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f48108f40 0x7f3f481a46e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.133+0000 7f3f4c9b3700 1 -- 172.21.15.53:0/2702536150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f481073b0 msgr2=0x7f3f4819d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.133+0000 7f3f4c9b3700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f481073b0 0x7f3f4819d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.133+0000 7f3f4c9b3700 1 -- 172.21.15.53:0/2702536150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f380095f0 con 0x7f3f481085a0 2023-11-06T01:35:04.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.134+0000 7f3f4c9b3700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f481085a0 0x7f3f4819d670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800ec10 tx=0x7f3f3800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:04.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.134+0000 7f3f3e7fc700 1 -- 172.21.15.53:0/2702536150 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800cde0 con 0x7f3f481085a0 2023-11-06T01:35:04.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.134+0000 7f3f3e7fc700 1 -- 172.21.15.53:0/2702536150 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f380044e0 con 0x7f3f481085a0 2023-11-06T01:35:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.134+0000 7f3f3e7fc700 1 -- 172.21.15.53:0/2702536150 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38010470 con 0x7f3f481085a0 2023-11-06T01:35:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.134+0000 7f3f4f418700 1 -- 172.21.15.53:0/2702536150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f44009520 con 0x7f3f481085a0 2023-11-06T01:35:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.134+0000 7f3f4f418700 1 -- 172.21.15.53:0/2702536150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4806aef0 con 0x7f3f481085a0 2023-11-06T01:35:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.137+0000 7f3f3e7fc700 1 -- 172.21.15.53:0/2702536150 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f3f38004020 con 0x7f3f481085a0 2023-11-06T01:35:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.138+0000 7f3f3e7fc700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f340c2600 0x7f3f340c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.138+0000 7f3f3e7fc700 1 -- 172.21.15.53:0/2702536150 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3f38014070 con 0x7f3f481085a0 2023-11-06T01:35:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.139+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f340c2600 0x7f3f340c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.140+0000 7f3f5b5a8b80 1 Processor -- start 2023-11-06T01:35:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.140+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f340c2600 0x7f3f340c4ac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009230 tx=0x7f3f440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.140+0000 7f3f5b5a8b80 1 -- start start 2023-11-06T01:35:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.141+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 0x55d070cd4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.141+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd4a70 0x55d070cd2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.141+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070cd33f0 0x55d070cd3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.141+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070c2aa10 con 0x55d070cd33f0 2023-11-06T01:35:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.141+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070c283d0 con 0x55d070cd40f0 2023-11-06T01:35:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.141+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070c23cc0 con 0x55d070cd4a70 2023-11-06T01:35:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.141+0000 7f3f4d1b4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 0x55d070cd4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.141+0000 7f3f4d1b4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 0x55d070cd4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51766/0 (socket says 172.21.15.53:51766) 2023-11-06T01:35:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.141+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/295403461 learned_addr learned my addr 172.21.15.53:0/295403461 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.141+0000 7f3f4d9b5700 1 --2- 172.21.15.53:0/295403461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070cd33f0 0x55d070cd3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.142+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/295403461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd4a70 msgr2=0x55d070cd2e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:35:04.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.142+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/295403461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd4a70 0x55d070cd2e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.142+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/295403461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070cd33f0 msgr2=0x55d070cd3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:04.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.142+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/295403461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070cd33f0 0x55d070cd3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.142+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/295403461 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d070a51de0 con 0x55d070cd40f0 2023-11-06T01:35:04.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.142+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/295403461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 0x55d070cd4530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f44016000 tx=0x7f3f4401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ec130299fb9396 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:04.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.142+0000 7f3f337fe700 1 -- 172.21.15.53:0/295403461 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44027070 con 0x55d070cd40f0 2023-11-06T01:35:04.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.142+0000 7f3f337fe700 1 -- 172.21.15.53:0/295403461 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f4400bb20 con 0x55d070cd40f0 2023-11-06T01:35:04.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.142+0000 7f3f337fe700 1 -- 172.21.15.53:0/295403461 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4401c070 con 0x55d070cd40f0 2023-11-06T01:35:04.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.143+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/295403461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 msgr2=0x55d070cd4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:04.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.143+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/295403461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 0x55d070cd4530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f44016000 tx=0x7f3f4401c000 comp rx=0 tx=0).stop 2023-11-06T01:35:04.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.144+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/295403461 shutdown_connections 2023-11-06T01:35:04.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.144+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/295403461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 0x55d070cd4530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.144+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/295403461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070cd33f0 0x55d070cd3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.144+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/295403461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd4a70 0x55d070cd2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.144+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/295403461 >> 172.21.15.53:0/295403461 conn(0x55d070c320d0 msgr2=0x55d070ccd940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:04.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.145+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/295403461 shutdown_connections 2023-11-06T01:35:04.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.145+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/295403461 wait complete. 2023-11-06T01:35:04.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.145+0000 7f3f5b5a8b80 1 -- start start 2023-11-06T01:35:04.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.145+0000 7f3f5b5a8b80 1 Processor -- start 2023-11-06T01:35:04.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.146+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070cdc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.146+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070cd40f0 0x55d070cdcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.146+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd4a70 0x55d070ce3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.146+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070bc0510 con 0x55d070cd40f0 2023-11-06T01:35:04.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.146+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070bc01b0 con 0x55d070cd4a70 2023-11-06T01:35:04.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.146+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070aa7920 con 0x55d070cd33f0 2023-11-06T01:35:04.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.146+0000 7f3f5b5a8b80 10 client.0 _ll_register_callbacks cb 0x55d070b0da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:04.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.146+0000 7f3f4d1b4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070cdc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.146+0000 7f3f4d1b4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070cdc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54572/0 (socket says 172.21.15.53:54572) 2023-11-06T01:35:04.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.146+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/1606228493 learned_addr learned my addr 172.21.15.53:0/1606228493 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:04.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.146+0000 7f3f4d9b5700 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd4a70 0x55d070ce3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.147+0000 7f3f4c9b3700 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070cd40f0 0x55d070cdcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.147+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/1606228493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd4a70 msgr2=0x55d070ce3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:04.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.147+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd4a70 0x55d070ce3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.147+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/1606228493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070cd40f0 msgr2=0x55d070cdcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:04.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.147+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070cd40f0 0x55d070cdcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.147+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/1606228493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f48110140 con 0x55d070cd33f0 2023-11-06T01:35:04.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.158+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070cdc9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009810 tx=0x7f3f44005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8747014c706a400 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:04.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.158+0000 7f3f30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:35:04.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.159+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1606228493 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44027040 con 0x55d070cd33f0 2023-11-06T01:35:04.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.159+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1606228493 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3f44003b30 con 0x55d070cd33f0 2023-11-06T01:35:04.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.159+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1606228493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d070c4ed10 con 0x55d070cd33f0 2023-11-06T01:35:04.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.164+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1606228493 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f44003cf0 con 0x55d070cd33f0 2023-11-06T01:35:04.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.165+0000 7f3f5b5a8b80 10 client.25315 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:04.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.165+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1606228493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d070a3ac60 con 0x55d070cd33f0 2023-11-06T01:35:04.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.166+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1606228493 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d070c4ed10 con 0x55d070cd33f0 2023-11-06T01:35:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.166+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1606228493 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3f4402b490 con 0x55d070cd33f0 2023-11-06T01:35:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.166+0000 7f3f5b5a8b80 10 client.25315 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.166+0000 7f3f5b5a8b80 10 client.25315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.166+0000 7f3f5b5a8b80 20 client.25315 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.166+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d070ce81e0 0x55d070cea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.166+0000 7f3f5b5a8b80 4 client.25315 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:35:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.166+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1606228493 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d070ca1e10 con 0x55d070ce81e0 2023-11-06T01:35:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.167+0000 7f3f4c9b3700 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d070ce81e0 0x55d070cea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.169+0000 7f3f30ff9700 10 client.25315 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:04.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.169+0000 7f3f4c9b3700 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d070ce81e0 0x55d070cea5d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc21dcc435f1480 server_cookie=7e6504f23abed97 in_seq=0 out_seq=0 2023-11-06T01:35:04.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.169+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1606228493 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d070ca1e10 con 0x55d070ce81e0 2023-11-06T01:35:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.169+0000 7f3f30ff9700 10 client.25315 handle_command_reply: tid=0 2023-11-06T01:35:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.169+0000 7f3f5b5a8b80 1 client.25315 shutdown 2023-11-06T01:35:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.170+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1606228493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 msgr2=0x55d070cdc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.170+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070cdc9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009810 tx=0x7f3f44005270 comp rx=0 tx=0).stop 2023-11-06T01:35:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.170+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1606228493 shutdown_connections 2023-11-06T01:35:04.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.170+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd4a70 0x55d070ce3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.170+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d070ce81e0 0x55d070cea5d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.170+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070cdc9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.170+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1606228493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070cd40f0 0x55d070cdcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.170+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1606228493 >> 172.21.15.53:0/1606228493 conn(0x55d070c320d0 msgr2=0x55d070ccd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.170+0000 7f3f30ff9700 0 client.25315 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.170+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1606228493 shutdown_connections 2023-11-06T01:35:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.170+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1606228493 wait complete. 2023-11-06T01:35:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.171+0000 7f3f5b5a8b80 20 client.25315 trim_cache size 0 max 16384 2023-11-06T01:35:04.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.199+0000 7f3f5b5a8b80 1 Processor -- start 2023-11-06T01:35:04.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.199+0000 7f3f5b5a8b80 1 -- start start 2023-11-06T01:35:04.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 0x55d070cda870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070ce3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 0x55d070cdadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070ce4f20 con 0x55d070c320d0 2023-11-06T01:35:04.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070ca1e10 con 0x55d070cd40f0 2023-11-06T01:35:04.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070c4ed10 con 0x55d070cd33f0 2023-11-06T01:35:04.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4d1b4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 0x55d070cda870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4c9b3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070ce3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4c9b3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070ce3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54580/0 (socket says 172.21.15.53:54580) 2023-11-06T01:35:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4d1b4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 0x55d070cda870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40340/0 (socket says 172.21.15.53:40340) 2023-11-06T01:35:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4c9b3700 1 -- 172.21.15.53:0/845320793 learned_addr learned my addr 172.21.15.53:0/845320793 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:04.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4d9b5700 1 --2- 172.21.15.53:0/845320793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 0x55d070cdadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/845320793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 msgr2=0x55d070ce3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:04.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/845320793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070ce3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/845320793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 msgr2=0x55d070cdadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:04.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/845320793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 0x55d070cdadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/845320793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d070a78250 con 0x55d070c320d0 2023-11-06T01:35:04.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4c9b3700 1 --2- 172.21.15.53:0/845320793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070ce3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:35:04.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.200+0000 7f3f4d9b5700 1 --2- 172.21.15.53:0/845320793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 0x55d070cdadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:35:04.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.201+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/845320793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 0x55d070cda870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400fa70 tx=0x7f3f44005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adefa39767fe8fe6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:04.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.201+0000 7f3f33fff700 1 -- 172.21.15.53:0/845320793 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44027030 con 0x55d070c320d0 2023-11-06T01:35:04.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.201+0000 7f3f33fff700 1 -- 172.21.15.53:0/845320793 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f4402bac0 con 0x55d070c320d0 2023-11-06T01:35:04.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.201+0000 7f3f33fff700 1 -- 172.21.15.53:0/845320793 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4401c040 con 0x55d070c320d0 2023-11-06T01:35:04.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.202+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/845320793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 msgr2=0x55d070cda870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.202+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/845320793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 0x55d070cda870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400fa70 tx=0x7f3f44005670 comp rx=0 tx=0).stop 2023-11-06T01:35:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.202+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/845320793 shutdown_connections 2023-11-06T01:35:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.202+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/845320793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd40f0 0x55d070cdadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.202+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/845320793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 0x55d070cda870 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.202+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/845320793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd33f0 0x55d070ce3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.202+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/845320793 >> 172.21.15.53:0/845320793 conn(0x55d070ccdc70 msgr2=0x55d070ce4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.202+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/845320793 shutdown_connections 2023-11-06T01:35:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.202+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/845320793 wait complete. 2023-11-06T01:35:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.203+0000 7f3f5b5a8b80 1 -- start start 2023-11-06T01:35:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.203+0000 7f3f5b5a8b80 1 Processor -- start 2023-11-06T01:35:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 0x55d070ce3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd33f0 0x55d070ce3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f5b5a8b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd40f0 0x55d070ce1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070ce2670 con 0x55d070c320d0 2023-11-06T01:35:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070ce27e0 con 0x55d070cd33f0 2023-11-06T01:35:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f5b5a8b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d070ce29a0 con 0x55d070cd40f0 2023-11-06T01:35:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f5b5a8b80 10 client.0 _ll_register_callbacks cb 0x7f3f40009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f4d1b4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 0x55d070ce3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f4c9b3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd33f0 0x55d070ce3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f4d1b4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 0x55d070ce3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40346/0 (socket says 172.21.15.53:40346) 2023-11-06T01:35:04.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f4c9b3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd33f0 0x55d070ce3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51786/0 (socket says 172.21.15.53:51786) 2023-11-06T01:35:04.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.204+0000 7f3f4d1b4700 1 -- 172.21.15.53:0/1758458731 learned_addr learned my addr 172.21.15.53:0/1758458731 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:04.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.205+0000 7f3f4d9b5700 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd40f0 0x55d070ce1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.205+0000 7f3f4d9b5700 1 -- 172.21.15.53:0/1758458731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd33f0 msgr2=0x55d070ce3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.205+0000 7f3f4d9b5700 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd33f0 0x55d070ce3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.205+0000 7f3f4d9b5700 1 -- 172.21.15.53:0/1758458731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 msgr2=0x55d070ce3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.205+0000 7f3f4d9b5700 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 0x55d070ce3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.205+0000 7f3f4d9b5700 1 -- 172.21.15.53:0/1758458731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4806aef0 con 0x55d070cd40f0 2023-11-06T01:35:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.205+0000 7f3f4d9b5700 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd40f0 0x55d070ce1f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55d070cc8820 tx=0x7f3f40004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa2b057fd443e9bf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.205+0000 7f3f30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:35:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.206+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1758458731 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4000e040 con 0x55d070cd40f0 2023-11-06T01:35:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.206+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1758458731 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3f400093a0 con 0x55d070cd40f0 2023-11-06T01:35:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.206+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1758458731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f4400bb20 con 0x55d070cd40f0 2023-11-06T01:35:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.207+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1758458731 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f40014bb0 con 0x55d070cd40f0 2023-11-06T01:35:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.207+0000 7f3f5b5a8b80 10 client.25321 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.207+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1758458731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f3f40014320 con 0x55d070cd40f0 2023-11-06T01:35:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.207+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1758458731 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f4400bb20 con 0x55d070cd40f0 2023-11-06T01:35:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.207+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1758458731 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3f40029050 con 0x55d070cd40f0 2023-11-06T01:35:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.207+0000 7f3f5b5a8b80 10 client.25321 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.207+0000 7f3f5b5a8b80 10 client.25321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:04.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.207+0000 7f3f5b5a8b80 20 client.25321 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:04.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.207+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d070cd4a70 0x55d070d4e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:04.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.207+0000 7f3f5b5a8b80 4 client.25321 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:35:04.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.207+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1758458731 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d070c1cd40 con 0x55d070cd4a70 2023-11-06T01:35:04.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.208+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d070cd4a70 0x55d070d4e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.208+0000 7f3f4d1b4700 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d070cd4a70 0x55d070d4e2a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b244f8c853414782 server_cookie=a7ca890224aa59e1 in_seq=0 out_seq=0 2023-11-06T01:35:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.208+0000 7f3f30ff9700 10 client.25321 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.208+0000 7f3f30ff9700 1 -- 172.21.15.53:0/1758458731 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d070c1cd40 con 0x55d070cd4a70 2023-11-06T01:35:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.208+0000 7f3f30ff9700 10 client.25321 handle_command_reply: tid=0 2023-11-06T01:35:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.208+0000 7f3f5b5a8b80 1 client.25321 shutdown 2023-11-06T01:35:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.209+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1758458731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd40f0 msgr2=0x55d070ce1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.209+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd40f0 0x55d070ce1f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55d070cc8820 tx=0x7f3f40004950 comp rx=0 tx=0).stop 2023-11-06T01:35:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.209+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1758458731 shutdown_connections 2023-11-06T01:35:04.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.209+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d070cd4a70 0x55d070d4e2a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.209+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d070cd40f0 0x55d070ce1f80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.209+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d070c320d0 0x55d070ce3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.209+0000 7f3f5b5a8b80 1 --2- 172.21.15.53:0/1758458731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d070cd33f0 0x55d070ce3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.209+0000 7f3f30ff9700 0 client.25321 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:04.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.209+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1758458731 >> 172.21.15.53:0/1758458731 conn(0x55d070ccdc70 msgr2=0x55d070cd73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:04.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.211+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1758458731 shutdown_connections 2023-11-06T01:35:04.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.211+0000 7f3f5b5a8b80 1 -- 172.21.15.53:0/1758458731 wait complete. 2023-11-06T01:35:04.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.211+0000 7f3f5b5a8b80 20 client.25321 trim_cache size 0 max 16384 2023-11-06T01:35:04.223 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:35:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:35:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:35:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:35:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:35:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:35:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:35:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:35:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:35:04.225 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:35:04.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 -- 172.21.15.53:0/2702536150 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f340c2600 msgr2=0x7f3f340c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:04.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f340c2600 0x7f3f340c4ac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009230 tx=0x7f3f440058e0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 -- 172.21.15.53:0/2702536150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f481085a0 msgr2=0x7f3f4819d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:04.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f481085a0 0x7f3f4819d670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800ec10 tx=0x7f3f3800c2d0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 -- 172.21.15.53:0/2702536150 shutdown_connections 2023-11-06T01:35:04.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f340c2600 0x7f3f340c4ac0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f48108f40 0x7f3f481a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f481073b0 0x7f3f4819d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 --2- 172.21.15.53:0/2702536150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f481085a0 0x7f3f4819d670 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:04.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 -- 172.21.15.53:0/2702536150 >> 172.21.15.53:0/2702536150 conn(0x7f3f48076370 msgr2=0x7f3f48078610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:04.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 -- 172.21.15.53:0/2702536150 shutdown_connections 2023-11-06T01:35:04.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:04.212+0000 7f3f4f418700 1 -- 172.21.15.53:0/2702536150 wait complete. 2023-11-06T01:35:04.233 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:35:05.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:05 smithi053 ceph-mon[99472]: pgmap v333: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 1.39k op/s 2023-11-06T01:35:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.617+0000 7f6dd1586700 1 -- 172.21.15.53:0/110625288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dcc1073f0 msgr2=0x7f6dcc107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.617+0000 7f6dd1586700 1 --2- 172.21.15.53:0/110625288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dcc1073f0 0x7f6dcc107810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400b0d0 tx=0x7f6dc400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.618+0000 7f6dd1586700 1 -- 172.21.15.53:0/110625288 shutdown_connections 2023-11-06T01:35:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.618+0000 7f6dd1586700 1 --2- 172.21.15.53:0/110625288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dcc108f80 0x7f6dcc10f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.618+0000 7f6dd1586700 1 --2- 172.21.15.53:0/110625288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dcc1073f0 0x7f6dcc107810 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.618+0000 7f6dd1586700 1 --2- 172.21.15.53:0/110625288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc1085e0 0x7f6dcc108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.618+0000 7f6dd1586700 1 -- 172.21.15.53:0/110625288 >> 172.21.15.53:0/110625288 conn(0x7f6dcc076370 msgr2=0x7f6dcc0787d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.618+0000 7f6dd1586700 1 -- 172.21.15.53:0/110625288 shutdown_connections 2023-11-06T01:35:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.618+0000 7f6dd1586700 1 -- 172.21.15.53:0/110625288 wait complete. 2023-11-06T01:35:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dd1586700 1 Processor -- start 2023-11-06T01:35:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dd1586700 1 -- start start 2023-11-06T01:35:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dd1586700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dcc1073f0 0x7f6dcc19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dd1586700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dcc1085e0 0x7f6dcc19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dd1586700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc108f80 0x7f6dcc19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dd1586700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc19eda0 con 0x7f6dcc1073f0 2023-11-06T01:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dd1586700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc19ef10 con 0x7f6dcc1085e0 2023-11-06T01:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dd1586700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc19f0d0 con 0x7f6dcc108f80 2023-11-06T01:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dcaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dcc1073f0 0x7f6dcc19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dca7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dcc1085e0 0x7f6dcc19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dcaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dcc1073f0 0x7f6dcc19d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40360/0 (socket says 172.21.15.53:40360) 2023-11-06T01:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dcaffd700 1 -- 172.21.15.53:0/3444846670 learned_addr learned my addr 172.21.15.53:0/3444846670 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dca7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dcc1085e0 0x7f6dcc19d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51798/0 (socket says 172.21.15.53:51798) 2023-11-06T01:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.619+0000 7f6dcb7fe700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc108f80 0x7f6dcc19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.620+0000 7f6dcaffd700 1 -- 172.21.15.53:0/3444846670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc108f80 msgr2=0x7f6dcc19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.620+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc108f80 0x7f6dcc19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.620+0000 7f6dcaffd700 1 -- 172.21.15.53:0/3444846670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dcc1085e0 msgr2=0x7f6dcc19d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.620+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dcc1085e0 0x7f6dcc19d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.620+0000 7f6dcaffd700 1 -- 172.21.15.53:0/3444846670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc009520 con 0x7f6dcc1073f0 2023-11-06T01:35:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.620+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dcc1073f0 0x7f6dcc19d380 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4000f80 tx=0x7f6dc4008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.620+0000 7f6dcb7fe700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc108f80 0x7f6dcc19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:35:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.622+0000 7f6db3fff700 1 -- 172.21.15.53:0/3444846670 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400e040 con 0x7f6dcc1073f0 2023-11-06T01:35:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.622+0000 7f6db3fff700 1 -- 172.21.15.53:0/3444846670 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc400be30 con 0x7f6dcc1073f0 2023-11-06T01:35:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.622+0000 7f6dd1586700 1 -- 172.21.15.53:0/3444846670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc4009d00 con 0x7f6dcc1073f0 2023-11-06T01:35:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.622+0000 7f6db3fff700 1 -- 172.21.15.53:0/3444846670 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc401bac0 con 0x7f6dcc1073f0 2023-11-06T01:35:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.622+0000 7f6dd1586700 1 -- 172.21.15.53:0/3444846670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc1a5990 con 0x7f6dcc1073f0 2023-11-06T01:35:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.626+0000 7f6db3fff700 1 -- 172.21.15.53:0/3444846670 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f6dc401d670 con 0x7f6dcc1073f0 2023-11-06T01:35:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.627+0000 7f6db3fff700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6db40a05a0 0x7f6db40a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.627+0000 7f6db3fff700 1 -- 172.21.15.53:0/3444846670 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6dc40cce50 con 0x7f6dcc1073f0 2023-11-06T01:35:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6db40a05a0 0x7f6db40a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6ddd716b80 1 Processor -- start 2023-11-06T01:35:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6ddd716b80 1 -- start start 2023-11-06T01:35:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 0x5569677f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 0x5569677f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f2ed0 0x5569677f3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569677499b0 con 0x5569677f1680 2023-11-06T01:35:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556967747370 con 0x5569677f1fe0 2023-11-06T01:35:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556967742c60 con 0x5569677f2ed0 2023-11-06T01:35:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6db40a05a0 0x7f6db40a2a60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009c60 tx=0x7f6dc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6dca7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 0x5569677f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6dcaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 0x5569677f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6dcaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 0x5569677f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40368/0 (socket says 172.21.15.53:40368) 2023-11-06T01:35:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6dca7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 0x5569677f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51808/0 (socket says 172.21.15.53:51808) 2023-11-06T01:35:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.628+0000 7f6dcaffd700 1 -- 172.21.15.53:0/2858790370 learned_addr learned my addr 172.21.15.53:0/2858790370 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.629+0000 7f6dcb7fe700 1 --2- 172.21.15.53:0/2858790370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f2ed0 0x5569677f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.629+0000 7f6dcaffd700 1 -- 172.21.15.53:0/2858790370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f2ed0 msgr2=0x5569677f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.629+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/2858790370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f2ed0 0x5569677f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.629+0000 7f6dcaffd700 1 -- 172.21.15.53:0/2858790370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 msgr2=0x5569677f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.629+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/2858790370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 0x5569677f28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.629+0000 7f6dcaffd700 1 -- 172.21.15.53:0/2858790370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556967570d80 con 0x5569677f1680 2023-11-06T01:35:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.629+0000 7f6dcb7fe700 1 --2- 172.21.15.53:0/2858790370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f2ed0 0x5569677f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:35:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.629+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/2858790370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 0x5569677f1aa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4007590 tx=0x7f6dc401b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd02fb8d29c4451 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.629+0000 7f6db17fa700 1 -- 172.21.15.53:0/2858790370 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc409ae20 con 0x5569677f1680 2023-11-06T01:35:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.629+0000 7f6db17fa700 1 -- 172.21.15.53:0/2858790370 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc409a720 con 0x5569677f1680 2023-11-06T01:35:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.629+0000 7f6db17fa700 1 -- 172.21.15.53:0/2858790370 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc402d2f0 con 0x5569677f1680 2023-11-06T01:35:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.631+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2858790370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 msgr2=0x5569677f1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.631+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/2858790370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 0x5569677f1aa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4007590 tx=0x7f6dc401b520 comp rx=0 tx=0).stop 2023-11-06T01:35:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.631+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2858790370 shutdown_connections 2023-11-06T01:35:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.631+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/2858790370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f2ed0 0x5569677f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.631+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/2858790370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 0x5569677f1aa0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.631+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/2858790370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 0x5569677f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.631+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2858790370 >> 172.21.15.53:0/2858790370 conn(0x556967751070 msgr2=0x5569677ee040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.632+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2858790370 shutdown_connections 2023-11-06T01:35:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.632+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2858790370 wait complete. 2023-11-06T01:35:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.632+0000 7f6ddd716b80 1 -- start start 2023-11-06T01:35:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6ddd716b80 1 Processor -- start 2023-11-06T01:35:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 0x5569677fb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 0x5569677fb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f2ed0 0x5569678027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569676df4b0 con 0x5569677f1680 2023-11-06T01:35:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569676df150 con 0x5569677f2ed0 2023-11-06T01:35:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569675c68c0 con 0x5569677f1fe0 2023-11-06T01:35:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6ddd716b80 10 client.0 _ll_register_callbacks cb 0x55696762ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6dcb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f2ed0 0x5569678027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6dcb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f2ed0 0x5569678027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51812/0 (socket says 172.21.15.53:51812) 2023-11-06T01:35:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6dcb7fe700 1 -- 172.21.15.53:0/119947765 learned_addr learned my addr 172.21.15.53:0/119947765 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.633+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 0x5569677fb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.634+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 0x5569677fb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.634+0000 7f6dca7fc700 1 -- 172.21.15.53:0/119947765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f2ed0 msgr2=0x5569678027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.634+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f2ed0 0x5569678027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.634+0000 7f6dca7fc700 1 -- 172.21.15.53:0/119947765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 msgr2=0x5569677fb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.634+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 0x5569677fb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.634+0000 7f6dca7fc700 1 -- 172.21.15.53:0/119947765 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc1a5990 con 0x5569677f1fe0 2023-11-06T01:35:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.634+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 0x5569677fb770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005950 tx=0x7f6dc00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08fe418ab544179 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.634+0000 7f6daeff5700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:35:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.634+0000 7f6daeff5700 1 -- 172.21.15.53:0/119947765 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0022070 con 0x5569677f1fe0 2023-11-06T01:35:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.634+0000 7f6ddd716b80 1 -- 172.21.15.53:0/119947765 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55696776dcb0 con 0x5569677f1fe0 2023-11-06T01:35:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.634+0000 7f6daeff5700 1 -- 172.21.15.53:0/119947765 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6dc001d620 con 0x5569677f1fe0 2023-11-06T01:35:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6daeff5700 1 -- 172.21.15.53:0/119947765 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc0004580 con 0x5569677f1fe0 2023-11-06T01:35:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6ddd716b80 10 client.25333 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6ddd716b80 1 -- 172.21.15.53:0/119947765 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556967559c00 con 0x5569677f1fe0 2023-11-06T01:35:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6daeff5700 1 -- 172.21.15.53:0/119947765 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55696776dcb0 con 0x5569677f1fe0 2023-11-06T01:35:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6daeff5700 1 -- 172.21.15.53:0/119947765 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6dc0005020 con 0x5569677f1fe0 2023-11-06T01:35:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6ddd716b80 10 client.25333 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6ddd716b80 10 client.25333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6ddd716b80 20 client.25333 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556967807190 0x5569678095b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6ddd716b80 4 client.25333 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:35:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6ddd716b80 1 -- 172.21.15.53:0/119947765 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569677c0db0 con 0x556967807190 2023-11-06T01:35:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.635+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556967807190 0x5569678095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.636+0000 7f6daeff5700 10 client.25333 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.636+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556967807190 0x5569678095b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219018457012f8ab server_cookie=80d0dafb9c704ee0 in_seq=0 out_seq=0 2023-11-06T01:35:05.655 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:05 smithi062 ceph-mon[103924]: pgmap v333: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 1.39k op/s 2023-11-06T01:35:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.637+0000 7f6daeff5700 1 -- 172.21.15.53:0/119947765 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5569677c0db0 con 0x556967807190 2023-11-06T01:35:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.637+0000 7f6daeff5700 10 client.25333 handle_command_reply: tid=0 2023-11-06T01:35:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.637+0000 7f6ddd716b80 1 client.25333 shutdown 2023-11-06T01:35:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.637+0000 7f6ddd716b80 1 -- 172.21.15.53:0/119947765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 msgr2=0x5569677fb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.637+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 0x5569677fb770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005950 tx=0x7f6dc00047c0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.638+0000 7f6ddd716b80 1 -- 172.21.15.53:0/119947765 shutdown_connections 2023-11-06T01:35:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.638+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f2ed0 0x5569678027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.638+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556967807190 0x5569678095b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.638+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f1680 0x5569677fb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.638+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/119947765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 0x5569677fb770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.638+0000 7f6daeff5700 0 client.25333 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.638+0000 7f6ddd716b80 1 -- 172.21.15.53:0/119947765 >> 172.21.15.53:0/119947765 conn(0x556967751070 msgr2=0x5569677ec3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.638+0000 7f6ddd716b80 1 -- 172.21.15.53:0/119947765 shutdown_connections 2023-11-06T01:35:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.638+0000 7f6ddd716b80 1 -- 172.21.15.53:0/119947765 wait complete. 2023-11-06T01:35:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.639+0000 7f6ddd716b80 20 client.25333 trim_cache size 0 max 16384 2023-11-06T01:35:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.666+0000 7f6ddd716b80 1 Processor -- start 2023-11-06T01:35:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.666+0000 7f6ddd716b80 1 -- start start 2023-11-06T01:35:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.666+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1680 0x556967809580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.666+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 0x55696780df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.666+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f2ed0 0x556967800760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.666+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569677f2d50 con 0x5569677f2ed0 2023-11-06T01:35:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.666+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569677e47a0 con 0x5569677f1fe0 2023-11-06T01:35:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.666+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569677e7500 con 0x5569677f1680 2023-11-06T01:35:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.666+0000 7f6dca7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 0x55696780df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.666+0000 7f6dca7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 0x55696780df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51824/0 (socket says 172.21.15.53:51824) 2023-11-06T01:35:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.666+0000 7f6dca7fc700 1 -- 172.21.15.53:0/3169946021 learned_addr learned my addr 172.21.15.53:0/3169946021 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.667+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/3169946021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1680 0x556967809580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.667+0000 7f6dca7fc700 1 -- 172.21.15.53:0/3169946021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1680 msgr2=0x556967809580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.667+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/3169946021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1680 0x556967809580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.667+0000 7f6dca7fc700 1 -- 172.21.15.53:0/3169946021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f2ed0 msgr2=0x556967800760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.667+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/3169946021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f2ed0 0x556967800760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.667+0000 7f6dca7fc700 1 -- 172.21.15.53:0/3169946021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569675971f0 con 0x5569677f1fe0 2023-11-06T01:35:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.667+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/3169946021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1680 0x556967809580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:35:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.667+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/3169946021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 0x55696780df10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005450 tx=0x7f6dc0009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce0c1f3c43801f2f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.668+0000 7f6db1ffb700 1 -- 172.21.15.53:0/3169946021 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0022040 con 0x5569677f1fe0 2023-11-06T01:35:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.668+0000 7f6db1ffb700 1 -- 172.21.15.53:0/3169946021 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc00055f0 con 0x5569677f1fe0 2023-11-06T01:35:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.668+0000 7f6db1ffb700 1 -- 172.21.15.53:0/3169946021 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0004d90 con 0x5569677f1fe0 2023-11-06T01:35:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.669+0000 7f6ddd716b80 1 -- 172.21.15.53:0/3169946021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 msgr2=0x55696780df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.669+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/3169946021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 0x55696780df10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005450 tx=0x7f6dc0009490 comp rx=0 tx=0).stop 2023-11-06T01:35:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.669+0000 7f6ddd716b80 1 -- 172.21.15.53:0/3169946021 shutdown_connections 2023-11-06T01:35:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.669+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/3169946021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f2ed0 0x556967800760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.669+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/3169946021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1680 0x556967809580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.669+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/3169946021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1fe0 0x55696780df10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.669+0000 7f6ddd716b80 1 -- 172.21.15.53:0/3169946021 >> 172.21.15.53:0/3169946021 conn(0x556967751070 msgr2=0x5569677edc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.669+0000 7f6ddd716b80 1 -- 172.21.15.53:0/3169946021 shutdown_connections 2023-11-06T01:35:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.669+0000 7f6ddd716b80 1 -- 172.21.15.53:0/3169946021 wait complete. 2023-11-06T01:35:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.669+0000 7f6ddd716b80 1 -- start start 2023-11-06T01:35:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.669+0000 7f6ddd716b80 1 Processor -- start 2023-11-06T01:35:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1680 0x556967852c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 0x5569678551c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6ddd716b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f2ed0 0x556967857850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556967857f90 con 0x5569677f2ed0 2023-11-06T01:35:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556967858100 con 0x5569677f1680 2023-11-06T01:35:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6ddd716b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569678692d0 con 0x5569677f1fe0 2023-11-06T01:35:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6ddd716b80 10 client.0 _ll_register_callbacks cb 0x55696762ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6dca7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 0x5569678551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6dca7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 0x5569678551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54628/0 (socket says 172.21.15.53:54628) 2023-11-06T01:35:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6dca7fc700 1 -- 172.21.15.53:0/2061786290 learned_addr learned my addr 172.21.15.53:0/2061786290 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1680 0x556967852c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.670+0000 7f6dcb7fe700 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f2ed0 0x556967857850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.671+0000 7f6dca7fc700 1 -- 172.21.15.53:0/2061786290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1680 msgr2=0x556967852c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.671+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1680 0x556967852c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.671+0000 7f6dca7fc700 1 -- 172.21.15.53:0/2061786290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f2ed0 msgr2=0x556967857850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.671+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f2ed0 0x556967857850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.671+0000 7f6dca7fc700 1 -- 172.21.15.53:0/2061786290 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556967570d80 con 0x5569677f1fe0 2023-11-06T01:35:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.671+0000 7f6dca7fc700 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 0x5569678551c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0000c00 tx=0x7f6dc001daf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f74552c1f36701da server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.671+0000 7f6daeff5700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:35:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6daeff5700 1 -- 172.21.15.53:0/2061786290 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0022070 con 0x5569677f1fe0 2023-11-06T01:35:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6daeff5700 1 -- 172.21.15.53:0/2061786290 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6dc00046e0 con 0x5569677f1fe0 2023-11-06T01:35:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2061786290 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dc409a720 con 0x5569677f1fe0 2023-11-06T01:35:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6daeff5700 1 -- 172.21.15.53:0/2061786290 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc0004b10 con 0x5569677f1fe0 2023-11-06T01:35:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6ddd716b80 10 client.25339 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2061786290 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556967559c00 con 0x5569677f1fe0 2023-11-06T01:35:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6daeff5700 1 -- 172.21.15.53:0/2061786290 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc409a720 con 0x5569677f1fe0 2023-11-06T01:35:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6daeff5700 1 -- 172.21.15.53:0/2061786290 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6dc0027820 con 0x5569677f1fe0 2023-11-06T01:35:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6ddd716b80 10 client.25339 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6ddd716b80 10 client.25339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6ddd716b80 20 client.25339 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55696786a9a0 0x55696786cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6ddd716b80 4 client.25339 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:35:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.672+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2061786290 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55696773bce0 con 0x55696786a9a0 2023-11-06T01:35:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.673+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55696786a9a0 0x55696786cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.673+0000 7f6daeff5700 10 client.25339 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.673+0000 7f6dcaffd700 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55696786a9a0 0x55696786cdc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e269c88c9437987e server_cookie=d1cb72ec40a1a58d in_seq=0 out_seq=0 2023-11-06T01:35:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.694+0000 7f6daeff5700 1 -- 172.21.15.53:0/2061786290 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55696773bce0 con 0x55696786a9a0 2023-11-06T01:35:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.694+0000 7f6daeff5700 10 client.25339 handle_command_reply: tid=0 2023-11-06T01:35:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.694+0000 7f6ddd716b80 1 client.25339 shutdown 2023-11-06T01:35:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.694+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2061786290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 msgr2=0x5569678551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.694+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 0x5569678551c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0000c00 tx=0x7f6dc001daf0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.695+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2061786290 shutdown_connections 2023-11-06T01:35:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.695+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569677f1680 0x556967852c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.695+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55696786a9a0 0x55696786cdc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.695+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569677f2ed0 0x556967857850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.695+0000 7f6ddd716b80 1 --2- 172.21.15.53:0/2061786290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569677f1fe0 0x5569678551c0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0000c00 tx=0x7f6dc001daf0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.695+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2061786290 >> 172.21.15.53:0/2061786290 conn(0x556967751070 msgr2=0x5569677f60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.695+0000 7f6daeff5700 0 client.25339 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.695+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2061786290 shutdown_connections 2023-11-06T01:35:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.696+0000 7f6ddd716b80 1 -- 172.21.15.53:0/2061786290 wait complete. 2023-11-06T01:35:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.696+0000 7f6ddd716b80 20 client.25339 trim_cache size 0 max 16384 2023-11-06T01:35:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:35:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:35:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:35:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:35:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.696+0000 7f6dd1586700 1 -- 172.21.15.53:0/3444846670 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6db40a05a0 msgr2=0x7f6db40a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.696+0000 7f6dd1586700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6db40a05a0 0x7f6db40a2a60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009c60 tx=0x7f6dc0009260 comp rx=0 tx=0).stop 2023-11-06T01:35:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.696+0000 7f6dd1586700 1 -- 172.21.15.53:0/3444846670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dcc1073f0 msgr2=0x7f6dcc19d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.696+0000 7f6dd1586700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dcc1073f0 0x7f6dcc19d380 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4000f80 tx=0x7f6dc4008ec0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.698+0000 7f6dd1586700 1 -- 172.21.15.53:0/3444846670 shutdown_connections 2023-11-06T01:35:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.698+0000 7f6dd1586700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6db40a05a0 0x7f6db40a2a60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.698+0000 7f6dd1586700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc108f80 0x7f6dcc19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.698+0000 7f6dd1586700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dcc1073f0 0x7f6dcc19d380 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.698+0000 7f6dd1586700 1 --2- 172.21.15.53:0/3444846670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dcc1085e0 0x7f6dcc19d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.698+0000 7f6dd1586700 1 -- 172.21.15.53:0/3444846670 >> 172.21.15.53:0/3444846670 conn(0x7f6dcc076370 msgr2=0x7f6dcc10bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.698+0000 7f6dd1586700 1 -- 172.21.15.53:0/3444846670 shutdown_connections 2023-11-06T01:35:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:05.698+0000 7f6dd1586700 1 -- 172.21.15.53:0/3444846670 wait complete. 2023-11-06T01:35:05.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:05 smithi163 ceph-mon[104416]: pgmap v333: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 1.39k op/s 2023-11-06T01:35:07.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 50.0 s, 4469.4 tps, lat 22.307 ms stddev 24.414 2023-11-06T01:35:07.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:07 smithi053 ceph-mon[99472]: pgmap v334: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 59 MiB/s wr, 2.35k op/s 2023-11-06T01:35:07.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:07 smithi062 ceph-mon[103924]: pgmap v334: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 59 MiB/s wr, 2.35k op/s 2023-11-06T01:35:07.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:07 smithi163 ceph-mon[104416]: pgmap v334: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 59 MiB/s wr, 2.35k op/s 2023-11-06T01:35:09.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:09 smithi053 ceph-mon[99472]: pgmap v335: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 53 MiB/s wr, 2.19k op/s 2023-11-06T01:35:09.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:09 smithi062 ceph-mon[103924]: pgmap v335: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 53 MiB/s wr, 2.19k op/s 2023-11-06T01:35:09.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:09 smithi163 ceph-mon[104416]: pgmap v335: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 53 MiB/s wr, 2.19k op/s 2023-11-06T01:35:11.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:11 smithi053 ceph-mon[99472]: pgmap v336: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 51 MiB/s wr, 2.15k op/s 2023-11-06T01:35:11.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:11 smithi062 ceph-mon[103924]: pgmap v336: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 51 MiB/s wr, 2.15k op/s 2023-11-06T01:35:11.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:11 smithi163 ceph-mon[104416]: pgmap v336: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 51 MiB/s wr, 2.15k op/s 2023-11-06T01:35:12.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 55.0 s, 4790.3 tps, lat 20.932 ms stddev 22.330 2023-11-06T01:35:13.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:13 smithi053 ceph-mon[99472]: pgmap v337: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 53 MiB/s wr, 2.21k op/s 2023-11-06T01:35:13.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:13 smithi062 ceph-mon[103924]: pgmap v337: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 53 MiB/s wr, 2.21k op/s 2023-11-06T01:35:13.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:13 smithi163 ceph-mon[104416]: pgmap v337: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 53 MiB/s wr, 2.21k op/s 2023-11-06T01:35:15.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:15 smithi053 ceph-mon[99472]: pgmap v338: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 46 MiB/s wr, 2.00k op/s 2023-11-06T01:35:15.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:15 smithi062 ceph-mon[103924]: pgmap v338: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 46 MiB/s wr, 2.00k op/s 2023-11-06T01:35:15.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:15 smithi163 ceph-mon[104416]: pgmap v338: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 46 MiB/s wr, 2.00k op/s 2023-11-06T01:35:17.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 60.0 s, 3736.2 tps, lat 26.765 ms stddev 30.814 2023-11-06T01:35:17.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:17 smithi053 ceph-mon[99472]: pgmap v339: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 45 MiB/s wr, 1.95k op/s 2023-11-06T01:35:17.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:17 smithi062 ceph-mon[103924]: pgmap v339: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 45 MiB/s wr, 1.95k op/s 2023-11-06T01:35:17.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:17 smithi163 ceph-mon[104416]: pgmap v339: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 45 MiB/s wr, 1.95k op/s 2023-11-06T01:35:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.867+0000 7f524ab83700 1 -- 172.21.15.53:0/389104671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52440ffce0 msgr2=0x7f5244100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.867+0000 7f524ab83700 1 --2- 172.21.15.53:0/389104671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52440ffce0 0x7f5244100100 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5238009990 tx=0x7f5238009ca0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.867+0000 7f524ab83700 1 -- 172.21.15.53:0/389104671 shutdown_connections 2023-11-06T01:35:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.867+0000 7f524ab83700 1 --2- 172.21.15.53:0/389104671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5244101870 0x7f5244068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.867+0000 7f524ab83700 1 --2- 172.21.15.53:0/389104671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52440ffce0 0x7f5244100100 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.867+0000 7f524ab83700 1 --2- 172.21.15.53:0/389104671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5244100ed0 0x7f5244101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.867+0000 7f524ab83700 1 -- 172.21.15.53:0/389104671 >> 172.21.15.53:0/389104671 conn(0x7f52440fb250 msgr2=0x7f52440fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.877+0000 7f524ab83700 1 -- 172.21.15.53:0/389104671 shutdown_connections 2023-11-06T01:35:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.880+0000 7f524ab83700 1 -- 172.21.15.53:0/389104671 wait complete. 2023-11-06T01:35:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.881+0000 7f524ab83700 1 Processor -- start 2023-11-06T01:35:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.882+0000 7f524ab83700 1 -- start start 2023-11-06T01:35:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.882+0000 7f524ab83700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52440ffce0 0x7f524410ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.882+0000 7f524ab83700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5244101870 0x7f5244110490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.882+0000 7f524ab83700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52441165c0 0x7f5244111360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.882+0000 7f524ab83700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5244116b30 con 0x7f52441165c0 2023-11-06T01:35:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.882+0000 7f524ab83700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5244116ca0 con 0x7f52440ffce0 2023-11-06T01:35:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.882+0000 7f524ab83700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52441118a0 con 0x7f5244101870 2023-11-06T01:35:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.882+0000 7f5243fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5244101870 0x7f5244110490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.882+0000 7f5243fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5244101870 0x7f5244110490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54642/0 (socket says 172.21.15.53:54642) 2023-11-06T01:35:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.882+0000 7f5243fff700 1 -- 172.21.15.53:0/936813064 learned_addr learned my addr 172.21.15.53:0/936813064 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.882+0000 7f5249120700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52441165c0 0x7f5244111360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.883+0000 7f5243fff700 1 -- 172.21.15.53:0/936813064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52440ffce0 msgr2=0x7f524410ff50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:35:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.883+0000 7f5243fff700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52440ffce0 0x7f524410ff50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.883+0000 7f5243fff700 1 -- 172.21.15.53:0/936813064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52441165c0 msgr2=0x7f5244111360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.883+0000 7f5243fff700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52441165c0 0x7f5244111360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.883+0000 7f5243fff700 1 -- 172.21.15.53:0/936813064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380095f0 con 0x7f5244101870 2023-11-06T01:35:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.884+0000 7f5243fff700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5244101870 0x7f5244110490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f523000b500 tx=0x7f523000b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.885+0000 7f5241ffb700 1 -- 172.21.15.53:0/936813064 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230011830 con 0x7f5244101870 2023-11-06T01:35:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.885+0000 7f5241ffb700 1 -- 172.21.15.53:0/936813064 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5230011e50 con 0x7f5244101870 2023-11-06T01:35:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.886+0000 7f5241ffb700 1 -- 172.21.15.53:0/936813064 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230010e40 con 0x7f5244101870 2023-11-06T01:35:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.886+0000 7f524ab83700 1 -- 172.21.15.53:0/936813064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5244111b80 con 0x7f5244101870 2023-11-06T01:35:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.886+0000 7f524ab83700 1 -- 172.21.15.53:0/936813064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52441120c0 con 0x7f5244101870 2023-11-06T01:35:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.889+0000 7f5241ffb700 1 -- 172.21.15.53:0/936813064 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5230011990 con 0x7f5244101870 2023-11-06T01:35:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.889+0000 7f5241ffb700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52340a06a0 0x7f52340a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.889+0000 7f5241ffb700 1 -- 172.21.15.53:0/936813064 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f52300cb3b0 con 0x7f5244101870 2023-11-06T01:35:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f524891f700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52340a06a0 0x7f52340a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f524891f700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52340a06a0 0x7f52340a2b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f52380095c0 tx=0x7f5238009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f5256d13b80 1 Processor -- start 2023-11-06T01:35:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f5256d13b80 1 -- start start 2023-11-06T01:35:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383a680 0x55c17383aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 0x55c17383b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383bed0 0x55c17383c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f5256d13b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1737929b0 con 0x55c17383bed0 2023-11-06T01:35:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f5256d13b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c173790370 con 0x55c17383afe0 2023-11-06T01:35:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f5256d13b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17378bc60 con 0x55c17383a680 2023-11-06T01:35:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f5249120700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383bed0 0x55c17383c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f524891f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383a680 0x55c17383aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.890+0000 7f5249120700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383bed0 0x55c17383c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40412/0 (socket says 172.21.15.53:40412) 2023-11-06T01:35:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.891+0000 7f5249120700 1 -- 172.21.15.53:0/1735139509 learned_addr learned my addr 172.21.15.53:0/1735139509 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.891+0000 7f5243fff700 1 --2- 172.21.15.53:0/1735139509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 0x55c17383b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.891+0000 7f5249120700 1 -- 172.21.15.53:0/1735139509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383a680 msgr2=0x55c17383aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.891+0000 7f5249120700 1 --2- 172.21.15.53:0/1735139509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383a680 0x55c17383aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.891+0000 7f5249120700 1 -- 172.21.15.53:0/1735139509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 msgr2=0x55c17383b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.891+0000 7f5249120700 1 --2- 172.21.15.53:0/1735139509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 0x55c17383b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.891+0000 7f5249120700 1 -- 172.21.15.53:0/1735139509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1735b9d80 con 0x55c17383bed0 2023-11-06T01:35:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.892+0000 7f5249120700 1 --2- 172.21.15.53:0/1735139509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383bed0 0x55c17383c350 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f523c009f20 tx=0x7f523c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816931ae3cda0c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.893+0000 7f522effd700 1 -- 172.21.15.53:0/1735139509 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c011070 con 0x55c17383bed0 2023-11-06T01:35:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.893+0000 7f522effd700 1 -- 172.21.15.53:0/1735139509 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f523c009a50 con 0x55c17383bed0 2023-11-06T01:35:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.893+0000 7f522effd700 1 -- 172.21.15.53:0/1735139509 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c015e70 con 0x55c17383bed0 2023-11-06T01:35:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.893+0000 7f5256d13b80 1 -- 172.21.15.53:0/1735139509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383bed0 msgr2=0x55c17383c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.893+0000 7f5256d13b80 1 --2- 172.21.15.53:0/1735139509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383bed0 0x55c17383c350 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f523c009f20 tx=0x7f523c009f50 comp rx=0 tx=0).stop 2023-11-06T01:35:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.893+0000 7f5256d13b80 1 -- 172.21.15.53:0/1735139509 shutdown_connections 2023-11-06T01:35:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.893+0000 7f5256d13b80 1 --2- 172.21.15.53:0/1735139509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383bed0 0x55c17383c350 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.893+0000 7f5256d13b80 1 --2- 172.21.15.53:0/1735139509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383a680 0x55c17383aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.893+0000 7f5256d13b80 1 --2- 172.21.15.53:0/1735139509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 0x55c17383b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.893+0000 7f5256d13b80 1 -- 172.21.15.53:0/1735139509 >> 172.21.15.53:0/1735139509 conn(0x55c17379a070 msgr2=0x55c173837040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f5256d13b80 1 -- 172.21.15.53:0/1735139509 shutdown_connections 2023-11-06T01:35:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f5256d13b80 1 -- 172.21.15.53:0/1735139509 wait complete. 2023-11-06T01:35:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f5256d13b80 1 -- start start 2023-11-06T01:35:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f5256d13b80 1 Processor -- start 2023-11-06T01:35:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383a680 0x55c1738444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 0x55c173844a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383bed0 0x55c17384ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f5256d13b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1737284b0 con 0x55c17383a680 2023-11-06T01:35:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f5256d13b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c173728150 con 0x55c17383afe0 2023-11-06T01:35:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f5256d13b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17360f8c0 con 0x55c17383bed0 2023-11-06T01:35:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f5256d13b80 10 client.0 _ll_register_callbacks cb 0x55c173675a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.894+0000 7f524891f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383a680 0x55c1738444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5243fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 0x55c173844a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5249120700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383bed0 0x55c17384ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f524891f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383a680 0x55c1738444f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40414/0 (socket says 172.21.15.53:40414) 2023-11-06T01:35:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5243fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 0x55c173844a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51854/0 (socket says 172.21.15.53:51854) 2023-11-06T01:35:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f524891f700 1 -- 172.21.15.53:0/1040371634 learned_addr learned my addr 172.21.15.53:0/1040371634 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5243fff700 1 -- 172.21.15.53:0/1040371634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383bed0 msgr2=0x55c17384ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5243fff700 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383bed0 0x55c17384ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5243fff700 1 -- 172.21.15.53:0/1040371634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383a680 msgr2=0x55c1738444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5243fff700 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383a680 0x55c1738444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5243fff700 1 -- 172.21.15.53:0/1040371634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c009bd0 con 0x55c17383afe0 2023-11-06T01:35:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5249120700 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383bed0 0x55c17384ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:35:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5243fff700 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 0x55c173844a30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f52300c8e90 tx=0x7f5230098620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2b66e8443735e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5223fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:35:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5223fff700 1 -- 172.21.15.53:0/1040371634 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523009ca20 con 0x55c17383afe0 2023-11-06T01:35:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.895+0000 7f5256d13b80 1 -- 172.21.15.53:0/1040371634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c173809db0 con 0x55c17383afe0 2023-11-06T01:35:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.896+0000 7f5223fff700 1 -- 172.21.15.53:0/1040371634 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f52300a0c60 con 0x55c17383afe0 2023-11-06T01:35:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.897+0000 7f5223fff700 1 -- 172.21.15.53:0/1040371634 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5230010510 con 0x55c17383afe0 2023-11-06T01:35:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.897+0000 7f5256d13b80 10 client.25262 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.897+0000 7f5256d13b80 1 -- 172.21.15.53:0/1040371634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c1735a2c00 con 0x55c17383afe0 2023-11-06T01:35:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.897+0000 7f5223fff700 1 -- 172.21.15.53:0/1040371634 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c173809db0 con 0x55c17383afe0 2023-11-06T01:35:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.898+0000 7f5223fff700 1 -- 172.21.15.53:0/1040371634 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f52300a07d0 con 0x55c17383afe0 2023-11-06T01:35:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.898+0000 7f5256d13b80 10 client.25262 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.898+0000 7f5256d13b80 10 client.25262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.898+0000 7f5256d13b80 20 client.25262 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.898+0000 7f5256d13b80 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c173850380 0x55c1738527a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.898+0000 7f5256d13b80 4 client.25262 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:35:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.898+0000 7f5256d13b80 1 -- 172.21.15.53:0/1040371634 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c173784ce0 con 0x55c173850380 2023-11-06T01:35:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.898+0000 7f524891f700 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c173850380 0x55c1738527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.899+0000 7f5223fff700 10 client.25262 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.899+0000 7f524891f700 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c173850380 0x55c1738527a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a231db3065b9864 server_cookie=9fd6d312221dea59 in_seq=0 out_seq=0 2023-11-06T01:35:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.900+0000 7f5223fff700 1 -- 172.21.15.53:0/1040371634 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c173784ce0 con 0x55c173850380 2023-11-06T01:35:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.900+0000 7f5223fff700 10 client.25262 handle_command_reply: tid=0 2023-11-06T01:35:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.900+0000 7f5256d13b80 1 client.25262 shutdown 2023-11-06T01:35:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.900+0000 7f5256d13b80 1 -- 172.21.15.53:0/1040371634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 msgr2=0x55c173844a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.900+0000 7f5256d13b80 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 0x55c173844a30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f52300c8e90 tx=0x7f5230098620 comp rx=0 tx=0).stop 2023-11-06T01:35:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.900+0000 7f5256d13b80 1 -- 172.21.15.53:0/1040371634 shutdown_connections 2023-11-06T01:35:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.901+0000 7f5256d13b80 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c173850380 0x55c1738527a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.901+0000 7f5256d13b80 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383bed0 0x55c17384ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.901+0000 7f5223fff700 0 client.25262 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.901+0000 7f5256d13b80 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383a680 0x55c1738444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.901+0000 7f5256d13b80 1 --2- 172.21.15.53:0/1040371634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383afe0 0x55c173844a30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.901+0000 7f5256d13b80 1 -- 172.21.15.53:0/1040371634 >> 172.21.15.53:0/1040371634 conn(0x55c17379a070 msgr2=0x55c1738353c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.901+0000 7f5256d13b80 1 -- 172.21.15.53:0/1040371634 shutdown_connections 2023-11-06T01:35:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.901+0000 7f5256d13b80 1 -- 172.21.15.53:0/1040371634 wait complete. 2023-11-06T01:35:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.901+0000 7f5256d13b80 20 client.25262 trim_cache size 0 max 16384 2023-11-06T01:35:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.925+0000 7f5256d13b80 1 Processor -- start 2023-11-06T01:35:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.925+0000 7f5256d13b80 1 -- start start 2023-11-06T01:35:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.926+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383a680 0x55c17383c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.926+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383afe0 0x55c17384ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.926+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383bed0 0x55c17383f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.926+0000 7f5256d13b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1738456e0 con 0x55c17383a680 2023-11-06T01:35:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.926+0000 7f5256d13b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17382d7a0 con 0x55c17383bed0 2023-11-06T01:35:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.926+0000 7f5256d13b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17384ff10 con 0x55c17383afe0 2023-11-06T01:35:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.926+0000 7f5249120700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383bed0 0x55c17383f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.926+0000 7f5249120700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383bed0 0x55c17383f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51862/0 (socket says 172.21.15.53:51862) 2023-11-06T01:35:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.926+0000 7f5249120700 1 -- 172.21.15.53:0/3151915034 learned_addr learned my addr 172.21.15.53:0/3151915034 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.926+0000 7f5243fff700 1 --2- 172.21.15.53:0/3151915034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383afe0 0x55c17384ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.926+0000 7f524891f700 1 --2- 172.21.15.53:0/3151915034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383a680 0x55c17383c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.927+0000 7f5243fff700 1 -- 172.21.15.53:0/3151915034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383bed0 msgr2=0x55c17383f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.927+0000 7f5243fff700 1 --2- 172.21.15.53:0/3151915034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383bed0 0x55c17383f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.927+0000 7f5243fff700 1 -- 172.21.15.53:0/3151915034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383a680 msgr2=0x55c17383c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.927+0000 7f5243fff700 1 --2- 172.21.15.53:0/3151915034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383a680 0x55c17383c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.927+0000 7f5243fff700 1 -- 172.21.15.53:0/3151915034 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1735e01f0 con 0x55c17383afe0 2023-11-06T01:35:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.927+0000 7f5243fff700 1 --2- 172.21.15.53:0/3151915034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383afe0 0x55c17384ba30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5230004840 tx=0x7f5230009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7caf113fa09b57f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.927+0000 7f522f7fe700 1 -- 172.21.15.53:0/3151915034 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523000d040 con 0x55c17383afe0 2023-11-06T01:35:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.927+0000 7f522f7fe700 1 -- 172.21.15.53:0/3151915034 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5230098340 con 0x55c17383afe0 2023-11-06T01:35:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.927+0000 7f522f7fe700 1 -- 172.21.15.53:0/3151915034 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52300105f0 con 0x55c17383afe0 2023-11-06T01:35:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.928+0000 7f5256d13b80 1 -- 172.21.15.53:0/3151915034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383afe0 msgr2=0x55c17384ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.928+0000 7f5256d13b80 1 --2- 172.21.15.53:0/3151915034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383afe0 0x55c17384ba30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5230004840 tx=0x7f5230009370 comp rx=0 tx=0).stop 2023-11-06T01:35:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.928+0000 7f5256d13b80 1 -- 172.21.15.53:0/3151915034 shutdown_connections 2023-11-06T01:35:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.928+0000 7f5256d13b80 1 --2- 172.21.15.53:0/3151915034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383bed0 0x55c17383f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.928+0000 7f5256d13b80 1 --2- 172.21.15.53:0/3151915034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17383a680 0x55c17383c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.928+0000 7f5256d13b80 1 --2- 172.21.15.53:0/3151915034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383afe0 0x55c17384ba30 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5230004840 tx=0x7f5230009370 comp rx=0 tx=0).stop 2023-11-06T01:35:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.928+0000 7f5256d13b80 1 -- 172.21.15.53:0/3151915034 >> 172.21.15.53:0/3151915034 conn(0x55c17379a070 msgr2=0x55c173843170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.929+0000 7f5256d13b80 1 -- 172.21.15.53:0/3151915034 shutdown_connections 2023-11-06T01:35:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.929+0000 7f5256d13b80 1 -- 172.21.15.53:0/3151915034 wait complete. 2023-11-06T01:35:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.929+0000 7f5256d13b80 1 -- start start 2023-11-06T01:35:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.930+0000 7f5256d13b80 1 Processor -- start 2023-11-06T01:35:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.930+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383a680 0x55c173849450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.930+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383bed0 0x55c173849990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.930+0000 7f5256d13b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17384a9a0 0x55c17384ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.930+0000 7f5256d13b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17384b360 con 0x55c17384a9a0 2023-11-06T01:35:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.930+0000 7f5256d13b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17384b4d0 con 0x55c17383a680 2023-11-06T01:35:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.930+0000 7f5256d13b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17384b690 con 0x55c17383bed0 2023-11-06T01:35:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.930+0000 7f5256d13b80 10 client.0 _ll_register_callbacks cb 0x55c173675a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.930+0000 7f5249120700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17384a9a0 0x55c17384ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.931+0000 7f5243fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383bed0 0x55c173849990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.931+0000 7f5249120700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17384a9a0 0x55c17384ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40430/0 (socket says 172.21.15.53:40430) 2023-11-06T01:35:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.931+0000 7f5249120700 1 -- 172.21.15.53:0/2761080855 learned_addr learned my addr 172.21.15.53:0/2761080855 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.931+0000 7f524891f700 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383a680 0x55c173849450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.931+0000 7f524891f700 1 -- 172.21.15.53:0/2761080855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383bed0 msgr2=0x55c173849990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.931+0000 7f524891f700 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383bed0 0x55c173849990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.931+0000 7f524891f700 1 -- 172.21.15.53:0/2761080855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17384a9a0 msgr2=0x55c17384ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.931+0000 7f524891f700 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17384a9a0 0x55c17384ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.931+0000 7f524891f700 1 -- 172.21.15.53:0/2761080855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1735a2c00 con 0x55c17383a680 2023-11-06T01:35:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.931+0000 7f524891f700 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383a680 0x55c173849450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55c173853040 tx=0x7f5238009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c9a7837d04151b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.931+0000 7f5223fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:35:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.932+0000 7f5223fff700 1 -- 172.21.15.53:0/2761080855 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238029040 con 0x55c17383a680 2023-11-06T01:35:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.932+0000 7f5256d13b80 1 -- 172.21.15.53:0/2761080855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1737b6cb0 con 0x55c17383a680 2023-11-06T01:35:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.932+0000 7f5223fff700 1 -- 172.21.15.53:0/2761080855 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5238033b00 con 0x55c17383a680 2023-11-06T01:35:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.932+0000 7f5223fff700 1 -- 172.21.15.53:0/2761080855 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f523801d070 con 0x55c17383a680 2023-11-06T01:35:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.932+0000 7f5256d13b80 10 client.25268 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.932+0000 7f5256d13b80 1 -- 172.21.15.53:0/2761080855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c1735b8250 con 0x55c17383a680 2023-11-06T01:35:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.932+0000 7f5223fff700 1 -- 172.21.15.53:0/2761080855 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c1737b6cb0 con 0x55c17383a680 2023-11-06T01:35:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.933+0000 7f5223fff700 1 -- 172.21.15.53:0/2761080855 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f523800f460 con 0x55c17383a680 2023-11-06T01:35:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.933+0000 7f5256d13b80 10 client.25268 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.933+0000 7f5256d13b80 10 client.25268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.933+0000 7f5256d13b80 20 client.25268 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.933+0000 7f5256d13b80 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c1738a2f50 0x55c1738a5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.933+0000 7f5256d13b80 4 client.25268 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:35:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.933+0000 7f5256d13b80 1 -- 172.21.15.53:0/2761080855 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c173784a00 con 0x55c1738a2f50 2023-11-06T01:35:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.933+0000 7f5243fff700 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c1738a2f50 0x55c1738a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.933+0000 7f5243fff700 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c1738a2f50 0x55c1738a5370 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=880a6e7ba98f7949 server_cookie=706c30ed5c5168b7 in_seq=0 out_seq=0 2023-11-06T01:35:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.933+0000 7f5223fff700 10 client.25268 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.965+0000 7f5223fff700 1 -- 172.21.15.53:0/2761080855 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c173784a00 con 0x55c1738a2f50 2023-11-06T01:35:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.965+0000 7f5223fff700 10 client.25268 handle_command_reply: tid=0 2023-11-06T01:35:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.965+0000 7f5256d13b80 1 client.25268 shutdown 2023-11-06T01:35:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5256d13b80 1 -- 172.21.15.53:0/2761080855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383a680 msgr2=0x55c173849450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5256d13b80 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383a680 0x55c173849450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55c173853040 tx=0x7f5238009310 comp rx=0 tx=0).stop 2023-11-06T01:35:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5256d13b80 1 -- 172.21.15.53:0/2761080855 shutdown_connections 2023-11-06T01:35:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5256d13b80 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17383a680 0x55c173849450 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5256d13b80 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c1738a2f50 0x55c1738a5370 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5256d13b80 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c17384a9a0 0x55c17384ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5256d13b80 1 --2- 172.21.15.53:0/2761080855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17383bed0 0x55c173849990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5256d13b80 1 -- 172.21.15.53:0/2761080855 >> 172.21.15.53:0/2761080855 conn(0x55c17379a070 msgr2=0x55c17383f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5223fff700 0 client.25268 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5256d13b80 1 -- 172.21.15.53:0/2761080855 shutdown_connections 2023-11-06T01:35:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5256d13b80 1 -- 172.21.15.53:0/2761080855 wait complete. 2023-11-06T01:35:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.966+0000 7f5256d13b80 20 client.25268 trim_cache size 0 max 16384 2023-11-06T01:35:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:35:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:35:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:35:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:35:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 -- 172.21.15.53:0/936813064 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52340a06a0 msgr2=0x7f52340a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52340a06a0 0x7f52340a2b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f52380095c0 tx=0x7f5238009510 comp rx=0 tx=0).stop 2023-11-06T01:35:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 -- 172.21.15.53:0/936813064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5244101870 msgr2=0x7f5244110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5244101870 0x7f5244110490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f523000b500 tx=0x7f523000b8c0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 -- 172.21.15.53:0/936813064 shutdown_connections 2023-11-06T01:35:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f52340a06a0 0x7f52340a2b60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52440ffce0 0x7f524410ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52441165c0 0x7f5244111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 --2- 172.21.15.53:0/936813064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5244101870 0x7f5244110490 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 -- 172.21.15.53:0/936813064 >> 172.21.15.53:0/936813064 conn(0x7f52440fb250 msgr2=0x7f52440fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 -- 172.21.15.53:0/936813064 shutdown_connections 2023-11-06T01:35:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:18.968+0000 7f524ab83700 1 -- 172.21.15.53:0/936813064 wait complete. 2023-11-06T01:35:19.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:19 smithi053 ceph-mon[99472]: pgmap v340: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 26 MiB/s wr, 1.07k op/s 2023-11-06T01:35:19.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:19 smithi062 ceph-mon[103924]: pgmap v340: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 26 MiB/s wr, 1.07k op/s 2023-11-06T01:35:19.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:19 smithi163 ceph-mon[104416]: pgmap v340: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 26 MiB/s wr, 1.07k op/s 2023-11-06T01:35:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:21 smithi053 ceph-mon[99472]: pgmap v341: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 24 MiB/s wr, 1.01k op/s 2023-11-06T01:35:21.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:21 smithi062 ceph-mon[103924]: pgmap v341: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 24 MiB/s wr, 1.01k op/s 2023-11-06T01:35:21.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:21 smithi163 ceph-mon[104416]: pgmap v341: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 24 MiB/s wr, 1.01k op/s 2023-11-06T01:35:22.184 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 65.0 s, 3905.6 tps, lat 25.500 ms stddev 28.243 2023-11-06T01:35:23.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:23 smithi053 ceph-mon[99472]: pgmap v342: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 32 MiB/s wr, 1.26k op/s 2023-11-06T01:35:23.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:23 smithi062 ceph-mon[103924]: pgmap v342: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 32 MiB/s wr, 1.26k op/s 2023-11-06T01:35:23.743 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:23 smithi163 ceph-mon[104416]: pgmap v342: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 32 MiB/s wr, 1.26k op/s 2023-11-06T01:35:25.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:25 smithi053 ceph-mon[99472]: pgmap v343: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 30 MiB/s wr, 1.16k op/s 2023-11-06T01:35:25.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:25 smithi062 ceph-mon[103924]: pgmap v343: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 30 MiB/s wr, 1.16k op/s 2023-11-06T01:35:25.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:25 smithi163 ceph-mon[104416]: pgmap v343: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 30 MiB/s wr, 1.16k op/s 2023-11-06T01:35:27.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 70.0 s, 3685.4 tps, lat 27.177 ms stddev 46.670 2023-11-06T01:35:27.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:27 smithi053 ceph-mon[99472]: pgmap v344: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 51 MiB/s wr, 2.11k op/s 2023-11-06T01:35:27.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:27 smithi062 ceph-mon[103924]: pgmap v344: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 51 MiB/s wr, 2.11k op/s 2023-11-06T01:35:27.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:27 smithi163 ceph-mon[104416]: pgmap v344: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 51 MiB/s wr, 2.11k op/s 2023-11-06T01:35:29.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:29 smithi053 ceph-mon[99472]: pgmap v345: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 52 MiB/s wr, 2.18k op/s 2023-11-06T01:35:29.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:29 smithi062 ceph-mon[103924]: pgmap v345: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 52 MiB/s wr, 2.18k op/s 2023-11-06T01:35:29.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:29 smithi163 ceph-mon[104416]: pgmap v345: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 52 MiB/s wr, 2.18k op/s 2023-11-06T01:35:31.605 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:31 smithi053 ceph-mon[99472]: pgmap v346: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 55 MiB/s wr, 2.25k op/s 2023-11-06T01:35:31.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:31 smithi062 ceph-mon[103924]: pgmap v346: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 55 MiB/s wr, 2.25k op/s 2023-11-06T01:35:31.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:31 smithi163 ceph-mon[104416]: pgmap v346: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 55 MiB/s wr, 2.25k op/s 2023-11-06T01:35:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.154+0000 7f37877af700 1 -- 172.21.15.53:0/3695042705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3780108f80 msgr2=0x7f378010f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.154+0000 7f37877af700 1 --2- 172.21.15.53:0/3695042705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3780108f80 0x7f378010f830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3778009fc0 tx=0x7f377800b330 comp rx=0 tx=0).stop 2023-11-06T01:35:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.154+0000 7f37877af700 1 -- 172.21.15.53:0/3695042705 shutdown_connections 2023-11-06T01:35:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.154+0000 7f37877af700 1 --2- 172.21.15.53:0/3695042705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3780108f80 0x7f378010f830 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.155+0000 7f37877af700 1 --2- 172.21.15.53:0/3695042705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37801073f0 0x7f3780107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.155+0000 7f37877af700 1 --2- 172.21.15.53:0/3695042705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37801085e0 0x7f3780108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.155+0000 7f37877af700 1 -- 172.21.15.53:0/3695042705 >> 172.21.15.53:0/3695042705 conn(0x7f3780076390 msgr2=0x7f37800787d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.155+0000 7f37877af700 1 -- 172.21.15.53:0/3695042705 shutdown_connections 2023-11-06T01:35:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.156+0000 7f37877af700 1 -- 172.21.15.53:0/3695042705 wait complete. 2023-11-06T01:35:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.156+0000 7f37877af700 1 Processor -- start 2023-11-06T01:35:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.156+0000 7f37877af700 1 -- start start 2023-11-06T01:35:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f37877af700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37801073f0 0x7f378019d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f37877af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37801085e0 0x7f378019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f37877af700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3780108f80 0x7f378019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f37877af700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378019eca0 con 0x7f37801085e0 2023-11-06T01:35:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f37877af700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378019ee10 con 0x7f37801073f0 2023-11-06T01:35:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f37877af700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378019efd0 con 0x7f3780108f80 2023-11-06T01:35:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f378554b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37801073f0 0x7f378019d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f378554b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37801073f0 0x7f378019d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51880/0 (socket says 172.21.15.53:51880) 2023-11-06T01:35:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f378554b700 1 -- 172.21.15.53:0/153180872 learned_addr learned my addr 172.21.15.53:0/153180872 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f3785d4c700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3780108f80 0x7f378019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f378554b700 1 -- 172.21.15.53:0/153180872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3780108f80 msgr2=0x7f378019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f378554b700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3780108f80 0x7f378019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f378554b700 1 -- 172.21.15.53:0/153180872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37801085e0 msgr2=0x7f378019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f378554b700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37801085e0 0x7f378019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.157+0000 7f378554b700 1 -- 172.21.15.53:0/153180872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37700095f0 con 0x7f37801073f0 2023-11-06T01:35:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.158+0000 7f378554b700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37801073f0 0x7f378019d280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f377c00e970 tx=0x7f377c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.158+0000 7f37767fc700 1 -- 172.21.15.53:0/153180872 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c009790 con 0x7f37801073f0 2023-11-06T01:35:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.158+0000 7f37767fc700 1 -- 172.21.15.53:0/153180872 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f377c00eec0 con 0x7f37801073f0 2023-11-06T01:35:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.158+0000 7f37767fc700 1 -- 172.21.15.53:0/153180872 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c003cf0 con 0x7f37801073f0 2023-11-06T01:35:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.158+0000 7f37877af700 1 -- 172.21.15.53:0/153180872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3778009c40 con 0x7f37801073f0 2023-11-06T01:35:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.158+0000 7f37877af700 1 -- 172.21.15.53:0/153180872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37801a5890 con 0x7f37801073f0 2023-11-06T01:35:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.162+0000 7f37767fc700 1 -- 172.21.15.53:0/153180872 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f377c00ccd0 con 0x7f37801073f0 2023-11-06T01:35:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.162+0000 7f37767fc700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f376c0a05a0 0x7f376c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.162+0000 7f37767fc700 1 -- 172.21.15.53:0/153180872 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f377c014070 con 0x7f37801073f0 2023-11-06T01:35:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.163+0000 7f379393fb80 1 Processor -- start 2023-11-06T01:35:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.163+0000 7f379393fb80 1 -- start start 2023-11-06T01:35:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f379393fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fc690 0x56110c6fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f379393fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fcff0 0x56110c6fd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f379393fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 0x56110c6fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f379393fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c6439b0 con 0x56110c6fdee0 2023-11-06T01:35:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f379393fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c641370 con 0x56110c6fcff0 2023-11-06T01:35:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f379393fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c63cc60 con 0x56110c6fc690 2023-11-06T01:35:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f3785d4c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 0x56110c6fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f3785d4c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 0x56110c6fe360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40452/0 (socket says 172.21.15.53:40452) 2023-11-06T01:35:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f3785d4c700 1 -- 172.21.15.53:0/4038630383 learned_addr learned my addr 172.21.15.53:0/4038630383 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f378554b700 1 --2- 172.21.15.53:0/4038630383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fc690 0x56110c6fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f3785d4c700 1 -- 172.21.15.53:0/4038630383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fc690 msgr2=0x56110c6fcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f3785d4c700 1 --2- 172.21.15.53:0/4038630383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fc690 0x56110c6fcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f3785d4c700 1 -- 172.21.15.53:0/4038630383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fcff0 msgr2=0x56110c6fd900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:35:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f3785d4c700 1 --2- 172.21.15.53:0/4038630383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fcff0 0x56110c6fd900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.164+0000 7f3785d4c700 1 -- 172.21.15.53:0/4038630383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56110c46ad80 con 0x56110c6fdee0 2023-11-06T01:35:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.165+0000 7f3785d4c700 1 --2- 172.21.15.53:0/4038630383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 0x56110c6fe360 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3778009c00 tx=0x7f37780092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5dd8bfd70ebf94 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.165+0000 7f3784d4a700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f376c0a05a0 0x7f376c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.165+0000 7f376b7fe700 1 -- 172.21.15.53:0/4038630383 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377800e040 con 0x56110c6fdee0 2023-11-06T01:35:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.165+0000 7f376b7fe700 1 -- 172.21.15.53:0/4038630383 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37780074a0 con 0x56110c6fdee0 2023-11-06T01:35:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.165+0000 7f376b7fe700 1 -- 172.21.15.53:0/4038630383 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377800bc30 con 0x56110c6fdee0 2023-11-06T01:35:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.166+0000 7f379393fb80 1 -- 172.21.15.53:0/4038630383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 msgr2=0x56110c6fe360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.166+0000 7f379393fb80 1 --2- 172.21.15.53:0/4038630383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 0x56110c6fe360 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3778009c00 tx=0x7f37780092e0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.166+0000 7f3784d4a700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f376c0a05a0 0x7f376c0a2a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f37700095c0 tx=0x7f3770009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.166+0000 7f379393fb80 1 -- 172.21.15.53:0/4038630383 shutdown_connections 2023-11-06T01:35:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.166+0000 7f379393fb80 1 --2- 172.21.15.53:0/4038630383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 0x56110c6fe360 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.166+0000 7f379393fb80 1 --2- 172.21.15.53:0/4038630383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fc690 0x56110c6fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.166+0000 7f379393fb80 1 --2- 172.21.15.53:0/4038630383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fcff0 0x56110c6fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.166+0000 7f379393fb80 1 -- 172.21.15.53:0/4038630383 >> 172.21.15.53:0/4038630383 conn(0x56110c64b070 msgr2=0x56110c6e8040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.167+0000 7f379393fb80 1 -- 172.21.15.53:0/4038630383 shutdown_connections 2023-11-06T01:35:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.167+0000 7f379393fb80 1 -- 172.21.15.53:0/4038630383 wait complete. 2023-11-06T01:35:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f379393fb80 1 -- start start 2023-11-06T01:35:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f379393fb80 1 Processor -- start 2023-11-06T01:35:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f379393fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 0x56110c7064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f379393fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 0x56110c706a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f379393fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fdee0 0x56110c70da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f379393fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c5d94b0 con 0x56110c6fc690 2023-11-06T01:35:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f379393fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c5d9150 con 0x56110c6fdee0 2023-11-06T01:35:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f379393fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c4c08c0 con 0x56110c6fcff0 2023-11-06T01:35:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f379393fb80 10 client.0 _ll_register_callbacks cb 0x56110c526a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f378554b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 0x56110c7064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f3784d4a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 0x56110c706a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f3785d4c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fdee0 0x56110c70da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f378554b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 0x56110c7064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40454/0 (socket says 172.21.15.53:40454) 2023-11-06T01:35:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.168+0000 7f378554b700 1 -- 172.21.15.53:0/1564490943 learned_addr learned my addr 172.21.15.53:0/1564490943 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.169+0000 7f3785d4c700 1 -- 172.21.15.53:0/1564490943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 msgr2=0x56110c706a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.169+0000 7f3785d4c700 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 0x56110c706a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.169+0000 7f3785d4c700 1 -- 172.21.15.53:0/1564490943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 msgr2=0x56110c7064f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.169+0000 7f3785d4c700 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 0x56110c7064f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.169+0000 7f3785d4c700 1 -- 172.21.15.53:0/1564490943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3780110180 con 0x56110c6fdee0 2023-11-06T01:35:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.169+0000 7f3784d4a700 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 0x56110c706a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:35:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.169+0000 7f3785d4c700 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fdee0 0x56110c70da70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f377800b6c0 tx=0x7f377800bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dac384e9498b58e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.169+0000 7f3768ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:35:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.169+0000 7f3768ff9700 1 -- 172.21.15.53:0/1564490943 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377802b070 con 0x56110c6fdee0 2023-11-06T01:35:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.169+0000 7f3768ff9700 1 -- 172.21.15.53:0/1564490943 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3778007610 con 0x56110c6fdee0 2023-11-06T01:35:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.170+0000 7f379393fb80 1 -- 172.21.15.53:0/1564490943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56110c667cb0 con 0x56110c6fdee0 2023-11-06T01:35:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.170+0000 7f3768ff9700 1 -- 172.21.15.53:0/1564490943 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f377800e040 con 0x56110c6fdee0 2023-11-06T01:35:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.170+0000 7f379393fb80 10 client.25277 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.170+0000 7f379393fb80 1 -- 172.21.15.53:0/1564490943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56110c453c00 con 0x56110c6fdee0 2023-11-06T01:35:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.170+0000 7f3768ff9700 1 -- 172.21.15.53:0/1564490943 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56110c667cb0 con 0x56110c6fdee0 2023-11-06T01:35:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.172+0000 7f3768ff9700 1 -- 172.21.15.53:0/1564490943 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f377802e020 con 0x56110c6fdee0 2023-11-06T01:35:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.172+0000 7f379393fb80 10 client.25277 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.172+0000 7f379393fb80 10 client.25277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.172+0000 7f379393fb80 20 client.25277 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.172+0000 7f379393fb80 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56110c711ec0 0x56110c7142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.172+0000 7f379393fb80 4 client.25277 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:35:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.172+0000 7f379393fb80 1 -- 172.21.15.53:0/1564490943 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56110c6badb0 con 0x56110c711ec0 2023-11-06T01:35:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.173+0000 7f378554b700 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56110c711ec0 0x56110c7142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.173+0000 7f3768ff9700 10 client.25277 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.173+0000 7f378554b700 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56110c711ec0 0x56110c7142e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=108b97b2b6fa83b8 server_cookie=4d1932a02e31d7ee in_seq=0 out_seq=0 2023-11-06T01:35:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f3768ff9700 1 -- 172.21.15.53:0/1564490943 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56110c6badb0 con 0x56110c711ec0 2023-11-06T01:35:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f3768ff9700 10 client.25277 handle_command_reply: tid=0 2023-11-06T01:35:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f379393fb80 1 client.25277 shutdown 2023-11-06T01:35:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f379393fb80 1 -- 172.21.15.53:0/1564490943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fdee0 msgr2=0x56110c70da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f379393fb80 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fdee0 0x56110c70da70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f377800b6c0 tx=0x7f377800bc20 comp rx=0 tx=0).stop 2023-11-06T01:35:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f379393fb80 1 -- 172.21.15.53:0/1564490943 shutdown_connections 2023-11-06T01:35:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f379393fb80 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fdee0 0x56110c70da70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f379393fb80 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56110c711ec0 0x56110c7142e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f379393fb80 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 0x56110c7064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f379393fb80 1 --2- 172.21.15.53:0/1564490943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 0x56110c706a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f379393fb80 1 -- 172.21.15.53:0/1564490943 >> 172.21.15.53:0/1564490943 conn(0x56110c64b070 msgr2=0x56110c6e63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.174+0000 7f3768ff9700 0 client.25277 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.175+0000 7f379393fb80 1 -- 172.21.15.53:0/1564490943 shutdown_connections 2023-11-06T01:35:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.175+0000 7f379393fb80 1 -- 172.21.15.53:0/1564490943 wait complete. 2023-11-06T01:35:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.175+0000 7f379393fb80 20 client.25277 trim_cache size 0 max 16384 2023-11-06T01:35:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 75.0 s, 4547.3 tps, lat 22.013 ms stddev 23.547 2023-11-06T01:35:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.199+0000 7f379393fb80 1 Processor -- start 2023-11-06T01:35:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.210+0000 7f379393fb80 1 -- start start 2023-11-06T01:35:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.210+0000 7f379393fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 0x56110c70ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.210+0000 7f379393fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 0x56110c6fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.210+0000 7f379393fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fdee0 0x56110c7010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.210+0000 7f379393fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c6de7a0 con 0x56110c6fc690 2023-11-06T01:35:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.210+0000 7f379393fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c7059a0 con 0x56110c6fdee0 2023-11-06T01:35:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.210+0000 7f379393fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c6e1500 con 0x56110c6fcff0 2023-11-06T01:35:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.211+0000 7f378554b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 0x56110c70ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.211+0000 7f3785d4c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fdee0 0x56110c7010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.211+0000 7f378554b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 0x56110c70ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40462/0 (socket says 172.21.15.53:40462) 2023-11-06T01:35:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.211+0000 7f378554b700 1 -- 172.21.15.53:0/129209354 learned_addr learned my addr 172.21.15.53:0/129209354 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.211+0000 7f378554b700 1 -- 172.21.15.53:0/129209354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 msgr2=0x56110c6fe2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:35:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.211+0000 7f378554b700 1 --2- 172.21.15.53:0/129209354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 0x56110c6fe2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.211+0000 7f378554b700 1 -- 172.21.15.53:0/129209354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fdee0 msgr2=0x56110c7010b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.211+0000 7f378554b700 1 --2- 172.21.15.53:0/129209354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fdee0 0x56110c7010b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.211+0000 7f378554b700 1 -- 172.21.15.53:0/129209354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56110c4911f0 con 0x56110c6fc690 2023-11-06T01:35:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.211+0000 7f378554b700 1 --2- 172.21.15.53:0/129209354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 0x56110c70ba20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f377c099360 tx=0x7f377c084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7330338af650c594 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.212+0000 7f376bfff700 1 -- 172.21.15.53:0/129209354 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c0993d0 con 0x56110c6fc690 2023-11-06T01:35:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.212+0000 7f376bfff700 1 -- 172.21.15.53:0/129209354 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f377c0ace90 con 0x56110c6fc690 2023-11-06T01:35:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.212+0000 7f376bfff700 1 -- 172.21.15.53:0/129209354 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c01fde0 con 0x56110c6fc690 2023-11-06T01:35:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.213+0000 7f379393fb80 1 -- 172.21.15.53:0/129209354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 msgr2=0x56110c70ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.213+0000 7f379393fb80 1 --2- 172.21.15.53:0/129209354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 0x56110c70ba20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f377c099360 tx=0x7f377c084040 comp rx=0 tx=0).stop 2023-11-06T01:35:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.213+0000 7f379393fb80 1 -- 172.21.15.53:0/129209354 shutdown_connections 2023-11-06T01:35:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.213+0000 7f379393fb80 1 --2- 172.21.15.53:0/129209354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fdee0 0x56110c7010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.213+0000 7f379393fb80 1 --2- 172.21.15.53:0/129209354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fc690 0x56110c70ba20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.213+0000 7f379393fb80 1 --2- 172.21.15.53:0/129209354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 0x56110c6fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.213+0000 7f379393fb80 1 -- 172.21.15.53:0/129209354 >> 172.21.15.53:0/129209354 conn(0x56110c64b070 msgr2=0x56110c6e7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.217+0000 7f379393fb80 1 -- 172.21.15.53:0/129209354 shutdown_connections 2023-11-06T01:35:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.217+0000 7f379393fb80 1 -- 172.21.15.53:0/129209354 wait complete. 2023-11-06T01:35:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.217+0000 7f379393fb80 1 -- start start 2023-11-06T01:35:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f379393fb80 1 Processor -- start 2023-11-06T01:35:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f379393fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fc690 0x56110c707c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f379393fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 0x56110c708190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f379393fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 0x56110c70abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f379393fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c70b130 con 0x56110c6fdee0 2023-11-06T01:35:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f379393fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c70b2a0 con 0x56110c6fc690 2023-11-06T01:35:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f379393fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110c70b460 con 0x56110c6fcff0 2023-11-06T01:35:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f379393fb80 10 client.0 _ll_register_callbacks cb 0x7f3778009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f378554b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fc690 0x56110c707c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f378554b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fc690 0x56110c707c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51904/0 (socket says 172.21.15.53:51904) 2023-11-06T01:35:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f378554b700 1 -- 172.21.15.53:0/56157077 learned_addr learned my addr 172.21.15.53:0/56157077 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.218+0000 7f3785d4c700 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 0x56110c70abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.219+0000 7f378554b700 1 -- 172.21.15.53:0/56157077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 msgr2=0x56110c708190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.219+0000 7f378554b700 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 0x56110c708190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.219+0000 7f378554b700 1 -- 172.21.15.53:0/56157077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 msgr2=0x56110c70abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.219+0000 7f378554b700 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 0x56110c70abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.219+0000 7f378554b700 1 -- 172.21.15.53:0/56157077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37801a5890 con 0x56110c6fc690 2023-11-06T01:35:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.219+0000 7f378554b700 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fc690 0x56110c707c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f377c084010 tx=0x7f377c00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92187bf2c2838ea5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.219+0000 7f3768ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:35:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.219+0000 7f3768ff9700 1 -- 172.21.15.53:0/56157077 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c0a1440 con 0x56110c6fc690 2023-11-06T01:35:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.219+0000 7f379393fb80 1 -- 172.21.15.53:0/56157077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37780074a0 con 0x56110c6fc690 2023-11-06T01:35:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.219+0000 7f3768ff9700 1 -- 172.21.15.53:0/56157077 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f377c01fde0 con 0x56110c6fc690 2023-11-06T01:35:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.220+0000 7f3768ff9700 1 -- 172.21.15.53:0/56157077 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f377c0b7280 con 0x56110c6fc690 2023-11-06T01:35:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.220+0000 7f379393fb80 10 client.25283 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.220+0000 7f379393fb80 1 -- 172.21.15.53:0/56157077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56110c469250 con 0x56110c6fc690 2023-11-06T01:35:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.220+0000 7f3768ff9700 1 -- 172.21.15.53:0/56157077 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37780074a0 con 0x56110c6fc690 2023-11-06T01:35:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.221+0000 7f3768ff9700 1 -- 172.21.15.53:0/56157077 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f377c0988c0 con 0x56110c6fc690 2023-11-06T01:35:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.221+0000 7f379393fb80 10 client.25283 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.221+0000 7f379393fb80 10 client.25283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.221+0000 7f379393fb80 20 client.25283 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.221+0000 7f379393fb80 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56110c775600 0x56110c777a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.221+0000 7f379393fb80 4 client.25283 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:35:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.221+0000 7f379393fb80 1 -- 172.21.15.53:0/56157077 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56110c635ce0 con 0x56110c775600 2023-11-06T01:35:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.221+0000 7f3784d4a700 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56110c775600 0x56110c777a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.222+0000 7f3784d4a700 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56110c775600 0x56110c777a20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382ab911190deed3 server_cookie=34704b6d9f0f8f3d in_seq=0 out_seq=0 2023-11-06T01:35:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.222+0000 7f3768ff9700 10 client.25283 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f3768ff9700 1 -- 172.21.15.53:0/56157077 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56110c635ce0 con 0x56110c775600 2023-11-06T01:35:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f3768ff9700 10 client.25283 handle_command_reply: tid=0 2023-11-06T01:35:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f379393fb80 1 client.25283 shutdown 2023-11-06T01:35:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f379393fb80 1 -- 172.21.15.53:0/56157077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fc690 msgr2=0x56110c707c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f379393fb80 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fc690 0x56110c707c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f377c084010 tx=0x7f377c00c6e0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f379393fb80 1 -- 172.21.15.53:0/56157077 shutdown_connections 2023-11-06T01:35:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f379393fb80 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56110c6fc690 0x56110c707c50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f379393fb80 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56110c775600 0x56110c777a20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f379393fb80 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56110c6fdee0 0x56110c70abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f379393fb80 1 --2- 172.21.15.53:0/56157077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56110c6fcff0 0x56110c708190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f379393fb80 1 -- 172.21.15.53:0/56157077 >> 172.21.15.53:0/56157077 conn(0x56110c64b070 msgr2=0x56110c7010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f3768ff9700 0 client.25283 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f379393fb80 1 -- 172.21.15.53:0/56157077 shutdown_connections 2023-11-06T01:35:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.237+0000 7f379393fb80 1 -- 172.21.15.53:0/56157077 wait complete. 2023-11-06T01:35:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f379393fb80 20 client.25283 trim_cache size 0 max 16384 2023-11-06T01:35:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:35:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:35:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:35:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:35:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f37877af700 1 -- 172.21.15.53:0/153180872 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f376c0a05a0 msgr2=0x7f376c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f37877af700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f376c0a05a0 0x7f376c0a2a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f37700095c0 tx=0x7f3770009510 comp rx=0 tx=0).stop 2023-11-06T01:35:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f37877af700 1 -- 172.21.15.53:0/153180872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37801073f0 msgr2=0x7f378019d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f37877af700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37801073f0 0x7f378019d280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f377c00e970 tx=0x7f377c00ec80 comp rx=0 tx=0).stop 2023-11-06T01:35:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f37877af700 1 -- 172.21.15.53:0/153180872 shutdown_connections 2023-11-06T01:35:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f37877af700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f376c0a05a0 0x7f376c0a2a60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f37877af700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37801073f0 0x7f378019d280 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f37877af700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3780108f80 0x7f378019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f37877af700 1 --2- 172.21.15.53:0/153180872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37801085e0 0x7f378019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f37877af700 1 -- 172.21.15.53:0/153180872 >> 172.21.15.53:0/153180872 conn(0x7f3780076390 msgr2=0x7f3780078510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.238+0000 7f37877af700 1 -- 172.21.15.53:0/153180872 shutdown_connections 2023-11-06T01:35:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:32.239+0000 7f37877af700 1 -- 172.21.15.53:0/153180872 wait complete. 2023-11-06T01:35:33.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:33 smithi053 ceph-mon[99472]: pgmap v347: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 MiB/s wr, 2.25k op/s 2023-11-06T01:35:33.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:33 smithi062 ceph-mon[103924]: pgmap v347: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 MiB/s wr, 2.25k op/s 2023-11-06T01:35:33.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:33 smithi163 ceph-mon[104416]: pgmap v347: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 MiB/s wr, 2.25k op/s 2023-11-06T01:35:34.234 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:35:34.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.430+0000 7f0c6e035700 1 -- 172.21.15.53:0/2809840166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c68075b00 msgr2=0x7f0c68077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:34.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.430+0000 7f0c6e035700 1 --2- 172.21.15.53:0/2809840166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c68075b00 0x7f0c68077f90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009f90 tx=0x7f0c6000b330 comp rx=0 tx=0).stop 2023-11-06T01:35:34.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.430+0000 7f0c6e035700 1 -- 172.21.15.53:0/2809840166 shutdown_connections 2023-11-06T01:35:34.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.430+0000 7f0c6e035700 1 --2- 172.21.15.53:0/2809840166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c68072c20 0x7f0c68073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.430+0000 7f0c6e035700 1 --2- 172.21.15.53:0/2809840166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c680784d0 0x7f0c6807a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.430+0000 7f0c6e035700 1 --2- 172.21.15.53:0/2809840166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c68075b00 0x7f0c68077f90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.430+0000 7f0c6e035700 1 -- 172.21.15.53:0/2809840166 >> 172.21.15.53:0/2809840166 conn(0x7f0c6806db40 msgr2=0x7f0c6806ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:34.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.431+0000 7f0c6e035700 1 -- 172.21.15.53:0/2809840166 shutdown_connections 2023-11-06T01:35:34.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.431+0000 7f0c6e035700 1 -- 172.21.15.53:0/2809840166 wait complete. 2023-11-06T01:35:34.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.432+0000 7f0c6e035700 1 Processor -- start 2023-11-06T01:35:34.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.432+0000 7f0c6e035700 1 -- start start 2023-11-06T01:35:34.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.432+0000 7f0c6e035700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c68072c20 0x7f0c681b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.432+0000 7f0c6e035700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c68075b00 0x7f0c681bb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.432+0000 7f0c6e035700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c680784d0 0x7f0c681bdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.432+0000 7f0c6e035700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c681be4c0 con 0x7f0c68072c20 2023-11-06T01:35:34.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.432+0000 7f0c6e035700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c681be600 con 0x7f0c680784d0 2023-11-06T01:35:34.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.432+0000 7f0c6e035700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c681be7c0 con 0x7f0c68075b00 2023-11-06T01:35:34.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.432+0000 7f0c677fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c68072c20 0x7f0c681b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.432+0000 7f0c677fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c68072c20 0x7f0c681b92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40478/0 (socket says 172.21.15.53:40478) 2023-11-06T01:35:34.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.432+0000 7f0c677fe700 1 -- 172.21.15.53:0/2319712510 learned_addr learned my addr 172.21.15.53:0/2319712510 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:34.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.433+0000 7f0c66ffd700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c68075b00 0x7f0c681bb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.433+0000 7f0c67fff700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c680784d0 0x7f0c681bdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.433+0000 7f0c66ffd700 1 -- 172.21.15.53:0/2319712510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c680784d0 msgr2=0x7f0c681bdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:34.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.433+0000 7f0c66ffd700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c680784d0 0x7f0c681bdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.433+0000 7f0c66ffd700 1 -- 172.21.15.53:0/2319712510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c68072c20 msgr2=0x7f0c681b92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:34.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.433+0000 7f0c66ffd700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c68072c20 0x7f0c681b92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.433+0000 7f0c66ffd700 1 -- 172.21.15.53:0/2319712510 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c60009c40 con 0x7f0c68075b00 2023-11-06T01:35:34.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.433+0000 7f0c66ffd700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c68075b00 0x7f0c681bb820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009f90 tx=0x7f0c6000ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:34.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.434+0000 7f0c64ff9700 1 -- 172.21.15.53:0/2319712510 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6000e070 con 0x7f0c68075b00 2023-11-06T01:35:34.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.434+0000 7f0c6e035700 1 -- 172.21.15.53:0/2319712510 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c681bea40 con 0x7f0c68075b00 2023-11-06T01:35:34.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.434+0000 7f0c64ff9700 1 -- 172.21.15.53:0/2319712510 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c6000cec0 con 0x7f0c68075b00 2023-11-06T01:35:34.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.434+0000 7f0c64ff9700 1 -- 172.21.15.53:0/2319712510 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6001a9f0 con 0x7f0c68075b00 2023-11-06T01:35:34.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.434+0000 7f0c6e035700 1 -- 172.21.15.53:0/2319712510 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c681bef40 con 0x7f0c68075b00 2023-11-06T01:35:34.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.437+0000 7f0c64ff9700 1 -- 172.21.15.53:0/2319712510 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f0c600052b0 con 0x7f0c68075b00 2023-11-06T01:35:34.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.437+0000 7f0c64ff9700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c500a0500 0x7f0c500a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.438+0000 7f0c677fe700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c500a0500 0x7f0c500a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.438+0000 7f0c64ff9700 1 -- 172.21.15.53:0/2319712510 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0c600ccd40 con 0x7f0c68075b00 2023-11-06T01:35:34.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.438+0000 7f0c677fe700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c500a0500 0x7f0c500a29c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005950 tx=0x7f0c5c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:34.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.439+0000 7f0c7a1c5b80 1 Processor -- start 2023-11-06T01:35:34.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.439+0000 7f0c7a1c5b80 1 -- start start 2023-11-06T01:35:34.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.439+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c126f6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c126f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 0x556c126f5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1264ca10 con 0x556c126f60f0 2023-11-06T01:35:34.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1264a3d0 con 0x556c126f53f0 2023-11-06T01:35:34.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c12645cc0 con 0x556c126f6a70 2023-11-06T01:35:34.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c677fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c126f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c677fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c126f6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40486/0 (socket says 172.21.15.53:40486) 2023-11-06T01:35:34.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c66ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c126f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c67fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 0x556c126f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c677fe700 1 -- 172.21.15.53:0/2748819918 learned_addr learned my addr 172.21.15.53:0/2748819918 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:34.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c677fe700 1 -- 172.21.15.53:0/2748819918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 msgr2=0x556c126f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:34.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c677fe700 1 --2- 172.21.15.53:0/2748819918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c126f4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c677fe700 1 -- 172.21.15.53:0/2748819918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 msgr2=0x556c126f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:34.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c677fe700 1 --2- 172.21.15.53:0/2748819918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 0x556c126f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c677fe700 1 -- 172.21.15.53:0/2748819918 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c12473de0 con 0x556c126f60f0 2023-11-06T01:35:34.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c677fe700 1 --2- 172.21.15.53:0/2748819918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c126f6530 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c004260 tx=0x7f0c5c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73263a1fd8cceb9c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:34.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c4dffb700 1 -- 172.21.15.53:0/2748819918 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c003d10 con 0x556c126f60f0 2023-11-06T01:35:34.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c4dffb700 1 -- 172.21.15.53:0/2748819918 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c5c003e70 con 0x556c126f60f0 2023-11-06T01:35:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.440+0000 7f0c4dffb700 1 -- 172.21.15.53:0/2748819918 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c01d680 con 0x556c126f60f0 2023-11-06T01:35:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.441+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2748819918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 msgr2=0x556c126f6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.441+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/2748819918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c126f6530 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c004260 tx=0x7f0c5c014000 comp rx=0 tx=0).stop 2023-11-06T01:35:34.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.442+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2748819918 shutdown_connections 2023-11-06T01:35:34.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.442+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/2748819918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 0x556c126f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.442+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/2748819918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c126f6530 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.442+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/2748819918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c126f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.442+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2748819918 >> 172.21.15.53:0/2748819918 conn(0x556c126540d0 msgr2=0x556c126ef940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:34.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.442+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2748819918 shutdown_connections 2023-11-06T01:35:34.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.442+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2748819918 wait complete. 2023-11-06T01:35:34.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c7a1c5b80 1 -- start start 2023-11-06T01:35:34.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c7a1c5b80 1 Processor -- start 2023-11-06T01:35:34.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 0x556c126fe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c126fef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c12705f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c125e2510 con 0x556c126f60f0 2023-11-06T01:35:34.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c125e21b0 con 0x556c126f53f0 2023-11-06T01:35:34.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c124c9920 con 0x556c126f6a70 2023-11-06T01:35:34.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c7a1c5b80 10 client.0 _ll_register_callbacks cb 0x556c1252fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:34.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c66ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c126fef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 0x556c126fe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c66ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c126fef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40494/0 (socket says 172.21.15.53:40494) 2023-11-06T01:35:34.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 0x556c126fe9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51930/0 (socket says 172.21.15.53:51930) 2023-11-06T01:35:34.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c66ffd700 1 -- 172.21.15.53:0/2701757384 learned_addr learned my addr 172.21.15.53:0/2701757384 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:34.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.443+0000 7f0c67fff700 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c12705f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.444+0000 7f0c677fe700 1 -- 172.21.15.53:0/2701757384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 msgr2=0x556c12705f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:34.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.444+0000 7f0c677fe700 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c12705f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.444+0000 7f0c677fe700 1 -- 172.21.15.53:0/2701757384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 msgr2=0x556c126fef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:34.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.444+0000 7f0c677fe700 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c126fef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.444+0000 7f0c677fe700 1 -- 172.21.15.53:0/2701757384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c017070 con 0x556c126f53f0 2023-11-06T01:35:34.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.444+0000 7f0c677fe700 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 0x556c126fe9f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c011ab0 tx=0x7f0c5c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f645343a99da893 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:34.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.444+0000 7f0c46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:35:34.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.444+0000 7f0c46ffd700 1 -- 172.21.15.53:0/2701757384 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c02c070 con 0x556c126f53f0 2023-11-06T01:35:34.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.444+0000 7f0c46ffd700 1 -- 172.21.15.53:0/2701757384 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0c5c01e420 con 0x556c126f53f0 2023-11-06T01:35:34.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.445+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2701757384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c12670d10 con 0x556c126f53f0 2023-11-06T01:35:34.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.445+0000 7f0c46ffd700 1 -- 172.21.15.53:0/2701757384 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c5c0056b0 con 0x556c126f53f0 2023-11-06T01:35:34.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.445+0000 7f0c7a1c5b80 10 client.25295 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:34.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.445+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2701757384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556c1245cc60 con 0x556c126f53f0 2023-11-06T01:35:34.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.445+0000 7f0c46ffd700 1 -- 172.21.15.53:0/2701757384 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c12670d10 con 0x556c126f53f0 2023-11-06T01:35:34.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.445+0000 7f0c46ffd700 1 -- 172.21.15.53:0/2701757384 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0c5c024020 con 0x556c126f53f0 2023-11-06T01:35:34.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.445+0000 7f0c7a1c5b80 10 client.25295 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:34.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.445+0000 7f0c7a1c5b80 10 client.25295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:34.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.445+0000 7f0c7a1c5b80 20 client.25295 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:34.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.446+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556c1270a6c0 0x556c1270cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.446+0000 7f0c7a1c5b80 4 client.25295 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:35:34.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.446+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2701757384 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c126c3e10 con 0x556c1270a6c0 2023-11-06T01:35:34.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.446+0000 7f0c66ffd700 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556c1270a6c0 0x556c1270cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.446+0000 7f0c46ffd700 10 client.25295 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:34.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.446+0000 7f0c66ffd700 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556c1270a6c0 0x556c1270cae0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a80e7d0a00e259 server_cookie=297b269802599dc9 in_seq=0 out_seq=0 2023-11-06T01:35:34.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.447+0000 7f0c46ffd700 1 -- 172.21.15.53:0/2701757384 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556c126c3e10 con 0x556c1270a6c0 2023-11-06T01:35:34.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.447+0000 7f0c46ffd700 10 client.25295 handle_command_reply: tid=0 2023-11-06T01:35:34.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.447+0000 7f0c7a1c5b80 1 client.25295 shutdown 2023-11-06T01:35:34.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.447+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2701757384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 msgr2=0x556c126fe9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:34.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.448+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 0x556c126fe9f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c011ab0 tx=0x7f0c5c005270 comp rx=0 tx=0).stop 2023-11-06T01:35:34.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.448+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2701757384 shutdown_connections 2023-11-06T01:35:34.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.448+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f53f0 0x556c126fe9f0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c011ab0 tx=0x7f0c5c005270 comp rx=0 tx=0).stop 2023-11-06T01:35:34.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.448+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556c1270a6c0 0x556c1270cae0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.448+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c12705f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.448+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/2701757384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c126fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.448+0000 7f0c46ffd700 0 client.25295 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:34.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.448+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2701757384 >> 172.21.15.53:0/2701757384 conn(0x556c126540d0 msgr2=0x556c126ef0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:34.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.448+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2701757384 shutdown_connections 2023-11-06T01:35:34.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.448+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/2701757384 wait complete. 2023-11-06T01:35:34.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.448+0000 7f0c7a1c5b80 20 client.25295 trim_cache size 0 max 16384 2023-11-06T01:35:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.470+0000 7f0c7a1c5b80 1 Processor -- start 2023-11-06T01:35:34.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.470+0000 7f0c7a1c5b80 1 -- start start 2023-11-06T01:35:34.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.470+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f60f0 0x556c126f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.470+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f6a70 0x556c12703f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.470+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126fe7a0 0x556c126fec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.470+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c126fe5c0 con 0x556c126fe7a0 2023-11-06T01:35:34.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.470+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c126c3e10 con 0x556c126f6a70 2023-11-06T01:35:34.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.470+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c12670d10 con 0x556c126f60f0 2023-11-06T01:35:34.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.470+0000 7f0c67fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126fe7a0 0x556c126fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.471+0000 7f0c677fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f60f0 0x556c126f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.471+0000 7f0c67fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126fe7a0 0x556c126fec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40502/0 (socket says 172.21.15.53:40502) 2023-11-06T01:35:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.471+0000 7f0c67fff700 1 -- 172.21.15.53:0/683912488 learned_addr learned my addr 172.21.15.53:0/683912488 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.471+0000 7f0c66ffd700 1 --2- 172.21.15.53:0/683912488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f6a70 0x556c12703f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.471+0000 7f0c677fe700 1 -- 172.21.15.53:0/683912488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f6a70 msgr2=0x556c12703f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.471+0000 7f0c677fe700 1 --2- 172.21.15.53:0/683912488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f6a70 0x556c12703f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.471+0000 7f0c677fe700 1 -- 172.21.15.53:0/683912488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126fe7a0 msgr2=0x556c126fec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:34.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.471+0000 7f0c677fe700 1 --2- 172.21.15.53:0/683912488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126fe7a0 0x556c126fec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.471+0000 7f0c677fe700 1 -- 172.21.15.53:0/683912488 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c1249a250 con 0x556c126f60f0 2023-11-06T01:35:34.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.471+0000 7f0c66ffd700 1 --2- 172.21.15.53:0/683912488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f6a70 0x556c12703f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:35:34.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.472+0000 7f0c677fe700 1 --2- 172.21.15.53:0/683912488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f60f0 0x556c126f4e60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c003e40 tx=0x7f0c5c004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f003f5ce67f18186 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:34.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.472+0000 7f0c4e7fc700 1 -- 172.21.15.53:0/683912488 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c040070 con 0x556c126f60f0 2023-11-06T01:35:34.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.472+0000 7f0c4e7fc700 1 -- 172.21.15.53:0/683912488 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c5c0056b0 con 0x556c126f60f0 2023-11-06T01:35:34.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.472+0000 7f0c4e7fc700 1 -- 172.21.15.53:0/683912488 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c02c070 con 0x556c126f60f0 2023-11-06T01:35:34.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.473+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/683912488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f60f0 msgr2=0x556c126f4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:34.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.473+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/683912488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f60f0 0x556c126f4e60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c003e40 tx=0x7f0c5c004cf0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.474+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/683912488 shutdown_connections 2023-11-06T01:35:34.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.474+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/683912488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126fe7a0 0x556c126fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.474+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/683912488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f60f0 0x556c126f4e60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.474+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/683912488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126f6a70 0x556c12703f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.474+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/683912488 >> 172.21.15.53:0/683912488 conn(0x556c126540d0 msgr2=0x556c12706860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:34.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.474+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/683912488 shutdown_connections 2023-11-06T01:35:34.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.474+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/683912488 wait complete. 2023-11-06T01:35:34.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c7a1c5b80 1 -- start start 2023-11-06T01:35:34.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c7a1c5b80 1 Processor -- start 2023-11-06T01:35:34.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c12754440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c12754980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c7a1c5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126fe7a0 0x556c1275b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1275beb0 con 0x556c126f60f0 2023-11-06T01:35:34.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1275bff0 con 0x556c126fe7a0 2023-11-06T01:35:34.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c7a1c5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1275c1b0 con 0x556c126f6a70 2023-11-06T01:35:34.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c7a1c5b80 10 client.0 _ll_register_callbacks cb 0x7f0c580093b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:34.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c677fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c12754440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c66ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c12754980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c677fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c12754440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40506/0 (socket says 172.21.15.53:40506) 2023-11-06T01:35:34.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c677fe700 1 -- 172.21.15.53:0/103076854 learned_addr learned my addr 172.21.15.53:0/103076854 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:34.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.475+0000 7f0c67fff700 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126fe7a0 0x556c1275b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c677fe700 1 -- 172.21.15.53:0/103076854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 msgr2=0x556c12754980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:34.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c677fe700 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c12754980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c677fe700 1 -- 172.21.15.53:0/103076854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126fe7a0 msgr2=0x556c1275b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:34.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c677fe700 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126fe7a0 0x556c1275b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c677fe700 1 -- 172.21.15.53:0/103076854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c1245cc60 con 0x556c126f60f0 2023-11-06T01:35:34.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c66ffd700 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c12754980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:35:34.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c67fff700 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126fe7a0 0x556c1275b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:35:34.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c677fe700 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c12754440 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009c10 tx=0x7f0c5c01e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9267f55d849da40 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:34.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:35:34.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c46ffd700 1 -- 172.21.15.53:0/103076854 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c02c070 con 0x556c126f60f0 2023-11-06T01:35:34.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c46ffd700 1 -- 172.21.15.53:0/103076854 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0c5c004dc0 con 0x556c126f60f0 2023-11-06T01:35:34.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/103076854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c5c003e70 con 0x556c126f60f0 2023-11-06T01:35:34.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c46ffd700 1 -- 172.21.15.53:0/103076854 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c5c040040 con 0x556c126f60f0 2023-11-06T01:35:34.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c7a1c5b80 10 client.15735 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:34.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.476+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/103076854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556c124722b0 con 0x556c126f60f0 2023-11-06T01:35:34.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.478+0000 7f0c46ffd700 1 -- 172.21.15.53:0/103076854 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c5c003e70 con 0x556c126f60f0 2023-11-06T01:35:34.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.478+0000 7f0c46ffd700 1 -- 172.21.15.53:0/103076854 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0c5c04d020 con 0x556c126f60f0 2023-11-06T01:35:34.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.478+0000 7f0c7a1c5b80 10 client.15735 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:34.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.478+0000 7f0c7a1c5b80 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:34.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.478+0000 7f0c7a1c5b80 20 client.15735 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:34.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.478+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556c1275d740 0x556c1275fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:34.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.478+0000 7f0c7a1c5b80 4 client.15735 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:35:34.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.478+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/103076854 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c1263ed40 con 0x556c1275d740 2023-11-06T01:35:34.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.478+0000 7f0c66ffd700 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556c1275d740 0x556c1275fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:34.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c46ffd700 10 client.15735 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:34.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c66ffd700 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556c1275d740 0x556c1275fb60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd107c8e96bf3a4 server_cookie=7e0966b53e485a7e in_seq=0 out_seq=0 2023-11-06T01:35:34.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c46ffd700 1 -- 172.21.15.53:0/103076854 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556c1263ed40 con 0x556c1275d740 2023-11-06T01:35:34.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c46ffd700 10 client.15735 handle_command_reply: tid=0 2023-11-06T01:35:34.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c7a1c5b80 1 client.15735 shutdown 2023-11-06T01:35:34.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/103076854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 msgr2=0x556c12754440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:34.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c12754440 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009c10 tx=0x7f0c5c01e930 comp rx=0 tx=0).stop 2023-11-06T01:35:34.494 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:35:34.494 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:35:34.494 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:35:34.494 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:35:34.495 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:35:34.495 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:35:34.495 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:35:34.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:35:34.497 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:35:34.497 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:35:34.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/103076854 shutdown_connections 2023-11-06T01:35:34.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c126fe7a0 0x556c1275b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556c1275d740 0x556c1275fb60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556c126f60f0 0x556c12754440 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c7a1c5b80 1 --2- 172.21.15.53:0/103076854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c126f6a70 0x556c12754980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.479+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/103076854 >> 172.21.15.53:0/103076854 conn(0x556c126540d0 msgr2=0x556c126f93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:34.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.480+0000 7f0c46ffd700 0 client.15735 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:34.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.480+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/103076854 shutdown_connections 2023-11-06T01:35:34.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.480+0000 7f0c7a1c5b80 1 -- 172.21.15.53:0/103076854 wait complete. 2023-11-06T01:35:34.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.480+0000 7f0c7a1c5b80 20 client.15735 trim_cache size 0 max 16384 2023-11-06T01:35:34.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 -- 172.21.15.53:0/2319712510 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c500a0500 msgr2=0x7f0c500a29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:34.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c500a0500 0x7f0c500a29c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005950 tx=0x7f0c5c009310 comp rx=0 tx=0).stop 2023-11-06T01:35:34.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 -- 172.21.15.53:0/2319712510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c68075b00 msgr2=0x7f0c681bb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:34.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c68075b00 0x7f0c681bb820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009f90 tx=0x7f0c6000ca00 comp rx=0 tx=0).stop 2023-11-06T01:35:34.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 -- 172.21.15.53:0/2319712510 shutdown_connections 2023-11-06T01:35:34.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0c500a0500 0x7f0c500a29c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c680784d0 0x7f0c681bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c68072c20 0x7f0c681b92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 --2- 172.21.15.53:0/2319712510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c68075b00 0x7f0c681bb820 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:34.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 -- 172.21.15.53:0/2319712510 >> 172.21.15.53:0/2319712510 conn(0x7f0c6806db40 msgr2=0x7f0c680799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:34.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 -- 172.21.15.53:0/2319712510 shutdown_connections 2023-11-06T01:35:34.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:35:34.481+0000 7f0c6e035700 1 -- 172.21.15.53:0/2319712510 wait complete. 2023-11-06T01:35:34.504 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:35:35.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:35 smithi053 ceph-mon[99472]: pgmap v348: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 47 MiB/s wr, 1.99k op/s 2023-11-06T01:35:35.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:35 smithi062 ceph-mon[103924]: pgmap v348: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 47 MiB/s wr, 1.99k op/s 2023-11-06T01:35:35.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:35 smithi163 ceph-mon[104416]: pgmap v348: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 47 MiB/s wr, 1.99k op/s 2023-11-06T01:35:37.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 80.0 s, 4405.2 tps, lat 22.745 ms stddev 24.332 2023-11-06T01:35:37.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:37 smithi053 ceph-mon[99472]: pgmap v349: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 MiB/s wr, 2.07k op/s 2023-11-06T01:35:37.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:37 smithi062 ceph-mon[103924]: pgmap v349: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 MiB/s wr, 2.07k op/s 2023-11-06T01:35:37.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:37 smithi163 ceph-mon[104416]: pgmap v349: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 MiB/s wr, 2.07k op/s 2023-11-06T01:35:38.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:35:38.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:35:38.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:35:38.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:38 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:35:38.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:35:38.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:35:38.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:35:38.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:38 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:35:38.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:35:38.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:35:38.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:35:38.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:38 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:35:39.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:39 smithi053 ceph-mon[99472]: pgmap v350: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 1.12k op/s 2023-11-06T01:35:39.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:39 smithi062 ceph-mon[103924]: pgmap v350: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 1.12k op/s 2023-11-06T01:35:39.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:39 smithi163 ceph-mon[104416]: pgmap v350: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 1.12k op/s 2023-11-06T01:35:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:41 smithi053 ceph-mon[99472]: pgmap v351: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 938 op/s 2023-11-06T01:35:41.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:41 smithi062 ceph-mon[103924]: pgmap v351: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 938 op/s 2023-11-06T01:35:41.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:41 smithi163 ceph-mon[104416]: pgmap v351: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 938 op/s 2023-11-06T01:35:42.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 85.0 s, 3478.7 tps, lat 28.639 ms stddev 31.568 2023-11-06T01:35:43.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:43 smithi062 ceph-mon[103924]: pgmap v352: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 977 op/s 2023-11-06T01:35:43.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:43 smithi163 ceph-mon[104416]: pgmap v352: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 977 op/s 2023-11-06T01:35:43.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:43 smithi053 ceph-mon[99472]: pgmap v352: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 977 op/s 2023-11-06T01:35:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.416+0000 7ff370de2700 1 -- 172.21.15.53:0/1053873524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff36c100fa0 msgr2=0x7ff36c107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.416+0000 7ff370de2700 1 --2- 172.21.15.53:0/1053873524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff36c100fa0 0x7ff36c107850 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff364009f90 tx=0x7ff36400b330 comp rx=0 tx=0).stop 2023-11-06T01:35:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.416+0000 7ff370de2700 1 -- 172.21.15.53:0/1053873524 shutdown_connections 2023-11-06T01:35:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.416+0000 7ff370de2700 1 --2- 172.21.15.53:0/1053873524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff36c100fa0 0x7ff36c107850 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.416+0000 7ff370de2700 1 --2- 172.21.15.53:0/1053873524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff36c0ff410 0x7ff36c0ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.416+0000 7ff370de2700 1 --2- 172.21.15.53:0/1053873524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff36c100600 0x7ff36c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.416+0000 7ff370de2700 1 -- 172.21.15.53:0/1053873524 >> 172.21.15.53:0/1053873524 conn(0x7ff36c0fa960 msgr2=0x7ff36c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.416+0000 7ff370de2700 1 -- 172.21.15.53:0/1053873524 shutdown_connections 2023-11-06T01:35:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.416+0000 7ff370de2700 1 -- 172.21.15.53:0/1053873524 wait complete. 2023-11-06T01:35:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.417+0000 7ff370de2700 1 Processor -- start 2023-11-06T01:35:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.417+0000 7ff370de2700 1 -- start start 2023-11-06T01:35:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff370de2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff36c0ff410 0x7ff36c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff370de2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff36c100600 0x7ff36c1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff370de2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff36c100fa0 0x7ff36c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff370de2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c1985a0 con 0x7ff36c100600 2023-11-06T01:35:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff370de2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c198710 con 0x7ff36c100fa0 2023-11-06T01:35:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff370de2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c1988d0 con 0x7ff36c0ff410 2023-11-06T01:35:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff36a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff36c0ff410 0x7ff36c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff369d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff36c100600 0x7ff36c1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff36ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff36c100fa0 0x7ff36c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff369d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff36c100600 0x7ff36c1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40520/0 (socket says 172.21.15.53:40520) 2023-11-06T01:35:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff369d9b700 1 -- 172.21.15.53:0/2311466741 learned_addr learned my addr 172.21.15.53:0/2311466741 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff369d9b700 1 -- 172.21.15.53:0/2311466741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff36c0ff410 msgr2=0x7ff36c196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff369d9b700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff36c0ff410 0x7ff36c196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff369d9b700 1 -- 172.21.15.53:0/2311466741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff36c100fa0 msgr2=0x7ff36c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff369d9b700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff36c100fa0 0x7ff36c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff369d9b700 1 -- 172.21.15.53:0/2311466741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff364009c40 con 0x7ff36c100600 2023-11-06T01:35:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff36ad9d700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff36c100fa0 0x7ff36c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:35:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.418+0000 7ff36a59c700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff36c0ff410 0x7ff36c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:35:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.419+0000 7ff369d9b700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff36c100600 0x7ff36c1970c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff36000b530 tx=0x7ff36000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.419+0000 7ff35b7fe700 1 -- 172.21.15.53:0/2311466741 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360011830 con 0x7ff36c100600 2023-11-06T01:35:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.419+0000 7ff35b7fe700 1 -- 172.21.15.53:0/2311466741 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff360011e50 con 0x7ff36c100600 2023-11-06T01:35:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.419+0000 7ff370de2700 1 -- 172.21.15.53:0/2311466741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36c198bb0 con 0x7ff36c100600 2023-11-06T01:35:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.419+0000 7ff35b7fe700 1 -- 172.21.15.53:0/2311466741 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36000f2e0 con 0x7ff36c100600 2023-11-06T01:35:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.419+0000 7ff370de2700 1 -- 172.21.15.53:0/2311466741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36c06aa10 con 0x7ff36c100600 2023-11-06T01:35:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.421+0000 7ff35b7fe700 1 -- 172.21.15.53:0/2311466741 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff360010380 con 0x7ff36c100600 2023-11-06T01:35:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.422+0000 7ff35b7fe700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff3540a0520 0x7ff3540a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.423+0000 7ff35b7fe700 1 -- 172.21.15.53:0/2311466741 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff3600cb550 con 0x7ff36c100600 2023-11-06T01:35:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.423+0000 7ff37cf72b80 1 Processor -- start 2023-11-06T01:35:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff37cf72b80 1 -- start start 2023-11-06T01:35:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff36a59c700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff3540a0520 0x7ff3540a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401610 0x555bfb401a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401f70 0x555bfb4028d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 0x555bfb4032e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb3599b0 con 0x555bfb401610 2023-11-06T01:35:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb357370 con 0x555bfb402e60 2023-11-06T01:35:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb352c60 con 0x555bfb401f70 2023-11-06T01:35:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff36ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 0x555bfb4032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff369d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401f70 0x555bfb4028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff36ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 0x555bfb4032e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51968/0 (socket says 172.21.15.53:51968) 2023-11-06T01:35:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff36ad9d700 1 -- 172.21.15.53:0/1663978077 learned_addr learned my addr 172.21.15.53:0/1663978077 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.424+0000 7ff36a59c700 1 --2- 172.21.15.53:0/1663978077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401610 0x555bfb401a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.425+0000 7ff369d9b700 1 -- 172.21.15.53:0/1663978077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 msgr2=0x555bfb4032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.425+0000 7ff369d9b700 1 --2- 172.21.15.53:0/1663978077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 0x555bfb4032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.425+0000 7ff369d9b700 1 -- 172.21.15.53:0/1663978077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401610 msgr2=0x555bfb401a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.425+0000 7ff36a59c700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff3540a0520 0x7ff3540a29e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff36c100460 tx=0x7ff35c010820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.425+0000 7ff369d9b700 1 --2- 172.21.15.53:0/1663978077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401610 0x555bfb401a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.425+0000 7ff369d9b700 1 -- 172.21.15.53:0/1663978077 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bfb180d80 con 0x555bfb401f70 2023-11-06T01:35:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.425+0000 7ff36ad9d700 1 --2- 172.21.15.53:0/1663978077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 0x555bfb4032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:35:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.425+0000 7ff369d9b700 1 --2- 172.21.15.53:0/1663978077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401f70 0x555bfb4028d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff3600c93a0 tx=0x7ff360004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26957cbca60b2edd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.425+0000 7ff358ff9700 1 -- 172.21.15.53:0/1663978077 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360098d90 con 0x555bfb401f70 2023-11-06T01:35:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.425+0000 7ff358ff9700 1 -- 172.21.15.53:0/1663978077 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36009cbc0 con 0x555bfb401f70 2023-11-06T01:35:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.425+0000 7ff358ff9700 1 -- 172.21.15.53:0/1663978077 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3600984a0 con 0x555bfb401f70 2023-11-06T01:35:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.426+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1663978077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401f70 msgr2=0x555bfb4028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.426+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/1663978077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401f70 0x555bfb4028d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff3600c93a0 tx=0x7ff360004750 comp rx=0 tx=0).stop 2023-11-06T01:35:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.426+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1663978077 shutdown_connections 2023-11-06T01:35:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.426+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/1663978077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 0x555bfb4032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.426+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/1663978077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401610 0x555bfb401a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.426+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/1663978077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401f70 0x555bfb4028d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.426+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1663978077 >> 172.21.15.53:0/1663978077 conn(0x555bfb361070 msgr2=0x555bfb3fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.427+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1663978077 shutdown_connections 2023-11-06T01:35:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.427+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1663978077 wait complete. 2023-11-06T01:35:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.427+0000 7ff37cf72b80 1 -- start start 2023-11-06T01:35:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.427+0000 7ff37cf72b80 1 Processor -- start 2023-11-06T01:35:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401610 0x555bfb40b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401f70 0x555bfb40ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 0x555bfb412a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb2ef4b0 con 0x555bfb401f70 2023-11-06T01:35:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb2ef150 con 0x555bfb402e60 2023-11-06T01:35:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb1d68c0 con 0x555bfb401610 2023-11-06T01:35:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff37cf72b80 10 client.0 _ll_register_callbacks cb 0x555bfb23ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff369d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401f70 0x555bfb40ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff36a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401610 0x555bfb40b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff369d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401f70 0x555bfb40ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40534/0 (socket says 172.21.15.53:40534) 2023-11-06T01:35:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff369d9b700 1 -- 172.21.15.53:0/898898640 learned_addr learned my addr 172.21.15.53:0/898898640 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff36ad9d700 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 0x555bfb412a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff369d9b700 1 -- 172.21.15.53:0/898898640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401610 msgr2=0x555bfb40b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff369d9b700 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401610 0x555bfb40b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff369d9b700 1 -- 172.21.15.53:0/898898640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 msgr2=0x555bfb412a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff369d9b700 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 0x555bfb412a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff369d9b700 1 -- 172.21.15.53:0/898898640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36c067c70 con 0x555bfb401f70 2023-11-06T01:35:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff369d9b700 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401f70 0x555bfb40ba40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff3600b6fd0 tx=0x7ff360095180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26967c3e293bccb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.428+0000 7ff351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:35:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff351ffb700 1 -- 172.21.15.53:0/898898640 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360098d90 con 0x555bfb401f70 2023-11-06T01:35:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff351ffb700 1 -- 172.21.15.53:0/898898640 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff360098280 con 0x555bfb401f70 2023-11-06T01:35:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff37cf72b80 1 -- 172.21.15.53:0/898898640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bfb37dcb0 con 0x555bfb401f70 2023-11-06T01:35:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff351ffb700 1 -- 172.21.15.53:0/898898640 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff360098440 con 0x555bfb401f70 2023-11-06T01:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff37cf72b80 10 client.15747 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff37cf72b80 1 -- 172.21.15.53:0/898898640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7ff36c06aa10 con 0x555bfb401f70 2023-11-06T01:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff351ffb700 1 -- 172.21.15.53:0/898898640 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36007b080 con 0x555bfb401f70 2023-11-06T01:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff351ffb700 1 -- 172.21.15.53:0/898898640 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff3600cb110 con 0x555bfb401f70 2023-11-06T01:35:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff37cf72b80 10 client.15747 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff37cf72b80 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff37cf72b80 20 client.15747 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555bfb4280a0 0x555bfb42a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff37cf72b80 4 client.15747 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:35:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.429+0000 7ff37cf72b80 1 -- 172.21.15.53:0/898898640 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bfb3d0db0 con 0x555bfb4280a0 2023-11-06T01:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.430+0000 7ff36a59c700 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555bfb4280a0 0x555bfb42a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.431+0000 7ff36a59c700 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555bfb4280a0 0x555bfb42a4c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a41730bacca63aea server_cookie=c591a446a32ceb06 in_seq=0 out_seq=0 2023-11-06T01:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.431+0000 7ff351ffb700 10 client.15747 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.431+0000 7ff351ffb700 1 -- 172.21.15.53:0/898898640 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555bfb3d0db0 con 0x555bfb4280a0 2023-11-06T01:35:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.431+0000 7ff351ffb700 10 client.15747 handle_command_reply: tid=0 2023-11-06T01:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.431+0000 7ff37cf72b80 1 client.15747 shutdown 2023-11-06T01:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff37cf72b80 1 -- 172.21.15.53:0/898898640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401f70 msgr2=0x555bfb40ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401f70 0x555bfb40ba40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff3600b6fd0 tx=0x7ff360095180 comp rx=0 tx=0).stop 2023-11-06T01:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff37cf72b80 1 -- 172.21.15.53:0/898898640 shutdown_connections 2023-11-06T01:35:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb402e60 0x555bfb412a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555bfb4280a0 0x555bfb42a4c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401610 0x555bfb40b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/898898640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401f70 0x555bfb40ba40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff37cf72b80 1 -- 172.21.15.53:0/898898640 >> 172.21.15.53:0/898898640 conn(0x555bfb361070 msgr2=0x555bfb3fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff351ffb700 0 client.15747 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff37cf72b80 1 -- 172.21.15.53:0/898898640 shutdown_connections 2023-11-06T01:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff37cf72b80 1 -- 172.21.15.53:0/898898640 wait complete. 2023-11-06T01:35:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.432+0000 7ff37cf72b80 20 client.15747 trim_cache size 0 max 16384 2023-11-06T01:35:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff37cf72b80 1 Processor -- start 2023-11-06T01:35:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff37cf72b80 1 -- start start 2023-11-06T01:35:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401610 0x555bfb410a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 0x555bfb40b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb402e60 0x555bfb40ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb40c400 con 0x555bfb402e60 2023-11-06T01:35:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb40a9b0 con 0x555bfb401f70 2023-11-06T01:35:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb3f7500 con 0x555bfb401610 2023-11-06T01:35:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff369d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 0x555bfb40b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff36a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401610 0x555bfb410a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff36ad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb402e60 0x555bfb40ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff369d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 0x555bfb40b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51980/0 (socket says 172.21.15.53:51980) 2023-11-06T01:35:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff36a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401610 0x555bfb410a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54782/0 (socket says 172.21.15.53:54782) 2023-11-06T01:35:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.451+0000 7ff369d9b700 1 -- 172.21.15.53:0/3741500331 learned_addr learned my addr 172.21.15.53:0/3741500331 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.452+0000 7ff369d9b700 1 -- 172.21.15.53:0/3741500331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401610 msgr2=0x555bfb410a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.452+0000 7ff369d9b700 1 --2- 172.21.15.53:0/3741500331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401610 0x555bfb410a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.452+0000 7ff369d9b700 1 -- 172.21.15.53:0/3741500331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb402e60 msgr2=0x555bfb40ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.452+0000 7ff369d9b700 1 --2- 172.21.15.53:0/3741500331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb402e60 0x555bfb40ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.452+0000 7ff369d9b700 1 -- 172.21.15.53:0/3741500331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bfb169c00 con 0x555bfb401f70 2023-11-06T01:35:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.452+0000 7ff369d9b700 1 --2- 172.21.15.53:0/3741500331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 0x555bfb40b4d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff360098d20 tx=0x7ff360097340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5b819d43c8eb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.452+0000 7ff3597fa700 1 -- 172.21.15.53:0/3741500331 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360061020 con 0x555bfb401f70 2023-11-06T01:35:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.452+0000 7ff3597fa700 1 -- 172.21.15.53:0/3741500331 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555bfb37dcb0 con 0x555bfb401f70 2023-11-06T01:35:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.452+0000 7ff3597fa700 1 -- 172.21.15.53:0/3741500331 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3600b8230 con 0x555bfb401f70 2023-11-06T01:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.453+0000 7ff37cf72b80 1 -- 172.21.15.53:0/3741500331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 msgr2=0x555bfb40b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.453+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/3741500331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 0x555bfb40b4d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff360098d20 tx=0x7ff360097340 comp rx=0 tx=0).stop 2023-11-06T01:35:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.454+0000 7ff37cf72b80 1 -- 172.21.15.53:0/3741500331 shutdown_connections 2023-11-06T01:35:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.454+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/3741500331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb402e60 0x555bfb40ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.454+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/3741500331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb401610 0x555bfb410a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.454+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/3741500331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 0x555bfb40b4d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.454+0000 7ff37cf72b80 1 -- 172.21.15.53:0/3741500331 >> 172.21.15.53:0/3741500331 conn(0x555bfb361070 msgr2=0x555bfb3fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.454+0000 7ff37cf72b80 1 -- 172.21.15.53:0/3741500331 shutdown_connections 2023-11-06T01:35:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.454+0000 7ff37cf72b80 1 -- 172.21.15.53:0/3741500331 wait complete. 2023-11-06T01:35:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.454+0000 7ff37cf72b80 1 -- start start 2023-11-06T01:35:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.454+0000 7ff37cf72b80 1 Processor -- start 2023-11-06T01:35:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401610 0x555bfb40ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 0x555bfb410500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff37cf72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb402e60 0x555bfb478140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb478680 con 0x555bfb401610 2023-11-06T01:35:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb4787c0 con 0x555bfb401f70 2023-11-06T01:35:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff37cf72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfb478950 con 0x555bfb402e60 2023-11-06T01:35:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff37cf72b80 10 client.0 _ll_register_callbacks cb 0x7ff36400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff36a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401610 0x555bfb40ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff369d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 0x555bfb410500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff36ad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb402e60 0x555bfb478140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff369d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 0x555bfb410500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:51988/0 (socket says 172.21.15.53:51988) 2023-11-06T01:35:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff369d9b700 1 -- 172.21.15.53:0/1260289055 learned_addr learned my addr 172.21.15.53:0/1260289055 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff36a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401610 0x555bfb40ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40548/0 (socket says 172.21.15.53:40548) 2023-11-06T01:35:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff36ad9d700 1 -- 172.21.15.53:0/1260289055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 msgr2=0x555bfb410500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff36ad9d700 1 --2- 172.21.15.53:0/1260289055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 0x555bfb410500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff36ad9d700 1 -- 172.21.15.53:0/1260289055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401610 msgr2=0x555bfb40ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff36ad9d700 1 --2- 172.21.15.53:0/1260289055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401610 0x555bfb40ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.455+0000 7ff36ad9d700 1 -- 172.21.15.53:0/1260289055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36c06aa10 con 0x555bfb402e60 2023-11-06T01:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.456+0000 7ff36ad9d700 1 --2- 172.21.15.53:0/1260289055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb402e60 0x555bfb478140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x555bfb427f70 tx=0x7ff36400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63f27628f68ed19e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.456+0000 7ff351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.456+0000 7ff351ffb700 1 -- 172.21.15.53:0/1260289055 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364021070 con 0x555bfb402e60 2023-11-06T01:35:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.456+0000 7ff351ffb700 1 -- 172.21.15.53:0/1260289055 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff3640076e0 con 0x555bfb402e60 2023-11-06T01:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1260289055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff36009cbc0 con 0x555bfb402e60 2023-11-06T01:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff351ffb700 1 -- 172.21.15.53:0/1260289055 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff36400e040 con 0x555bfb402e60 2023-11-06T01:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff37cf72b80 10 client.25399 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1260289055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555bfb1a71f0 con 0x555bfb402e60 2023-11-06T01:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff351ffb700 1 -- 172.21.15.53:0/1260289055 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36009cbc0 con 0x555bfb402e60 2023-11-06T01:35:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff351ffb700 1 -- 172.21.15.53:0/1260289055 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff36402c020 con 0x555bfb402e60 2023-11-06T01:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff37cf72b80 10 client.25399 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff37cf72b80 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff37cf72b80 20 client.25399 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/1260289055 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555bfb47a4c0 0x555bfb47c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff37cf72b80 4 client.25399 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.457+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1260289055 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555bfb34bce0 con 0x555bfb47a4c0 2023-11-06T01:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.458+0000 7ff36a59c700 1 --2- 172.21.15.53:0/1260289055 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555bfb47a4c0 0x555bfb47c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.458+0000 7ff351ffb700 10 client.25399 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.458+0000 7ff36a59c700 1 --2- 172.21.15.53:0/1260289055 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555bfb47a4c0 0x555bfb47c8e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5882e55d913686ea server_cookie=ea183a3590cdf639 in_seq=0 out_seq=0 2023-11-06T01:35:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.481+0000 7ff351ffb700 1 -- 172.21.15.53:0/1260289055 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555bfb34bce0 con 0x555bfb47a4c0 2023-11-06T01:35:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.481+0000 7ff351ffb700 10 client.25399 handle_command_reply: tid=0 2023-11-06T01:35:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.481+0000 7ff37cf72b80 1 client.25399 shutdown 2023-11-06T01:35:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1260289055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb402e60 msgr2=0x555bfb478140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/1260289055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb402e60 0x555bfb478140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x555bfb427f70 tx=0x7ff36400bf10 comp rx=0 tx=0).stop 2023-11-06T01:35:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1260289055 shutdown_connections 2023-11-06T01:35:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/1260289055 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555bfb47a4c0 0x555bfb47c8e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/1260289055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555bfb402e60 0x555bfb478140 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/1260289055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555bfb401610 0x555bfb40ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff37cf72b80 1 --2- 172.21.15.53:0/1260289055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bfb401f70 0x555bfb410500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff351ffb700 0 client.25399 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1260289055 >> 172.21.15.53:0/1260289055 conn(0x555bfb361070 msgr2=0x555bfb40b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1260289055 shutdown_connections 2023-11-06T01:35:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff37cf72b80 1 -- 172.21.15.53:0/1260289055 wait complete. 2023-11-06T01:35:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.482+0000 7ff37cf72b80 20 client.25399 trim_cache size 0 max 16384 2023-11-06T01:35:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:35:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:35:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:35:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:35:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 -- 172.21.15.53:0/2311466741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff3540a0520 msgr2=0x7ff3540a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff3540a0520 0x7ff3540a29e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff36c100460 tx=0x7ff35c010820 comp rx=0 tx=0).stop 2023-11-06T01:35:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 -- 172.21.15.53:0/2311466741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff36c100600 msgr2=0x7ff36c1970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff36c100600 0x7ff36c1970c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff36000b530 tx=0x7ff36000b840 comp rx=0 tx=0).stop 2023-11-06T01:35:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 -- 172.21.15.53:0/2311466741 shutdown_connections 2023-11-06T01:35:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff3540a0520 0x7ff3540a29e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff36c100fa0 0x7ff36c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff36c0ff410 0x7ff36c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 --2- 172.21.15.53:0/2311466741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff36c100600 0x7ff36c1970c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 -- 172.21.15.53:0/2311466741 >> 172.21.15.53:0/2311466741 conn(0x7ff36c0fa960 msgr2=0x7ff36c103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 -- 172.21.15.53:0/2311466741 shutdown_connections 2023-11-06T01:35:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:45.483+0000 7ff370de2700 1 -- 172.21.15.53:0/2311466741 wait complete. 2023-11-06T01:35:45.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:45 smithi062 ceph-mon[103924]: pgmap v353: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 875 op/s 2023-11-06T01:35:45.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:45 smithi163 ceph-mon[104416]: pgmap v353: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 875 op/s 2023-11-06T01:35:45.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:45 smithi053 ceph-mon[99472]: pgmap v353: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 875 op/s 2023-11-06T01:35:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 90.0 s, 3739.6 tps, lat 25.729 ms stddev 27.521 2023-11-06T01:35:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:47 smithi062 ceph-mon[103924]: pgmap v354: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 1.07k op/s 2023-11-06T01:35:47.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:47 smithi163 ceph-mon[104416]: pgmap v354: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 1.07k op/s 2023-11-06T01:35:47.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:47 smithi053 ceph-mon[99472]: pgmap v354: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 1.07k op/s 2023-11-06T01:35:49.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:49 smithi062 ceph-mon[103924]: pgmap v355: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 28 MiB/s wr, 1.07k op/s 2023-11-06T01:35:49.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:49 smithi163 ceph-mon[104416]: pgmap v355: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 28 MiB/s wr, 1.07k op/s 2023-11-06T01:35:49.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:49 smithi053 ceph-mon[99472]: pgmap v355: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 28 MiB/s wr, 1.07k op/s 2023-11-06T01:35:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:51 smithi062 ceph-mon[103924]: pgmap v356: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 37 MiB/s wr, 1.44k op/s 2023-11-06T01:35:51.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:51 smithi163 ceph-mon[104416]: pgmap v356: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 37 MiB/s wr, 1.44k op/s 2023-11-06T01:35:51.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:51 smithi053 ceph-mon[99472]: pgmap v356: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 37 MiB/s wr, 1.44k op/s 2023-11-06T01:35:52.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 95.0 s, 3674.8 tps, lat 28.289 ms stddev 52.027 2023-11-06T01:35:53.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:53 smithi062 ceph-mon[103924]: pgmap v357: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 MiB/s wr, 2.14k op/s 2023-11-06T01:35:53.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:53 smithi163 ceph-mon[104416]: pgmap v357: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 MiB/s wr, 2.14k op/s 2023-11-06T01:35:53.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:53 smithi053 ceph-mon[99472]: pgmap v357: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 MiB/s wr, 2.14k op/s 2023-11-06T01:35:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:55 smithi062 ceph-mon[103924]: pgmap v358: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 50 MiB/s wr, 1.95k op/s 2023-11-06T01:35:55.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:55 smithi163 ceph-mon[104416]: pgmap v358: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 50 MiB/s wr, 1.95k op/s 2023-11-06T01:35:55.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:55 smithi053 ceph-mon[99472]: pgmap v358: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 50 MiB/s wr, 1.95k op/s 2023-11-06T01:35:57.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 100.0 s, 4013.0 tps, lat 24.943 ms stddev 25.798 2023-11-06T01:35:57.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:57 smithi062 ceph-mon[103924]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 2.09k op/s 2023-11-06T01:35:57.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:57 smithi163 ceph-mon[104416]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 2.09k op/s 2023-11-06T01:35:57.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:57 smithi053 ceph-mon[99472]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 2.09k op/s 2023-11-06T01:35:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.655+0000 7f3fec45f700 1 -- 172.21.15.53:0/2392219474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe410d0d0 msgr2=0x7f3fe410d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.655+0000 7f3fec45f700 1 --2- 172.21.15.53:0/2392219474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe410d0d0 0x7f3fe410d550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc01c0d0 tx=0x7f3fdc01c3e0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.655+0000 7f3fec45f700 1 -- 172.21.15.53:0/2392219474 shutdown_connections 2023-11-06T01:35:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.655+0000 7f3fec45f700 1 --2- 172.21.15.53:0/2392219474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe410d0d0 0x7f3fe410d550 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.655+0000 7f3fec45f700 1 --2- 172.21.15.53:0/2392219474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4107d90 0x7f3fe410a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.655+0000 7f3fec45f700 1 --2- 172.21.15.53:0/2392219474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe410a700 0x7f3fe410cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.655+0000 7f3fec45f700 1 -- 172.21.15.53:0/2392219474 >> 172.21.15.53:0/2392219474 conn(0x7f3fe406db80 msgr2=0x7f3fe406ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.655+0000 7f3fec45f700 1 -- 172.21.15.53:0/2392219474 shutdown_connections 2023-11-06T01:35:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.656+0000 7f3fec45f700 1 -- 172.21.15.53:0/2392219474 wait complete. 2023-11-06T01:35:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.656+0000 7f3fec45f700 1 Processor -- start 2023-11-06T01:35:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.656+0000 7f3fec45f700 1 -- start start 2023-11-06T01:35:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.657+0000 7f3fec45f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4107d90 0x7f3fe419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.657+0000 7f3fec45f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe410a700 0x7f3fe419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.657+0000 7f3fec45f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe410d0d0 0x7f3fe41a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.657+0000 7f3fec45f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe41a4b60 con 0x7f3fe4107d90 2023-11-06T01:35:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.657+0000 7f3fec45f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe41a4ca0 con 0x7f3fe410a700 2023-11-06T01:35:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.657+0000 7f3fec45f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe41a4e30 con 0x7f3fe410d0d0 2023-11-06T01:35:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.657+0000 7f3fea1fb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4107d90 0x7f3fe419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.657+0000 7f3fea1fb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4107d90 0x7f3fe419cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40562/0 (socket says 172.21.15.53:40562) 2023-11-06T01:35:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.657+0000 7f3fea1fb700 1 -- 172.21.15.53:0/169486036 learned_addr learned my addr 172.21.15.53:0/169486036 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.658+0000 7f3fea1fb700 1 -- 172.21.15.53:0/169486036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe410d0d0 msgr2=0x7f3fe41a4590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:35:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.658+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe410d0d0 0x7f3fe41a4590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.658+0000 7f3fea1fb700 1 -- 172.21.15.53:0/169486036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe410a700 msgr2=0x7f3fe419d520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:35:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.658+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe410a700 0x7f3fe419d520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.658+0000 7f3fea1fb700 1 -- 172.21.15.53:0/169486036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc01ad10 con 0x7f3fe4107d90 2023-11-06T01:35:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.658+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4107d90 0x7f3fe419cfe0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000d950 tx=0x7f3fe000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.658+0000 7f3fd77fe700 1 -- 172.21.15.53:0/169486036 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0009b40 con 0x7f3fe4107d90 2023-11-06T01:35:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.659+0000 7f3fd77fe700 1 -- 172.21.15.53:0/169486036 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fe0009ca0 con 0x7f3fe4107d90 2023-11-06T01:35:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.659+0000 7f3fec45f700 1 -- 172.21.15.53:0/169486036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe41a5110 con 0x7f3fe4107d90 2023-11-06T01:35:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.659+0000 7f3fec45f700 1 -- 172.21.15.53:0/169486036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe41a5620 con 0x7f3fe4107d90 2023-11-06T01:35:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.659+0000 7f3fd77fe700 1 -- 172.21.15.53:0/169486036 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000f620 con 0x7f3fe4107d90 2023-11-06T01:35:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.663+0000 7f3fd77fe700 1 -- 172.21.15.53:0/169486036 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f3fe000f780 con 0x7f3fe4107d90 2023-11-06T01:35:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.664+0000 7f3ff85efb80 1 Processor -- start 2023-11-06T01:35:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.664+0000 7f3ff85efb80 1 -- start start 2023-11-06T01:35:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.664+0000 7f3fd77fe700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3fd00a0520 0x7f3fd00a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.664+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c3baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.664+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 0x55cb70c3c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.664+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3ced0 0x55cb70c3d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.664+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70b939b0 con 0x55cb70c3bfe0 2023-11-06T01:35:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.664+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70b91370 con 0x55cb70c3b680 2023-11-06T01:35:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.664+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70b8cc60 con 0x55cb70c3ced0 2023-11-06T01:35:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.664+0000 7f3fd77fe700 1 -- 172.21.15.53:0/169486036 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3fe00cbd70 con 0x7f3fe4107d90 2023-11-06T01:35:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.664+0000 7f3fea1fb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c3baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.665+0000 7f3fea9fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3ced0 0x55cb70c3d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.665+0000 7f3fea9fc700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3fd00a0520 0x7f3fd00a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.665+0000 7f3fea9fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3ced0 0x55cb70c3d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54810/0 (socket says 172.21.15.53:54810) 2023-11-06T01:35:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.665+0000 7f3fea9fc700 1 -- 172.21.15.53:0/383635325 learned_addr learned my addr 172.21.15.53:0/383635325 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.666+0000 7f3fe99fa700 1 --2- 172.21.15.53:0/383635325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 0x55cb70c3c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.666+0000 7f3fea1fb700 1 -- 172.21.15.53:0/383635325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3ced0 msgr2=0x55cb70c3d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.666+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/383635325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3ced0 0x55cb70c3d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.666+0000 7f3fea1fb700 1 -- 172.21.15.53:0/383635325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 msgr2=0x55cb70c3c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.666+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/383635325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 0x55cb70c3c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.666+0000 7f3fea1fb700 1 -- 172.21.15.53:0/383635325 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb709bad80 con 0x55cb70c3b680 2023-11-06T01:35:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.666+0000 7f3fe99fa700 1 --2- 172.21.15.53:0/383635325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 0x55cb70c3c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:35:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.667+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/383635325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c3baa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00c9850 tx=0x7f3fe0095a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49744ed03ff48a11 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.667+0000 7f3fea9fc700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3fd00a0520 0x7f3fd00a29e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55cb70c3c6b0 tx=0x7f3fdc01a9f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.667+0000 7f3fd4daa700 1 -- 172.21.15.53:0/383635325 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0099cb0 con 0x55cb70c3b680 2023-11-06T01:35:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.667+0000 7f3fd4daa700 1 -- 172.21.15.53:0/383635325 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fe001a780 con 0x55cb70c3b680 2023-11-06T01:35:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.667+0000 7f3fd4daa700 1 -- 172.21.15.53:0/383635325 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00850b0 con 0x55cb70c3b680 2023-11-06T01:35:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.668+0000 7f3ff85efb80 1 -- 172.21.15.53:0/383635325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 msgr2=0x55cb70c3baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.668+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/383635325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c3baa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00c9850 tx=0x7f3fe0095a20 comp rx=0 tx=0).stop 2023-11-06T01:35:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.668+0000 7f3ff85efb80 1 -- 172.21.15.53:0/383635325 shutdown_connections 2023-11-06T01:35:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.668+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/383635325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c3baa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.668+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/383635325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3ced0 0x55cb70c3d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.668+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/383635325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 0x55cb70c3c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.668+0000 7f3ff85efb80 1 -- 172.21.15.53:0/383635325 >> 172.21.15.53:0/383635325 conn(0x55cb70b9b070 msgr2=0x55cb70c38040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.669+0000 7f3ff85efb80 1 -- 172.21.15.53:0/383635325 shutdown_connections 2023-11-06T01:35:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.669+0000 7f3ff85efb80 1 -- 172.21.15.53:0/383635325 wait complete. 2023-11-06T01:35:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.669+0000 7f3ff85efb80 1 -- start start 2023-11-06T01:35:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.669+0000 7f3ff85efb80 1 Processor -- start 2023-11-06T01:35:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3b680 0x55cb70c45490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3bfe0 0x55cb70c459d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3ced0 0x55cb70c4ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70b294b0 con 0x55cb70c3ced0 2023-11-06T01:35:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70b29150 con 0x55cb70c3bfe0 2023-11-06T01:35:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70a108c0 con 0x55cb70c3b680 2023-11-06T01:35:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3ff85efb80 10 client.0 _ll_register_callbacks cb 0x55cb70a76a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3fe99fa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3bfe0 0x55cb70c459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3fea1fb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3b680 0x55cb70c45490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3fe99fa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3bfe0 0x55cb70c459d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52012/0 (socket says 172.21.15.53:52012) 2023-11-06T01:35:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3fea1fb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3b680 0x55cb70c45490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54814/0 (socket says 172.21.15.53:54814) 2023-11-06T01:35:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3fe99fa700 1 -- 172.21.15.53:0/3953110756 learned_addr learned my addr 172.21.15.53:0/3953110756 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3fe99fa700 1 -- 172.21.15.53:0/3953110756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3b680 msgr2=0x55cb70c45490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3fe99fa700 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3b680 0x55cb70c45490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3fe99fa700 1 -- 172.21.15.53:0/3953110756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3ced0 msgr2=0x55cb70c4ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3fe99fa700 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3ced0 0x55cb70c4ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.670+0000 7f3fe99fa700 1 -- 172.21.15.53:0/3953110756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe41a5620 con 0x55cb70c3bfe0 2023-11-06T01:35:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.671+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3b680 0x55cb70c45490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:35:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.671+0000 7f3fe99fa700 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3bfe0 0x55cb70c459d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8010620 tx=0x7f3fd80109e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5f17bb9aa42d4e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.671+0000 7f3fcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:35:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.671+0000 7f3fcdffb700 1 -- 172.21.15.53:0/3953110756 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8004a30 con 0x55cb70c3bfe0 2023-11-06T01:35:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.672+0000 7f3fcdffb700 1 -- 172.21.15.53:0/3953110756 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3fd8016070 con 0x55cb70c3bfe0 2023-11-06T01:35:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.672+0000 7f3ff85efb80 1 -- 172.21.15.53:0/3953110756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb70bb7cb0 con 0x55cb70c3bfe0 2023-11-06T01:35:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.672+0000 7f3fcdffb700 1 -- 172.21.15.53:0/3953110756 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd8004b90 con 0x55cb70c3bfe0 2023-11-06T01:35:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.672+0000 7f3ff85efb80 10 client.25322 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.672+0000 7f3ff85efb80 1 -- 172.21.15.53:0/3953110756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55cb709a3c00 con 0x55cb70c3bfe0 2023-11-06T01:35:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.673+0000 7f3fcdffb700 1 -- 172.21.15.53:0/3953110756 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb70bb7cb0 con 0x55cb70c3bfe0 2023-11-06T01:35:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.673+0000 7f3fcdffb700 1 -- 172.21.15.53:0/3953110756 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3fd8005180 con 0x55cb70c3bfe0 2023-11-06T01:35:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.673+0000 7f3ff85efb80 10 client.25322 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.673+0000 7f3ff85efb80 10 client.25322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.673+0000 7f3ff85efb80 20 client.25322 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.673+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cb70c51350 0x55cb70c53770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.673+0000 7f3ff85efb80 4 client.25322 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:35:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.673+0000 7f3ff85efb80 1 -- 172.21.15.53:0/3953110756 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb70c0adb0 con 0x55cb70c51350 2023-11-06T01:35:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.673+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cb70c51350 0x55cb70c53770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.673+0000 7f3fcdffb700 10 client.25322 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.673+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cb70c51350 0x55cb70c53770 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782a8a96013894dd server_cookie=a6b8166c64b2776b in_seq=0 out_seq=0 2023-11-06T01:35:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.674+0000 7f3fcdffb700 1 -- 172.21.15.53:0/3953110756 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cb70c0adb0 con 0x55cb70c51350 2023-11-06T01:35:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.674+0000 7f3fcdffb700 10 client.25322 handle_command_reply: tid=0 2023-11-06T01:35:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.674+0000 7f3ff85efb80 1 client.25322 shutdown 2023-11-06T01:35:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.674+0000 7f3ff85efb80 1 -- 172.21.15.53:0/3953110756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3bfe0 msgr2=0x55cb70c459d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.674+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3bfe0 0x55cb70c459d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8010620 tx=0x7f3fd80109e0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.675+0000 7f3ff85efb80 1 -- 172.21.15.53:0/3953110756 shutdown_connections 2023-11-06T01:35:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.675+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cb70c51350 0x55cb70c53770 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.675+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3ced0 0x55cb70c4ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.675+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3b680 0x55cb70c45490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.675+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/3953110756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3bfe0 0x55cb70c459d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.675+0000 7f3ff85efb80 1 -- 172.21.15.53:0/3953110756 >> 172.21.15.53:0/3953110756 conn(0x55cb70b9b070 msgr2=0x55cb70c363c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.675+0000 7f3fcdffb700 0 client.25322 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.675+0000 7f3ff85efb80 1 -- 172.21.15.53:0/3953110756 shutdown_connections 2023-11-06T01:35:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.675+0000 7f3ff85efb80 1 -- 172.21.15.53:0/3953110756 wait complete. 2023-11-06T01:35:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.675+0000 7f3ff85efb80 20 client.25322 trim_cache size 0 max 16384 2023-11-06T01:35:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3ff85efb80 1 Processor -- start 2023-11-06T01:35:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3ff85efb80 1 -- start start 2023-11-06T01:35:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c4a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3bfe0 0x55cb70c43320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3ced0 0x55cb70c53360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70c45000 con 0x55cb70c3ced0 2023-11-06T01:35:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70c37840 con 0x55cb70c3b680 2023-11-06T01:35:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70c463e0 con 0x55cb70c3bfe0 2023-11-06T01:35:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3fe99fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3bfe0 0x55cb70c43320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3fea9fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3ced0 0x55cb70c53360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3fe99fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3bfe0 0x55cb70c43320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54824/0 (socket says 172.21.15.53:54824) 2023-11-06T01:35:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3fe99fa700 1 -- 172.21.15.53:0/2514261240 learned_addr learned my addr 172.21.15.53:0/2514261240 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.701+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/2514261240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c4a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.702+0000 7f3fea9fc700 1 -- 172.21.15.53:0/2514261240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3bfe0 msgr2=0x55cb70c43320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.702+0000 7f3fea9fc700 1 --2- 172.21.15.53:0/2514261240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3bfe0 0x55cb70c43320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.702+0000 7f3fea9fc700 1 -- 172.21.15.53:0/2514261240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 msgr2=0x55cb70c4a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.702+0000 7f3fea9fc700 1 --2- 172.21.15.53:0/2514261240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c4a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.702+0000 7f3fea9fc700 1 -- 172.21.15.53:0/2514261240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb709e11f0 con 0x55cb70c3ced0 2023-11-06T01:35:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.702+0000 7f3fe99fa700 1 --2- 172.21.15.53:0/2514261240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3bfe0 0x55cb70c43320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:35:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.702+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/2514261240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c4a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:35:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.702+0000 7f3fea9fc700 1 --2- 172.21.15.53:0/2514261240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3ced0 0x55cb70c53360 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc01a860 tx=0x7f3fdc01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e441e365b27626c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.703+0000 7f3fd55ab700 1 -- 172.21.15.53:0/2514261240 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc01f070 con 0x55cb70c3ced0 2023-11-06T01:35:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.703+0000 7f3fd55ab700 1 -- 172.21.15.53:0/2514261240 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fdc01a6b0 con 0x55cb70c3ced0 2023-11-06T01:35:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.703+0000 7f3fd55ab700 1 -- 172.21.15.53:0/2514261240 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc02d760 con 0x55cb70c3ced0 2023-11-06T01:35:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.704+0000 7f3ff85efb80 1 -- 172.21.15.53:0/2514261240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3ced0 msgr2=0x55cb70c53360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.704+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/2514261240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3ced0 0x55cb70c53360 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc01a860 tx=0x7f3fdc01c730 comp rx=0 tx=0).stop 2023-11-06T01:35:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.704+0000 7f3ff85efb80 1 -- 172.21.15.53:0/2514261240 shutdown_connections 2023-11-06T01:35:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.704+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/2514261240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c4a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.704+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/2514261240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3ced0 0x55cb70c53360 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.704+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/2514261240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3bfe0 0x55cb70c43320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.704+0000 7f3ff85efb80 1 -- 172.21.15.53:0/2514261240 >> 172.21.15.53:0/2514261240 conn(0x55cb70b9b070 msgr2=0x55cb70c37c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.704+0000 7f3ff85efb80 1 -- 172.21.15.53:0/2514261240 shutdown_connections 2023-11-06T01:35:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.704+0000 7f3ff85efb80 1 -- 172.21.15.53:0/2514261240 wait complete. 2023-11-06T01:35:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.704+0000 7f3ff85efb80 1 -- start start 2023-11-06T01:35:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.705+0000 7f3ff85efb80 1 Processor -- start 2023-11-06T01:35:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.705+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c9ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.705+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 0x55cb70c9f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.705+0000 7f3ff85efb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3ced0 0x55cb70ca18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.705+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70ca1e10 con 0x55cb70c3bfe0 2023-11-06T01:35:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.705+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70ca1f50 con 0x55cb70c3b680 2023-11-06T01:35:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.705+0000 7f3ff85efb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb70ca2110 con 0x55cb70c3ced0 2023-11-06T01:35:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.705+0000 7f3ff85efb80 10 client.0 _ll_register_callbacks cb 0x7f3fd8009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:35:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.706+0000 7f3fe99fa700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 0x55cb70c9f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.706+0000 7f3fea1fb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c9ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.706+0000 7f3fe99fa700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 0x55cb70c9f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40588/0 (socket says 172.21.15.53:40588) 2023-11-06T01:35:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.706+0000 7f3fe99fa700 1 -- 172.21.15.53:0/141450018 learned_addr learned my addr 172.21.15.53:0/141450018 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:35:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.706+0000 7f3fea9fc700 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3ced0 0x55cb70ca18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.706+0000 7f3fe99fa700 1 -- 172.21.15.53:0/141450018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3ced0 msgr2=0x55cb70ca18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.706+0000 7f3fe99fa700 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3ced0 0x55cb70ca18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.706+0000 7f3fe99fa700 1 -- 172.21.15.53:0/141450018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 msgr2=0x55cb70c9ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:35:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.706+0000 7f3fe99fa700 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c9ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.706+0000 7f3fe99fa700 1 -- 172.21.15.53:0/141450018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb709bad80 con 0x55cb70c3bfe0 2023-11-06T01:35:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.707+0000 7f3fe99fa700 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 0x55cb70c9f380 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8000c00 tx=0x7f3fd801ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f08f647eec40c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:35:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.707+0000 7f3fcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:35:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.707+0000 7f3fcdffb700 1 -- 172.21.15.53:0/141450018 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8005270 con 0x55cb70c3bfe0 2023-11-06T01:35:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.707+0000 7f3ff85efb80 1 -- 172.21.15.53:0/141450018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fe001a780 con 0x55cb70c3bfe0 2023-11-06T01:35:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.707+0000 7f3fcdffb700 1 -- 172.21.15.53:0/141450018 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3fd8016040 con 0x55cb70c3bfe0 2023-11-06T01:35:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.707+0000 7f3fcdffb700 1 -- 172.21.15.53:0/141450018 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd801a430 con 0x55cb70c3bfe0 2023-11-06T01:35:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.707+0000 7f3ff85efb80 10 client.15765 fetch_fsmap learned FSMap version 33 2023-11-06T01:35:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.707+0000 7f3ff85efb80 1 -- 172.21.15.53:0/141450018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55cb709a3c00 con 0x55cb70c3bfe0 2023-11-06T01:35:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.707+0000 7f3fcdffb700 1 -- 172.21.15.53:0/141450018 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fd803d080 con 0x55cb70c3bfe0 2023-11-06T01:35:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.708+0000 7f3fcdffb700 1 -- 172.21.15.53:0/141450018 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3fd8037030 con 0x55cb70c3bfe0 2023-11-06T01:35:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.708+0000 7f3ff85efb80 10 client.15765 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:35:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.708+0000 7f3ff85efb80 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:35:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.708+0000 7f3ff85efb80 20 client.15765 populate_metadata read hostname 'smithi053' 2023-11-06T01:35:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.708+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cb70ca3bd0 0x55cb70ca5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:35:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.708+0000 7f3ff85efb80 4 client.15765 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:35:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.708+0000 7f3ff85efb80 1 -- 172.21.15.53:0/141450018 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb70b85ce0 con 0x55cb70ca3bd0 2023-11-06T01:35:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.708+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cb70ca3bd0 0x55cb70ca5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:35:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.708+0000 7f3fcdffb700 10 client.15765 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.708+0000 7f3fea1fb700 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cb70ca3bd0 0x55cb70ca5ff0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57439f7c542a2f99 server_cookie=25c8024844fc2add in_seq=0 out_seq=0 2023-11-06T01:35:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.723+0000 7f3fcdffb700 1 -- 172.21.15.53:0/141450018 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb70b85ce0 con 0x55cb70ca3bd0 2023-11-06T01:35:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.723+0000 7f3fcdffb700 10 client.15765 handle_command_reply: tid=0 2023-11-06T01:35:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3ff85efb80 1 client.15765 shutdown 2023-11-06T01:35:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3ff85efb80 1 -- 172.21.15.53:0/141450018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 msgr2=0x55cb70c9f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 0x55cb70c9f380 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8000c00 tx=0x7f3fd801ab90 comp rx=0 tx=0).stop 2023-11-06T01:35:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3ff85efb80 1 -- 172.21.15.53:0/141450018 shutdown_connections 2023-11-06T01:35:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb70c3b680 0x55cb70c9ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cb70ca3bd0 0x55cb70ca5ff0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb70c3ced0 0x55cb70ca18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3ff85efb80 1 --2- 172.21.15.53:0/141450018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb70c3bfe0 0x55cb70c9f380 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3fcdffb700 0 client.15765 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:35:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3ff85efb80 1 -- 172.21.15.53:0/141450018 >> 172.21.15.53:0/141450018 conn(0x55cb70b9b070 msgr2=0x55cb70c4d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3ff85efb80 1 -- 172.21.15.53:0/141450018 shutdown_connections 2023-11-06T01:35:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.724+0000 7f3ff85efb80 1 -- 172.21.15.53:0/141450018 wait complete. 2023-11-06T01:35:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.725+0000 7f3ff85efb80 20 client.15765 trim_cache size 0 max 16384 2023-11-06T01:35:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:35:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:35:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:35:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:35:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.725+0000 7f3fec45f700 1 -- 172.21.15.53:0/169486036 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3fd00a0520 msgr2=0x7f3fd00a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.725+0000 7f3fec45f700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3fd00a0520 0x7f3fd00a29e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55cb70c3c6b0 tx=0x7f3fdc01a9f0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.725+0000 7f3fec45f700 1 -- 172.21.15.53:0/169486036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4107d90 msgr2=0x7f3fe419cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:35:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.725+0000 7f3fec45f700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4107d90 0x7f3fe419cfe0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000d950 tx=0x7f3fe000dd10 comp rx=0 tx=0).stop 2023-11-06T01:35:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.726+0000 7f3fec45f700 1 -- 172.21.15.53:0/169486036 shutdown_connections 2023-11-06T01:35:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.726+0000 7f3fec45f700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3fd00a0520 0x7f3fd00a29e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.726+0000 7f3fec45f700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe410d0d0 0x7f3fe41a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.726+0000 7f3fec45f700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4107d90 0x7f3fe419cfe0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.726+0000 7f3fec45f700 1 --2- 172.21.15.53:0/169486036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe410a700 0x7f3fe419d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:35:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.726+0000 7f3fec45f700 1 -- 172.21.15.53:0/169486036 >> 172.21.15.53:0/169486036 conn(0x7f3fe406db80 msgr2=0x7f3fe4108790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:35:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.726+0000 7f3fec45f700 1 -- 172.21.15.53:0/169486036 shutdown_connections 2023-11-06T01:35:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:35:58.726+0000 7f3fec45f700 1 -- 172.21.15.53:0/169486036 wait complete. 2023-11-06T01:35:59.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:35:59 smithi062 ceph-mon[103924]: pgmap v360: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 MiB/s wr, 1.95k op/s 2023-11-06T01:35:59.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:35:59 smithi163 ceph-mon[104416]: pgmap v360: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 MiB/s wr, 1.95k op/s 2023-11-06T01:35:59.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:35:59 smithi053 ceph-mon[99472]: pgmap v360: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 MiB/s wr, 1.95k op/s 2023-11-06T01:36:01.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:01 smithi062 ceph-mon[103924]: pgmap v361: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 MiB/s wr, 1.83k op/s 2023-11-06T01:36:01.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:01 smithi163 ceph-mon[104416]: pgmap v361: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 MiB/s wr, 1.83k op/s 2023-11-06T01:36:01.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:01 smithi053 ceph-mon[99472]: pgmap v361: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 MiB/s wr, 1.83k op/s 2023-11-06T01:36:02.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 105.0 s, 4091.4 tps, lat 24.369 ms stddev 27.867 2023-11-06T01:36:03.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:03 smithi163 ceph-mon[104416]: pgmap v362: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 1.58k op/s 2023-11-06T01:36:03.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:03 smithi053 ceph-mon[99472]: pgmap v362: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 1.58k op/s 2023-11-06T01:36:03.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:03 smithi062 ceph-mon[103924]: pgmap v362: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 1.58k op/s 2023-11-06T01:36:04.506 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:36:04.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.700+0000 7fa878366700 1 -- 172.21.15.53:0/2534424249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87010d4e0 msgr2=0x7fa87010f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:04.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.700+0000 7fa878366700 1 --2- 172.21.15.53:0/2534424249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87010d4e0 0x7fa87010f9a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa8600074e0 tx=0x7fa86000af90 comp rx=0 tx=0).stop 2023-11-06T01:36:04.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.701+0000 7fa878366700 1 -- 172.21.15.53:0/2534424249 shutdown_connections 2023-11-06T01:36:04.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.701+0000 7fa878366700 1 --2- 172.21.15.53:0/2534424249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa870108000 0x7fa87010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.701+0000 7fa878366700 1 --2- 172.21.15.53:0/2534424249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87010d4e0 0x7fa87010f9a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.701+0000 7fa878366700 1 --2- 172.21.15.53:0/2534424249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87010a9c0 0x7fa87010ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.701+0000 7fa878366700 1 -- 172.21.15.53:0/2534424249 >> 172.21.15.53:0/2534424249 conn(0x7fa87006df90 msgr2=0x7fa8700703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:04.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.701+0000 7fa878366700 1 -- 172.21.15.53:0/2534424249 shutdown_connections 2023-11-06T01:36:04.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.701+0000 7fa878366700 1 -- 172.21.15.53:0/2534424249 wait complete. 2023-11-06T01:36:04.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.701+0000 7fa878366700 1 Processor -- start 2023-11-06T01:36:04.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.701+0000 7fa878366700 1 -- start start 2023-11-06T01:36:04.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa878366700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa870108000 0x7fa8701a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa878366700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87010a9c0 0x7fa8701a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa878366700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87010d4e0 0x7fa8701ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa878366700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8701ad440 con 0x7fa87010a9c0 2023-11-06T01:36:04.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa878366700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8701ad580 con 0x7fa870108000 2023-11-06T01:36:04.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa878366700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8701ad710 con 0x7fa87010d4e0 2023-11-06T01:36:04.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa875901700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87010a9c0 0x7fa8701a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa875901700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87010a9c0 0x7fa8701a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40604/0 (socket says 172.21.15.53:40604) 2023-11-06T01:36:04.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa875901700 1 -- 172.21.15.53:0/2949650949 learned_addr learned my addr 172.21.15.53:0/2949650949 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:04.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa876102700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa870108000 0x7fa8701a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa876903700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87010d4e0 0x7fa8701ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa875901700 1 -- 172.21.15.53:0/2949650949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87010d4e0 msgr2=0x7fa8701ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:04.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa875901700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87010d4e0 0x7fa8701ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa875901700 1 -- 172.21.15.53:0/2949650949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa870108000 msgr2=0x7fa8701a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:04.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa875901700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa870108000 0x7fa8701a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.702+0000 7fa875901700 1 -- 172.21.15.53:0/2949650949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600071c0 con 0x7fa87010a9c0 2023-11-06T01:36:04.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.704+0000 7fa875901700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87010a9c0 0x7fa8701a5e00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00e9a0 tx=0x7fa86c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:04.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.705+0000 7fa8677fe700 1 -- 172.21.15.53:0/2949650949 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c00ca10 con 0x7fa87010a9c0 2023-11-06T01:36:04.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.705+0000 7fa8677fe700 1 -- 172.21.15.53:0/2949650949 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa86c00cb70 con 0x7fa87010a9c0 2023-11-06T01:36:04.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.705+0000 7fa8677fe700 1 -- 172.21.15.53:0/2949650949 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c018680 con 0x7fa87010a9c0 2023-11-06T01:36:04.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.705+0000 7fa878366700 1 -- 172.21.15.53:0/2949650949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8701ad9f0 con 0x7fa87010a9c0 2023-11-06T01:36:04.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.705+0000 7fa878366700 1 -- 172.21.15.53:0/2949650949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8701adf00 con 0x7fa87010a9c0 2023-11-06T01:36:04.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.709+0000 7fa8677fe700 1 -- 172.21.15.53:0/2949650949 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa86c00cce0 con 0x7fa87010a9c0 2023-11-06T01:36:04.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.709+0000 7fa8677fe700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa85c0a0670 0x7fa85c0a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.710+0000 7fa8677fe700 1 -- 172.21.15.53:0/2949650949 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa86c014070 con 0x7fa87010a9c0 2023-11-06T01:36:04.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.710+0000 7fa876102700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa85c0a0670 0x7fa85c0a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.710+0000 7fa8844f6b80 1 Processor -- start 2023-11-06T01:36:04.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.710+0000 7fa8844f6b80 1 -- start start 2023-11-06T01:36:04.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c19530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c19a70 0x561440c17e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c18870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561440b6fa10 con 0x561440c19a70 2023-11-06T01:36:04.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561440b6d3d0 con 0x561440c183f0 2023-11-06T01:36:04.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561440b68cc0 con 0x561440c190f0 2023-11-06T01:36:04.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa876102700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c19530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa876102700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c19530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54850/0 (socket says 172.21.15.53:54850) 2023-11-06T01:36:04.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa876102700 1 -- 172.21.15.53:0/1018579501 learned_addr learned my addr 172.21.15.53:0/1018579501 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:04.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa876102700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa85c0a0670 0x7fa85c0a2b30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa868009550 tx=0x7fa8680091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:04.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa876102700 1 -- 172.21.15.53:0/1018579501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 msgr2=0x561440c18870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:36:04.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa876102700 1 --2- 172.21.15.53:0/1018579501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c18870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa876102700 1 -- 172.21.15.53:0/1018579501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c19a70 msgr2=0x561440c17e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:04.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa876102700 1 --2- 172.21.15.53:0/1018579501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c19a70 0x561440c17e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa876102700 1 -- 172.21.15.53:0/1018579501 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561440996de0 con 0x561440c190f0 2023-11-06T01:36:04.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.711+0000 7fa876102700 1 --2- 172.21.15.53:0/1018579501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c19530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa868004730 tx=0x7fa868013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da1915626135062d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:04.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.712+0000 7fa864daa700 1 -- 172.21.15.53:0/1018579501 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868021640 con 0x561440c190f0 2023-11-06T01:36:04.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.712+0000 7fa864daa700 1 -- 172.21.15.53:0/1018579501 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa86801a070 con 0x561440c190f0 2023-11-06T01:36:04.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.712+0000 7fa864daa700 1 -- 172.21.15.53:0/1018579501 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868015b60 con 0x561440c190f0 2023-11-06T01:36:04.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.713+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1018579501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 msgr2=0x561440c19530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:04.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.713+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/1018579501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c19530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa868004730 tx=0x7fa868013000 comp rx=0 tx=0).stop 2023-11-06T01:36:04.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.713+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1018579501 shutdown_connections 2023-11-06T01:36:04.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.713+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/1018579501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c18870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.713+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/1018579501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c19530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.713+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/1018579501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c19a70 0x561440c17e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.714+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1018579501 >> 172.21.15.53:0/1018579501 conn(0x561440b770d0 msgr2=0x561440c12940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:04.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.714+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1018579501 shutdown_connections 2023-11-06T01:36:04.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.714+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1018579501 wait complete. 2023-11-06T01:36:04.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.714+0000 7fa8844f6b80 1 -- start start 2023-11-06T01:36:04.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.714+0000 7fa8844f6b80 1 Processor -- start 2023-11-06T01:36:04.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c183f0 0x561440c21ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c22000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c19a70 0x561440c29040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561440b05510 con 0x561440c183f0 2023-11-06T01:36:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561440b051b0 con 0x561440c19a70 2023-11-06T01:36:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614409ec920 con 0x561440c190f0 2023-11-06T01:36:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa8844f6b80 10 client.0 _ll_register_callbacks cb 0x561440a52a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:04.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa875901700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c22000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa875901700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c22000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54856/0 (socket says 172.21.15.53:54856) 2023-11-06T01:36:04.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa875901700 1 -- 172.21.15.53:0/1075307851 learned_addr learned my addr 172.21.15.53:0/1075307851 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:04.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa876102700 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c183f0 0x561440c21ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.715+0000 7fa876903700 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c19a70 0x561440c29040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.716+0000 7fa876903700 1 -- 172.21.15.53:0/1075307851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 msgr2=0x561440c22000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:04.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.716+0000 7fa876903700 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c22000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.716+0000 7fa876903700 1 -- 172.21.15.53:0/1075307851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c183f0 msgr2=0x561440c21ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:04.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.716+0000 7fa876903700 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c183f0 0x561440c21ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.716+0000 7fa876903700 1 -- 172.21.15.53:0/1075307851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8680043e0 con 0x561440c19a70 2023-11-06T01:36:04.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.716+0000 7fa876903700 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c19a70 0x561440c29040 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa8600039a0 tx=0x7fa860005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95c6716fd79f0af server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:04.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.716+0000 7fa859ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:36:04.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.716+0000 7fa859ffb700 1 -- 172.21.15.53:0/1075307851 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000f070 con 0x561440c19a70 2023-11-06T01:36:04.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.717+0000 7fa859ffb700 1 -- 172.21.15.53:0/1075307851 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa860005240 con 0x561440c19a70 2023-11-06T01:36:04.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.717+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1075307851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561440b93d10 con 0x561440c19a70 2023-11-06T01:36:04.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.717+0000 7fa859ffb700 1 -- 172.21.15.53:0/1075307851 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa860005400 con 0x561440c19a70 2023-11-06T01:36:04.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.717+0000 7fa8844f6b80 10 client.25337 fetch_fsmap learned FSMap version 33 2023-11-06T01:36:04.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.717+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1075307851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56144097fc60 con 0x561440c19a70 2023-11-06T01:36:04.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.718+0000 7fa859ffb700 1 -- 172.21.15.53:0/1075307851 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561440b93d10 con 0x561440c19a70 2023-11-06T01:36:04.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.718+0000 7fa859ffb700 1 -- 172.21.15.53:0/1075307851 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa860004770 con 0x561440c19a70 2023-11-06T01:36:04.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.718+0000 7fa8844f6b80 10 client.25337 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:36:04.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.718+0000 7fa8844f6b80 10 client.25337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:04.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.718+0000 7fa8844f6b80 20 client.25337 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:04.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.718+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561440c2da10 0x561440c2fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.718+0000 7fa8844f6b80 4 client.25337 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:36:04.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.718+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1075307851 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561440be6e10 con 0x561440c2da10 2023-11-06T01:36:04.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.718+0000 7fa876102700 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561440c2da10 0x561440c2fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.719+0000 7fa876102700 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561440c2da10 0x561440c2fe30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45487c781ea6b05a server_cookie=86ce348b7c80e8a5 in_seq=0 out_seq=0 2023-11-06T01:36:04.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.719+0000 7fa859ffb700 10 client.25337 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:04.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa859ffb700 1 -- 172.21.15.53:0/1075307851 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561440be6e10 con 0x561440c2da10 2023-11-06T01:36:04.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa859ffb700 10 client.25337 handle_command_reply: tid=0 2023-11-06T01:36:04.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 1 client.25337 shutdown 2023-11-06T01:36:04.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1075307851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c19a70 msgr2=0x561440c29040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:04.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c19a70 0x561440c29040 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa8600039a0 tx=0x7fa860005dc0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1075307851 shutdown_connections 2023-11-06T01:36:04.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c19a70 0x561440c29040 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561440c2da10 0x561440c2fe30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c183f0 0x561440c21ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/1075307851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c22000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1075307851 >> 172.21.15.53:0/1075307851 conn(0x561440b770d0 msgr2=0x561440c120b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:04.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa859ffb700 0 client.25337 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:04.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1075307851 shutdown_connections 2023-11-06T01:36:04.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 1 -- 172.21.15.53:0/1075307851 wait complete. 2023-11-06T01:36:04.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.720+0000 7fa8844f6b80 20 client.25337 trim_cache size 0 max 16384 2023-11-06T01:36:04.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:04 smithi163 ceph-mon[104416]: pgmap v363: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 794 op/s 2023-11-06T01:36:04.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.743+0000 7fa8844f6b80 1 Processor -- start 2023-11-06T01:36:04.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.743+0000 7fa8844f6b80 1 -- start start 2023-11-06T01:36:04.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.743+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c17e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.743+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c26ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.743+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c19a70 0x561440c21ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.743+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561440b93d10 con 0x561440c19a70 2023-11-06T01:36:04.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.743+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561440c22cb0 con 0x561440c183f0 2023-11-06T01:36:04.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.743+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561440c2d550 con 0x561440c190f0 2023-11-06T01:36:04.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.743+0000 7fa876102700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c17e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.743+0000 7fa876102700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c17e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52062/0 (socket says 172.21.15.53:52062) 2023-11-06T01:36:04.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.744+0000 7fa876102700 1 -- 172.21.15.53:0/935609614 learned_addr learned my addr 172.21.15.53:0/935609614 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:04.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.744+0000 7fa876903700 1 --2- 172.21.15.53:0/935609614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c19a70 0x561440c21ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.744+0000 7fa876102700 1 -- 172.21.15.53:0/935609614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 msgr2=0x561440c26ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:04.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.744+0000 7fa876102700 1 --2- 172.21.15.53:0/935609614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c26ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.744+0000 7fa876102700 1 -- 172.21.15.53:0/935609614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c19a70 msgr2=0x561440c21ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:04.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.744+0000 7fa876102700 1 --2- 172.21.15.53:0/935609614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c19a70 0x561440c21ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.744+0000 7fa876102700 1 -- 172.21.15.53:0/935609614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561440c18aa0 con 0x561440c183f0 2023-11-06T01:36:04.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.744+0000 7fa876102700 1 --2- 172.21.15.53:0/935609614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c17e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa868009690 tx=0x7fa868015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca8b4573d0f6471 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:04.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.744+0000 7fa8655ab700 1 -- 172.21.15.53:0/935609614 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868015980 con 0x561440c183f0 2023-11-06T01:36:04.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.745+0000 7fa8655ab700 1 -- 172.21.15.53:0/935609614 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa86800f040 con 0x561440c183f0 2023-11-06T01:36:04.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.745+0000 7fa8655ab700 1 -- 172.21.15.53:0/935609614 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868035de0 con 0x561440c183f0 2023-11-06T01:36:04.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.745+0000 7fa8844f6b80 1 -- 172.21.15.53:0/935609614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 msgr2=0x561440c17e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:04.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.745+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/935609614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c17e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa868009690 tx=0x7fa868015000 comp rx=0 tx=0).stop 2023-11-06T01:36:04.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.746+0000 7fa8844f6b80 1 -- 172.21.15.53:0/935609614 shutdown_connections 2023-11-06T01:36:04.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.746+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/935609614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c17e60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.746+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/935609614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c19a70 0x561440c21ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.746+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/935609614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c190f0 0x561440c26ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.746+0000 7fa8844f6b80 1 -- 172.21.15.53:0/935609614 >> 172.21.15.53:0/935609614 conn(0x561440b770d0 msgr2=0x561440c1c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:04.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.746+0000 7fa8844f6b80 1 -- 172.21.15.53:0/935609614 shutdown_connections 2023-11-06T01:36:04.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.746+0000 7fa8844f6b80 1 -- 172.21.15.53:0/935609614 wait complete. 2023-11-06T01:36:04.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.746+0000 7fa8844f6b80 1 -- start start 2023-11-06T01:36:04.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa8844f6b80 1 Processor -- start 2023-11-06T01:36:04.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c88740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c190f0 0x561440c88c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa8844f6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c19a70 0x561440c8f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561440c8fdd0 con 0x561440c190f0 2023-11-06T01:36:04.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561440c8ff10 con 0x561440c183f0 2023-11-06T01:36:04.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa8844f6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561440c900d0 con 0x561440c19a70 2023-11-06T01:36:04.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa8844f6b80 10 client.0 _ll_register_callbacks cb 0x7fa860016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:04.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa876102700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c88740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa876102700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c88740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52068/0 (socket says 172.21.15.53:52068) 2023-11-06T01:36:04.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa876102700 1 -- 172.21.15.53:0/2174051217 learned_addr learned my addr 172.21.15.53:0/2174051217 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:04.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa875901700 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c190f0 0x561440c88c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.747+0000 7fa876903700 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c19a70 0x561440c8f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.748+0000 7fa876102700 1 -- 172.21.15.53:0/2174051217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c19a70 msgr2=0x561440c8f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:04.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.748+0000 7fa876102700 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c19a70 0x561440c8f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.748+0000 7fa876102700 1 -- 172.21.15.53:0/2174051217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c190f0 msgr2=0x561440c88c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:04.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.748+0000 7fa876102700 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c190f0 0x561440c88c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.748+0000 7fa876102700 1 -- 172.21.15.53:0/2174051217 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561440996de0 con 0x561440c183f0 2023-11-06T01:36:04.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.748+0000 7fa876102700 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c88740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa868035a50 tx=0x7fa868015440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61528b8edf47eec3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:04.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.748+0000 7fa859ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:36:04.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.748+0000 7fa859ffb700 1 -- 172.21.15.53:0/2174051217 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868035760 con 0x561440c183f0 2023-11-06T01:36:04.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.748+0000 7fa859ffb700 1 -- 172.21.15.53:0/2174051217 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa868004a40 con 0x561440c183f0 2023-11-06T01:36:04.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.748+0000 7fa8844f6b80 1 -- 172.21.15.53:0/2174051217 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa86801a070 con 0x561440c183f0 2023-11-06T01:36:04.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.750+0000 7fa859ffb700 1 -- 172.21.15.53:0/2174051217 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa868004c00 con 0x561440c183f0 2023-11-06T01:36:04.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.750+0000 7fa8844f6b80 10 client.25343 fetch_fsmap learned FSMap version 33 2023-11-06T01:36:04.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.750+0000 7fa8844f6b80 1 -- 172.21.15.53:0/2174051217 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5614409bd250 con 0x561440c183f0 2023-11-06T01:36:04.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.751+0000 7fa859ffb700 1 -- 172.21.15.53:0/2174051217 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa868046080 con 0x561440c183f0 2023-11-06T01:36:04.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.751+0000 7fa859ffb700 1 -- 172.21.15.53:0/2174051217 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa868040030 con 0x561440c183f0 2023-11-06T01:36:04.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.751+0000 7fa8844f6b80 10 client.25343 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:36:04.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.751+0000 7fa8844f6b80 10 client.25343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:04.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.751+0000 7fa8844f6b80 20 client.25343 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:04.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.751+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561440c91770 0x561440c93b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:04.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.751+0000 7fa8844f6b80 4 client.25343 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:36:04.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.751+0000 7fa8844f6b80 1 -- 172.21.15.53:0/2174051217 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561440b61d40 con 0x561440c91770 2023-11-06T01:36:04.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.752+0000 7fa875901700 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561440c91770 0x561440c93b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:04.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.752+0000 7fa875901700 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561440c91770 0x561440c93b90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98477ea71a7c7904 server_cookie=682fb97889b5f972 in_seq=0 out_seq=0 2023-11-06T01:36:04.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.752+0000 7fa859ffb700 10 client.25343 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:04.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.753+0000 7fa859ffb700 1 -- 172.21.15.53:0/2174051217 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561440b61d40 con 0x561440c91770 2023-11-06T01:36:04.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.753+0000 7fa859ffb700 10 client.25343 handle_command_reply: tid=0 2023-11-06T01:36:04.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.753+0000 7fa8844f6b80 1 client.25343 shutdown 2023-11-06T01:36:04.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.753+0000 7fa8844f6b80 1 -- 172.21.15.53:0/2174051217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 msgr2=0x561440c88740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:04.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.753+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c88740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa868035a50 tx=0x7fa868015440 comp rx=0 tx=0).stop 2023-11-06T01:36:04.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.754+0000 7fa8844f6b80 1 -- 172.21.15.53:0/2174051217 shutdown_connections 2023-11-06T01:36:04.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.754+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561440c183f0 0x561440c88740 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.754+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561440c91770 0x561440c93b90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.754+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561440c19a70 0x561440c8f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.754+0000 7fa8844f6b80 1 --2- 172.21.15.53:0/2174051217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561440c190f0 0x561440c88c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.754+0000 7fa8844f6b80 1 -- 172.21.15.53:0/2174051217 >> 172.21.15.53:0/2174051217 conn(0x561440b770d0 msgr2=0x561440c21a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:04.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.754+0000 7fa859ffb700 0 client.25343 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:04.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.754+0000 7fa8844f6b80 1 -- 172.21.15.53:0/2174051217 shutdown_connections 2023-11-06T01:36:04.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.754+0000 7fa8844f6b80 1 -- 172.21.15.53:0/2174051217 wait complete. 2023-11-06T01:36:04.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.754+0000 7fa8844f6b80 20 client.25343 trim_cache size 0 max 16384 2023-11-06T01:36:04.767 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:36:04.768 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:36:04.768 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:36:04.768 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:36:04.768 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:36:04.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:36:04.769 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:36:04.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:36:04.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:36:04.769 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:36:04.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.755+0000 7fa878366700 1 -- 172.21.15.53:0/2949650949 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa85c0a0670 msgr2=0x7fa85c0a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:04.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.755+0000 7fa878366700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa85c0a0670 0x7fa85c0a2b30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa868009550 tx=0x7fa8680091b0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.755+0000 7fa878366700 1 -- 172.21.15.53:0/2949650949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87010a9c0 msgr2=0x7fa8701a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.755+0000 7fa878366700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87010a9c0 0x7fa8701a5e00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00e9a0 tx=0x7fa86c00ecb0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.755+0000 7fa878366700 1 -- 172.21.15.53:0/2949650949 shutdown_connections 2023-11-06T01:36:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.755+0000 7fa878366700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa85c0a0670 0x7fa85c0a2b30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.755+0000 7fa878366700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa870108000 0x7fa8701a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.755+0000 7fa878366700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87010d4e0 0x7fa8701ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.755+0000 7fa878366700 1 --2- 172.21.15.53:0/2949650949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87010a9c0 0x7fa8701a5e00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.755+0000 7fa878366700 1 -- 172.21.15.53:0/2949650949 >> 172.21.15.53:0/2949650949 conn(0x7fa87006df90 msgr2=0x7fa870108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.755+0000 7fa878366700 1 -- 172.21.15.53:0/2949650949 shutdown_connections 2023-11-06T01:36:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:04.756+0000 7fa878366700 1 -- 172.21.15.53:0/2949650949 wait complete. 2023-11-06T01:36:04.777 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:04 smithi053 ceph-mon[99472]: pgmap v363: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 794 op/s 2023-11-06T01:36:04.778 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:36:04.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:04 smithi062 ceph-mon[103924]: pgmap v363: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 794 op/s 2023-11-06T01:36:07.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 110.0 s, 3140.4 tps, lat 31.914 ms stddev 35.562 2023-11-06T01:36:07.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:07 smithi053 ceph-mon[99472]: pgmap v364: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 946 op/s 2023-11-06T01:36:07.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:07 smithi062 ceph-mon[103924]: pgmap v364: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 946 op/s 2023-11-06T01:36:07.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:07 smithi163 ceph-mon[104416]: pgmap v364: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 946 op/s 2023-11-06T01:36:09.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:09 smithi053 ceph-mon[99472]: pgmap v365: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 825 op/s 2023-11-06T01:36:09.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:09 smithi062 ceph-mon[103924]: pgmap v365: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 825 op/s 2023-11-06T01:36:09.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:09 smithi163 ceph-mon[104416]: pgmap v365: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 825 op/s 2023-11-06T01:36:11.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:11 smithi053 ceph-mon[99472]: pgmap v366: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 852 op/s 2023-11-06T01:36:11.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:11 smithi062 ceph-mon[103924]: pgmap v366: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 852 op/s 2023-11-06T01:36:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:11 smithi163 ceph-mon[104416]: pgmap v366: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 852 op/s 2023-11-06T01:36:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.902+0000 7fda26891700 1 -- 172.21.15.53:0/2687167391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20108130 msgr2=0x7fda20108550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.902+0000 7fda26891700 1 --2- 172.21.15.53:0/2687167391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20108130 0x7fda20108550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda1000a930 tx=0x7fda1000ac40 comp rx=0 tx=0).stop 2023-11-06T01:36:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.903+0000 7fda26891700 1 -- 172.21.15.53:0/2687167391 shutdown_connections 2023-11-06T01:36:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.903+0000 7fda26891700 1 --2- 172.21.15.53:0/2687167391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20108130 0x7fda20108550 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.903+0000 7fda26891700 1 --2- 172.21.15.53:0/2687167391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda20109800 0x7fda20110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.903+0000 7fda26891700 1 --2- 172.21.15.53:0/2687167391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda20108e40 0x7fda201092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.903+0000 7fda26891700 1 -- 172.21.15.53:0/2687167391 >> 172.21.15.53:0/2687167391 conn(0x7fda2006dbc0 msgr2=0x7fda20070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.903+0000 7fda26891700 1 -- 172.21.15.53:0/2687167391 shutdown_connections 2023-11-06T01:36:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.903+0000 7fda26891700 1 -- 172.21.15.53:0/2687167391 wait complete. 2023-11-06T01:36:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.903+0000 7fda26891700 1 Processor -- start 2023-11-06T01:36:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.904+0000 7fda26891700 1 -- start start 2023-11-06T01:36:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.904+0000 7fda26891700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda20108130 0x7fda2019d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.904+0000 7fda26891700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda20109800 0x7fda2019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.904+0000 7fda26891700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2019e750 0x7fda2019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.904+0000 7fda26891700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2019f1a0 con 0x7fda20108130 2023-11-06T01:36:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.904+0000 7fda26891700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2019f310 con 0x7fda2019e750 2023-11-06T01:36:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.904+0000 7fda26891700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2019f4d0 con 0x7fda20109800 2023-11-06T01:36:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.905+0000 7fda24e2e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2019e750 0x7fda2019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.905+0000 7fda24e2e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2019e750 0x7fda2019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52080/0 (socket says 172.21.15.53:52080) 2023-11-06T01:36:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.905+0000 7fda24e2e700 1 -- 172.21.15.53:0/528539851 learned_addr learned my addr 172.21.15.53:0/528539851 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.905+0000 7fda1f7fe700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda20109800 0x7fda2019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.905+0000 7fda24e2e700 1 -- 172.21.15.53:0/528539851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda20109800 msgr2=0x7fda2019d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.905+0000 7fda24e2e700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda20109800 0x7fda2019d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.905+0000 7fda24e2e700 1 -- 172.21.15.53:0/528539851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda20108130 msgr2=0x7fda2019d390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:36:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.905+0000 7fda24e2e700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda20108130 0x7fda2019d390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.905+0000 7fda24e2e700 1 -- 172.21.15.53:0/528539851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1000a540 con 0x7fda2019e750 2023-11-06T01:36:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.905+0000 7fda24e2e700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2019e750 0x7fda2019ebd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fda1800eec0 tx=0x7fda180106a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.906+0000 7fda1d7fa700 1 -- 172.21.15.53:0/528539851 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1800e190 con 0x7fda2019e750 2023-11-06T01:36:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.906+0000 7fda1d7fa700 1 -- 172.21.15.53:0/528539851 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda18011040 con 0x7fda2019e750 2023-11-06T01:36:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.906+0000 7fda26891700 1 -- 172.21.15.53:0/528539851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda201a5a00 con 0x7fda2019e750 2023-11-06T01:36:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.906+0000 7fda1d7fa700 1 -- 172.21.15.53:0/528539851 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18009a30 con 0x7fda2019e750 2023-11-06T01:36:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.906+0000 7fda26891700 1 -- 172.21.15.53:0/528539851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda201a5f90 con 0x7fda2019e750 2023-11-06T01:36:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.910+0000 7fda1d7fa700 1 -- 172.21.15.53:0/528539851 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fda18009b90 con 0x7fda2019e750 2023-11-06T01:36:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.911+0000 7fda1d7fa700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda080a06a0 0x7fda080a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.911+0000 7fda1d7fa700 1 -- 172.21.15.53:0/528539851 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fda180cefc0 con 0x7fda2019e750 2023-11-06T01:36:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.911+0000 7fda32a21b80 1 Processor -- start 2023-11-06T01:36:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda32a21b80 1 -- start start 2023-11-06T01:36:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda1ffff700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda080a06a0 0x7fda080a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145030 0x5643c3145450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145990 0x5643c31462f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 0x5643c3146d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda32a21b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c309f9b0 con 0x5643c3145030 2023-11-06T01:36:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda32a21b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c309d370 con 0x5643c3145990 2023-11-06T01:36:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda32a21b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c3098c60 con 0x5643c3146880 2023-11-06T01:36:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda1ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145030 0x5643c3145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda1f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145990 0x5643c31462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda24e2e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 0x5643c3146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda1f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145990 0x5643c31462f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52086/0 (socket says 172.21.15.53:52086) 2023-11-06T01:36:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda1f7fe700 1 -- 172.21.15.53:0/2809541109 learned_addr learned my addr 172.21.15.53:0/2809541109 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.912+0000 7fda24e2e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 0x5643c3146d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54890/0 (socket says 172.21.15.53:54890) 2023-11-06T01:36:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.913+0000 7fda1ffff700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda080a06a0 0x7fda080a2b60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fda10017040 tx=0x7fda1000b8f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.913+0000 7fda1ffff700 1 -- 172.21.15.53:0/2809541109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 msgr2=0x5643c3146d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.913+0000 7fda1ffff700 1 --2- 172.21.15.53:0/2809541109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 0x5643c3146d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.913+0000 7fda1ffff700 1 -- 172.21.15.53:0/2809541109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145990 msgr2=0x5643c31462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.913+0000 7fda1ffff700 1 --2- 172.21.15.53:0/2809541109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145990 0x5643c31462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.913+0000 7fda1ffff700 1 -- 172.21.15.53:0/2809541109 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643c2ec6d80 con 0x5643c3145030 2023-11-06T01:36:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.913+0000 7fda24e2e700 1 --2- 172.21.15.53:0/2809541109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 0x5643c3146d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.913+0000 7fda1ffff700 1 --2- 172.21.15.53:0/2809541109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145030 0x5643c3145450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fda1000dd50 tx=0x7fda10027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0464000c71279d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.913+0000 7fda067fc700 1 -- 172.21.15.53:0/2809541109 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1000d440 con 0x5643c3145030 2023-11-06T01:36:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.913+0000 7fda067fc700 1 -- 172.21.15.53:0/2809541109 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1000da60 con 0x5643c3145030 2023-11-06T01:36:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.913+0000 7fda067fc700 1 -- 172.21.15.53:0/2809541109 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10024450 con 0x5643c3145030 2023-11-06T01:36:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.914+0000 7fda32a21b80 1 -- 172.21.15.53:0/2809541109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145030 msgr2=0x5643c3145450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.914+0000 7fda32a21b80 1 --2- 172.21.15.53:0/2809541109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145030 0x5643c3145450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fda1000dd50 tx=0x7fda10027000 comp rx=0 tx=0).stop 2023-11-06T01:36:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.914+0000 7fda32a21b80 1 -- 172.21.15.53:0/2809541109 shutdown_connections 2023-11-06T01:36:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.914+0000 7fda32a21b80 1 --2- 172.21.15.53:0/2809541109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 0x5643c3146d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.914+0000 7fda32a21b80 1 --2- 172.21.15.53:0/2809541109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145030 0x5643c3145450 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.914+0000 7fda32a21b80 1 --2- 172.21.15.53:0/2809541109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145990 0x5643c31462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.914+0000 7fda32a21b80 1 -- 172.21.15.53:0/2809541109 >> 172.21.15.53:0/2809541109 conn(0x5643c30a7070 msgr2=0x5643c3144040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.915+0000 7fda32a21b80 1 -- 172.21.15.53:0/2809541109 shutdown_connections 2023-11-06T01:36:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.915+0000 7fda32a21b80 1 -- 172.21.15.53:0/2809541109 wait complete. 2023-11-06T01:36:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.915+0000 7fda32a21b80 1 -- start start 2023-11-06T01:36:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda32a21b80 1 Processor -- start 2023-11-06T01:36:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c3151470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31519b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3146880 0x5643c31589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda32a21b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c30354b0 con 0x5643c3146880 2023-11-06T01:36:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda32a21b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c3035150 con 0x5643c3145030 2023-11-06T01:36:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda32a21b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c2f1c8c0 con 0x5643c3145990 2023-11-06T01:36:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda32a21b80 10 client.0 _ll_register_callbacks cb 0x5643c2f82a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda1f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c3151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda1f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54896/0 (socket says 172.21.15.53:54896) 2023-11-06T01:36:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda1f7fe700 1 -- 172.21.15.53:0/2400108265 learned_addr learned my addr 172.21.15.53:0/2400108265 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.916+0000 7fda24e2e700 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3146880 0x5643c31589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fda1f7fe700 1 -- 172.21.15.53:0/2400108265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 msgr2=0x5643c3151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fda1f7fe700 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c3151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fda1f7fe700 1 -- 172.21.15.53:0/2400108265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3146880 msgr2=0x5643c31589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fda1f7fe700 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3146880 0x5643c31589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fda1f7fe700 1 -- 172.21.15.53:0/2400108265 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda201a5f90 con 0x5643c3145990 2023-11-06T01:36:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fda1ffff700 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c3151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fda1f7fe700 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31519b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fda140104b0 tx=0x7fda140107c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f86197d1bd0b3e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fd9ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:36:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/2400108265 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14014430 con 0x5643c3145990 2023-11-06T01:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/2400108265 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fda14014590 con 0x5643c3145990 2023-11-06T01:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fda32a21b80 1 -- 172.21.15.53:0/2400108265 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643c30c3cb0 con 0x5643c3145990 2023-11-06T01:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/2400108265 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda14014750 con 0x5643c3145990 2023-11-06T01:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fda32a21b80 10 client.25438 fetch_fsmap learned FSMap version 33 2023-11-06T01:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.917+0000 7fda32a21b80 1 -- 172.21.15.53:0/2400108265 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fda201109c0 con 0x5643c3145990 2023-11-06T01:36:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.918+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/2400108265 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5643c30c3cb0 con 0x5643c3145990 2023-11-06T01:36:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.918+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/2400108265 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fda14013a00 con 0x5643c3145990 2023-11-06T01:36:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.918+0000 7fda32a21b80 10 client.25438 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:36:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.918+0000 7fda32a21b80 10 client.25438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.918+0000 7fda32a21b80 20 client.25438 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.918+0000 7fda32a21b80 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5643c315d3c0 0x5643c315f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.918+0000 7fda32a21b80 4 client.25438 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:36:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.918+0000 7fda32a21b80 1 -- 172.21.15.53:0/2400108265 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643c3116db0 con 0x5643c315d3c0 2023-11-06T01:36:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.918+0000 7fda1ffff700 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5643c315d3c0 0x5643c315f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.918+0000 7fd9ff7fe700 10 client.25438 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.918+0000 7fda1ffff700 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5643c315d3c0 0x5643c315f7e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a6814bb84ebcf2 server_cookie=e6b9294471a7ba1a in_seq=0 out_seq=0 2023-11-06T01:36:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/2400108265 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5643c3116db0 con 0x5643c315d3c0 2023-11-06T01:36:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fd9ff7fe700 10 client.25438 handle_command_reply: tid=0 2023-11-06T01:36:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fda32a21b80 1 client.25438 shutdown 2023-11-06T01:36:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fda32a21b80 1 -- 172.21.15.53:0/2400108265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 msgr2=0x5643c31519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fda32a21b80 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31519b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fda140104b0 tx=0x7fda140107c0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fda32a21b80 1 -- 172.21.15.53:0/2400108265 shutdown_connections 2023-11-06T01:36:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fda32a21b80 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c3151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fda32a21b80 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5643c315d3c0 0x5643c315f7e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fda32a21b80 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3146880 0x5643c31589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fda32a21b80 1 --2- 172.21.15.53:0/2400108265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31519b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fda32a21b80 1 -- 172.21.15.53:0/2400108265 >> 172.21.15.53:0/2400108265 conn(0x5643c30a7070 msgr2=0x5643c31423c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.919+0000 7fd9ff7fe700 0 client.25438 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.921+0000 7fda32a21b80 1 -- 172.21.15.53:0/2400108265 shutdown_connections 2023-11-06T01:36:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.921+0000 7fda32a21b80 1 -- 172.21.15.53:0/2400108265 wait complete. 2023-11-06T01:36:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.921+0000 7fda32a21b80 20 client.25438 trim_cache size 0 max 16384 2023-11-06T01:36:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.942+0000 7fda32a21b80 1 Processor -- start 2023-11-06T01:36:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.942+0000 7fda32a21b80 1 -- start start 2023-11-06T01:36:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.943+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c31569a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.943+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145990 0x5643c3146c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.943+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 0x5643c3143c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.943+0000 7fda32a21b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c315a660 con 0x5643c3145990 2023-11-06T01:36:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.943+0000 7fda32a21b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c314b960 con 0x5643c3145030 2023-11-06T01:36:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.943+0000 7fda32a21b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c313ad00 con 0x5643c3146880 2023-11-06T01:36:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.943+0000 7fda1f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145990 0x5643c3146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.943+0000 7fda1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c31569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.943+0000 7fda1f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145990 0x5643c3146c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40664/0 (socket says 172.21.15.53:40664) 2023-11-06T01:36:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.943+0000 7fda1f7fe700 1 -- 172.21.15.53:0/3364309406 learned_addr learned my addr 172.21.15.53:0/3364309406 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.943+0000 7fda24e2e700 1 --2- 172.21.15.53:0/3364309406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 0x5643c3143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.944+0000 7fda1f7fe700 1 -- 172.21.15.53:0/3364309406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 msgr2=0x5643c3143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.944+0000 7fda1f7fe700 1 --2- 172.21.15.53:0/3364309406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 0x5643c3143c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.944+0000 7fda1f7fe700 1 -- 172.21.15.53:0/3364309406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 msgr2=0x5643c31569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.944+0000 7fda1f7fe700 1 --2- 172.21.15.53:0/3364309406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c31569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.944+0000 7fda1f7fe700 1 -- 172.21.15.53:0/3364309406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643c2eafc00 con 0x5643c3145990 2023-11-06T01:36:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.944+0000 7fda1f7fe700 1 --2- 172.21.15.53:0/3364309406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145990 0x5643c3146c60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fda14007fd0 tx=0x7fda14007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58eb5e8f256e25ca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.944+0000 7fda06ffd700 1 -- 172.21.15.53:0/3364309406 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14010b10 con 0x5643c3145990 2023-11-06T01:36:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.944+0000 7fda06ffd700 1 -- 172.21.15.53:0/3364309406 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1401c3e0 con 0x5643c3145990 2023-11-06T01:36:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.944+0000 7fda06ffd700 1 -- 172.21.15.53:0/3364309406 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140139c0 con 0x5643c3145990 2023-11-06T01:36:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.945+0000 7fda32a21b80 1 -- 172.21.15.53:0/3364309406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145990 msgr2=0x5643c3146c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.945+0000 7fda32a21b80 1 --2- 172.21.15.53:0/3364309406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145990 0x5643c3146c60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fda14007fd0 tx=0x7fda14007500 comp rx=0 tx=0).stop 2023-11-06T01:36:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.945+0000 7fda32a21b80 1 -- 172.21.15.53:0/3364309406 shutdown_connections 2023-11-06T01:36:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.945+0000 7fda32a21b80 1 --2- 172.21.15.53:0/3364309406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c31569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.945+0000 7fda32a21b80 1 --2- 172.21.15.53:0/3364309406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3146880 0x5643c3143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.945+0000 7fda32a21b80 1 --2- 172.21.15.53:0/3364309406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3145990 0x5643c3146c60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.945+0000 7fda32a21b80 1 -- 172.21.15.53:0/3364309406 >> 172.21.15.53:0/3364309406 conn(0x5643c30a7070 msgr2=0x5643c3151430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.946+0000 7fda32a21b80 1 -- 172.21.15.53:0/3364309406 shutdown_connections 2023-11-06T01:36:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.946+0000 7fda32a21b80 1 -- 172.21.15.53:0/3364309406 wait complete. 2023-11-06T01:36:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.946+0000 7fda32a21b80 1 -- start start 2023-11-06T01:36:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.946+0000 7fda32a21b80 1 Processor -- start 2023-11-06T01:36:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c3154be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda32a21b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3146880 0x5643c3155e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda32a21b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c31563a0 con 0x5643c3146880 2023-11-06T01:36:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda32a21b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c3156510 con 0x5643c3145030 2023-11-06T01:36:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda32a21b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c31566d0 con 0x5643c3145990 2023-11-06T01:36:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda32a21b80 10 client.0 _ll_register_callbacks cb 0x7fda14007ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda1f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda1f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54908/0 (socket says 172.21.15.53:54908) 2023-11-06T01:36:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda1f7fe700 1 -- 172.21.15.53:0/848628844 learned_addr learned my addr 172.21.15.53:0/848628844 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda24e2e700 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3146880 0x5643c3155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.947+0000 7fda1ffff700 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c3154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fda1f7fe700 1 -- 172.21.15.53:0/848628844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 msgr2=0x5643c3154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fda1f7fe700 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c3154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fda1f7fe700 1 -- 172.21.15.53:0/848628844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3146880 msgr2=0x5643c3155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fda1f7fe700 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3146880 0x5643c3155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fda1f7fe700 1 -- 172.21.15.53:0/848628844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda201109c0 con 0x5643c3145990 2023-11-06T01:36:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fda1ffff700 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c3154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fda1f7fe700 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31551a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fda14007010 tx=0x7fda14013500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b55fb169e5754de0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fd9ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:36:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/848628844 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14013b50 con 0x5643c3145990 2023-11-06T01:36:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/848628844 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fda14005a50 con 0x5643c3145990 2023-11-06T01:36:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fda32a21b80 1 -- 172.21.15.53:0/848628844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda1000da60 con 0x5643c3145990 2023-11-06T01:36:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.948+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/848628844 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda140100b0 con 0x5643c3145990 2023-11-06T01:36:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.949+0000 7fda32a21b80 10 client.25444 fetch_fsmap learned FSMap version 33 2023-11-06T01:36:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.949+0000 7fda32a21b80 1 -- 172.21.15.53:0/848628844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5643c2eed1f0 con 0x5643c3145990 2023-11-06T01:36:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.949+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/848628844 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1000da60 con 0x5643c3145990 2023-11-06T01:36:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.950+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/848628844 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fda14006030 con 0x5643c3145990 2023-11-06T01:36:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.950+0000 7fda32a21b80 10 client.25444 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:36:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.950+0000 7fda32a21b80 10 client.25444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.950+0000 7fda32a21b80 20 client.25444 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.950+0000 7fda32a21b80 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5643c31afbc0 0x5643c31b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.950+0000 7fda32a21b80 4 client.25444 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:36:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.950+0000 7fda32a21b80 1 -- 172.21.15.53:0/848628844 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5643c3091ce0 con 0x5643c31afbc0 2023-11-06T01:36:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.951+0000 7fda1ffff700 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5643c31afbc0 0x5643c31b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.951+0000 7fd9ff7fe700 10 client.25444 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.951+0000 7fda1ffff700 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5643c31afbc0 0x5643c31b1fe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6486a067730103c7 server_cookie=ecedaaf49b324d57 in_seq=0 out_seq=0 2023-11-06T01:36:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fd9ff7fe700 1 -- 172.21.15.53:0/848628844 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5643c3091ce0 con 0x5643c31afbc0 2023-11-06T01:36:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fd9ff7fe700 10 client.25444 handle_command_reply: tid=0 2023-11-06T01:36:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fda32a21b80 1 client.25444 shutdown 2023-11-06T01:36:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fda32a21b80 1 -- 172.21.15.53:0/848628844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 msgr2=0x5643c31551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fda32a21b80 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31551a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fda14007010 tx=0x7fda14013500 comp rx=0 tx=0).stop 2023-11-06T01:36:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fda32a21b80 1 -- 172.21.15.53:0/848628844 shutdown_connections 2023-11-06T01:36:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fda32a21b80 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643c3145030 0x5643c3154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fda32a21b80 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5643c31afbc0 0x5643c31b1fe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fda32a21b80 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643c3146880 0x5643c3155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fd9ff7fe700 0 client.25444 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fda32a21b80 1 --2- 172.21.15.53:0/848628844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5643c3145990 0x5643c31551a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fda32a21b80 1 -- 172.21.15.53:0/848628844 >> 172.21.15.53:0/848628844 conn(0x5643c30a7070 msgr2=0x5643c3143c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fda32a21b80 1 -- 172.21.15.53:0/848628844 shutdown_connections 2023-11-06T01:36:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.965+0000 7fda32a21b80 1 -- 172.21.15.53:0/848628844 wait complete. 2023-11-06T01:36:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda32a21b80 20 client.25444 trim_cache size 0 max 16384 2023-11-06T01:36:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:36:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:36:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:36:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:36:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 -- 172.21.15.53:0/528539851 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda080a06a0 msgr2=0x7fda080a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda080a06a0 0x7fda080a2b60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fda10017040 tx=0x7fda1000b8f0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 -- 172.21.15.53:0/528539851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2019e750 msgr2=0x7fda2019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2019e750 0x7fda2019ebd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fda1800eec0 tx=0x7fda180106a0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 -- 172.21.15.53:0/528539851 shutdown_connections 2023-11-06T01:36:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda080a06a0 0x7fda080a2b60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2019e750 0x7fda2019ebd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda20108130 0x7fda2019d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 --2- 172.21.15.53:0/528539851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda20109800 0x7fda2019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 -- 172.21.15.53:0/528539851 >> 172.21.15.53:0/528539851 conn(0x7fda2006dbc0 msgr2=0x7fda20070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 -- 172.21.15.53:0/528539851 shutdown_connections 2023-11-06T01:36:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:11.966+0000 7fda26891700 1 -- 172.21.15.53:0/528539851 wait complete. 2023-11-06T01:36:12.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 115.0 s, 3697.2 tps, lat 27.075 ms stddev 28.873 2023-11-06T01:36:13.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:13 smithi053 ceph-mon[99472]: pgmap v367: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 1.03k op/s 2023-11-06T01:36:13.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:13 smithi062 ceph-mon[103924]: pgmap v367: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 1.03k op/s 2023-11-06T01:36:13.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:13 smithi163 ceph-mon[104416]: pgmap v367: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 1.03k op/s 2023-11-06T01:36:15.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:15 smithi062 ceph-mon[103924]: pgmap v368: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 904 op/s 2023-11-06T01:36:15.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:15 smithi163 ceph-mon[104416]: pgmap v368: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 904 op/s 2023-11-06T01:36:15.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:15 smithi053 ceph-mon[99472]: pgmap v368: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 904 op/s 2023-11-06T01:36:17.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 120.0 s, 3028.3 tps, lat 32.977 ms stddev 58.178 2023-11-06T01:36:17.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:17 smithi062 ceph-mon[103924]: pgmap v369: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 MiB/s wr, 1.71k op/s 2023-11-06T01:36:17.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:17 smithi163 ceph-mon[104416]: pgmap v369: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 MiB/s wr, 1.71k op/s 2023-11-06T01:36:17.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:17 smithi053 ceph-mon[99472]: pgmap v369: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 MiB/s wr, 1.71k op/s 2023-11-06T01:36:19.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:19 smithi062 ceph-mon[103924]: pgmap v370: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 MiB/s wr, 1.83k op/s 2023-11-06T01:36:19.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:19 smithi163 ceph-mon[104416]: pgmap v370: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 MiB/s wr, 1.83k op/s 2023-11-06T01:36:19.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:19 smithi053 ceph-mon[99472]: pgmap v370: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 MiB/s wr, 1.83k op/s 2023-11-06T01:36:21.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:21 smithi163 ceph-mon[104416]: pgmap v371: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 50 MiB/s wr, 2.02k op/s 2023-11-06T01:36:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:21 smithi053 ceph-mon[99472]: pgmap v371: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 50 MiB/s wr, 2.02k op/s 2023-11-06T01:36:21.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:21 smithi062 ceph-mon[103924]: pgmap v371: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 50 MiB/s wr, 2.02k op/s 2023-11-06T01:36:22.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 125.0 s, 3657.8 tps, lat 27.364 ms stddev 29.786 2023-11-06T01:36:23.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:23 smithi163 ceph-mon[104416]: pgmap v372: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 MiB/s wr, 2.19k op/s 2023-11-06T01:36:23.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:23 smithi053 ceph-mon[99472]: pgmap v372: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 MiB/s wr, 2.19k op/s 2023-11-06T01:36:23.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:23 smithi062 ceph-mon[103924]: pgmap v372: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 MiB/s wr, 2.19k op/s 2023-11-06T01:36:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.130+0000 7f3cbb095700 1 -- 172.21.15.53:0/2980449428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cb4101360 msgr2=0x7f3cb4101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.130+0000 7f3cbb095700 1 --2- 172.21.15.53:0/2980449428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cb4101360 0x7f3cb4101780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009950 tx=0x7f3ca8009c60 comp rx=0 tx=0).stop 2023-11-06T01:36:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.150+0000 7f3cbb095700 1 -- 172.21.15.53:0/2980449428 shutdown_connections 2023-11-06T01:36:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.150+0000 7f3cbb095700 1 --2- 172.21.15.53:0/2980449428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cb4101360 0x7f3cb4101780 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.150+0000 7f3cbb095700 1 --2- 172.21.15.53:0/2980449428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb41008e0 0x7f3cb4108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.150+0000 7f3cbb095700 1 --2- 172.21.15.53:0/2980449428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb40fff20 0x7f3cb41003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.150+0000 7f3cbb095700 1 -- 172.21.15.53:0/2980449428 >> 172.21.15.53:0/2980449428 conn(0x7f3cb40fa9d0 msgr2=0x7f3cb40fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.165+0000 7f3cbb095700 1 -- 172.21.15.53:0/2980449428 shutdown_connections 2023-11-06T01:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.166+0000 7f3cbb095700 1 -- 172.21.15.53:0/2980449428 wait complete. 2023-11-06T01:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.167+0000 7f3cbb095700 1 Processor -- start 2023-11-06T01:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.167+0000 7f3cbb095700 1 -- start start 2023-11-06T01:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.168+0000 7f3cbb095700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cb40fff20 0x7f3cb406f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.168+0000 7f3cbb095700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb41008e0 0x7f3cb406fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.168+0000 7f3cbb095700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb4101360 0x7f3cb4070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.168+0000 7f3cbb095700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb410db10 con 0x7f3cb41008e0 2023-11-06T01:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.168+0000 7f3cbb095700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb410dc80 con 0x7f3cb40fff20 2023-11-06T01:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.168+0000 7f3cbb095700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40710b0 con 0x7f3cb4101360 2023-11-06T01:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.169+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb41008e0 0x7f3cb406fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.169+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb41008e0 0x7f3cb406fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40684/0 (socket says 172.21.15.53:40684) 2023-11-06T01:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.169+0000 7f3cb3fff700 1 -- 172.21.15.53:0/612247401 learned_addr learned my addr 172.21.15.53:0/612247401 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.169+0000 7f3cb8e31700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cb40fff20 0x7f3cb406f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.169+0000 7f3cb9632700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb4101360 0x7f3cb4070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.169+0000 7f3cb8e31700 1 -- 172.21.15.53:0/612247401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb4101360 msgr2=0x7f3cb4070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.169+0000 7f3cb8e31700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb4101360 0x7f3cb4070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.169+0000 7f3cb8e31700 1 -- 172.21.15.53:0/612247401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb41008e0 msgr2=0x7f3cb406fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.169+0000 7f3cb8e31700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb41008e0 0x7f3cb406fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.169+0000 7f3cb8e31700 1 -- 172.21.15.53:0/612247401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca8009520 con 0x7f3cb40fff20 2023-11-06T01:36:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.170+0000 7f3cb8e31700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cb40fff20 0x7f3cb406f6e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009fd0 tx=0x7f3ca80055e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.171+0000 7f3cb1ffb700 1 -- 172.21.15.53:0/612247401 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8024070 con 0x7f3cb40fff20 2023-11-06T01:36:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.171+0000 7f3cb1ffb700 1 -- 172.21.15.53:0/612247401 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ca8004bd0 con 0x7f3cb40fff20 2023-11-06T01:36:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.171+0000 7f3cb1ffb700 1 -- 172.21.15.53:0/612247401 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8016b10 con 0x7f3cb40fff20 2023-11-06T01:36:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.173+0000 7f3cbb095700 1 -- 172.21.15.53:0/612247401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb4071330 con 0x7f3cb40fff20 2023-11-06T01:36:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.173+0000 7f3cbb095700 1 -- 172.21.15.53:0/612247401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb4071760 con 0x7f3cb40fff20 2023-11-06T01:36:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.178+0000 7f3cb1ffb700 1 -- 172.21.15.53:0/612247401 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f3ca8016c70 con 0x7f3cb40fff20 2023-11-06T01:36:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.178+0000 7f3cc7225b80 1 Processor -- start 2023-11-06T01:36:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.179+0000 7f3cc7225b80 1 -- start start 2023-11-06T01:36:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.179+0000 7f3cb1ffb700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3ca40a0620 0x7f3ca40a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.179+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc04aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.179+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04fe0 0x55759fc058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.179+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc05ed0 0x55759fc06350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.179+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759fb5c9b0 con 0x55759fc04680 2023-11-06T01:36:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.179+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759fb5a370 con 0x55759fc04fe0 2023-11-06T01:36:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.179+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759fb55c60 con 0x55759fc05ed0 2023-11-06T01:36:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.179+0000 7f3cb1ffb700 1 -- 172.21.15.53:0/612247401 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3ca80cca40 con 0x7f3cb40fff20 2023-11-06T01:36:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.180+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04fe0 0x55759fc058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.180+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04fe0 0x55759fc058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52126/0 (socket says 172.21.15.53:52126) 2023-11-06T01:36:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.180+0000 7f3cb3fff700 1 -- 172.21.15.53:0/2469296750 learned_addr learned my addr 172.21.15.53:0/2469296750 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.180+0000 7f3cb3fff700 1 -- 172.21.15.53:0/2469296750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc05ed0 msgr2=0x55759fc06350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:36:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.180+0000 7f3cb3fff700 1 --2- 172.21.15.53:0/2469296750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc05ed0 0x55759fc06350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.180+0000 7f3cb3fff700 1 -- 172.21.15.53:0/2469296750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 msgr2=0x55759fc04aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.180+0000 7f3cb3fff700 1 --2- 172.21.15.53:0/2469296750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc04aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.180+0000 7f3cb3fff700 1 -- 172.21.15.53:0/2469296750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55759f983d80 con 0x55759fc04fe0 2023-11-06T01:36:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.180+0000 7f3cb3fff700 1 --2- 172.21.15.53:0/2469296750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04fe0 0x55759fc058f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000be60 tx=0x7f3ca00106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a280f7e8fb9b50 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.180+0000 7f3cb9632700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3ca40a0620 0x7f3ca40a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.181+0000 7f3c9effd700 1 -- 172.21.15.53:0/2469296750 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0011440 con 0x55759fc04fe0 2023-11-06T01:36:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.182+0000 7f3cb9632700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3ca40a0620 0x7f3ca40a2ae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55759fc056b0 tx=0x7f3cac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.182+0000 7f3c9effd700 1 -- 172.21.15.53:0/2469296750 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ca0015070 con 0x55759fc04fe0 2023-11-06T01:36:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.182+0000 7f3c9effd700 1 -- 172.21.15.53:0/2469296750 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0019790 con 0x55759fc04fe0 2023-11-06T01:36:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.183+0000 7f3cc7225b80 1 -- 172.21.15.53:0/2469296750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04fe0 msgr2=0x55759fc058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.183+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/2469296750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04fe0 0x55759fc058f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000be60 tx=0x7f3ca00106b0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.183+0000 7f3cc7225b80 1 -- 172.21.15.53:0/2469296750 shutdown_connections 2023-11-06T01:36:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.183+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/2469296750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc05ed0 0x55759fc06350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.183+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/2469296750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc04aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.183+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/2469296750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04fe0 0x55759fc058f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.183+0000 7f3cc7225b80 1 -- 172.21.15.53:0/2469296750 >> 172.21.15.53:0/2469296750 conn(0x55759fb64070 msgr2=0x55759fc01040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.183+0000 7f3cc7225b80 1 -- 172.21.15.53:0/2469296750 shutdown_connections 2023-11-06T01:36:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.183+0000 7f3cc7225b80 1 -- 172.21.15.53:0/2469296750 wait complete. 2023-11-06T01:36:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.184+0000 7f3cc7225b80 1 -- start start 2023-11-06T01:36:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.184+0000 7f3cc7225b80 1 Processor -- start 2023-11-06T01:36:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.184+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc0e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.184+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 0x55759fc0e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.184+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc05ed0 0x55759fc15a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.184+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759faf24b0 con 0x55759fc04680 2023-11-06T01:36:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.184+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759faf2150 con 0x55759fc05ed0 2023-11-06T01:36:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.184+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759f9d98c0 con 0x55759fc04fe0 2023-11-06T01:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.184+0000 7f3cc7225b80 10 client.0 _ll_register_callbacks cb 0x55759fa3fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.185+0000 7f3cb8e31700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc0e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.185+0000 7f3cb8e31700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc0e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40696/0 (socket says 172.21.15.53:40696) 2023-11-06T01:36:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.185+0000 7f3cb8e31700 1 -- 172.21.15.53:0/1681078972 learned_addr learned my addr 172.21.15.53:0/1681078972 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.186+0000 7f3cb8e31700 1 -- 172.21.15.53:0/1681078972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 msgr2=0x55759fc0e9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:36:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.186+0000 7f3cb8e31700 1 --2- 172.21.15.53:0/1681078972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 0x55759fc0e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.186+0000 7f3cb8e31700 1 -- 172.21.15.53:0/1681078972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc05ed0 msgr2=0x55759fc15a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:36:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.186+0000 7f3cb8e31700 1 --2- 172.21.15.53:0/1681078972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc05ed0 0x55759fc15a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.186+0000 7f3cb8e31700 1 -- 172.21.15.53:0/1681078972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca000ba90 con 0x55759fc04680 2023-11-06T01:36:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.186+0000 7f3cb8e31700 1 --2- 172.21.15.53:0/1681078972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc0e4a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3ca809a240 tx=0x7f3ca8005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5382eac8223809f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.186+0000 7f3c93fff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:36:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.186+0000 7f3c93fff700 1 -- 172.21.15.53:0/1681078972 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca809a2b0 con 0x55759fc04680 2023-11-06T01:36:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.186+0000 7f3c93fff700 1 -- 172.21.15.53:0/1681078972 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3ca809e0e0 con 0x55759fc04680 2023-11-06T01:36:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.186+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1681078972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55759fb80cb0 con 0x55759fc04680 2023-11-06T01:36:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.187+0000 7f3c93fff700 1 -- 172.21.15.53:0/1681078972 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ca80a2290 con 0x55759fc04680 2023-11-06T01:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.187+0000 7f3cc7225b80 10 client.15801 fetch_fsmap learned FSMap version 33 2023-11-06T01:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.187+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1681078972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55759f96cc00 con 0x55759fc04680 2023-11-06T01:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.187+0000 7f3c93fff700 1 -- 172.21.15.53:0/1681078972 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55759fb80cb0 con 0x55759fc04680 2023-11-06T01:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.187+0000 7f3c93fff700 1 -- 172.21.15.53:0/1681078972 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3ca80a1e00 con 0x55759fc04680 2023-11-06T01:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.187+0000 7f3cc7225b80 10 client.15801 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.187+0000 7f3cc7225b80 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.187+0000 7f3cc7225b80 20 client.15801 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.187+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1681078972 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55759fc19f90 0x55759fc1c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.188+0000 7f3cc7225b80 4 client.15801 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:36:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.188+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1681078972 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55759fbd3db0 con 0x55759fc19f90 2023-11-06T01:36:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.188+0000 7f3cb3fff700 1 --2- 172.21.15.53:0/1681078972 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55759fc19f90 0x55759fc1c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.188+0000 7f3cb3fff700 1 --2- 172.21.15.53:0/1681078972 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55759fc19f90 0x55759fc1c3b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb0c1fd3460a016 server_cookie=701221df2c94a2a9 in_seq=0 out_seq=0 2023-11-06T01:36:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.188+0000 7f3c93fff700 10 client.15801 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.189+0000 7f3c93fff700 1 -- 172.21.15.53:0/1681078972 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55759fbd3db0 con 0x55759fc19f90 2023-11-06T01:36:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.189+0000 7f3c93fff700 10 client.15801 handle_command_reply: tid=0 2023-11-06T01:36:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.189+0000 7f3cc7225b80 1 client.15801 shutdown 2023-11-06T01:36:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.189+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1681078972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 msgr2=0x55759fc0e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.189+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1681078972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc0e4a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3ca809a240 tx=0x7f3ca8005bf0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.190+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1681078972 shutdown_connections 2023-11-06T01:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.190+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1681078972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc05ed0 0x55759fc15a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.190+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1681078972 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55759fc19f90 0x55759fc1c3b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.190+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1681078972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc0e4a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.190+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1681078972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 0x55759fc0e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.190+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1681078972 >> 172.21.15.53:0/1681078972 conn(0x55759fb64070 msgr2=0x55759fbff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.190+0000 7f3c93fff700 0 client.15801 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.190+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1681078972 shutdown_connections 2023-11-06T01:36:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.190+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1681078972 wait complete. 2023-11-06T01:36:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.190+0000 7f3cc7225b80 20 client.15801 trim_cache size 0 max 16384 2023-11-06T01:36:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.210+0000 7f3cc7225b80 1 Processor -- start 2023-11-06T01:36:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.210+0000 7f3cc7225b80 1 -- start start 2023-11-06T01:36:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.210+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc139d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.210+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 0x55759fc0db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.210+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc05ed0 0x55759fc1bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.210+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759fbf77e0 con 0x55759fc04680 2023-11-06T01:36:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.210+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759fbfa530 con 0x55759fc05ed0 2023-11-06T01:36:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.210+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759fc1c990 con 0x55759fc04fe0 2023-11-06T01:36:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.210+0000 7f3cb9632700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc05ed0 0x55759fc1bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.210+0000 7f3cb9632700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc05ed0 0x55759fc1bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52140/0 (socket says 172.21.15.53:52140) 2023-11-06T01:36:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.210+0000 7f3cb9632700 1 -- 172.21.15.53:0/3639728667 learned_addr learned my addr 172.21.15.53:0/3639728667 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.211+0000 7f3cb9632700 1 -- 172.21.15.53:0/3639728667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 msgr2=0x55759fc0db50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:36:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.211+0000 7f3cb9632700 1 --2- 172.21.15.53:0/3639728667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 0x55759fc0db50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.211+0000 7f3cb9632700 1 -- 172.21.15.53:0/3639728667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 msgr2=0x55759fc139d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.211+0000 7f3cb9632700 1 --2- 172.21.15.53:0/3639728667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.211+0000 7f3cb9632700 1 -- 172.21.15.53:0/3639728667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55759f9aa1f0 con 0x55759fc05ed0 2023-11-06T01:36:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.212+0000 7f3cb9632700 1 --2- 172.21.15.53:0/3639728667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc05ed0 0x55759fc1bfa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3cac017790 tx=0x7f3cac014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80239b1f42ae02f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.213+0000 7f3c9f7fe700 1 -- 172.21.15.53:0/3639728667 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00e040 con 0x55759fc05ed0 2023-11-06T01:36:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.213+0000 7f3c9f7fe700 1 -- 172.21.15.53:0/3639728667 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cac00f040 con 0x55759fc05ed0 2023-11-06T01:36:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.213+0000 7f3c9f7fe700 1 -- 172.21.15.53:0/3639728667 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac004280 con 0x55759fc05ed0 2023-11-06T01:36:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.214+0000 7f3cc7225b80 1 -- 172.21.15.53:0/3639728667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc05ed0 msgr2=0x55759fc1bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.214+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/3639728667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc05ed0 0x55759fc1bfa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3cac017790 tx=0x7f3cac014000 comp rx=0 tx=0).stop 2023-11-06T01:36:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.214+0000 7f3cc7225b80 1 -- 172.21.15.53:0/3639728667 shutdown_connections 2023-11-06T01:36:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.214+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/3639728667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc05ed0 0x55759fc1bfa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.214+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/3639728667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc04680 0x55759fc139d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.214+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/3639728667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 0x55759fc0db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.214+0000 7f3cc7225b80 1 -- 172.21.15.53:0/3639728667 >> 172.21.15.53:0/3639728667 conn(0x55759fb64070 msgr2=0x55759fc00c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.214+0000 7f3cc7225b80 1 -- 172.21.15.53:0/3639728667 shutdown_connections 2023-11-06T01:36:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.214+0000 7f3cc7225b80 1 -- 172.21.15.53:0/3639728667 wait complete. 2023-11-06T01:36:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.214+0000 7f3cc7225b80 1 -- start start 2023-11-06T01:36:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cc7225b80 1 Processor -- start 2023-11-06T01:36:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04680 0x55759fc65a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 0x55759fc67f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cc7225b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc05ed0 0x55759fc6a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759fc6ac60 con 0x55759fc05ed0 2023-11-06T01:36:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759fc6add0 con 0x55759fc04680 2023-11-06T01:36:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cc7225b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759fc6af90 con 0x55759fc04fe0 2023-11-06T01:36:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cc7225b80 10 client.0 _ll_register_callbacks cb 0x7f3ca8099df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cb8e31700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04680 0x55759fc65a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cb9632700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc05ed0 0x55759fc6a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cb8e31700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04680 0x55759fc65a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52146/0 (socket says 172.21.15.53:52146) 2023-11-06T01:36:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cb9632700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc05ed0 0x55759fc6a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40712/0 (socket says 172.21.15.53:40712) 2023-11-06T01:36:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.215+0000 7f3cb8e31700 1 -- 172.21.15.53:0/1711445618 learned_addr learned my addr 172.21.15.53:0/1711445618 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.216+0000 7f3cb3fff700 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 0x55759fc67f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.216+0000 7f3cb8e31700 1 -- 172.21.15.53:0/1711445618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 msgr2=0x55759fc67f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.216+0000 7f3cb8e31700 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 0x55759fc67f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.216+0000 7f3cb8e31700 1 -- 172.21.15.53:0/1711445618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc05ed0 msgr2=0x55759fc6a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.216+0000 7f3cb8e31700 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc05ed0 0x55759fc6a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.216+0000 7f3cb8e31700 1 -- 172.21.15.53:0/1711445618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac017440 con 0x55759fc04680 2023-11-06T01:36:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.216+0000 7f3cb8e31700 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04680 0x55759fc65a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8083b00 tx=0x7f3ca8016560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e049fb22b0ef1f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.216+0000 7f3c93fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:36:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.216+0000 7f3c93fff700 1 -- 172.21.15.53:0/1711445618 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80965f0 con 0x55759fc04680 2023-11-06T01:36:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.216+0000 7f3c93fff700 1 -- 172.21.15.53:0/1711445618 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3ca809e0e0 con 0x55759fc04680 2023-11-06T01:36:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.216+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1711445618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ca0015070 con 0x55759fc04680 2023-11-06T01:36:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.217+0000 7f3c93fff700 1 -- 172.21.15.53:0/1711445618 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ca80a2290 con 0x55759fc04680 2023-11-06T01:36:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.217+0000 7f3cc7225b80 10 client.25376 fetch_fsmap learned FSMap version 33 2023-11-06T01:36:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.217+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1711445618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55759f982250 con 0x55759fc04680 2023-11-06T01:36:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.217+0000 7f3c93fff700 1 -- 172.21.15.53:0/1711445618 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ca0015070 con 0x55759fc04680 2023-11-06T01:36:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.217+0000 7f3c93fff700 1 -- 172.21.15.53:0/1711445618 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3ca80960a0 con 0x55759fc04680 2023-11-06T01:36:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.217+0000 7f3cc7225b80 10 client.25376 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:36:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.217+0000 7f3cc7225b80 10 client.25376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.217+0000 7f3cc7225b80 20 client.25376 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.217+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55759fc6c420 0x55759fc6e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.217+0000 7f3cc7225b80 4 client.25376 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:36:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.217+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1711445618 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55759fb4ece0 con 0x55759fc6c420 2023-11-06T01:36:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.218+0000 7f3cb3fff700 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55759fc6c420 0x55759fc6e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.219+0000 7f3cb3fff700 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55759fc6c420 0x55759fc6e840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69beba332a6ba074 server_cookie=4b0324cfcac02605 in_seq=0 out_seq=0 2023-11-06T01:36:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.219+0000 7f3c93fff700 10 client.25376 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.235+0000 7f3c93fff700 1 -- 172.21.15.53:0/1711445618 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55759fb4ece0 con 0x55759fc6c420 2023-11-06T01:36:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.235+0000 7f3c93fff700 10 client.25376 handle_command_reply: tid=0 2023-11-06T01:36:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3cc7225b80 1 client.25376 shutdown 2023-11-06T01:36:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1711445618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04680 msgr2=0x55759fc65a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04680 0x55759fc65a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8083b00 tx=0x7f3ca8016560 comp rx=0 tx=0).stop 2023-11-06T01:36:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1711445618 shutdown_connections 2023-11-06T01:36:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55759fc04680 0x55759fc65a30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55759fc6c420 0x55759fc6e840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55759fc05ed0 0x55759fc6a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3c93fff700 0 client.25376 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3cc7225b80 1 --2- 172.21.15.53:0/1711445618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55759fc04fe0 0x55759fc67f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1711445618 >> 172.21.15.53:0/1711445618 conn(0x55759fb64070 msgr2=0x55759fc1bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1711445618 shutdown_connections 2023-11-06T01:36:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.236+0000 7f3cc7225b80 1 -- 172.21.15.53:0/1711445618 wait complete. 2023-11-06T01:36:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cc7225b80 20 client.25376 trim_cache size 0 max 16384 2023-11-06T01:36:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:36:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:36:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:36:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:36:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cbb095700 1 -- 172.21.15.53:0/612247401 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3ca40a0620 msgr2=0x7f3ca40a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cbb095700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3ca40a0620 0x7f3ca40a2ae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55759fc056b0 tx=0x7f3cac00b040 comp rx=0 tx=0).stop 2023-11-06T01:36:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cbb095700 1 -- 172.21.15.53:0/612247401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cb40fff20 msgr2=0x7f3cb406f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cbb095700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cb40fff20 0x7f3cb406f6e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009fd0 tx=0x7f3ca80055e0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cbb095700 1 -- 172.21.15.53:0/612247401 shutdown_connections 2023-11-06T01:36:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cbb095700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3ca40a0620 0x7f3ca40a2ae0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cbb095700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cb40fff20 0x7f3cb406f6e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cbb095700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cb4101360 0x7f3cb4070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cbb095700 1 --2- 172.21.15.53:0/612247401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb41008e0 0x7f3cb406fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cbb095700 1 -- 172.21.15.53:0/612247401 >> 172.21.15.53:0/612247401 conn(0x7f3cb40fa9d0 msgr2=0x7f3cb41044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.237+0000 7f3cbb095700 1 -- 172.21.15.53:0/612247401 shutdown_connections 2023-11-06T01:36:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:25.238+0000 7f3cbb095700 1 -- 172.21.15.53:0/612247401 wait complete. 2023-11-06T01:36:25.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:25 smithi163 ceph-mon[104416]: pgmap v373: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 MiB/s wr, 2.03k op/s 2023-11-06T01:36:25.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:25 smithi053 ceph-mon[99472]: pgmap v373: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 MiB/s wr, 2.03k op/s 2023-11-06T01:36:25.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:25 smithi062 ceph-mon[103924]: pgmap v373: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 MiB/s wr, 2.03k op/s 2023-11-06T01:36:27.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 130.0 s, 3873.4 tps, lat 25.745 ms stddev 26.627 2023-11-06T01:36:27.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:27 smithi163 ceph-mon[104416]: pgmap v374: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 51 MiB/s wr, 2.20k op/s 2023-11-06T01:36:27.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:27 smithi053 ceph-mon[99472]: pgmap v374: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 51 MiB/s wr, 2.20k op/s 2023-11-06T01:36:27.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:27 smithi062 ceph-mon[103924]: pgmap v374: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 51 MiB/s wr, 2.20k op/s 2023-11-06T01:36:29.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:29 smithi163 ceph-mon[104416]: pgmap v375: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.45k op/s 2023-11-06T01:36:29.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:29 smithi053 ceph-mon[99472]: pgmap v375: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.45k op/s 2023-11-06T01:36:29.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:29 smithi062 ceph-mon[103924]: pgmap v375: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.45k op/s 2023-11-06T01:36:31.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:31 smithi163 ceph-mon[104416]: pgmap v376: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 1.23k op/s 2023-11-06T01:36:31.599 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:31 smithi053 ceph-mon[99472]: pgmap v376: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 1.23k op/s 2023-11-06T01:36:31.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:31 smithi062 ceph-mon[103924]: pgmap v376: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 1.23k op/s 2023-11-06T01:36:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 135.0 s, 2533.2 tps, lat 39.567 ms stddev 47.620 2023-11-06T01:36:33.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:33 smithi163 ceph-mon[104416]: pgmap v377: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 1.22k op/s 2023-11-06T01:36:33.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:33 smithi053 ceph-mon[99472]: pgmap v377: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 1.22k op/s 2023-11-06T01:36:33.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:33 smithi062 ceph-mon[103924]: pgmap v377: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 1.22k op/s 2023-11-06T01:36:34.780 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:36:35.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.038+0000 7f70a7b9a700 1 -- 172.21.15.53:0/3267845604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70a0107d90 msgr2=0x7f70a010a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:35.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.038+0000 7f70a7b9a700 1 --2- 172.21.15.53:0/3267845604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70a0107d90 0x7f70a010a1c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f709c0098f0 tx=0x7f709c009c00 comp rx=0 tx=0).stop 2023-11-06T01:36:35.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.039+0000 7f70a7b9a700 1 -- 172.21.15.53:0/3267845604 shutdown_connections 2023-11-06T01:36:35.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.039+0000 7f70a7b9a700 1 --2- 172.21.15.53:0/3267845604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a010ab80 0x7f70a0113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.039+0000 7f70a7b9a700 1 --2- 172.21.15.53:0/3267845604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70a0107d90 0x7f70a010a1c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.039+0000 7f70a7b9a700 1 --2- 172.21.15.53:0/3267845604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010a700 0x7f70a01114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.039+0000 7f70a7b9a700 1 -- 172.21.15.53:0/3267845604 >> 172.21.15.53:0/3267845604 conn(0x7f70a006db40 msgr2=0x7f70a006ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:35.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.044+0000 7f70a7b9a700 1 -- 172.21.15.53:0/3267845604 shutdown_connections 2023-11-06T01:36:35.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.045+0000 7f70a7b9a700 1 -- 172.21.15.53:0/3267845604 wait complete. 2023-11-06T01:36:35.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.045+0000 7f70a7b9a700 1 Processor -- start 2023-11-06T01:36:35.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.045+0000 7f70a7b9a700 1 -- start start 2023-11-06T01:36:35.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a7b9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a0107d90 0x7f70a0118450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a7b9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70a010a700 0x7f70a0118990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a7b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010ab80 0x7f70a011d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a7b9a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a011dfc0 con 0x7f70a010a700 2023-11-06T01:36:35.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a7b9a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a011e130 con 0x7f70a010ab80 2023-11-06T01:36:35.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a7b9a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a011e2f0 con 0x7f70a0107d90 2023-11-06T01:36:35.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a5936700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a0107d90 0x7f70a0118450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a5936700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a0107d90 0x7f70a0118450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:54962/0 (socket says 172.21.15.53:54962) 2023-11-06T01:36:35.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a5936700 1 -- 172.21.15.53:0/2587300567 learned_addr learned my addr 172.21.15.53:0/2587300567 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:35.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a6137700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010ab80 0x7f70a011d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a5936700 1 -- 172.21.15.53:0/2587300567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010ab80 msgr2=0x7f70a011d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:35.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a5936700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010ab80 0x7f70a011d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a5936700 1 -- 172.21.15.53:0/2587300567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70a010a700 msgr2=0x7f70a0118990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:35.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a5936700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70a010a700 0x7f70a0118990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.046+0000 7f70a5936700 1 -- 172.21.15.53:0/2587300567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c009520 con 0x7f70a0107d90 2023-11-06T01:36:35.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.047+0000 7f70a5936700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a0107d90 0x7f70a0118450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f709c009230 tx=0x7f709c00bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:35.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.047+0000 7f7096ffd700 1 -- 172.21.15.53:0/2587300567 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c0055e0 con 0x7f70a0107d90 2023-11-06T01:36:35.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.047+0000 7f7096ffd700 1 -- 172.21.15.53:0/2587300567 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c003b40 con 0x7f70a0107d90 2023-11-06T01:36:35.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.047+0000 7f70a7b9a700 1 -- 172.21.15.53:0/2587300567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a00750c0 con 0x7f70a0107d90 2023-11-06T01:36:35.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.047+0000 7f7096ffd700 1 -- 172.21.15.53:0/2587300567 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c017ae0 con 0x7f70a0107d90 2023-11-06T01:36:35.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.048+0000 7f70a7b9a700 1 -- 172.21.15.53:0/2587300567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a0075540 con 0x7f70a0107d90 2023-11-06T01:36:35.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.051+0000 7f7096ffd700 1 -- 172.21.15.53:0/2587300567 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f709c003cb0 con 0x7f70a0107d90 2023-11-06T01:36:35.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.053+0000 7f7096ffd700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f708c0a0720 0x7f708c0a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.053+0000 7f7096ffd700 1 -- 172.21.15.53:0/2587300567 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f709c0cd070 con 0x7f70a0107d90 2023-11-06T01:36:35.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.053+0000 7f70a5135700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f708c0a0720 0x7f708c0a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.054+0000 7f70a5135700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f708c0a0720 0x7f708c0a2be0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7090009770 tx=0x7f7090006cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:35.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.056+0000 7f70b3d2ab80 1 Processor -- start 2023-11-06T01:36:35.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.057+0000 7f70b3d2ab80 1 -- start start 2023-11-06T01:36:35.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.057+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f40f0 0x5572c35f4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.057+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f4a70 0x5572c35f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.057+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35f3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.057+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c354aa10 con 0x5572c35f33f0 2023-11-06T01:36:35.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.057+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c35483d0 con 0x5572c35f40f0 2023-11-06T01:36:35.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.057+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c3543cc0 con 0x5572c35f4a70 2023-11-06T01:36:35.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.057+0000 7f70a6137700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.058+0000 7f70a6137700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35f3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40732/0 (socket says 172.21.15.53:40732) 2023-11-06T01:36:35.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.058+0000 7f70a6137700 1 -- 172.21.15.53:0/2209057013 learned_addr learned my addr 172.21.15.53:0/2209057013 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:35.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.058+0000 7f70a5936700 1 --2- 172.21.15.53:0/2209057013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f40f0 0x5572c35f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.058+0000 7f70a6137700 1 -- 172.21.15.53:0/2209057013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f4a70 msgr2=0x5572c35f2e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:36:35.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.058+0000 7f70a6137700 1 --2- 172.21.15.53:0/2209057013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f4a70 0x5572c35f2e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.058+0000 7f70a6137700 1 -- 172.21.15.53:0/2209057013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f40f0 msgr2=0x5572c35f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:35.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.058+0000 7f70a6137700 1 --2- 172.21.15.53:0/2209057013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f40f0 0x5572c35f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.058+0000 7f70a6137700 1 -- 172.21.15.53:0/2209057013 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572c3371de0 con 0x5572c35f33f0 2023-11-06T01:36:35.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.059+0000 7f70a5936700 1 --2- 172.21.15.53:0/2209057013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f40f0 0x5572c35f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:35.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.059+0000 7f70a6137700 1 --2- 172.21.15.53:0/2209057013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35f3870 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7098009d30 tx=0x7f70980078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a162eac2473be8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:35.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.059+0000 7f708bfff700 1 -- 172.21.15.53:0/2209057013 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098014610 con 0x5572c35f33f0 2023-11-06T01:36:35.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.059+0000 7f708bfff700 1 -- 172.21.15.53:0/2209057013 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709800f050 con 0x5572c35f33f0 2023-11-06T01:36:35.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.059+0000 7f708bfff700 1 -- 172.21.15.53:0/2209057013 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098013960 con 0x5572c35f33f0 2023-11-06T01:36:35.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.060+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/2209057013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 msgr2=0x5572c35f3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:35.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.060+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/2209057013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35f3870 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7098009d30 tx=0x7f70980078a0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.061+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/2209057013 shutdown_connections 2023-11-06T01:36:35.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.061+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/2209057013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f40f0 0x5572c35f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.061+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/2209057013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35f3870 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.061+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/2209057013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f4a70 0x5572c35f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.061+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/2209057013 >> 172.21.15.53:0/2209057013 conn(0x5572c35520d0 msgr2=0x5572c35ed940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:35.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.061+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/2209057013 shutdown_connections 2023-11-06T01:36:35.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.061+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/2209057013 wait complete. 2023-11-06T01:36:35.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.061+0000 7f70b3d2ab80 1 -- start start 2023-11-06T01:36:35.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.062+0000 7f70b3d2ab80 1 Processor -- start 2023-11-06T01:36:35.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35fc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f40f0 0x5572c35fced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c3603f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c34e0510 con 0x5572c35f33f0 2023-11-06T01:36:35.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c34e01b0 con 0x5572c35f4a70 2023-11-06T01:36:35.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c33c7920 con 0x5572c35f40f0 2023-11-06T01:36:35.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70b3d2ab80 10 client.0 _ll_register_callbacks cb 0x5572c342da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:35.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70a5936700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70a5936700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35fc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40738/0 (socket says 172.21.15.53:40738) 2023-11-06T01:36:35.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70a5936700 1 -- 172.21.15.53:0/4091783063 learned_addr learned my addr 172.21.15.53:0/4091783063 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:35.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70a5135700 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f40f0 0x5572c35fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70a5936700 1 -- 172.21.15.53:0/4091783063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f40f0 msgr2=0x5572c35fced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:35.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70a5936700 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f40f0 0x5572c35fced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70a5936700 1 -- 172.21.15.53:0/4091783063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 msgr2=0x5572c3603f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:36:35.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70a5936700 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c3603f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70a5936700 1 -- 172.21.15.53:0/4091783063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a0075540 con 0x5572c35f33f0 2023-11-06T01:36:35.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.063+0000 7f70a5936700 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35fc990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f709c005d90 tx=0x7f709c09a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcd704fe9de34f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:35.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.066+0000 7f70897fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:36:35.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.066+0000 7f70897fa700 1 -- 172.21.15.53:0/4091783063 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c099fd0 con 0x5572c35f33f0 2023-11-06T01:36:35.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.066+0000 7f70897fa700 1 -- 172.21.15.53:0/4091783063 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f709c09a8e0 con 0x5572c35f33f0 2023-11-06T01:36:35.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.066+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/4091783063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572c356ed10 con 0x5572c35f33f0 2023-11-06T01:36:35.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.068+0000 7f70897fa700 1 -- 172.21.15.53:0/4091783063 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f709c09e710 con 0x5572c35f33f0 2023-11-06T01:36:35.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.068+0000 7f70b3d2ab80 10 client.15813 fetch_fsmap learned FSMap version 33 2023-11-06T01:36:35.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.068+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/4091783063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f7098009960 con 0x5572c35f33f0 2023-11-06T01:36:35.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.069+0000 7f70897fa700 1 -- 172.21.15.53:0/4091783063 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5572c356ed10 con 0x5572c35f33f0 2023-11-06T01:36:35.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.069+0000 7f70897fa700 1 -- 172.21.15.53:0/4091783063 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f709c0a2b10 con 0x5572c35f33f0 2023-11-06T01:36:35.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.069+0000 7f70b3d2ab80 10 client.15813 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:36:35.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.069+0000 7f70b3d2ab80 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:35.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.069+0000 7f70b3d2ab80 20 client.15813 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:35.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.069+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5572c3608360 0x5572c360a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.069+0000 7f70b3d2ab80 4 client.15813 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:36:35.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.069+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/4091783063 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572c35c1e10 con 0x5572c3608360 2023-11-06T01:36:35.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.071+0000 7f70a5135700 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5572c3608360 0x5572c360a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.073+0000 7f70897fa700 10 client.15813 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:35.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.073+0000 7f70a5135700 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5572c3608360 0x5572c360a780 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2479d862e51decb server_cookie=70fac32dfa5eb16e in_seq=0 out_seq=0 2023-11-06T01:36:35.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70897fa700 1 -- 172.21.15.53:0/4091783063 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5572c35c1e10 con 0x5572c3608360 2023-11-06T01:36:35.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70897fa700 10 client.15813 handle_command_reply: tid=0 2023-11-06T01:36:35.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 1 client.15813 shutdown 2023-11-06T01:36:35.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/4091783063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 msgr2=0x5572c35fc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:35.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35fc990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f709c005d90 tx=0x7f709c09a240 comp rx=0 tx=0).stop 2023-11-06T01:36:35.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/4091783063 shutdown_connections 2023-11-06T01:36:35.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c3603f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5572c3608360 0x5572c360a780 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f33f0 0x5572c35fc990 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/4091783063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f40f0 0x5572c35fced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70897fa700 0 client.15813 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:35.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/4091783063 >> 172.21.15.53:0/4091783063 conn(0x5572c35520d0 msgr2=0x5572c35ed0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:35.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/4091783063 shutdown_connections 2023-11-06T01:36:35.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/4091783063 wait complete. 2023-11-06T01:36:35.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.074+0000 7f70b3d2ab80 20 client.15813 trim_cache size 0 max 16384 2023-11-06T01:36:35.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.103+0000 7f70b3d2ab80 1 Processor -- start 2023-11-06T01:36:35.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.103+0000 7f70b3d2ab80 1 -- start start 2023-11-06T01:36:35.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.103+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 0x5572c35f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.103+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f40f0 0x5572c3603ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.103+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c35fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.103+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c35fc560 con 0x5572c35f40f0 2023-11-06T01:36:35.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.103+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c35c1e10 con 0x5572c35f4a70 2023-11-06T01:36:35.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.103+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c356ed10 con 0x5572c35f33f0 2023-11-06T01:36:35.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.104+0000 7f70a5936700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 0x5572c35f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.104+0000 7f70a6137700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c35fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.104+0000 7f70a6137700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c35fa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52184/0 (socket says 172.21.15.53:52184) 2023-11-06T01:36:35.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.104+0000 7f70a6137700 1 -- 172.21.15.53:0/2515148135 learned_addr learned my addr 172.21.15.53:0/2515148135 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:35.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.104+0000 7f70a5936700 1 -- 172.21.15.53:0/2515148135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 msgr2=0x5572c35fa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:35.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.104+0000 7f70a5936700 1 --2- 172.21.15.53:0/2515148135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c35fa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.104+0000 7f70a5936700 1 -- 172.21.15.53:0/2515148135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f40f0 msgr2=0x5572c3603ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:35.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.104+0000 7f70a5936700 1 --2- 172.21.15.53:0/2515148135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f40f0 0x5572c3603ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.104+0000 7f70a5936700 1 -- 172.21.15.53:0/2515148135 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572c335ac60 con 0x5572c35f33f0 2023-11-06T01:36:35.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.104+0000 7f70a6137700 1 --2- 172.21.15.53:0/2515148135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c35fa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:35.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.105+0000 7f70a5936700 1 --2- 172.21.15.53:0/2515148135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 0x5572c35f2e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f709c096de0 tx=0x7f709c017500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53e48cc23452c40 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:35.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.105+0000 7f7094daa700 1 -- 172.21.15.53:0/2515148135 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c09a8e0 con 0x5572c35f33f0 2023-11-06T01:36:35.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.105+0000 7f7094daa700 1 -- 172.21.15.53:0/2515148135 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c09e710 con 0x5572c35f33f0 2023-11-06T01:36:35.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.105+0000 7f7094daa700 1 -- 172.21.15.53:0/2515148135 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c0a2690 con 0x5572c35f33f0 2023-11-06T01:36:35.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.106+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/2515148135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 msgr2=0x5572c35f2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:35.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.106+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/2515148135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 0x5572c35f2e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f709c096de0 tx=0x7f709c017500 comp rx=0 tx=0).stop 2023-11-06T01:36:35.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.106+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/2515148135 shutdown_connections 2023-11-06T01:36:35.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.106+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/2515148135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c35fa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.106+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/2515148135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 0x5572c35f2e60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.106+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/2515148135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f40f0 0x5572c3603ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.106+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/2515148135 >> 172.21.15.53:0/2515148135 conn(0x5572c35520d0 msgr2=0x5572c3604800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:35.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.106+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/2515148135 shutdown_connections 2023-11-06T01:36:35.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.106+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/2515148135 wait complete. 2023-11-06T01:36:35.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.106+0000 7f70b3d2ab80 1 -- start start 2023-11-06T01:36:35.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.106+0000 7f70b3d2ab80 1 Processor -- start 2023-11-06T01:36:35.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 0x5572c3600060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f40f0 0x5572c36005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70b3d2ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c3603430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c3603970 con 0x5572c35f40f0 2023-11-06T01:36:35.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c3603ae0 con 0x5572c35f4a70 2023-11-06T01:36:35.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70b3d2ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572c3603ca0 con 0x5572c35f33f0 2023-11-06T01:36:35.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70b3d2ab80 10 client.0 _ll_register_callbacks cb 0x7f709c09a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:35.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70a6137700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c3603430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70a5135700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f40f0 0x5572c36005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70a6137700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c3603430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52192/0 (socket says 172.21.15.53:52192) 2023-11-06T01:36:35.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70a6137700 1 -- 172.21.15.53:0/1373458200 learned_addr learned my addr 172.21.15.53:0/1373458200 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:35.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70a5936700 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 0x5572c3600060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70a5135700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f40f0 0x5572c36005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40752/0 (socket says 172.21.15.53:40752) 2023-11-06T01:36:35.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70a6137700 1 -- 172.21.15.53:0/1373458200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 msgr2=0x5572c3600060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:35.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70a6137700 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 0x5572c3600060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70a6137700 1 -- 172.21.15.53:0/1373458200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f40f0 msgr2=0x5572c36005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:35.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.107+0000 7f70a6137700 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f40f0 0x5572c36005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.108+0000 7f70a6137700 1 -- 172.21.15.53:0/1373458200 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572c3371de0 con 0x5572c35f4a70 2023-11-06T01:36:35.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.108+0000 7f70a5936700 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 0x5572c3600060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:35.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.108+0000 7f70a6137700 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c3603430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7098000f80 tx=0x7f70980137e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41885e00398675d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:35.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.108+0000 7f70897fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:36:35.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.108+0000 7f70897fa700 1 -- 172.21.15.53:0/1373458200 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098013c00 con 0x5572c35f4a70 2023-11-06T01:36:35.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.108+0000 7f70897fa700 1 -- 172.21.15.53:0/1373458200 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7098013370 con 0x5572c35f4a70 2023-11-06T01:36:35.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.108+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/1373458200 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f709800f050 con 0x5572c35f4a70 2023-11-06T01:36:35.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.110+0000 7f70897fa700 1 -- 172.21.15.53:0/1373458200 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70980090b0 con 0x5572c35f4a70 2023-11-06T01:36:35.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.110+0000 7f70b3d2ab80 10 client.25388 fetch_fsmap learned FSMap version 33 2023-11-06T01:36:35.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.110+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/1373458200 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f7098009960 con 0x5572c35f4a70 2023-11-06T01:36:35.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.110+0000 7f70897fa700 1 -- 172.21.15.53:0/1373458200 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709800f050 con 0x5572c35f4a70 2023-11-06T01:36:35.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.111+0000 7f70897fa700 1 -- 172.21.15.53:0/1373458200 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7098030020 con 0x5572c35f4a70 2023-11-06T01:36:35.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.111+0000 7f70b3d2ab80 10 client.25388 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:36:35.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.111+0000 7f70b3d2ab80 10 client.25388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:35.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.111+0000 7f70b3d2ab80 20 client.25388 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:35.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.111+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5572c365b0f0 0x5572c365d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:35.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.111+0000 7f70b3d2ab80 4 client.25388 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:36:35.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.111+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/1373458200 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5572c353cd40 con 0x5572c365b0f0 2023-11-06T01:36:35.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.111+0000 7f70a5936700 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5572c365b0f0 0x5572c365d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:35.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.111+0000 7f70a5936700 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5572c365b0f0 0x5572c365d510 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4463299c02f627d7 server_cookie=ccbdbb8ec5d58531 in_seq=0 out_seq=0 2023-11-06T01:36:35.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.111+0000 7f70897fa700 10 client.25388 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:35.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70897fa700 1 -- 172.21.15.53:0/1373458200 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5572c353cd40 con 0x5572c365b0f0 2023-11-06T01:36:35.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70897fa700 10 client.25388 handle_command_reply: tid=0 2023-11-06T01:36:35.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 1 client.25388 shutdown 2023-11-06T01:36:35.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/1373458200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 msgr2=0x5572c3603430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:35.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c3603430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7098000f80 tx=0x7f70980137e0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/1373458200 shutdown_connections 2023-11-06T01:36:35.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572c35f4a70 0x5572c3603430 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5572c365b0f0 0x5572c365d510 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5572c35f33f0 0x5572c3600060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 1 --2- 172.21.15.53:0/1373458200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572c35f40f0 0x5572c36005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/1373458200 >> 172.21.15.53:0/1373458200 conn(0x5572c35520d0 msgr2=0x5572c35fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:35.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70897fa700 0 client.25388 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:35.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/1373458200 shutdown_connections 2023-11-06T01:36:35.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 1 -- 172.21.15.53:0/1373458200 wait complete. 2023-11-06T01:36:35.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.112+0000 7f70b3d2ab80 20 client.25388 trim_cache size 0 max 16384 2023-11-06T01:36:35.134 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:36:35.135 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:36:35.135 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:36:35.135 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:36:35.135 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:36:35.136 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:36:35.136 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:36:35.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:36:35.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:36:35.136 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:36:35.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 -- 172.21.15.53:0/2587300567 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f708c0a0720 msgr2=0x7f708c0a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:35.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f708c0a0720 0x7f708c0a2be0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7090009770 tx=0x7f7090006cd0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 -- 172.21.15.53:0/2587300567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a0107d90 msgr2=0x7f70a0118450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:35.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a0107d90 0x7f70a0118450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f709c009230 tx=0x7f709c00bc60 comp rx=0 tx=0).stop 2023-11-06T01:36:35.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 -- 172.21.15.53:0/2587300567 shutdown_connections 2023-11-06T01:36:35.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f708c0a0720 0x7f708c0a2be0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a010ab80 0x7f70a011d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a0107d90 0x7f70a0118450 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 --2- 172.21.15.53:0/2587300567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70a010a700 0x7f70a0118990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:35.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 -- 172.21.15.53:0/2587300567 >> 172.21.15.53:0/2587300567 conn(0x7f70a006db40 msgr2=0x7f70a006f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:35.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 -- 172.21.15.53:0/2587300567 shutdown_connections 2023-11-06T01:36:35.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:36:35.113+0000 7f70a7b9a700 1 -- 172.21.15.53:0/2587300567 wait complete. 2023-11-06T01:36:35.141 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:36:35.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:35 smithi053 ceph-mon[99472]: pgmap v378: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 993 op/s 2023-11-06T01:36:35.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:35 smithi062 ceph-mon[103924]: pgmap v378: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 993 op/s 2023-11-06T01:36:35.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:35 smithi163 ceph-mon[104416]: pgmap v378: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 993 op/s 2023-11-06T01:36:37.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 140.0 s, 2330.7 tps, lat 40.317 ms stddev 41.354 2023-11-06T01:36:37.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:37 smithi053 ceph-mon[99472]: pgmap v379: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 1.21k op/s 2023-11-06T01:36:37.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:37 smithi062 ceph-mon[103924]: pgmap v379: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 1.21k op/s 2023-11-06T01:36:37.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:37 smithi163 ceph-mon[104416]: pgmap v379: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 1.21k op/s 2023-11-06T01:36:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.412+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/2389979793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc102940 msgr2=0x7fe5cc1091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.412+0000 7fe5d1d1a700 1 --2- 172.21.15.53:0/2389979793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc102940 0x7fe5cc1091f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400b0d0 tx=0x7fe5c400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.412+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/2389979793 shutdown_connections 2023-11-06T01:36:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.412+0000 7fe5d1d1a700 1 --2- 172.21.15.53:0/2389979793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc102940 0x7fe5cc1091f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.412+0000 7fe5d1d1a700 1 --2- 172.21.15.53:0/2389979793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5cc100db0 0x7fe5cc1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.412+0000 7fe5d1d1a700 1 --2- 172.21.15.53:0/2389979793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.412+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/2389979793 >> 172.21.15.53:0/2389979793 conn(0x7fe5cc0fc340 msgr2=0x7fe5cc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.413+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/2389979793 shutdown_connections 2023-11-06T01:36:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.413+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/2389979793 wait complete. 2023-11-06T01:36:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.413+0000 7fe5d1d1a700 1 Processor -- start 2023-11-06T01:36:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.413+0000 7fe5d1d1a700 1 -- start start 2023-11-06T01:36:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5d1d1a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5cc100db0 0x7fe5cc19d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5d1d1a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5d1d1a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5cc102940 0x7fe5cc19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5d1d1a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc19ecd0 con 0x7fe5cc101fa0 2023-11-06T01:36:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5d1d1a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc19ee40 con 0x7fe5cc100db0 2023-11-06T01:36:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5d1d1a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc19f000 con 0x7fe5cc102940 2023-11-06T01:36:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5c3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5cb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5cc100db0 0x7fe5cc19d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5c3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc19d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40766/0 (socket says 172.21.15.53:40766) 2023-11-06T01:36:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5c3fff700 1 -- 172.21.15.53:0/4237695201 learned_addr learned my addr 172.21.15.53:0/4237695201 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5cbfff700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5cc102940 0x7fe5cc19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5c3fff700 1 -- 172.21.15.53:0/4237695201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5cc102940 msgr2=0x7fe5cc19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5c3fff700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5cc102940 0x7fe5cc19e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5c3fff700 1 -- 172.21.15.53:0/4237695201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5cc100db0 msgr2=0x7fe5cc19d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5c3fff700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5cc100db0 0x7fe5cc19d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5c3fff700 1 -- 172.21.15.53:0/4237695201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c4009d00 con 0x7fe5cc101fa0 2023-11-06T01:36:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.414+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5cc100db0 0x7fe5cc19d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.415+0000 7fe5c3fff700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc19d7f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400e9a0 tx=0x7fe5b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.416+0000 7fe5c97fa700 1 -- 172.21.15.53:0/4237695201 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4009790 con 0x7fe5cc101fa0 2023-11-06T01:36:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.416+0000 7fe5c97fa700 1 -- 172.21.15.53:0/4237695201 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5b4010430 con 0x7fe5cc101fa0 2023-11-06T01:36:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.416+0000 7fe5c97fa700 1 -- 172.21.15.53:0/4237695201 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4018680 con 0x7fe5cc101fa0 2023-11-06T01:36:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.416+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/4237695201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5cc19f2e0 con 0x7fe5cc101fa0 2023-11-06T01:36:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.416+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/4237695201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5cc1a5aa0 con 0x7fe5cc101fa0 2023-11-06T01:36:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.421+0000 7fe5ddeaab80 1 Processor -- start 2023-11-06T01:36:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.422+0000 7fe5ddeaab80 1 -- start start 2023-11-06T01:36:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.422+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb815aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.422+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815fe0 0x55c8bb8168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.422+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb817350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.422+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb76d9b0 con 0x55c8bb815680 2023-11-06T01:36:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.422+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb76b370 con 0x55c8bb815fe0 2023-11-06T01:36:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.422+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb766c60 con 0x55c8bb816ed0 2023-11-06T01:36:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.422+0000 7fe5cbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb815aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.422+0000 7fe5cbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb815aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40770/0 (socket says 172.21.15.53:40770) 2023-11-06T01:36:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.422+0000 7fe5cbfff700 1 -- 172.21.15.53:0/4954647 learned_addr learned my addr 172.21.15.53:0/4954647 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.423+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/4954647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815fe0 0x55c8bb8168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.423+0000 7fe5c3fff700 1 --2- 172.21.15.53:0/4954647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb817350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.423+0000 7fe5cbfff700 1 -- 172.21.15.53:0/4954647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 msgr2=0x55c8bb817350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.423+0000 7fe5cbfff700 1 --2- 172.21.15.53:0/4954647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb817350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.423+0000 7fe5cbfff700 1 -- 172.21.15.53:0/4954647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815fe0 msgr2=0x55c8bb8168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.423+0000 7fe5cbfff700 1 --2- 172.21.15.53:0/4954647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815fe0 0x55c8bb8168f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.423+0000 7fe5cbfff700 1 -- 172.21.15.53:0/4954647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8bb594d80 con 0x55c8bb815680 2023-11-06T01:36:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.423+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/4954647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815fe0 0x55c8bb8168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:36:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.423+0000 7fe5c3fff700 1 --2- 172.21.15.53:0/4954647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb817350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:36:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.423+0000 7fe5cbfff700 1 --2- 172.21.15.53:0/4954647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb815aa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4006060 tx=0x7fe5c4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902447b23b2f2ed9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.424+0000 7fe5c1ffb700 1 -- 172.21.15.53:0/4954647 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400e040 con 0x55c8bb815680 2023-11-06T01:36:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.424+0000 7fe5c1ffb700 1 -- 172.21.15.53:0/4954647 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5c400baa0 con 0x55c8bb815680 2023-11-06T01:36:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.424+0000 7fe5c1ffb700 1 -- 172.21.15.53:0/4954647 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4005370 con 0x55c8bb815680 2023-11-06T01:36:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.424+0000 7fe5c97fa700 1 -- 172.21.15.53:0/4237695201 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe5b40105a0 con 0x7fe5cc101fa0 2023-11-06T01:36:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.425+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/4954647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 msgr2=0x55c8bb815aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.425+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/4954647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb815aa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4006060 tx=0x7fe5c4009b80 comp rx=0 tx=0).stop 2023-11-06T01:36:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.425+0000 7fe5c97fa700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe5b80a0670 0x7fe5b80a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.425+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/4954647 shutdown_connections 2023-11-06T01:36:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.426+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/4954647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb817350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.426+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/4954647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb815aa0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.426+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/4954647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815fe0 0x55c8bb8168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.426+0000 7fe5c97fa700 1 -- 172.21.15.53:0/4237695201 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe5b4014070 con 0x7fe5cc101fa0 2023-11-06T01:36:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.426+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/4954647 >> 172.21.15.53:0/4954647 conn(0x55c8bb775070 msgr2=0x55c8bb812040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.426+0000 7fe5cbfff700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe5b80a0670 0x7fe5b80a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.426+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/4954647 shutdown_connections 2023-11-06T01:36:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.426+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/4954647 wait complete. 2023-11-06T01:36:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5cbfff700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe5b80a0670 0x7fe5b80a2b30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40062a0 tx=0x7fe5c400b3e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5ddeaab80 1 -- start start 2023-11-06T01:36:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5ddeaab80 1 Processor -- start 2023-11-06T01:36:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815680 0x55c8bb830430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 0x55c8bb830970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb8379b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb7034b0 con 0x55c8bb815fe0 2023-11-06T01:36:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb703150 con 0x55c8bb815680 2023-11-06T01:36:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb5ea8c0 con 0x55c8bb816ed0 2023-11-06T01:36:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5ddeaab80 10 client.0 _ll_register_callbacks cb 0x55c8bb650a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5c3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 0x55c8bb830970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5cbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb8379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.427+0000 7fe5c3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 0x55c8bb830970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40778/0 (socket says 172.21.15.53:40778) 2023-11-06T01:36:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5c3fff700 1 -- 172.21.15.53:0/2285616562 learned_addr learned my addr 172.21.15.53:0/2285616562 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815680 0x55c8bb830430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5c3fff700 1 -- 172.21.15.53:0/2285616562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 msgr2=0x55c8bb8379b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5c3fff700 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb8379b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5c3fff700 1 -- 172.21.15.53:0/2285616562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815680 msgr2=0x55c8bb830430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5c3fff700 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815680 0x55c8bb830430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5c3fff700 1 -- 172.21.15.53:0/2285616562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5cc109b40 con 0x55c8bb815fe0 2023-11-06T01:36:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5cbfff700 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb8379b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815680 0x55c8bb830430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5c3fff700 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 0x55c8bb830970 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400c2d0 tx=0x7fe5b40998a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759ccd0f3c43f5de server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:36:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2285616562 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4099a60 con 0x55c8bb815fe0 2023-11-06T01:36:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2285616562 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe5b4099360 con 0x55c8bb815fe0 2023-11-06T01:36:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.428+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2285616562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8bb791cb0 con 0x55c8bb815fe0 2023-11-06T01:36:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.429+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2285616562 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5b409d190 con 0x55c8bb815fe0 2023-11-06T01:36:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.429+0000 7fe5ddeaab80 10 client.15828 fetch_fsmap learned FSMap version 33 2023-11-06T01:36:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.429+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2285616562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c8bb57dc00 con 0x55c8bb815fe0 2023-11-06T01:36:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.430+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2285616562 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5b40b5080 con 0x55c8bb815fe0 2023-11-06T01:36:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.430+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2285616562 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe5b4082bb0 con 0x55c8bb815fe0 2023-11-06T01:36:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.430+0000 7fe5ddeaab80 10 client.15828 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:36:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.430+0000 7fe5ddeaab80 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.430+0000 7fe5ddeaab80 20 client.15828 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.430+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c8bb83c220 0x55c8bb83e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.430+0000 7fe5ddeaab80 4 client.15828 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:36:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.430+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2285616562 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8bb7e4db0 con 0x55c8bb83c220 2023-11-06T01:36:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.430+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c8bb83c220 0x55c8bb83e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.431+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c8bb83c220 0x55c8bb83e640 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4c46095be3373f9 server_cookie=87ffb26f283ab821 in_seq=0 out_seq=0 2023-11-06T01:36:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.431+0000 7fe5aaffd700 10 client.15828 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.431+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2285616562 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c8bb7e4db0 con 0x55c8bb83c220 2023-11-06T01:36:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.431+0000 7fe5aaffd700 10 client.15828 handle_command_reply: tid=0 2023-11-06T01:36:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 1 client.15828 shutdown 2023-11-06T01:36:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2285616562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 msgr2=0x55c8bb830970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 0x55c8bb830970 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400c2d0 tx=0x7fe5b40998a0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2285616562 shutdown_connections 2023-11-06T01:36:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815680 0x55c8bb830430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c8bb83c220 0x55c8bb83e640 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb8379b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5aaffd700 0 client.15828 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2285616562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 0x55c8bb830970 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2285616562 >> 172.21.15.53:0/2285616562 conn(0x55c8bb775070 msgr2=0x55c8bb8103c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2285616562 shutdown_connections 2023-11-06T01:36:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2285616562 wait complete. 2023-11-06T01:36:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.432+0000 7fe5ddeaab80 20 client.15828 trim_cache size 0 max 16384 2023-11-06T01:36:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.455+0000 7fe5ddeaab80 1 Processor -- start 2023-11-06T01:36:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.455+0000 7fe5ddeaab80 1 -- start start 2023-11-06T01:36:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.455+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb815680 0x55c8bb835960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.455+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 0x55c8bb8172b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.455+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb816ed0 0x55c8bb81a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.455+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb82ef40 con 0x55c8bb815fe0 2023-11-06T01:36:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.455+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb816d50 con 0x55c8bb816ed0 2023-11-06T01:36:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.455+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb80b500 con 0x55c8bb815680 2023-11-06T01:36:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5cb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb815680 0x55c8bb835960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5cbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb816ed0 0x55c8bb81a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5cb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb815680 0x55c8bb835960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55026/0 (socket says 172.21.15.53:55026) 2023-11-06T01:36:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5c3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 0x55c8bb8172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5cbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb816ed0 0x55c8bb81a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52226/0 (socket says 172.21.15.53:52226) 2023-11-06T01:36:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5cb7fe700 1 -- 172.21.15.53:0/3028455111 learned_addr learned my addr 172.21.15.53:0/3028455111 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5cb7fe700 1 -- 172.21.15.53:0/3028455111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb816ed0 msgr2=0x55c8bb81a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/3028455111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb816ed0 0x55c8bb81a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5cb7fe700 1 -- 172.21.15.53:0/3028455111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 msgr2=0x55c8bb8172b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/3028455111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 0x55c8bb8172b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5cb7fe700 1 -- 172.21.15.53:0/3028455111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8bb5bb1f0 con 0x55c8bb815680 2023-11-06T01:36:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/3028455111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb815680 0x55c8bb835960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55c8bb83ef50 tx=0x7fe5bc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6df3f981a16f94d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5c27fc700 1 -- 172.21.15.53:0/3028455111 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc016040 con 0x55c8bb815680 2023-11-06T01:36:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5c27fc700 1 -- 172.21.15.53:0/3028455111 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5bc017040 con 0x55c8bb815680 2023-11-06T01:36:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.456+0000 7fe5c27fc700 1 -- 172.21.15.53:0/3028455111 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc01ed50 con 0x55c8bb815680 2023-11-06T01:36:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.458+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/3028455111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb815680 msgr2=0x55c8bb835960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.458+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/3028455111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb815680 0x55c8bb835960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55c8bb83ef50 tx=0x7fe5bc009260 comp rx=0 tx=0).stop 2023-11-06T01:36:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.458+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/3028455111 shutdown_connections 2023-11-06T01:36:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.458+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/3028455111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb816ed0 0x55c8bb81a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.458+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/3028455111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb815680 0x55c8bb835960 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.458+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/3028455111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815fe0 0x55c8bb8172b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.458+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/3028455111 >> 172.21.15.53:0/3028455111 conn(0x55c8bb775070 msgr2=0x55c8bb811c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.458+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/3028455111 shutdown_connections 2023-11-06T01:36:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.458+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/3028455111 wait complete. 2023-11-06T01:36:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.458+0000 7fe5ddeaab80 1 -- start start 2023-11-06T01:36:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.458+0000 7fe5ddeaab80 1 Processor -- start 2023-11-06T01:36:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.467+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb885f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.467+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815fe0 0x55c8bb886480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.467+0000 7fe5ddeaab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb88ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.467+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb88d3a0 con 0x55c8bb815680 2023-11-06T01:36:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.467+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb88d4e0 con 0x55c8bb815fe0 2023-11-06T01:36:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.467+0000 7fe5ddeaab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bb88d670 con 0x55c8bb816ed0 2023-11-06T01:36:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.467+0000 7fe5ddeaab80 10 client.0 _ll_register_callbacks cb 0x55c8bb650a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.472+0000 7fe5c3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815fe0 0x55c8bb886480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.472+0000 7fe5cb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb885f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.472+0000 7fe5cb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb885f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40794/0 (socket says 172.21.15.53:40794) 2023-11-06T01:36:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.472+0000 7fe5cb7fe700 1 -- 172.21.15.53:0/2983154304 learned_addr learned my addr 172.21.15.53:0/2983154304 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.473+0000 7fe5cb7fe700 1 -- 172.21.15.53:0/2983154304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 msgr2=0x55c8bb88ce60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:36:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.473+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/2983154304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb88ce60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.473+0000 7fe5cb7fe700 1 -- 172.21.15.53:0/2983154304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815fe0 msgr2=0x55c8bb886480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.473+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/2983154304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815fe0 0x55c8bb886480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.473+0000 7fe5cb7fe700 1 -- 172.21.15.53:0/2983154304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5bc010990 con 0x55c8bb815680 2023-11-06T01:36:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.475+0000 7fe5cb7fe700 1 --2- 172.21.15.53:0/2983154304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb885f40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0094f0 tx=0x7fe5bc01e250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8338e52d0bba9827 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.477+0000 7fe5aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:36:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.477+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2983154304 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc016070 con 0x55c8bb815680 2023-11-06T01:36:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.477+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2983154304 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe5bc005040 con 0x55c8bb815680 2023-11-06T01:36:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.477+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2983154304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5c400baa0 con 0x55c8bb815680 2023-11-06T01:36:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.477+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2983154304 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5bc005200 con 0x55c8bb815680 2023-11-06T01:36:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.477+0000 7fe5ddeaab80 10 client.15834 fetch_fsmap learned FSMap version 33 2023-11-06T01:36:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.477+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2983154304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c8bb593250 con 0x55c8bb815680 2023-11-06T01:36:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.477+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2983154304 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5bc038080 con 0x55c8bb815680 2023-11-06T01:36:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.479+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2983154304 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe5bc032030 con 0x55c8bb815680 2023-11-06T01:36:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.479+0000 7fe5ddeaab80 10 client.15834 fetch_fsmap finished waiting for FSMap version 33 2023-11-06T01:36:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.479+0000 7fe5ddeaab80 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.479+0000 7fe5ddeaab80 20 client.15834 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.479+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2983154304 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c8bb88eff0 0x55c8bb891410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.479+0000 7fe5ddeaab80 4 client.15834 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:36:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.479+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2983154304 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c8bb75fce0 con 0x55c8bb88eff0 2023-11-06T01:36:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.479+0000 7fe5c3fff700 1 --2- 172.21.15.53:0/2983154304 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c8bb88eff0 0x55c8bb891410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.480+0000 7fe5aaffd700 10 client.15834 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.480+0000 7fe5c3fff700 1 --2- 172.21.15.53:0/2983154304 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c8bb88eff0 0x55c8bb891410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=494e234a2ca6632e server_cookie=ba3f9e16e44c43e2 in_seq=0 out_seq=0 2023-11-06T01:36:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5aaffd700 1 -- 172.21.15.53:0/2983154304 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c8bb75fce0 con 0x55c8bb88eff0 2023-11-06T01:36:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5aaffd700 10 client.15834 handle_command_reply: tid=0 2023-11-06T01:36:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5ddeaab80 1 client.15834 shutdown 2023-11-06T01:36:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2983154304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 msgr2=0x55c8bb885f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2983154304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb885f40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0094f0 tx=0x7fe5bc01e250 comp rx=0 tx=0).stop 2023-11-06T01:36:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2983154304 shutdown_connections 2023-11-06T01:36:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2983154304 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c8bb88eff0 0x55c8bb891410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2983154304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c8bb816ed0 0x55c8bb88ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2983154304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bb815680 0x55c8bb885f40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5ddeaab80 1 --2- 172.21.15.53:0/2983154304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8bb815fe0 0x55c8bb886480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5aaffd700 0 client.15834 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.498+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2983154304 >> 172.21.15.53:0/2983154304 conn(0x55c8bb775070 msgr2=0x55c8bb81a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2983154304 shutdown_connections 2023-11-06T01:36:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5ddeaab80 1 -- 172.21.15.53:0/2983154304 wait complete. 2023-11-06T01:36:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5ddeaab80 20 client.15834 trim_cache size 0 max 16384 2023-11-06T01:36:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:36:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:36:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:36:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:36:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/4237695201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe5b80a0670 msgr2=0x7fe5b80a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe5b80a0670 0x7fe5b80a2b30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40062a0 tx=0x7fe5c400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/4237695201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101fa0 msgr2=0x7fe5cc19d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc19d7f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400e9a0 tx=0x7fe5b400ecb0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/4237695201 shutdown_connections 2023-11-06T01:36:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe5b80a0670 0x7fe5b80a2b30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5cc100db0 0x7fe5cc19d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5cc102940 0x7fe5cc19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 --2- 172.21.15.53:0/4237695201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc19d7f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/4237695201 >> 172.21.15.53:0/4237695201 conn(0x7fe5cc0fc340 msgr2=0x7fe5cc10cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/4237695201 shutdown_connections 2023-11-06T01:36:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:38.499+0000 7fe5d1d1a700 1 -- 172.21.15.53:0/4237695201 wait complete. 2023-11-06T01:36:39.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:39 smithi053 ceph-mon[99472]: pgmap v380: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 1.19k op/s 2023-11-06T01:36:39.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:36:39.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:36:39.597 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:39 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:36:39.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:39 smithi062 ceph-mon[103924]: pgmap v380: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 1.19k op/s 2023-11-06T01:36:39.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:36:39.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:36:39.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:39 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:36:39.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:39 smithi163 ceph-mon[104416]: pgmap v380: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 1.19k op/s 2023-11-06T01:36:39.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:36:39.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:36:39.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:39 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:36:40.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:36:40.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:36:40.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:40 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:36:40.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:40 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:36:40.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:40 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:36:40.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:40 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:36:40.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:40 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:36:40.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:40 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:36:40.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:40 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:36:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:41 smithi053 ceph-mon[99472]: pgmap v381: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 MiB/s wr, 1.58k op/s 2023-11-06T01:36:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:41 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:36:41.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:41 smithi062 ceph-mon[103924]: pgmap v381: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 MiB/s wr, 1.58k op/s 2023-11-06T01:36:41.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:41 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:36:41.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:41 smithi163 ceph-mon[104416]: pgmap v381: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 MiB/s wr, 1.58k op/s 2023-11-06T01:36:41.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:41 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:36:42.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 145.0 s, 2908.0 tps, lat 36.528 ms stddev 63.734 2023-11-06T01:36:42.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:42 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:36:42.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:42 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:36:42.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:42 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:36:43.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:43 smithi062 ceph-mon[103924]: pgmap v382: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 2.36k op/s 2023-11-06T01:36:43.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:43 smithi163 ceph-mon[104416]: pgmap v382: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 2.36k op/s 2023-11-06T01:36:43.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:43 smithi053 ceph-mon[99472]: pgmap v382: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 2.36k op/s 2023-11-06T01:36:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:45 smithi062 ceph-mon[103924]: pgmap v383: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 48 MiB/s wr, 2.13k op/s 2023-11-06T01:36:45.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:45 smithi163 ceph-mon[104416]: pgmap v383: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 48 MiB/s wr, 2.13k op/s 2023-11-06T01:36:45.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:45 smithi053 ceph-mon[99472]: pgmap v383: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 48 MiB/s wr, 2.13k op/s 2023-11-06T01:36:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 150.0 s, 3166.3 tps, lat 31.413 ms stddev 34.436 2023-11-06T01:36:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:47 smithi062 ceph-mon[103924]: pgmap v384: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 51 MiB/s wr, 2.23k op/s 2023-11-06T01:36:47.744 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:47 smithi163 ceph-mon[104416]: pgmap v384: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 51 MiB/s wr, 2.23k op/s 2023-11-06T01:36:47.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:47 smithi053 ceph-mon[99472]: pgmap v384: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 51 MiB/s wr, 2.23k op/s 2023-11-06T01:36:49.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:49 smithi062 ceph-mon[103924]: pgmap v385: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 MiB/s wr, 1.99k op/s 2023-11-06T01:36:49.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:49 smithi163 ceph-mon[104416]: pgmap v385: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 MiB/s wr, 1.99k op/s 2023-11-06T01:36:49.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:49 smithi053 ceph-mon[99472]: pgmap v385: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 MiB/s wr, 1.99k op/s 2023-11-06T01:36:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.679+0000 7ff972e45700 1 -- 172.21.15.53:0/4179266791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff96c109800 msgr2=0x7ff96c110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.679+0000 7ff972e45700 1 --2- 172.21.15.53:0/4179266791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff96c109800 0x7ff96c110090 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009990 tx=0x7ff95c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.680+0000 7ff972e45700 1 -- 172.21.15.53:0/4179266791 shutdown_connections 2023-11-06T01:36:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.680+0000 7ff972e45700 1 --2- 172.21.15.53:0/4179266791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff96c109800 0x7ff96c110090 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.680+0000 7ff972e45700 1 --2- 172.21.15.53:0/4179266791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96c108130 0x7ff96c108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.680+0000 7ff972e45700 1 --2- 172.21.15.53:0/4179266791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff96c108e40 0x7ff96c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.680+0000 7ff972e45700 1 -- 172.21.15.53:0/4179266791 >> 172.21.15.53:0/4179266791 conn(0x7ff96c06dbc0 msgr2=0x7ff96c070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.680+0000 7ff972e45700 1 -- 172.21.15.53:0/4179266791 shutdown_connections 2023-11-06T01:36:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.680+0000 7ff972e45700 1 -- 172.21.15.53:0/4179266791 wait complete. 2023-11-06T01:36:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff972e45700 1 Processor -- start 2023-11-06T01:36:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff972e45700 1 -- start start 2023-11-06T01:36:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff972e45700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff96c108130 0x7ff96c1a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff972e45700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff96c108e40 0x7ff96c1a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff972e45700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96c109800 0x7ff96c1a6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff972e45700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c1a72e0 con 0x7ff96c108e40 2023-11-06T01:36:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff972e45700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c1a7450 con 0x7ff96c108130 2023-11-06T01:36:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff972e45700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c1a7610 con 0x7ff96c109800 2023-11-06T01:36:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff970be1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff96c108130 0x7ff96c1a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff96bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff96c108e40 0x7ff96c1a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff9713e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96c109800 0x7ff96c1a6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff970be1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff96c108130 0x7ff96c1a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52240/0 (socket says 172.21.15.53:52240) 2023-11-06T01:36:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff96bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff96c108e40 0x7ff96c1a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40802/0 (socket says 172.21.15.53:40802) 2023-11-06T01:36:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff970be1700 1 -- 172.21.15.53:0/4049599308 learned_addr learned my addr 172.21.15.53:0/4049599308 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff970be1700 1 -- 172.21.15.53:0/4049599308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96c109800 msgr2=0x7ff96c1a6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff970be1700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96c109800 0x7ff96c1a6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff970be1700 1 -- 172.21.15.53:0/4049599308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff96c108e40 msgr2=0x7ff96c1a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff970be1700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff96c108e40 0x7ff96c1a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff970be1700 1 -- 172.21.15.53:0/4049599308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c0095f0 con 0x7ff96c108130 2023-11-06T01:36:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff96bfff700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff96c108e40 0x7ff96c1a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.681+0000 7ff9713e2700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96c109800 0x7ff96c1a6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.683+0000 7ff970be1700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff96c108130 0x7ff96c1a58c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff96000ea80 tx=0x7ff96000ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.683+0000 7ff969ffb700 1 -- 172.21.15.53:0/4049599308 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000ccd0 con 0x7ff96c108130 2023-11-06T01:36:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.683+0000 7ff969ffb700 1 -- 172.21.15.53:0/4049599308 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff96000ce30 con 0x7ff96c108130 2023-11-06T01:36:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.683+0000 7ff969ffb700 1 -- 172.21.15.53:0/4049599308 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960015390 con 0x7ff96c108130 2023-11-06T01:36:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.684+0000 7ff972e45700 1 -- 172.21.15.53:0/4049599308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c1a78f0 con 0x7ff96c108130 2023-11-06T01:36:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.684+0000 7ff972e45700 1 -- 172.21.15.53:0/4049599308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c1ae0b0 con 0x7ff96c108130 2023-11-06T01:36:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.688+0000 7ff969ffb700 1 -- 172.21.15.53:0/4049599308 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff960003680 con 0x7ff96c108130 2023-11-06T01:36:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff97efd5b80 1 Processor -- start 2023-11-06T01:36:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff97efd5b80 1 -- start start 2023-11-06T01:36:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125946aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 0x5611259478f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff969ffb700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff9540a0650 0x7ff9540a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x561125948350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112589e9b0 con 0x561125946fe0 2023-11-06T01:36:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112589c370 con 0x561125947ed0 2023-11-06T01:36:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561125897c60 con 0x561125946680 2023-11-06T01:36:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff9713e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125946aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff970be1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 0x5611259478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff9713e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125946aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55046/0 (socket says 172.21.15.53:55046) 2023-11-06T01:36:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff9713e2700 1 -- 172.21.15.53:0/637716986 learned_addr learned my addr 172.21.15.53:0/637716986 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff969ffb700 1 -- 172.21.15.53:0/4049599308 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff9600cb820 con 0x7ff96c108130 2023-11-06T01:36:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff9713e2700 1 --2- 172.21.15.53:0/637716986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x561125948350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.689+0000 7ff96bfff700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff9540a0650 0x7ff9540a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.691+0000 7ff9713e2700 1 -- 172.21.15.53:0/637716986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 msgr2=0x561125948350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.691+0000 7ff9713e2700 1 --2- 172.21.15.53:0/637716986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x561125948350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.691+0000 7ff9713e2700 1 -- 172.21.15.53:0/637716986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 msgr2=0x5611259478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.691+0000 7ff9713e2700 1 --2- 172.21.15.53:0/637716986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 0x5611259478f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.691+0000 7ff9713e2700 1 -- 172.21.15.53:0/637716986 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611256c5d80 con 0x561125946680 2023-11-06T01:36:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.691+0000 7ff9713e2700 1 --2- 172.21.15.53:0/637716986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125946aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5611259476b0 tx=0x7ff95c01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7575be21347ee2b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.691+0000 7ff96bfff700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff9540a0650 0x7ff9540a2b10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff96c107e50 tx=0x7ff96400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.691+0000 7ff952ffd700 1 -- 172.21.15.53:0/637716986 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00f460 con 0x561125946680 2023-11-06T01:36:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.691+0000 7ff952ffd700 1 -- 172.21.15.53:0/637716986 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff95c00fa80 con 0x561125946680 2023-11-06T01:36:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.691+0000 7ff952ffd700 1 -- 172.21.15.53:0/637716986 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0179c0 con 0x561125946680 2023-11-06T01:36:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.692+0000 7ff97efd5b80 1 -- 172.21.15.53:0/637716986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 msgr2=0x561125946aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.692+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/637716986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125946aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5611259476b0 tx=0x7ff95c01a040 comp rx=0 tx=0).stop 2023-11-06T01:36:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.693+0000 7ff97efd5b80 1 -- 172.21.15.53:0/637716986 shutdown_connections 2023-11-06T01:36:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.693+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/637716986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x561125948350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.693+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/637716986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125946aa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.693+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/637716986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 0x5611259478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.693+0000 7ff97efd5b80 1 -- 172.21.15.53:0/637716986 >> 172.21.15.53:0/637716986 conn(0x5611258a6070 msgr2=0x561125943040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.693+0000 7ff97efd5b80 1 -- 172.21.15.53:0/637716986 shutdown_connections 2023-11-06T01:36:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.693+0000 7ff97efd5b80 1 -- 172.21.15.53:0/637716986 wait complete. 2023-11-06T01:36:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.693+0000 7ff97efd5b80 1 -- start start 2023-11-06T01:36:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.693+0000 7ff97efd5b80 1 Processor -- start 2023-11-06T01:36:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125950540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 0x561125950a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x561125957ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611258344b0 con 0x561125946fe0 2023-11-06T01:36:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561125834150 con 0x561125947ed0 2023-11-06T01:36:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112571b8c0 con 0x561125946680 2023-11-06T01:36:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff97efd5b80 10 client.0 _ll_register_callbacks cb 0x561125781a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff970be1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125950540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff970be1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125950540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55054/0 (socket says 172.21.15.53:55054) 2023-11-06T01:36:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff970be1700 1 -- 172.21.15.53:0/208952280 learned_addr learned my addr 172.21.15.53:0/208952280 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff9713e2700 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x561125957ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff96bfff700 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 0x561125950a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff970be1700 1 -- 172.21.15.53:0/208952280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 msgr2=0x561125957ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff970be1700 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x561125957ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff970be1700 1 -- 172.21.15.53:0/208952280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 msgr2=0x561125950a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff970be1700 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 0x561125950a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff970be1700 1 -- 172.21.15.53:0/208952280 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c020040 con 0x561125946680 2023-11-06T01:36:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.694+0000 7ff9713e2700 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x561125957ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:36:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.695+0000 7ff970be1700 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125950540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff960095560 tx=0x7ff96001bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d56dfce8fa40eff server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.695+0000 7ff94bfff700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:36:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.695+0000 7ff94bfff700 1 -- 172.21.15.53:0/208952280 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9600989c0 con 0x561125946680 2023-11-06T01:36:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.695+0000 7ff94bfff700 1 -- 172.21.15.53:0/208952280 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff960099760 con 0x561125946680 2023-11-06T01:36:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.696+0000 7ff97efd5b80 1 -- 172.21.15.53:0/208952280 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611258c2cb0 con 0x561125946680 2023-11-06T01:36:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.696+0000 7ff94bfff700 1 -- 172.21.15.53:0/208952280 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9600795e0 con 0x561125946680 2023-11-06T01:36:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.696+0000 7ff97efd5b80 10 client.25492 fetch_fsmap learned FSMap version 36 2023-11-06T01:36:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.696+0000 7ff97efd5b80 1 -- 172.21.15.53:0/208952280 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5611256aec00 con 0x561125946680 2023-11-06T01:36:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.697+0000 7ff94bfff700 1 -- 172.21.15.53:0/208952280 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5611258c2cb0 con 0x561125946680 2023-11-06T01:36:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.697+0000 7ff94bfff700 1 -- 172.21.15.53:0/208952280 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff960098550 con 0x561125946680 2023-11-06T01:36:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.697+0000 7ff97efd5b80 10 client.25492 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T01:36:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.697+0000 7ff97efd5b80 10 client.25492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.697+0000 7ff97efd5b80 20 client.25492 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.697+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56112595c350 0x56112595e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.697+0000 7ff97efd5b80 4 client.25492 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.697+0000 7ff97efd5b80 1 -- 172.21.15.53:0/208952280 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561125915db0 con 0x56112595c350 2023-11-06T01:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.697+0000 7ff96bfff700 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56112595c350 0x56112595e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.699+0000 7ff96bfff700 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56112595c350 0x56112595e770 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5dc10090dd4fb84 server_cookie=d02f21e6cbfc67cf in_seq=0 out_seq=0 2023-11-06T01:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.699+0000 7ff94bfff700 10 client.25492 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff94bfff700 1 -- 172.21.15.53:0/208952280 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561125915db0 con 0x56112595c350 2023-11-06T01:36:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff94bfff700 10 client.25492 handle_command_reply: tid=0 2023-11-06T01:36:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff97efd5b80 1 client.25492 shutdown 2023-11-06T01:36:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff97efd5b80 1 -- 172.21.15.53:0/208952280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 msgr2=0x561125950540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125950540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff960095560 tx=0x7ff96001bef0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff97efd5b80 1 -- 172.21.15.53:0/208952280 shutdown_connections 2023-11-06T01:36:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x561125957ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56112595c350 0x56112595e770 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x561125950540 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff94bfff700 0 client.25492 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/208952280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 0x561125950a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff97efd5b80 1 -- 172.21.15.53:0/208952280 >> 172.21.15.53:0/208952280 conn(0x5611258a6070 msgr2=0x5611259413c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.701+0000 7ff97efd5b80 1 -- 172.21.15.53:0/208952280 shutdown_connections 2023-11-06T01:36:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.702+0000 7ff97efd5b80 1 -- 172.21.15.53:0/208952280 wait complete. 2023-11-06T01:36:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.703+0000 7ff97efd5b80 20 client.25492 trim_cache size 0 max 16384 2023-11-06T01:36:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.725+0000 7ff97efd5b80 1 Processor -- start 2023-11-06T01:36:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.726+0000 7ff97efd5b80 1 -- start start 2023-11-06T01:36:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.726+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946680 0x561125955a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.726+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125946fe0 0x5611259582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.726+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125947ed0 0x56112594b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.726+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611258c2ce0 con 0x561125946680 2023-11-06T01:36:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.726+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611259477b0 con 0x561125946fe0 2023-11-06T01:36:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.726+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611259397a0 con 0x561125947ed0 2023-11-06T01:36:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.726+0000 7ff9713e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125947ed0 0x56112594b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.726+0000 7ff9713e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125947ed0 0x56112594b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55064/0 (socket says 172.21.15.53:55064) 2023-11-06T01:36:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.726+0000 7ff9713e2700 1 -- 172.21.15.53:0/2784714154 learned_addr learned my addr 172.21.15.53:0/2784714154 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.727+0000 7ff96bfff700 1 --2- 172.21.15.53:0/2784714154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125946fe0 0x5611259582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.727+0000 7ff970be1700 1 --2- 172.21.15.53:0/2784714154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946680 0x561125955a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.727+0000 7ff9713e2700 1 -- 172.21.15.53:0/2784714154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125946fe0 msgr2=0x5611259582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.727+0000 7ff9713e2700 1 --2- 172.21.15.53:0/2784714154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125946fe0 0x5611259582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.727+0000 7ff9713e2700 1 -- 172.21.15.53:0/2784714154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946680 msgr2=0x561125955a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.727+0000 7ff9713e2700 1 --2- 172.21.15.53:0/2784714154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946680 0x561125955a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.727+0000 7ff9713e2700 1 -- 172.21.15.53:0/2784714154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611256ec1f0 con 0x561125947ed0 2023-11-06T01:36:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.728+0000 7ff9713e2700 1 --2- 172.21.15.53:0/2784714154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125947ed0 0x56112594b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5611259476b0 tx=0x7ff95c005cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66601c3dc63d5388 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.728+0000 7ff9537fe700 1 -- 172.21.15.53:0/2784714154 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00f460 con 0x561125947ed0 2023-11-06T01:36:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.728+0000 7ff9537fe700 1 -- 172.21.15.53:0/2784714154 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff95c005040 con 0x561125947ed0 2023-11-06T01:36:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.728+0000 7ff9537fe700 1 -- 172.21.15.53:0/2784714154 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c004890 con 0x561125947ed0 2023-11-06T01:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.729+0000 7ff97efd5b80 1 -- 172.21.15.53:0/2784714154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125947ed0 msgr2=0x56112594b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.729+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/2784714154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125947ed0 0x56112594b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5611259476b0 tx=0x7ff95c005cd0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.729+0000 7ff97efd5b80 1 -- 172.21.15.53:0/2784714154 shutdown_connections 2023-11-06T01:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.729+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/2784714154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125947ed0 0x56112594b0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.729+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/2784714154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946680 0x561125955a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.729+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/2784714154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125946fe0 0x5611259582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.729+0000 7ff97efd5b80 1 -- 172.21.15.53:0/2784714154 >> 172.21.15.53:0/2784714154 conn(0x5611258a6070 msgr2=0x56112594e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.729+0000 7ff97efd5b80 1 -- 172.21.15.53:0/2784714154 shutdown_connections 2023-11-06T01:36:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.730+0000 7ff97efd5b80 1 -- 172.21.15.53:0/2784714154 wait complete. 2023-11-06T01:36:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.730+0000 7ff97efd5b80 1 -- start start 2023-11-06T01:36:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.730+0000 7ff97efd5b80 1 Processor -- start 2023-11-06T01:36:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x5611259a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 0x5611259aa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff97efd5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x5611259ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561125955da0 con 0x561125946fe0 2023-11-06T01:36:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611259ace10 con 0x561125947ed0 2023-11-06T01:36:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff97efd5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611259acf50 con 0x561125946680 2023-11-06T01:36:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff97efd5b80 10 client.0 _ll_register_callbacks cb 0x7ff960093b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:36:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff9713e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x5611259ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff9713e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x5611259ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52266/0 (socket says 172.21.15.53:52266) 2023-11-06T01:36:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff9713e2700 1 -- 172.21.15.53:0/1315425089 learned_addr learned my addr 172.21.15.53:0/1315425089 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff970be1700 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x5611259a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff9713e2700 1 -- 172.21.15.53:0/1315425089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 msgr2=0x5611259a7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff9713e2700 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x5611259a7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff9713e2700 1 -- 172.21.15.53:0/1315425089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 msgr2=0x5611259aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:36:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff9713e2700 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 0x5611259aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.731+0000 7ff9713e2700 1 -- 172.21.15.53:0/1315425089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c02d040 con 0x561125947ed0 2023-11-06T01:36:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.732+0000 7ff9713e2700 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x5611259ac8d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0045c0 tx=0x7ff95c005490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=530f13a8396c4401 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:36:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.732+0000 7ff94bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:36:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.732+0000 7ff94bfff700 1 -- 172.21.15.53:0/1315425089 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00f670 con 0x561125947ed0 2023-11-06T01:36:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.732+0000 7ff97efd5b80 1 -- 172.21.15.53:0/1315425089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff95c00fa80 con 0x561125947ed0 2023-11-06T01:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.732+0000 7ff94bfff700 1 -- 172.21.15.53:0/1315425089 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff95c00f7d0 con 0x561125947ed0 2023-11-06T01:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.732+0000 7ff94bfff700 1 -- 172.21.15.53:0/1315425089 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff95c004810 con 0x561125947ed0 2023-11-06T01:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.732+0000 7ff97efd5b80 10 client.25421 fetch_fsmap learned FSMap version 36 2023-11-06T01:36:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.732+0000 7ff97efd5b80 1 -- 172.21.15.53:0/1315425089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5611256c4250 con 0x561125947ed0 2023-11-06T01:36:51.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:51 smithi163 ceph-mon[104416]: pgmap v386: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 MiB/s wr, 1.89k op/s 2023-11-06T01:36:51.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:51 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:36:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.733+0000 7ff94bfff700 1 -- 172.21.15.53:0/1315425089 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff95c00fa80 con 0x561125947ed0 2023-11-06T01:36:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.733+0000 7ff94bfff700 1 -- 172.21.15.53:0/1315425089 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff95c043020 con 0x561125947ed0 2023-11-06T01:36:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.733+0000 7ff97efd5b80 10 client.25421 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T01:36:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.733+0000 7ff97efd5b80 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:36:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.733+0000 7ff97efd5b80 20 client.25421 populate_metadata read hostname 'smithi053' 2023-11-06T01:36:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.733+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5611259ae4d0 0x5611259b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:36:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.733+0000 7ff97efd5b80 4 client.25421 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:36:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.733+0000 7ff97efd5b80 1 -- 172.21.15.53:0/1315425089 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561125890ce0 con 0x5611259ae4d0 2023-11-06T01:36:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.733+0000 7ff970be1700 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5611259ae4d0 0x5611259b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:36:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.734+0000 7ff94bfff700 10 client.25421 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.734+0000 7ff970be1700 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5611259ae4d0 0x5611259b08f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc99d8253e178095 server_cookie=bfc066fb62534914 in_seq=0 out_seq=0 2023-11-06T01:36:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.748+0000 7ff94bfff700 1 -- 172.21.15.53:0/1315425089 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561125890ce0 con 0x5611259ae4d0 2023-11-06T01:36:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.748+0000 7ff94bfff700 10 client.25421 handle_command_reply: tid=0 2023-11-06T01:36:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.748+0000 7ff97efd5b80 1 client.25421 shutdown 2023-11-06T01:36:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff97efd5b80 1 -- 172.21.15.53:0/1315425089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 msgr2=0x5611259ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x5611259ac8d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0045c0 tx=0x7ff95c005490 comp rx=0 tx=0).stop 2023-11-06T01:36:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff97efd5b80 1 -- 172.21.15.53:0/1315425089 shutdown_connections 2023-11-06T01:36:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561125947ed0 0x5611259ac8d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5611259ae4d0 0x5611259b08f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561125946680 0x5611259a7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff94bfff700 0 client.25421 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:36:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff97efd5b80 1 --2- 172.21.15.53:0/1315425089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561125946fe0 0x5611259aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff97efd5b80 1 -- 172.21.15.53:0/1315425089 >> 172.21.15.53:0/1315425089 conn(0x5611258a6070 msgr2=0x5611259582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff97efd5b80 1 -- 172.21.15.53:0/1315425089 shutdown_connections 2023-11-06T01:36:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff97efd5b80 1 -- 172.21.15.53:0/1315425089 wait complete. 2023-11-06T01:36:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.749+0000 7ff97efd5b80 20 client.25421 trim_cache size 0 max 16384 2023-11-06T01:36:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:36:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:36:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:36:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:36:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 -- 172.21.15.53:0/4049599308 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff9540a0650 msgr2=0x7ff9540a2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff9540a0650 0x7ff9540a2b10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff96c107e50 tx=0x7ff96400b040 comp rx=0 tx=0).stop 2023-11-06T01:36:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 -- 172.21.15.53:0/4049599308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff96c108130 msgr2=0x7ff96c1a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:36:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff96c108130 0x7ff96c1a58c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff96000ea80 tx=0x7ff96000ed90 comp rx=0 tx=0).stop 2023-11-06T01:36:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 -- 172.21.15.53:0/4049599308 shutdown_connections 2023-11-06T01:36:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff9540a0650 0x7ff9540a2b10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff96c108130 0x7ff96c1a58c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96c109800 0x7ff96c1a6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 --2- 172.21.15.53:0/4049599308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff96c108e40 0x7ff96c1a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:36:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 -- 172.21.15.53:0/4049599308 >> 172.21.15.53:0/4049599308 conn(0x7ff96c06dbc0 msgr2=0x7ff96c070e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:36:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 -- 172.21.15.53:0/4049599308 shutdown_connections 2023-11-06T01:36:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:36:51.750+0000 7ff972e45700 1 -- 172.21.15.53:0/4049599308 wait complete. 2023-11-06T01:36:51.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:51 smithi053 ceph-mon[99472]: pgmap v386: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 MiB/s wr, 1.89k op/s 2023-11-06T01:36:51.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:51 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:36:51.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:51 smithi062 ceph-mon[103924]: pgmap v386: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 MiB/s wr, 1.89k op/s 2023-11-06T01:36:51.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:51 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:36:52.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 155.0 s, 3288.0 tps, lat 30.493 ms stddev 33.839 2023-11-06T01:36:53.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:53 smithi163 ceph-mon[104416]: pgmap v387: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.65k op/s 2023-11-06T01:36:53.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:53 smithi053 ceph-mon[99472]: pgmap v387: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.65k op/s 2023-11-06T01:36:53.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:53 smithi062 ceph-mon[103924]: pgmap v387: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.65k op/s 2023-11-06T01:36:55.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:55 smithi163 ceph-mon[104416]: pgmap v388: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 816 op/s 2023-11-06T01:36:55.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:55 smithi053 ceph-mon[99472]: pgmap v388: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 816 op/s 2023-11-06T01:36:55.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:55 smithi062 ceph-mon[103924]: pgmap v388: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 816 op/s 2023-11-06T01:36:57.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 160.0 s, 1944.2 tps, lat 51.308 ms stddev 58.884 2023-11-06T01:36:57.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:57 smithi053 ceph-mon[99472]: pgmap v389: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 1.09k op/s 2023-11-06T01:36:57.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:57 smithi062 ceph-mon[103924]: pgmap v389: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 1.09k op/s 2023-11-06T01:36:57.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:57 smithi163 ceph-mon[104416]: pgmap v389: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 1.09k op/s 2023-11-06T01:36:59.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:36:59 smithi053 ceph-mon[99472]: pgmap v390: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 1.05k op/s 2023-11-06T01:36:59.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:36:59 smithi062 ceph-mon[103924]: pgmap v390: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 1.05k op/s 2023-11-06T01:36:59.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:36:59 smithi163 ceph-mon[104416]: pgmap v390: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 1.05k op/s 2023-11-06T01:37:01.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:01 smithi053 ceph-mon[99472]: pgmap v391: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 1.11k op/s 2023-11-06T01:37:01.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:01 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:01.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:01 smithi062 ceph-mon[103924]: pgmap v391: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 1.11k op/s 2023-11-06T01:37:01.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:01 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:01.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:01 smithi163 ceph-mon[104416]: pgmap v391: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 1.11k op/s 2023-11-06T01:37:01.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:01 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:02.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 165.0 s, 2057.0 tps, lat 48.172 ms stddev 52.811 2023-11-06T01:37:03.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:03 smithi062 ceph-mon[103924]: pgmap v392: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 1.26k op/s 2023-11-06T01:37:03.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:03 smithi163 ceph-mon[104416]: pgmap v392: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 1.26k op/s 2023-11-06T01:37:03.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:03 smithi053 ceph-mon[99472]: pgmap v392: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 1.26k op/s 2023-11-06T01:37:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.919+0000 7f93edf3c700 1 -- 172.21.15.53:0/1277575297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e8101830 msgr2=0x7f93e81080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.919+0000 7f93edf3c700 1 --2- 172.21.15.53:0/1277575297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e8101830 0x7f93e81080e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f93e000b0a0 tx=0x7f93e000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.919+0000 7f93edf3c700 1 -- 172.21.15.53:0/1277575297 shutdown_connections 2023-11-06T01:37:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.919+0000 7f93edf3c700 1 --2- 172.21.15.53:0/1277575297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93e80ffca0 0x7f93e81000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.919+0000 7f93edf3c700 1 --2- 172.21.15.53:0/1277575297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e8101830 0x7f93e81080e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.919+0000 7f93edf3c700 1 --2- 172.21.15.53:0/1277575297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93e8100e90 0x7f93e81012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.919+0000 7f93edf3c700 1 -- 172.21.15.53:0/1277575297 >> 172.21.15.53:0/1277575297 conn(0x7f93e80fb230 msgr2=0x7f93e80fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.919+0000 7f93edf3c700 1 -- 172.21.15.53:0/1277575297 shutdown_connections 2023-11-06T01:37:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.919+0000 7f93edf3c700 1 -- 172.21.15.53:0/1277575297 wait complete. 2023-11-06T01:37:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93edf3c700 1 Processor -- start 2023-11-06T01:37:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93edf3c700 1 -- start start 2023-11-06T01:37:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93edf3c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93e80ffca0 0x7f93e8194ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93edf3c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e8100e90 0x7f93e8195000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93edf3c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93e8101830 0x7f93e8195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93edf3c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e81964e0 con 0x7f93e8100e90 2023-11-06T01:37:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93edf3c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8196650 con 0x7f93e8101830 2023-11-06T01:37:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93edf3c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8196810 con 0x7f93e80ffca0 2023-11-06T01:37:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93e6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e8100e90 0x7f93e8195000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93e7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93e8101830 0x7f93e8195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93e6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e8100e90 0x7f93e8195000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40844/0 (socket says 172.21.15.53:40844) 2023-11-06T01:37:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93e77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93e80ffca0 0x7f93e8194ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93e7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93e8101830 0x7f93e8195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52284/0 (socket says 172.21.15.53:52284) 2023-11-06T01:37:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.920+0000 7f93e6ffd700 1 -- 172.21.15.53:0/1398622400 learned_addr learned my addr 172.21.15.53:0/1398622400 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93e6ffd700 1 -- 172.21.15.53:0/1398622400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93e80ffca0 msgr2=0x7f93e8194ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93e80ffca0 0x7f93e8194ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93e6ffd700 1 -- 172.21.15.53:0/1398622400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93e8101830 msgr2=0x7f93e8195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93e8101830 0x7f93e8195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93e6ffd700 1 -- 172.21.15.53:0/1398622400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e0009d00 con 0x7f93e8100e90 2023-11-06T01:37:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93e7fff700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93e8101830 0x7f93e8195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:37:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e8100e90 0x7f93e8195000 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e9a0 tx=0x7f93dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93e4ff9700 1 -- 172.21.15.53:0/1398622400 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc009790 con 0x7f93e8100e90 2023-11-06T01:37:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93edf3c700 1 -- 172.21.15.53:0/1398622400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e8196af0 con 0x7f93e8100e90 2023-11-06T01:37:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93e4ff9700 1 -- 172.21.15.53:0/1398622400 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93dc010430 con 0x7f93e8100e90 2023-11-06T01:37:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93e4ff9700 1 -- 172.21.15.53:0/1398622400 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc018680 con 0x7f93e8100e90 2023-11-06T01:37:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.921+0000 7f93edf3c700 1 -- 172.21.15.53:0/1398622400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e819d2b0 con 0x7f93e8100e90 2023-11-06T01:37:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.926+0000 7f93e4ff9700 1 -- 172.21.15.53:0/1398622400 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f93dc0105a0 con 0x7f93e8100e90 2023-11-06T01:37:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.926+0000 7f93fa0ccb80 1 Processor -- start 2023-11-06T01:37:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.926+0000 7f93fa0ccb80 1 -- start start 2023-11-06T01:37:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.926+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69680 0x55dacde69aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.926+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde69fe0 0x55dacde6a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.926+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde6aed0 0x55dacde6b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.926+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacddc19b0 con 0x55dacde6aed0 2023-11-06T01:37:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.926+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacddbf370 con 0x55dacde69680 2023-11-06T01:37:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.926+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacddbac60 con 0x55dacde69fe0 2023-11-06T01:37:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69680 0x55dacde69aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69680 0x55dacde69aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52286/0 (socket says 172.21.15.53:52286) 2023-11-06T01:37:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e7fff700 1 -- 172.21.15.53:0/1944301485 learned_addr learned my addr 172.21.15.53:0/1944301485 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e77fe700 1 --2- 172.21.15.53:0/1944301485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde69fe0 0x55dacde6a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e7fff700 1 -- 172.21.15.53:0/1944301485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde69fe0 msgr2=0x55dacde6a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e7fff700 1 --2- 172.21.15.53:0/1944301485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde69fe0 0x55dacde6a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e7fff700 1 -- 172.21.15.53:0/1944301485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde6aed0 msgr2=0x55dacde6b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e7fff700 1 --2- 172.21.15.53:0/1944301485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde6aed0 0x55dacde6b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e7fff700 1 -- 172.21.15.53:0/1944301485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dacdbe8d80 con 0x55dacde69680 2023-11-06T01:37:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e4ff9700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93d00a4930 0x7f93d00a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e7fff700 1 --2- 172.21.15.53:0/1944301485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69680 0x55dacde69aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f93e000b0a0 tx=0x7f93e00093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfcc84de191f15bf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.927+0000 7f93e4ff9700 1 -- 172.21.15.53:0/1398622400 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f93dc014070 con 0x7f93e8100e90 2023-11-06T01:37:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.928+0000 7f93cdffb700 1 -- 172.21.15.53:0/1944301485 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e000e040 con 0x55dacde69680 2023-11-06T01:37:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.928+0000 7f93cdffb700 1 -- 172.21.15.53:0/1944301485 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e0007760 con 0x55dacde69680 2023-11-06T01:37:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.928+0000 7f93cdffb700 1 -- 172.21.15.53:0/1944301485 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0005460 con 0x55dacde69680 2023-11-06T01:37:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.928+0000 7f93e77fe700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93d00a4930 0x7f93d00a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.929+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1944301485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69680 msgr2=0x55dacde69aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.929+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1944301485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69680 0x55dacde69aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f93e000b0a0 tx=0x7f93e00093f0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.929+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1944301485 shutdown_connections 2023-11-06T01:37:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.929+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1944301485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69680 0x55dacde69aa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.929+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1944301485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde6aed0 0x55dacde6b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.929+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1944301485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde69fe0 0x55dacde6a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.929+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1944301485 >> 172.21.15.53:0/1944301485 conn(0x55dacddc9070 msgr2=0x55dacde66040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.929+0000 7f93e77fe700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93d00a4930 0x7f93d00a6df0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f93e8100cf0 tx=0x7f93d800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.929+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1944301485 shutdown_connections 2023-11-06T01:37:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.929+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1944301485 wait complete. 2023-11-06T01:37:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.929+0000 7f93fa0ccb80 1 -- start start 2023-11-06T01:37:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93fa0ccb80 1 Processor -- start 2023-11-06T01:37:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde69fe0 0x55dacde73a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde6aed0 0x55dacde7aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacdd574b0 con 0x55dacde69680 2023-11-06T01:37:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacdd57150 con 0x55dacde6aed0 2023-11-06T01:37:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacdc3e8c0 con 0x55dacde69fe0 2023-11-06T01:37:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93fa0ccb80 10 client.0 _ll_register_callbacks cb 0x55dacdca4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93e77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93e77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde734d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40856/0 (socket says 172.21.15.53:40856) 2023-11-06T01:37:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93e77fe700 1 -- 172.21.15.53:0/1091900581 learned_addr learned my addr 172.21.15.53:0/1091900581 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93e7fff700 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde6aed0 0x55dacde7aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde69fe0 0x55dacde73a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93e77fe700 1 -- 172.21.15.53:0/1091900581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde69fe0 msgr2=0x55dacde73a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93e77fe700 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde69fe0 0x55dacde73a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93e77fe700 1 -- 172.21.15.53:0/1091900581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde6aed0 msgr2=0x55dacde7aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93e77fe700 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde6aed0 0x55dacde7aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93e77fe700 1 -- 172.21.15.53:0/1091900581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e80693d0 con 0x55dacde69680 2023-11-06T01:37:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.930+0000 7f93e7fff700 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde6aed0 0x55dacde7aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:37:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.931+0000 7f93e77fe700 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde734d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f93d80109e0 tx=0x7f93d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be420109c59f3b3d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.931+0000 7f93c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:37:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.931+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1091900581 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8012a50 con 0x55dacde69680 2023-11-06T01:37:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.931+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1091900581 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f93d8004830 con 0x55dacde69680 2023-11-06T01:37:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.931+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1091900581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dacdde5cb0 con 0x55dacde69680 2023-11-06T01:37:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.933+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1091900581 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93d80049f0 con 0x55dacde69680 2023-11-06T01:37:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.933+0000 7f93fa0ccb80 10 client.15858 fetch_fsmap learned FSMap version 37 2023-11-06T01:37:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.933+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1091900581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55dacdbd1c00 con 0x55dacde69680 2023-11-06T01:37:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.933+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1091900581 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dacdde5cb0 con 0x55dacde69680 2023-11-06T01:37:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.933+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1091900581 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f93d801bac0 con 0x55dacde69680 2023-11-06T01:37:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.933+0000 7f93fa0ccb80 10 client.15858 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T01:37:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.933+0000 7f93fa0ccb80 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.933+0000 7f93fa0ccb80 20 client.15858 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.934+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dacde7f3b0 0x55dacde817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.934+0000 7f93fa0ccb80 4 client.15858 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:37:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.934+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1091900581 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dacde38db0 con 0x55dacde7f3b0 2023-11-06T01:37:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.934+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dacde7f3b0 0x55dacde817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.934+0000 7f93c6ffd700 10 client.15858 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.934+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dacde7f3b0 0x55dacde817d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416bf537a7b1923f server_cookie=10d2edd64cadd66a in_seq=0 out_seq=0 2023-11-06T01:37:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1091900581 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55dacde38db0 con 0x55dacde7f3b0 2023-11-06T01:37:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93c6ffd700 10 client.15858 handle_command_reply: tid=0 2023-11-06T01:37:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93fa0ccb80 1 client.15858 shutdown 2023-11-06T01:37:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1091900581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 msgr2=0x55dacde734d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde734d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f93d80109e0 tx=0x7f93d8004750 comp rx=0 tx=0).stop 2023-11-06T01:37:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1091900581 shutdown_connections 2023-11-06T01:37:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde6aed0 0x55dacde7aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dacde7f3b0 0x55dacde817d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde734d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93c6ffd700 0 client.15858 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1091900581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde69fe0 0x55dacde73a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1091900581 >> 172.21.15.53:0/1091900581 conn(0x55dacddc9070 msgr2=0x55dacde643c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1091900581 shutdown_connections 2023-11-06T01:37:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.935+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1091900581 wait complete. 2023-11-06T01:37:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.936+0000 7f93fa0ccb80 20 client.15858 trim_cache size 0 max 16384 2023-11-06T01:37:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.955+0000 7f93fa0ccb80 1 Processor -- start 2023-11-06T01:37:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.955+0000 7f93fa0ccb80 1 -- start start 2023-11-06T01:37:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.955+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde69a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.955+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 0x55dacde78a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.955+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde6aed0 0x55dacde72b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.955+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacde71f30 con 0x55dacde69680 2023-11-06T01:37:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.955+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacde5f500 con 0x55dacde69fe0 2023-11-06T01:37:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.955+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacde5cec0 con 0x55dacde6aed0 2023-11-06T01:37:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.956+0000 7f93e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 0x55dacde78a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.956+0000 7f93e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 0x55dacde78a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52302/0 (socket says 172.21.15.53:52302) 2023-11-06T01:37:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.956+0000 7f93e77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde69a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.956+0000 7f93e6ffd700 1 -- 172.21.15.53:0/3244850110 learned_addr learned my addr 172.21.15.53:0/3244850110 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.956+0000 7f93e6ffd700 1 -- 172.21.15.53:0/3244850110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde6aed0 msgr2=0x55dacde72b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:37:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.956+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/3244850110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde6aed0 0x55dacde72b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.956+0000 7f93e6ffd700 1 -- 172.21.15.53:0/3244850110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 msgr2=0x55dacde69a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.956+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/3244850110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde69a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.956+0000 7f93e6ffd700 1 -- 172.21.15.53:0/3244850110 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dacdc0f1f0 con 0x55dacde69fe0 2023-11-06T01:37:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.956+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/3244850110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 0x55dacde78a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0958b0 tx=0x7f93dc01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83270ff0e674ff17 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.957+0000 7f93ce7fc700 1 -- 172.21.15.53:0/3244850110 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc099ab0 con 0x55dacde69fe0 2023-11-06T01:37:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.957+0000 7f93ce7fc700 1 -- 172.21.15.53:0/3244850110 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93dc0993b0 con 0x55dacde69fe0 2023-11-06T01:37:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.957+0000 7f93ce7fc700 1 -- 172.21.15.53:0/3244850110 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc097820 con 0x55dacde69fe0 2023-11-06T01:37:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.958+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/3244850110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 msgr2=0x55dacde78a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.958+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/3244850110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 0x55dacde78a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0958b0 tx=0x7f93dc01a040 comp rx=0 tx=0).stop 2023-11-06T01:37:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.958+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/3244850110 shutdown_connections 2023-11-06T01:37:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.958+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/3244850110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde6aed0 0x55dacde72b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.958+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/3244850110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde69a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.958+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/3244850110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 0x55dacde78a00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.958+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/3244850110 >> 172.21.15.53:0/3244850110 conn(0x55dacddc9070 msgr2=0x55dacde72150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.958+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/3244850110 shutdown_connections 2023-11-06T01:37:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.958+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/3244850110 wait complete. 2023-11-06T01:37:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.959+0000 7f93fa0ccb80 1 -- start start 2023-11-06T01:37:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.959+0000 7f93fa0ccb80 1 Processor -- start 2023-11-06T01:37:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.959+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.959+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 0x55dacded5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.959+0000 7f93fa0ccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde6aed0 0x55dacded7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.959+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacded8440 con 0x55dacde69680 2023-11-06T01:37:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.959+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacded85b0 con 0x55dacde69fe0 2023-11-06T01:37:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.959+0000 7f93fa0ccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacded8770 con 0x55dacde6aed0 2023-11-06T01:37:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.959+0000 7f93fa0ccb80 10 client.0 _ll_register_callbacks cb 0x7f93d8012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.959+0000 7f93e77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 0x55dacded5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93e77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde769f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40868/0 (socket says 172.21.15.53:40868) 2023-11-06T01:37:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93e77fe700 1 -- 172.21.15.53:0/1710033574 learned_addr learned my addr 172.21.15.53:0/1710033574 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93e7fff700 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde6aed0 0x55dacded7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93e6ffd700 1 -- 172.21.15.53:0/1710033574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde6aed0 msgr2=0x55dacded7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde6aed0 0x55dacded7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93e6ffd700 1 -- 172.21.15.53:0/1710033574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 msgr2=0x55dacde769f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde769f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93e6ffd700 1 -- 172.21.15.53:0/1710033574 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e819d2b0 con 0x55dacde69fe0 2023-11-06T01:37:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93e6ffd700 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 0x55dacded5670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0958b0 tx=0x7f93dc099520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969e54eb8f3b749d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:37:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1710033574 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc09d1e0 con 0x55dacde69fe0 2023-11-06T01:37:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1710033574 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93e0007760 con 0x55dacde69fe0 2023-11-06T01:37:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1710033574 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f93dc0a1420 con 0x55dacde69fe0 2023-11-06T01:37:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1710033574 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93dc099ab0 con 0x55dacde69fe0 2023-11-06T01:37:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93fa0ccb80 10 client.25436 fetch_fsmap learned FSMap version 37 2023-11-06T01:37:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.960+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1710033574 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55dacdbe7250 con 0x55dacde69fe0 2023-11-06T01:37:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.961+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1710033574 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e0007760 con 0x55dacde69fe0 2023-11-06T01:37:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.962+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1710033574 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f93dc0b7230 con 0x55dacde69fe0 2023-11-06T01:37:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.962+0000 7f93fa0ccb80 10 client.25436 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T01:37:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.962+0000 7f93fa0ccb80 10 client.25436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.962+0000 7f93fa0ccb80 20 client.25436 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.962+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dacded9d40 0x55dacdedc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.962+0000 7f93fa0ccb80 4 client.25436 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:37:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.962+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1710033574 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dacddb3ce0 con 0x55dacded9d40 2023-11-06T01:37:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.962+0000 7f93e77fe700 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dacded9d40 0x55dacdedc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.968+0000 7f93e77fe700 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dacded9d40 0x55dacdedc160 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5fbcadaf3997c2a server_cookie=abff65e2e8f95a3c in_seq=0 out_seq=0 2023-11-06T01:37:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:04.968+0000 7f93c6ffd700 10 client.25436 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.001+0000 7f93c6ffd700 1 -- 172.21.15.53:0/1710033574 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dacddb3ce0 con 0x55dacded9d40 2023-11-06T01:37:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.001+0000 7f93c6ffd700 10 client.25436 handle_command_reply: tid=0 2023-11-06T01:37:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.001+0000 7f93fa0ccb80 1 client.25436 shutdown 2023-11-06T01:37:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1710033574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 msgr2=0x55dacded5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 0x55dacded5670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0958b0 tx=0x7f93dc099520 comp rx=0 tx=0).stop 2023-11-06T01:37:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1710033574 shutdown_connections 2023-11-06T01:37:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dacded9d40 0x55dacdedc160 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dacde6aed0 0x55dacded7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacde69680 0x55dacde769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93fa0ccb80 1 --2- 172.21.15.53:0/1710033574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dacde69fe0 0x55dacded5670 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1710033574 >> 172.21.15.53:0/1710033574 conn(0x55dacddc9070 msgr2=0x55dacde72150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93c6ffd700 0 client.25436 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1710033574 shutdown_connections 2023-11-06T01:37:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93fa0ccb80 1 -- 172.21.15.53:0/1710033574 wait complete. 2023-11-06T01:37:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93fa0ccb80 20 client.25436 trim_cache size 0 max 16384 2023-11-06T01:37:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:37:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:37:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:37:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:37:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93edf3c700 1 -- 172.21.15.53:0/1398622400 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93d00a4930 msgr2=0x7f93d00a6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93edf3c700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93d00a4930 0x7f93d00a6df0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f93e8100cf0 tx=0x7f93d800a380 comp rx=0 tx=0).stop 2023-11-06T01:37:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93edf3c700 1 -- 172.21.15.53:0/1398622400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e8100e90 msgr2=0x7f93e8195000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93edf3c700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e8100e90 0x7f93e8195000 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e9a0 tx=0x7f93dc00ecb0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93edf3c700 1 -- 172.21.15.53:0/1398622400 shutdown_connections 2023-11-06T01:37:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93edf3c700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f93d00a4930 0x7f93d00a6df0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93edf3c700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93e8101830 0x7f93e8195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93edf3c700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93e80ffca0 0x7f93e8194ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.002+0000 7f93edf3c700 1 --2- 172.21.15.53:0/1398622400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e8100e90 0x7f93e8195000 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.003+0000 7f93edf3c700 1 -- 172.21.15.53:0/1398622400 >> 172.21.15.53:0/1398622400 conn(0x7f93e80fb230 msgr2=0x7f93e8104330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.003+0000 7f93edf3c700 1 -- 172.21.15.53:0/1398622400 shutdown_connections 2023-11-06T01:37:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:05.003+0000 7f93edf3c700 1 -- 172.21.15.53:0/1398622400 wait complete. 2023-11-06T01:37:05.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:37:05.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.360+0000 7f34c8a8d700 1 -- 172.21.15.53:0/610985858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34c4067280 msgr2=0x7f34c41001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.360+0000 7f34c8a8d700 1 --2- 172.21.15.53:0/610985858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34c4067280 0x7f34c41001a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f34b80098f0 tx=0x7f34b8009c00 comp rx=0 tx=0).stop 2023-11-06T01:37:05.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.360+0000 7f34c8a8d700 1 -- 172.21.15.53:0/610985858 shutdown_connections 2023-11-06T01:37:05.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.360+0000 7f34c8a8d700 1 --2- 172.21.15.53:0/610985858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34c41006e0 0x7f34c4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.360+0000 7f34c8a8d700 1 --2- 172.21.15.53:0/610985858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34c40fefb0 0x7f34c40ff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.360+0000 7f34c8a8d700 1 --2- 172.21.15.53:0/610985858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34c4067280 0x7f34c41001a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.360+0000 7f34c8a8d700 1 -- 172.21.15.53:0/610985858 >> 172.21.15.53:0/610985858 conn(0x7f34c40fa520 msgr2=0x7f34c40fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:05.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.360+0000 7f34c8a8d700 1 -- 172.21.15.53:0/610985858 shutdown_connections 2023-11-06T01:37:05.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.360+0000 7f34c8a8d700 1 -- 172.21.15.53:0/610985858 wait complete. 2023-11-06T01:37:05.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c8a8d700 1 Processor -- start 2023-11-06T01:37:05.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c8a8d700 1 -- start start 2023-11-06T01:37:05.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c8a8d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34c4067280 0x7f34c410e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c8a8d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34c40fefb0 0x7f34c410e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c8a8d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34c41006e0 0x7f34c410ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c8a8d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c410b3b0 con 0x7f34c40fefb0 2023-11-06T01:37:05.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c8a8d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c410b520 con 0x7f34c4067280 2023-11-06T01:37:05.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c8a8d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c410b6e0 con 0x7f34c41006e0 2023-11-06T01:37:05.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34c41006e0 0x7f34c410ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34c4067280 0x7f34c410e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34c41006e0 0x7f34c410ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55124/0 (socket says 172.21.15.53:55124) 2023-11-06T01:37:05.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34c4067280 0x7f34c410e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52320/0 (socket says 172.21.15.53:52320) 2023-11-06T01:37:05.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.361+0000 7f34c2d9d700 1 -- 172.21.15.53:0/1148142689 learned_addr learned my addr 172.21.15.53:0/1148142689 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:05.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.362+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34c40fefb0 0x7f34c410e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.362+0000 7f34c2d9d700 1 -- 172.21.15.53:0/1148142689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34c4067280 msgr2=0x7f34c410e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.362+0000 7f34c2d9d700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34c4067280 0x7f34c410e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.362+0000 7f34c2d9d700 1 -- 172.21.15.53:0/1148142689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34c40fefb0 msgr2=0x7f34c410e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.362+0000 7f34c2d9d700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34c40fefb0 0x7f34c410e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.362+0000 7f34c2d9d700 1 -- 172.21.15.53:0/1148142689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b8009520 con 0x7f34c41006e0 2023-11-06T01:37:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.363+0000 7f34c2d9d700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34c41006e0 0x7f34c410ac10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00c110 tx=0x7f34bc00c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:05.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.363+0000 7f34b37fe700 1 -- 172.21.15.53:0/1148142689 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc017070 con 0x7f34c41006e0 2023-11-06T01:37:05.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.363+0000 7f34b37fe700 1 -- 172.21.15.53:0/1148142689 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34bc00e050 con 0x7f34c41006e0 2023-11-06T01:37:05.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.363+0000 7f34c8a8d700 1 -- 172.21.15.53:0/1148142689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c4198ac0 con 0x7f34c41006e0 2023-11-06T01:37:05.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.363+0000 7f34b37fe700 1 -- 172.21.15.53:0/1148142689 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc007940 con 0x7f34c41006e0 2023-11-06T01:37:05.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.363+0000 7f34c8a8d700 1 -- 172.21.15.53:0/1148142689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c4198fd0 con 0x7f34c41006e0 2023-11-06T01:37:05.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.368+0000 7f34d4c1db80 1 Processor -- start 2023-11-06T01:37:05.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.368+0000 7f34d4c1db80 1 -- start start 2023-11-06T01:37:05.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.369+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a790f0 0x561d48a79530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.369+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a79a70 0x561d48a77e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.369+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 0x561d48a78870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.369+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d489cfa10 con 0x561d48a783f0 2023-11-06T01:37:05.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.369+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d489cd3d0 con 0x561d48a790f0 2023-11-06T01:37:05.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.369+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d489c8cc0 con 0x561d48a79a70 2023-11-06T01:37:05.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.376+0000 7f34b37fe700 1 -- 172.21.15.53:0/1148142689 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f34bc004ad0 con 0x7f34c41006e0 2023-11-06T01:37:05.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.377+0000 7f34b37fe700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f34ac0a05a0 0x7f34ac0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.378+0000 7f34c2d9d700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f34ac0a05a0 0x7f34ac0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.378+0000 7f34b37fe700 1 -- 172.21.15.53:0/1148142689 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f34bc0cb480 con 0x7f34c41006e0 2023-11-06T01:37:05.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.378+0000 7f34c1d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 0x561d48a78870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.378+0000 7f34c1d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 0x561d48a78870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40892/0 (socket says 172.21.15.53:40892) 2023-11-06T01:37:05.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.378+0000 7f34c1d9b700 1 -- 172.21.15.53:0/2949417983 learned_addr learned my addr 172.21.15.53:0/2949417983 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:05.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.378+0000 7f34c2d9d700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f34ac0a05a0 0x7f34ac0a2a60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0956d0 tx=0x7f34bc04e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:05.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.379+0000 7f34c1d9b700 1 -- 172.21.15.53:0/2949417983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a79a70 msgr2=0x561d48a77e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:37:05.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.379+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/2949417983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a79a70 0x561d48a77e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.379+0000 7f34c1d9b700 1 -- 172.21.15.53:0/2949417983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a790f0 msgr2=0x561d48a79530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:05.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.379+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/2949417983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a790f0 0x561d48a79530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.379+0000 7f34c1d9b700 1 -- 172.21.15.53:0/2949417983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d487f6de0 con 0x561d48a783f0 2023-11-06T01:37:05.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.379+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/2949417983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 0x561d48a78870 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f34b80098f0 tx=0x7f34b8009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2899480e2c5628e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:05.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.379+0000 7f34b0ff9700 1 -- 172.21.15.53:0/2949417983 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b800f460 con 0x561d48a783f0 2023-11-06T01:37:05.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.379+0000 7f34b0ff9700 1 -- 172.21.15.53:0/2949417983 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34b801d050 con 0x561d48a783f0 2023-11-06T01:37:05.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.380+0000 7f34b0ff9700 1 -- 172.21.15.53:0/2949417983 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b80052a0 con 0x561d48a783f0 2023-11-06T01:37:05.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.381+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2949417983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 msgr2=0x561d48a78870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.381+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/2949417983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 0x561d48a78870 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f34b80098f0 tx=0x7f34b8009460 comp rx=0 tx=0).stop 2023-11-06T01:37:05.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.381+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2949417983 shutdown_connections 2023-11-06T01:37:05.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.381+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/2949417983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a790f0 0x561d48a79530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.381+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/2949417983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 0x561d48a78870 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.381+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/2949417983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a79a70 0x561d48a77e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.381+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2949417983 >> 172.21.15.53:0/2949417983 conn(0x561d489d70d0 msgr2=0x561d48a72940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:05.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.389+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2949417983 shutdown_connections 2023-11-06T01:37:05.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.389+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2949417983 wait complete. 2023-11-06T01:37:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.390+0000 7f34d4c1db80 1 -- start start 2023-11-06T01:37:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.390+0000 7f34d4c1db80 1 Processor -- start 2023-11-06T01:37:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.390+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a783f0 0x561d48a81980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.390+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 0x561d48a81ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.390+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a79a70 0x561d48a88f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.390+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d48965510 con 0x561d48a790f0 2023-11-06T01:37:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.390+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d489651b0 con 0x561d48a79a70 2023-11-06T01:37:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.390+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4884c920 con 0x561d48a783f0 2023-11-06T01:37:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.391+0000 7f34d4c1db80 10 client.0 _ll_register_callbacks cb 0x561d488b2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.391+0000 7f34c1d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a783f0 0x561d48a81980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.391+0000 7f34c1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a79a70 0x561d48a88f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.391+0000 7f34c1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a79a70 0x561d48a88f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52336/0 (socket says 172.21.15.53:52336) 2023-11-06T01:37:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.391+0000 7f34c1d9b700 1 -- 172.21.15.53:0/2669010541 learned_addr learned my addr 172.21.15.53:0/2669010541 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.391+0000 7f34c259c700 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 0x561d48a81ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34c259c700 1 -- 172.21.15.53:0/2669010541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a783f0 msgr2=0x561d48a81980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:05.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34c259c700 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a783f0 0x561d48a81980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34c259c700 1 -- 172.21.15.53:0/2669010541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a79a70 msgr2=0x561d48a88f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:05.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34c259c700 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a79a70 0x561d48a88f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34c259c700 1 -- 172.21.15.53:0/2669010541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c41018e0 con 0x561d48a790f0 2023-11-06T01:37:05.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34c259c700 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 0x561d48a81ec0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f34c4100000 tx=0x7f34b40119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4f5dde95fcac5e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:05.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:37:05.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34a9ffb700 1 -- 172.21.15.53:0/2669010541 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4004020 con 0x561d48a790f0 2023-11-06T01:37:05.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34a9ffb700 1 -- 172.21.15.53:0/2669010541 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f34b4004180 con 0x561d48a790f0 2023-11-06T01:37:05.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2669010541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d489f3d10 con 0x561d48a790f0 2023-11-06T01:37:05.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34a9ffb700 1 -- 172.21.15.53:0/2669010541 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34b4004380 con 0x561d48a790f0 2023-11-06T01:37:05.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34d4c1db80 10 client.15876 fetch_fsmap learned FSMap version 37 2023-11-06T01:37:05.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.392+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2669010541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x561d487dfc60 con 0x561d48a790f0 2023-11-06T01:37:05.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.393+0000 7f34a9ffb700 1 -- 172.21.15.53:0/2669010541 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561d489f3d10 con 0x561d48a790f0 2023-11-06T01:37:05.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.393+0000 7f34a9ffb700 1 -- 172.21.15.53:0/2669010541 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f34b40052f0 con 0x561d48a790f0 2023-11-06T01:37:05.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.393+0000 7f34d4c1db80 10 client.15876 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T01:37:05.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.393+0000 7f34d4c1db80 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:05.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.393+0000 7f34d4c1db80 20 client.15876 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:05.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.393+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561d48a8d7e0 0x561d48a8fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.393+0000 7f34d4c1db80 4 client.15876 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:37:05.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.393+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2669010541 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d48a46e10 con 0x561d48a8d7e0 2023-11-06T01:37:05.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.394+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561d48a8d7e0 0x561d48a8fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.394+0000 7f34a9ffb700 10 client.15876 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:05.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.394+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561d48a8d7e0 0x561d48a8fc00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42896c2d8e529f3 server_cookie=a2635cc749355a48 in_seq=0 out_seq=0 2023-11-06T01:37:05.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34a9ffb700 1 -- 172.21.15.53:0/2669010541 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561d48a46e10 con 0x561d48a8d7e0 2023-11-06T01:37:05.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34a9ffb700 10 client.15876 handle_command_reply: tid=0 2023-11-06T01:37:05.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 1 client.15876 shutdown 2023-11-06T01:37:05.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2669010541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 msgr2=0x561d48a81ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 0x561d48a81ec0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f34c4100000 tx=0x7f34b40119a0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2669010541 shutdown_connections 2023-11-06T01:37:05.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a79a70 0x561d48a88f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561d48a8d7e0 0x561d48a8fc00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a783f0 0x561d48a81980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34a9ffb700 0 client.15876 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:05.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/2669010541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 0x561d48a81ec0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2669010541 >> 172.21.15.53:0/2669010541 conn(0x561d489d70d0 msgr2=0x561d48a720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:05.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2669010541 shutdown_connections 2023-11-06T01:37:05.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 1 -- 172.21.15.53:0/2669010541 wait complete. 2023-11-06T01:37:05.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.395+0000 7f34d4c1db80 20 client.15876 trim_cache size 0 max 16384 2023-11-06T01:37:05.406 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:05 smithi062 ceph-mon[103924]: pgmap v393: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.29k op/s 2023-11-06T01:37:05.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.419+0000 7f34d4c1db80 1 Processor -- start 2023-11-06T01:37:05.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.419+0000 7f34d4c1db80 1 -- start start 2023-11-06T01:37:05.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.419+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 0x561d48a77e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.419+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a790f0 0x561d48a86eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.419+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a79a70 0x561d48a7c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.419+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d48a6d440 con 0x561d48a783f0 2023-11-06T01:37:05.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.419+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d48a46e40 con 0x561d48a790f0 2023-11-06T01:37:05.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.419+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d48a80e30 con 0x561d48a79a70 2023-11-06T01:37:05.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.420+0000 7f34c259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a790f0 0x561d48a86eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.420+0000 7f34c259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a790f0 0x561d48a86eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52342/0 (socket says 172.21.15.53:52342) 2023-11-06T01:37:05.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.420+0000 7f34c259c700 1 -- 172.21.15.53:0/420413807 learned_addr learned my addr 172.21.15.53:0/420413807 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:05.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.420+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/420413807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a79a70 0x561d48a7c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.420+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/420413807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 0x561d48a77e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.420+0000 7f34c1d9b700 1 -- 172.21.15.53:0/420413807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a790f0 msgr2=0x561d48a86eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:05.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.420+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/420413807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a790f0 0x561d48a86eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.420+0000 7f34c1d9b700 1 -- 172.21.15.53:0/420413807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 msgr2=0x561d48a77e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:05.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.420+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/420413807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 0x561d48a77e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.420+0000 7f34c1d9b700 1 -- 172.21.15.53:0/420413807 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d4881d250 con 0x561d48a79a70 2023-11-06T01:37:05.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.420+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/420413807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a79a70 0x561d48a7c370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x561d48a904d0 tx=0x7f34b8009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b7aea7a97867f76 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:05.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.421+0000 7f34b17fa700 1 -- 172.21.15.53:0/420413807 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b800b8c0 con 0x561d48a79a70 2023-11-06T01:37:05.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.421+0000 7f34b17fa700 1 -- 172.21.15.53:0/420413807 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34b800ba20 con 0x561d48a79a70 2023-11-06T01:37:05.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.421+0000 7f34b17fa700 1 -- 172.21.15.53:0/420413807 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8004a10 con 0x561d48a79a70 2023-11-06T01:37:05.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.422+0000 7f34d4c1db80 1 -- 172.21.15.53:0/420413807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a79a70 msgr2=0x561d48a7c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.422+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/420413807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a79a70 0x561d48a7c370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x561d48a904d0 tx=0x7f34b8009460 comp rx=0 tx=0).stop 2023-11-06T01:37:05.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 -- 172.21.15.53:0/420413807 shutdown_connections 2023-11-06T01:37:05.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/420413807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a79a70 0x561d48a7c370 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/420413807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a783f0 0x561d48a77e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/420413807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a790f0 0x561d48a86eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 -- 172.21.15.53:0/420413807 >> 172.21.15.53:0/420413807 conn(0x561d489d70d0 msgr2=0x561d48a81980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 -- 172.21.15.53:0/420413807 shutdown_connections 2023-11-06T01:37:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 -- 172.21.15.53:0/420413807 wait complete. 2023-11-06T01:37:05.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 -- start start 2023-11-06T01:37:05.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 Processor -- start 2023-11-06T01:37:05.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a783f0 0x561d48a86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 0x561d48a86ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a79a70 0x561d48a84d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d48a85390 con 0x561d48a790f0 2023-11-06T01:37:05.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d48a85500 con 0x561d48a79a70 2023-11-06T01:37:05.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d48a856c0 con 0x561d48a783f0 2023-11-06T01:37:05.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.423+0000 7f34d4c1db80 10 client.0 _ll_register_callbacks cb 0x7f34b4009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34c259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 0x561d48a86ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34c1d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a783f0 0x561d48a86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34c259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 0x561d48a86ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40910/0 (socket says 172.21.15.53:40910) 2023-11-06T01:37:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34c259c700 1 -- 172.21.15.53:0/3716043843 learned_addr learned my addr 172.21.15.53:0/3716043843 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a79a70 0x561d48a84d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34c259c700 1 -- 172.21.15.53:0/3716043843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a783f0 msgr2=0x561d48a86760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34c259c700 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a783f0 0x561d48a86760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34c259c700 1 -- 172.21.15.53:0/3716043843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a79a70 msgr2=0x561d48a84d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34c259c700 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a79a70 0x561d48a84d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34c259c700 1 -- 172.21.15.53:0/3716043843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d487f6de0 con 0x561d48a790f0 2023-11-06T01:37:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34c259c700 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 0x561d48a86ca0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f34b4005750 tx=0x7f34b400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dddd999287ce059 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:37:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34a9ffb700 1 -- 172.21.15.53:0/3716043843 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b40092d0 con 0x561d48a790f0 2023-11-06T01:37:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34a9ffb700 1 -- 172.21.15.53:0/3716043843 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f34b4000e40 con 0x561d48a790f0 2023-11-06T01:37:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34d4c1db80 1 -- 172.21.15.53:0/3716043843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34b801d050 con 0x561d48a790f0 2023-11-06T01:37:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34a9ffb700 1 -- 172.21.15.53:0/3716043843 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34b400eec0 con 0x561d48a790f0 2023-11-06T01:37:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34d4c1db80 10 client.15882 fetch_fsmap learned FSMap version 37 2023-11-06T01:37:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.424+0000 7f34d4c1db80 1 -- 172.21.15.53:0/3716043843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x561d487f52b0 con 0x561d48a790f0 2023-11-06T01:37:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.425+0000 7f34a9ffb700 1 -- 172.21.15.53:0/3716043843 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34b801d050 con 0x561d48a790f0 2023-11-06T01:37:05.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.425+0000 7f34a9ffb700 1 -- 172.21.15.53:0/3716043843 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f34b4038050 con 0x561d48a790f0 2023-11-06T01:37:05.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.425+0000 7f34d4c1db80 10 client.15882 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T01:37:05.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.425+0000 7f34d4c1db80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:05.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.425+0000 7f34d4c1db80 20 client.15882 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:05.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.425+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561d48ae0660 0x561d48ae2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:05.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.425+0000 7f34d4c1db80 4 client.15882 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:37:05.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.425+0000 7f34d4c1db80 1 -- 172.21.15.53:0/3716043843 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561d489c1d40 con 0x561d48ae0660 2023-11-06T01:37:05.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.425+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561d48ae0660 0x561d48ae2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:05.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.425+0000 7f34a9ffb700 10 client.15882 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:05.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.425+0000 7f34c1d9b700 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561d48ae0660 0x561d48ae2a80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=743a17d2d2fd9b75 server_cookie=c3291834f1473559 in_seq=0 out_seq=0 2023-11-06T01:37:05.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.426+0000 7f34a9ffb700 1 -- 172.21.15.53:0/3716043843 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561d489c1d40 con 0x561d48ae0660 2023-11-06T01:37:05.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.426+0000 7f34a9ffb700 10 client.15882 handle_command_reply: tid=0 2023-11-06T01:37:05.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.426+0000 7f34d4c1db80 1 client.15882 shutdown 2023-11-06T01:37:05.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.427+0000 7f34d4c1db80 1 -- 172.21.15.53:0/3716043843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 msgr2=0x561d48a86ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.427+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 0x561d48a86ca0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f34b4005750 tx=0x7f34b400ecd0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.427+0000 7f34d4c1db80 1 -- 172.21.15.53:0/3716043843 shutdown_connections 2023-11-06T01:37:05.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.427+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d48a79a70 0x561d48a84d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.427+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561d48ae0660 0x561d48ae2a80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.427+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d48a783f0 0x561d48a86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.427+0000 7f34d4c1db80 1 --2- 172.21.15.53:0/3716043843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d48a790f0 0x561d48a86ca0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.427+0000 7f34a9ffb700 0 client.15882 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:05.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.427+0000 7f34d4c1db80 1 -- 172.21.15.53:0/3716043843 >> 172.21.15.53:0/3716043843 conn(0x561d489d70d0 msgr2=0x561d48a81920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:05.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.427+0000 7f34d4c1db80 1 -- 172.21.15.53:0/3716043843 shutdown_connections 2023-11-06T01:37:05.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.427+0000 7f34d4c1db80 1 -- 172.21.15.53:0/3716043843 wait complete. 2023-11-06T01:37:05.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34d4c1db80 20 client.15882 trim_cache size 0 max 16384 2023-11-06T01:37:05.445 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:37:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:37:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:37:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:37:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:37:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:37:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:37:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:37:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:37:05.448 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:37:05.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 -- 172.21.15.53:0/1148142689 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f34ac0a05a0 msgr2=0x7f34ac0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f34ac0a05a0 0x7f34ac0a2a60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0956d0 tx=0x7f34bc04e040 comp rx=0 tx=0).stop 2023-11-06T01:37:05.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 -- 172.21.15.53:0/1148142689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34c41006e0 msgr2=0x7f34c410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:05.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34c41006e0 0x7f34c410ac10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00c110 tx=0x7f34bc00c420 comp rx=0 tx=0).stop 2023-11-06T01:37:05.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 -- 172.21.15.53:0/1148142689 shutdown_connections 2023-11-06T01:37:05.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f34ac0a05a0 0x7f34ac0a2a60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34c4067280 0x7f34c410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34c41006e0 0x7f34c410ac10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 --2- 172.21.15.53:0/1148142689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34c40fefb0 0x7f34c410e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:05.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 -- 172.21.15.53:0/1148142689 >> 172.21.15.53:0/1148142689 conn(0x7f34c40fa520 msgr2=0x7f34c40fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:05.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 -- 172.21.15.53:0/1148142689 shutdown_connections 2023-11-06T01:37:05.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:05.428+0000 7f34c8a8d700 1 -- 172.21.15.53:0/1148142689 wait complete. 2023-11-06T01:37:05.452 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:05 smithi053 ceph-mon[99472]: pgmap v393: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.29k op/s 2023-11-06T01:37:05.453 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:37:05.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:05 smithi163 ceph-mon[104416]: pgmap v393: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.29k op/s 2023-11-06T01:37:07.188 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 170.0 s, 2353.2 tps, lat 43.020 ms stddev 67.110 2023-11-06T01:37:07.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:07 smithi062 ceph-mon[103924]: pgmap v394: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 50 MiB/s wr, 2.19k op/s 2023-11-06T01:37:07.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:07 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:07.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:07 smithi163 ceph-mon[104416]: pgmap v394: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 50 MiB/s wr, 2.19k op/s 2023-11-06T01:37:07.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:07 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:07.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:07 smithi053 ceph-mon[99472]: pgmap v394: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 50 MiB/s wr, 2.19k op/s 2023-11-06T01:37:07.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:07 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:09.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:09 smithi062 ceph-mon[103924]: pgmap v395: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 47 MiB/s wr, 2.09k op/s 2023-11-06T01:37:09.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:09 smithi163 ceph-mon[104416]: pgmap v395: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 47 MiB/s wr, 2.09k op/s 2023-11-06T01:37:09.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:09 smithi053 ceph-mon[99472]: pgmap v395: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 47 MiB/s wr, 2.09k op/s 2023-11-06T01:37:11.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:11 smithi062 ceph-mon[103924]: pgmap v396: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 47 MiB/s wr, 2.09k op/s 2023-11-06T01:37:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:11 smithi163 ceph-mon[104416]: pgmap v396: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 47 MiB/s wr, 2.09k op/s 2023-11-06T01:37:11.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:11 smithi053 ceph-mon[99472]: pgmap v396: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 47 MiB/s wr, 2.09k op/s 2023-11-06T01:37:12.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 175.0 s, 3019.6 tps, lat 33.065 ms stddev 35.812 2023-11-06T01:37:13.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:13 smithi062 ceph-mon[103924]: pgmap v397: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 47 MiB/s wr, 2.10k op/s 2023-11-06T01:37:13.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:13 smithi163 ceph-mon[104416]: pgmap v397: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 47 MiB/s wr, 2.10k op/s 2023-11-06T01:37:13.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:13 smithi053 ceph-mon[99472]: pgmap v397: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 47 MiB/s wr, 2.10k op/s 2023-11-06T01:37:15.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:15 smithi062 ceph-mon[103924]: pgmap v398: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 MiB/s wr, 1.90k op/s 2023-11-06T01:37:15.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:15 smithi163 ceph-mon[104416]: pgmap v398: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 MiB/s wr, 1.90k op/s 2023-11-06T01:37:15.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:15 smithi053 ceph-mon[99472]: pgmap v398: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 MiB/s wr, 1.90k op/s 2023-11-06T01:37:17.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 180.0 s, 2898.9 tps, lat 34.552 ms stddev 39.178 2023-11-06T01:37:17.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:17 smithi062 ceph-mon[103924]: pgmap v399: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 MiB/s wr, 1.82k op/s 2023-11-06T01:37:17.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:17 smithi163 ceph-mon[104416]: pgmap v399: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 MiB/s wr, 1.82k op/s 2023-11-06T01:37:17.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:17 smithi053 ceph-mon[99472]: pgmap v399: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 MiB/s wr, 1.82k op/s 2023-11-06T01:37:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.169+0000 7fa6fe741700 1 -- 172.21.15.53:0/2049044717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f8108e40 msgr2=0x7fa6f81092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.169+0000 7fa6fe741700 1 --2- 172.21.15.53:0/2049044717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f8108e40 0x7fa6f81092c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009920 tx=0x7fa6ec009c30 comp rx=0 tx=0).stop 2023-11-06T01:37:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.169+0000 7fa6fe741700 1 -- 172.21.15.53:0/2049044717 shutdown_connections 2023-11-06T01:37:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.169+0000 7fa6fe741700 1 --2- 172.21.15.53:0/2049044717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f8109800 0x7fa6f8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.169+0000 7fa6fe741700 1 --2- 172.21.15.53:0/2049044717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f8108130 0x7fa6f8108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.169+0000 7fa6fe741700 1 --2- 172.21.15.53:0/2049044717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f8108e40 0x7fa6f81092c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.169+0000 7fa6fe741700 1 -- 172.21.15.53:0/2049044717 >> 172.21.15.53:0/2049044717 conn(0x7fa6f806dbc0 msgr2=0x7fa6f8070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.169+0000 7fa6fe741700 1 -- 172.21.15.53:0/2049044717 shutdown_connections 2023-11-06T01:37:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.169+0000 7fa6fe741700 1 -- 172.21.15.53:0/2049044717 wait complete. 2023-11-06T01:37:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6fe741700 1 Processor -- start 2023-11-06T01:37:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6fe741700 1 -- start start 2023-11-06T01:37:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6fe741700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f8108130 0x7fa6f81a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6fe741700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f8108e40 0x7fa6f81a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6fe741700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f8109800 0x7fa6f81a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6fe741700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81a7100 con 0x7fa6f8108e40 2023-11-06T01:37:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6fe741700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81a7240 con 0x7fa6f8109800 2023-11-06T01:37:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6fe741700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81a7400 con 0x7fa6f8108130 2023-11-06T01:37:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f8108e40 0x7fa6f81a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6fccde700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f8109800 0x7fa6f81a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6f7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f8108130 0x7fa6f81a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6fccde700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f8109800 0x7fa6f81a6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52364/0 (socket says 172.21.15.53:52364) 2023-11-06T01:37:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f8108e40 0x7fa6f81a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40924/0 (socket says 172.21.15.53:40924) 2023-11-06T01:37:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6fccde700 1 -- 172.21.15.53:0/529861201 learned_addr learned my addr 172.21.15.53:0/529861201 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.170+0000 7fa6f7fff700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f8108130 0x7fa6f81a2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55162/0 (socket says 172.21.15.53:55162) 2023-11-06T01:37:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.171+0000 7fa6f77fe700 1 -- 172.21.15.53:0/529861201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f8108130 msgr2=0x7fa6f81a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.171+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f8108130 0x7fa6f81a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.171+0000 7fa6f77fe700 1 -- 172.21.15.53:0/529861201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f8109800 msgr2=0x7fa6f81a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.171+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f8109800 0x7fa6f81a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.171+0000 7fa6f77fe700 1 -- 172.21.15.53:0/529861201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec009520 con 0x7fa6f8108e40 2023-11-06T01:37:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.171+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f8108e40 0x7fa6f81a45e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec000c00 tx=0x7fa6ec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.172+0000 7fa6f57fa700 1 -- 172.21.15.53:0/529861201 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec01c070 con 0x7fa6f8108e40 2023-11-06T01:37:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.172+0000 7fa6f57fa700 1 -- 172.21.15.53:0/529861201 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6ec00f7d0 con 0x7fa6f8108e40 2023-11-06T01:37:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.172+0000 7fa6f57fa700 1 -- 172.21.15.53:0/529861201 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec003a40 con 0x7fa6f8108e40 2023-11-06T01:37:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.172+0000 7fa6fe741700 1 -- 172.21.15.53:0/529861201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f81a7680 con 0x7fa6f8108e40 2023-11-06T01:37:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.172+0000 7fa6fe741700 1 -- 172.21.15.53:0/529861201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f81a7ab0 con 0x7fa6f8108e40 2023-11-06T01:37:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.179+0000 7fa6f57fa700 1 -- 172.21.15.53:0/529861201 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa6ec020030 con 0x7fa6f8108e40 2023-11-06T01:37:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa70a8d1b80 1 Processor -- start 2023-11-06T01:37:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa70a8d1b80 1 -- start start 2023-11-06T01:37:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x564553888aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 0x5645538898f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 0x56455388a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645537e09b0 con 0x564553888680 2023-11-06T01:37:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645537de370 con 0x564553888fe0 2023-11-06T01:37:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645537d9c60 con 0x564553889ed0 2023-11-06T01:37:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 0x56455388a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 0x56455388a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55168/0 (socket says 172.21.15.53:55168) 2023-11-06T01:37:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa6f77fe700 1 -- 172.21.15.53:0/131840690 learned_addr learned my addr 172.21.15.53:0/131840690 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.180+0000 7fa6f7fff700 1 --2- 172.21.15.53:0/131840690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 0x5645538898f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.181+0000 7fa6f77fe700 1 -- 172.21.15.53:0/131840690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 msgr2=0x5645538898f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.181+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/131840690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 0x5645538898f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.181+0000 7fa6f77fe700 1 -- 172.21.15.53:0/131840690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 msgr2=0x564553888aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:37:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.181+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/131840690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x564553888aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.181+0000 7fa6f77fe700 1 -- 172.21.15.53:0/131840690 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564553607d80 con 0x564553889ed0 2023-11-06T01:37:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.181+0000 7fa6f57fa700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6e00a0670 0x7fa6e00a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.181+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/131840690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 0x56455388a350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0d2a80 tx=0x7fa6ec0d2ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4dc4fc96485a165 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.181+0000 7fa6f57fa700 1 -- 172.21.15.53:0/529861201 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa6ec0cc8c0 con 0x7fa6f8108e40 2023-11-06T01:37:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.181+0000 7fa6de7fc700 1 -- 172.21.15.53:0/131840690 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0a0110 con 0x564553889ed0 2023-11-06T01:37:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.182+0000 7fa6de7fc700 1 -- 172.21.15.53:0/131840690 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6ec0964d0 con 0x564553889ed0 2023-11-06T01:37:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.182+0000 7fa6de7fc700 1 -- 172.21.15.53:0/131840690 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec099cd0 con 0x564553889ed0 2023-11-06T01:37:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.182+0000 7fa6fccde700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6e00a0670 0x7fa6e00a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.183+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/131840690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 msgr2=0x56455388a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.183+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/131840690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 0x56455388a350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0d2a80 tx=0x7fa6ec0d2ab0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.183+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/131840690 shutdown_connections 2023-11-06T01:37:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.183+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/131840690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 0x56455388a350 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.183+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/131840690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x564553888aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.183+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/131840690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 0x5645538898f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.183+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/131840690 >> 172.21.15.53:0/131840690 conn(0x5645537e8070 msgr2=0x564553885040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.183+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/131840690 shutdown_connections 2023-11-06T01:37:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.183+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/131840690 wait complete. 2023-11-06T01:37:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.183+0000 7fa6fccde700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6e00a0670 0x7fa6e00a2b30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009bd0 tx=0x7fa6f0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.183+0000 7fa70a8d1b80 1 -- start start 2023-11-06T01:37:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa70a8d1b80 1 Processor -- start 2023-11-06T01:37:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888680 0x56455389a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553888fe0 0x56455389ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553889ed0 0x5645538a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645537764b0 con 0x564553889ed0 2023-11-06T01:37:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564553776150 con 0x564553888680 2023-11-06T01:37:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455365d8c0 con 0x564553888fe0 2023-11-06T01:37:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa70a8d1b80 10 client.0 _ll_register_callbacks cb 0x5645536c3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa6f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888680 0x56455389a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553888fe0 0x56455389ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa6f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888680 0x56455389a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52372/0 (socket says 172.21.15.53:52372) 2023-11-06T01:37:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553888fe0 0x56455389ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55174/0 (socket says 172.21.15.53:55174) 2023-11-06T01:37:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.184+0000 7fa6f7fff700 1 -- 172.21.15.53:0/3151198150 learned_addr learned my addr 172.21.15.53:0/3151198150 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.185+0000 7fa6f77fe700 1 -- 172.21.15.53:0/3151198150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888680 msgr2=0x56455389a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.185+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/3151198150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888680 0x56455389a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.185+0000 7fa6f77fe700 1 -- 172.21.15.53:0/3151198150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553889ed0 msgr2=0x5645538a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.185+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/3151198150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553889ed0 0x5645538a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.185+0000 7fa6f77fe700 1 -- 172.21.15.53:0/3151198150 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec0cfe70 con 0x564553888fe0 2023-11-06T01:37:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.185+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/3151198150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553888fe0 0x56455389ae60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5645538896b0 tx=0x7fa6ec005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b16703c501e8ae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.185+0000 7fa6d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:37:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.185+0000 7fa6d77fe700 1 -- 172.21.15.53:0/3151198150 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec099af0 con 0x564553888fe0 2023-11-06T01:37:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.185+0000 7fa6d77fe700 1 -- 172.21.15.53:0/3151198150 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa6ec09a890 con 0x564553888fe0 2023-11-06T01:37:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.185+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/3151198150 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564553804cb0 con 0x564553888fe0 2023-11-06T01:37:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.186+0000 7fa6d77fe700 1 -- 172.21.15.53:0/3151198150 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6ec099cd0 con 0x564553888fe0 2023-11-06T01:37:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.186+0000 7fa70a8d1b80 10 client.25540 fetch_fsmap learned FSMap version 38 2023-11-06T01:37:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.186+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/3151198150 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5645535f0c00 con 0x564553888fe0 2023-11-06T01:37:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.186+0000 7fa6d77fe700 1 -- 172.21.15.53:0/3151198150 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6ec0b8030 con 0x564553888fe0 2023-11-06T01:37:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.186+0000 7fa6d77fe700 1 -- 172.21.15.53:0/3151198150 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa6ec0d4050 con 0x564553888fe0 2023-11-06T01:37:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.186+0000 7fa70a8d1b80 10 client.25540 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T01:37:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.186+0000 7fa70a8d1b80 10 client.25540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.186+0000 7fa70a8d1b80 20 client.25540 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.186+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/3151198150 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5645538a61b0 0x5645538a85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.186+0000 7fa70a8d1b80 4 client.25540 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:37:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.186+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/3151198150 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564553857db0 con 0x5645538a61b0 2023-11-06T01:37:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.187+0000 7fa6f7fff700 1 --2- 172.21.15.53:0/3151198150 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5645538a61b0 0x5645538a85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.188+0000 7fa6d77fe700 10 client.25540 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.188+0000 7fa6f7fff700 1 --2- 172.21.15.53:0/3151198150 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5645538a61b0 0x5645538a85d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5609d3856615d3f1 server_cookie=cc2a8a1248f6811d in_seq=0 out_seq=0 2023-11-06T01:37:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.188+0000 7fa6d77fe700 1 -- 172.21.15.53:0/3151198150 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564553857db0 con 0x5645538a61b0 2023-11-06T01:37:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.188+0000 7fa6d77fe700 10 client.25540 handle_command_reply: tid=0 2023-11-06T01:37:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.188+0000 7fa70a8d1b80 1 client.25540 shutdown 2023-11-06T01:37:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.189+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/3151198150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553888fe0 msgr2=0x56455389ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.189+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/3151198150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553888fe0 0x56455389ae60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5645538896b0 tx=0x7fa6ec005e20 comp rx=0 tx=0).stop 2023-11-06T01:37:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.190+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/3151198150 shutdown_connections 2023-11-06T01:37:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.190+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/3151198150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888680 0x56455389a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.190+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/3151198150 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5645538a61b0 0x5645538a85d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.190+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/3151198150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553889ed0 0x5645538a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.190+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/3151198150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553888fe0 0x56455389ae60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.190+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/3151198150 >> 172.21.15.53:0/3151198150 conn(0x5645537e8070 msgr2=0x5645538833c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.190+0000 7fa6d77fe700 0 client.25540 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.190+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/3151198150 shutdown_connections 2023-11-06T01:37:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.190+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/3151198150 wait complete. 2023-11-06T01:37:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.191+0000 7fa70a8d1b80 20 client.25540 trim_cache size 0 max 16384 2023-11-06T01:37:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.214+0000 7fa70a8d1b80 1 Processor -- start 2023-11-06T01:37:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.214+0000 7fa70a8d1b80 1 -- start start 2023-11-06T01:37:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.214+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x56455389fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 0x56455388a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 0x564553884c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455387b7a0 con 0x564553888680 2023-11-06T01:37:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564553899dd0 con 0x564553888fe0 2023-11-06T01:37:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455387e530 con 0x564553889ed0 2023-11-06T01:37:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 0x56455388a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 0x56455388a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52382/0 (socket says 172.21.15.53:52382) 2023-11-06T01:37:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa6f77fe700 1 -- 172.21.15.53:0/2452223164 learned_addr learned my addr 172.21.15.53:0/2452223164 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa6fccde700 1 --2- 172.21.15.53:0/2452223164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 0x564553884c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa6f7fff700 1 --2- 172.21.15.53:0/2452223164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x56455389fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa6f77fe700 1 -- 172.21.15.53:0/2452223164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 msgr2=0x564553884c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/2452223164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 0x564553884c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa6f77fe700 1 -- 172.21.15.53:0/2452223164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 msgr2=0x56455389fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/2452223164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x56455389fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.215+0000 7fa6f77fe700 1 -- 172.21.15.53:0/2452223164 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56455362e1f0 con 0x564553888fe0 2023-11-06T01:37:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.216+0000 7fa6f7fff700 1 --2- 172.21.15.53:0/2452223164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x56455389fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:37:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.216+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/2452223164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 0x56455388a310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0984e0 tx=0x7fa6ec0ccc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66f1e98c8c7219f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.216+0000 7fa6deffd700 1 -- 172.21.15.53:0/2452223164 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0215e0 con 0x564553888fe0 2023-11-06T01:37:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.216+0000 7fa6deffd700 1 -- 172.21.15.53:0/2452223164 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6ec0b8040 con 0x564553888fe0 2023-11-06T01:37:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.216+0000 7fa6deffd700 1 -- 172.21.15.53:0/2452223164 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0d2a80 con 0x564553888fe0 2023-11-06T01:37:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.217+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2452223164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 msgr2=0x56455388a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.217+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/2452223164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 0x56455388a310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0984e0 tx=0x7fa6ec0ccc00 comp rx=0 tx=0).stop 2023-11-06T01:37:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.217+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2452223164 shutdown_connections 2023-11-06T01:37:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.217+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/2452223164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564553889ed0 0x564553884c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.217+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/2452223164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x56455389fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.217+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/2452223164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553888fe0 0x56455388a310 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0984e0 tx=0x7fa6ec0ccc00 comp rx=0 tx=0).stop 2023-11-06T01:37:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.217+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2452223164 >> 172.21.15.53:0/2452223164 conn(0x5645537e8070 msgr2=0x56455387fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.219+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2452223164 shutdown_connections 2023-11-06T01:37:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.219+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2452223164 wait complete. 2023-11-06T01:37:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.219+0000 7fa70a8d1b80 1 -- start start 2023-11-06T01:37:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa70a8d1b80 1 Processor -- start 2023-11-06T01:37:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x56455389dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553889ed0 0x56455389e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa70a8d1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56455389ea70 0x56455389eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455389f5b0 con 0x564553888680 2023-11-06T01:37:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455389f720 con 0x564553889ed0 2023-11-06T01:37:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa70a8d1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455389f8e0 con 0x56455389ea70 2023-11-06T01:37:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa70a8d1b80 10 client.0 _ll_register_callbacks cb 0x7fa6e8009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553889ed0 0x56455389e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa6f7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x56455389dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa6fccde700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56455389ea70 0x56455389eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553889ed0 0x56455389e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52388/0 (socket says 172.21.15.53:52388) 2023-11-06T01:37:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa6f77fe700 1 -- 172.21.15.53:0/2898822239 learned_addr learned my addr 172.21.15.53:0/2898822239 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa6f77fe700 1 -- 172.21.15.53:0/2898822239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56455389ea70 msgr2=0x56455389eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/2898822239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56455389ea70 0x56455389eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa6f77fe700 1 -- 172.21.15.53:0/2898822239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 msgr2=0x56455389dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/2898822239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x56455389dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.220+0000 7fa6f77fe700 1 -- 172.21.15.53:0/2898822239 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec0cfe70 con 0x564553889ed0 2023-11-06T01:37:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.221+0000 7fa6f77fe700 1 --2- 172.21.15.53:0/2898822239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553889ed0 0x56455389e530 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00bb10 tx=0x7fa6ec0a2170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b73e4c89d92f870b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.221+0000 7fa6d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:37:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.221+0000 7fa6d77fe700 1 -- 172.21.15.53:0/2898822239 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec099cd0 con 0x564553889ed0 2023-11-06T01:37:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.221+0000 7fa6d77fe700 1 -- 172.21.15.53:0/2898822239 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa6ec0d2a80 con 0x564553889ed0 2023-11-06T01:37:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2898822239 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6ec0964d0 con 0x564553889ed0 2023-11-06T01:37:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa6d77fe700 1 -- 172.21.15.53:0/2898822239 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6ec099af0 con 0x564553889ed0 2023-11-06T01:37:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa70a8d1b80 10 client.25469 fetch_fsmap learned FSMap version 38 2023-11-06T01:37:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2898822239 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x564553606250 con 0x564553889ed0 2023-11-06T01:37:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa6d77fe700 1 -- 172.21.15.53:0/2898822239 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6ec0964d0 con 0x564553889ed0 2023-11-06T01:37:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa6d77fe700 1 -- 172.21.15.53:0/2898822239 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa6ec0d4050 con 0x564553889ed0 2023-11-06T01:37:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa70a8d1b80 10 client.25469 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T01:37:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa70a8d1b80 10 client.25469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa70a8d1b80 20 client.25469 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/2898822239 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564553895710 0x564553897b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa70a8d1b80 4 client.25469 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:37:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.222+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2898822239 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645537d2ce0 con 0x564553895710 2023-11-06T01:37:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.223+0000 7fa6f7fff700 1 --2- 172.21.15.53:0/2898822239 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564553895710 0x564553897b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.223+0000 7fa6d77fe700 10 client.25469 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.223+0000 7fa6f7fff700 1 --2- 172.21.15.53:0/2898822239 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564553895710 0x564553897b30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afd266a17d5910b9 server_cookie=558c619390840d7c in_seq=0 out_seq=0 2023-11-06T01:37:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.236+0000 7fa6d77fe700 1 -- 172.21.15.53:0/2898822239 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645537d2ce0 con 0x564553895710 2023-11-06T01:37:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.236+0000 7fa6d77fe700 10 client.25469 handle_command_reply: tid=0 2023-11-06T01:37:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.236+0000 7fa70a8d1b80 1 client.25469 shutdown 2023-11-06T01:37:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.236+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2898822239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553889ed0 msgr2=0x56455389e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.236+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/2898822239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553889ed0 0x56455389e530 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00bb10 tx=0x7fa6ec0a2170 comp rx=0 tx=0).stop 2023-11-06T01:37:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2898822239 shutdown_connections 2023-11-06T01:37:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/2898822239 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564553895710 0x564553897b30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/2898822239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56455389ea70 0x56455389eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/2898822239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564553888680 0x56455389dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa70a8d1b80 1 --2- 172.21.15.53:0/2898822239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564553889ed0 0x56455389e530 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2898822239 >> 172.21.15.53:0/2898822239 conn(0x5645537e8070 msgr2=0x564553884c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa6d77fe700 0 client.25469 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2898822239 shutdown_connections 2023-11-06T01:37:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa70a8d1b80 1 -- 172.21.15.53:0/2898822239 wait complete. 2023-11-06T01:37:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa70a8d1b80 20 client.25469 trim_cache size 0 max 16384 2023-11-06T01:37:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:37:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:37:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:37:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:37:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa6fe741700 1 -- 172.21.15.53:0/529861201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6e00a0670 msgr2=0x7fa6e00a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa6fe741700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6e00a0670 0x7fa6e00a2b30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009bd0 tx=0x7fa6f0009040 comp rx=0 tx=0).stop 2023-11-06T01:37:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa6fe741700 1 -- 172.21.15.53:0/529861201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f8108e40 msgr2=0x7fa6f81a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.237+0000 7fa6fe741700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f8108e40 0x7fa6f81a45e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec000c00 tx=0x7fa6ec00b840 comp rx=0 tx=0).stop 2023-11-06T01:37:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.238+0000 7fa6fe741700 1 -- 172.21.15.53:0/529861201 shutdown_connections 2023-11-06T01:37:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.238+0000 7fa6fe741700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa6e00a0670 0x7fa6e00a2b30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.238+0000 7fa6fe741700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f8109800 0x7fa6f81a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.238+0000 7fa6fe741700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f8108130 0x7fa6f81a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.238+0000 7fa6fe741700 1 --2- 172.21.15.53:0/529861201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6f8108e40 0x7fa6f81a45e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.238+0000 7fa6fe741700 1 -- 172.21.15.53:0/529861201 >> 172.21.15.53:0/529861201 conn(0x7fa6f806dbc0 msgr2=0x7fa6f80722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.238+0000 7fa6fe741700 1 -- 172.21.15.53:0/529861201 shutdown_connections 2023-11-06T01:37:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:18.238+0000 7fa6fe741700 1 -- 172.21.15.53:0/529861201 wait complete. 2023-11-06T01:37:19.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:19 smithi062 ceph-mon[103924]: pgmap v400: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 881 op/s 2023-11-06T01:37:19.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:19 smithi163 ceph-mon[104416]: pgmap v400: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 881 op/s 2023-11-06T01:37:19.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:19 smithi053 ceph-mon[99472]: pgmap v400: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 881 op/s 2023-11-06T01:37:21.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:21 smithi062 ceph-mon[103924]: pgmap v401: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 779 op/s 2023-11-06T01:37:21.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:21 smithi163 ceph-mon[104416]: pgmap v401: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 779 op/s 2023-11-06T01:37:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:21 smithi053 ceph-mon[99472]: pgmap v401: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 779 op/s 2023-11-06T01:37:22.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 185.0 s, 1925.1 tps, lat 51.447 ms stddev 61.081 2023-11-06T01:37:23.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:23 smithi062 ceph-mon[103924]: pgmap v402: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 877 op/s 2023-11-06T01:37:23.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:23 smithi163 ceph-mon[104416]: pgmap v402: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 877 op/s 2023-11-06T01:37:23.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:23 smithi053 ceph-mon[99472]: pgmap v402: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 877 op/s 2023-11-06T01:37:25.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:25 smithi163 ceph-mon[104416]: pgmap v403: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 834 op/s 2023-11-06T01:37:25.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:25 smithi053 ceph-mon[99472]: pgmap v403: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 834 op/s 2023-11-06T01:37:25.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:25 smithi062 ceph-mon[103924]: pgmap v403: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 834 op/s 2023-11-06T01:37:27.186 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 190.0 s, 2161.4 tps, lat 46.664 ms stddev 50.475 2023-11-06T01:37:27.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:27 smithi163 ceph-mon[104416]: pgmap v404: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 MiB/s wr, 1.12k op/s 2023-11-06T01:37:27.598 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:27 smithi053 ceph-mon[99472]: pgmap v404: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 MiB/s wr, 1.12k op/s 2023-11-06T01:37:27.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:27 smithi062 ceph-mon[103924]: pgmap v404: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 MiB/s wr, 1.12k op/s 2023-11-06T01:37:29.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:29 smithi163 ceph-mon[104416]: pgmap v405: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 994 op/s 2023-11-06T01:37:29.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:29 smithi053 ceph-mon[99472]: pgmap v405: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 994 op/s 2023-11-06T01:37:29.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:29 smithi062 ceph-mon[103924]: pgmap v405: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 994 op/s 2023-11-06T01:37:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.417+0000 7f3356686700 1 -- 172.21.15.53:0/2269637179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3350100da0 msgr2=0x7f33501011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.417+0000 7f3356686700 1 --2- 172.21.15.53:0/2269637179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3350100da0 0x7f33501011c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f334800b0d0 tx=0x7f334800b3e0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.417+0000 7f3356686700 1 -- 172.21.15.53:0/2269637179 shutdown_connections 2023-11-06T01:37:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.417+0000 7f3356686700 1 --2- 172.21.15.53:0/2269637179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3350102930 0x7f33501091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.417+0000 7f3356686700 1 --2- 172.21.15.53:0/2269637179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3350100da0 0x7f33501011c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.417+0000 7f3356686700 1 --2- 172.21.15.53:0/2269637179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350101f90 0x7f33501023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.417+0000 7f3356686700 1 -- 172.21.15.53:0/2269637179 >> 172.21.15.53:0/2269637179 conn(0x7f33500fc310 msgr2=0x7f33500fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.417+0000 7f3356686700 1 -- 172.21.15.53:0/2269637179 shutdown_connections 2023-11-06T01:37:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.417+0000 7f3356686700 1 -- 172.21.15.53:0/2269637179 wait complete. 2023-11-06T01:37:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.418+0000 7f3356686700 1 Processor -- start 2023-11-06T01:37:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.418+0000 7f3356686700 1 -- start start 2023-11-06T01:37:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f3356686700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3350101f90 0x7f3350073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f3356686700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3350102930 0x7f33500736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f3356686700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350074cd0 0x7f3350073c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f3356686700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335019adb0 con 0x7f3350101f90 2023-11-06T01:37:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f3356686700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33501a5760 con 0x7f3350074cd0 2023-11-06T01:37:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f3356686700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33501a5920 con 0x7f3350102930 2023-11-06T01:37:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f334ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3350101f90 0x7f3350073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f334ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3350101f90 0x7f3350073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40962/0 (socket says 172.21.15.53:40962) 2023-11-06T01:37:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f334ffff700 1 -- 172.21.15.53:0/2943103521 learned_addr learned my addr 172.21.15.53:0/2943103521 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f334f7fe700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3350102930 0x7f33500736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f3354c23700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350074cd0 0x7f3350073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f334ffff700 1 -- 172.21.15.53:0/2943103521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3350102930 msgr2=0x7f33500736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f334ffff700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3350102930 0x7f33500736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f334ffff700 1 -- 172.21.15.53:0/2943103521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350074cd0 msgr2=0x7f3350073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f334ffff700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350074cd0 0x7f3350073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f334ffff700 1 -- 172.21.15.53:0/2943103521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3348009d00 con 0x7f3350101f90 2023-11-06T01:37:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f3354c23700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350074cd0 0x7f3350073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:37:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.419+0000 7f334f7fe700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3350102930 0x7f33500736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:37:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.420+0000 7f334ffff700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3350101f90 0x7f3350073180 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3348000f80 tx=0x7f3348009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.420+0000 7f334d7fa700 1 -- 172.21.15.53:0/2943103521 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334800e040 con 0x7f3350101f90 2023-11-06T01:37:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.420+0000 7f334d7fa700 1 -- 172.21.15.53:0/2943103521 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3348003e10 con 0x7f3350101f90 2023-11-06T01:37:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.420+0000 7f334d7fa700 1 -- 172.21.15.53:0/2943103521 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348013a30 con 0x7f3350101f90 2023-11-06T01:37:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.420+0000 7f3356686700 1 -- 172.21.15.53:0/2943103521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33501a5ba0 con 0x7f3350101f90 2023-11-06T01:37:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.420+0000 7f3356686700 1 -- 172.21.15.53:0/2943103521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33501a5f70 con 0x7f3350101f90 2023-11-06T01:37:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.424+0000 7f334d7fa700 1 -- 172.21.15.53:0/2943103521 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f33480056a0 con 0x7f3350101f90 2023-11-06T01:37:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.425+0000 7f3362816b80 1 Processor -- start 2023-11-06T01:37:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.425+0000 7f3362816b80 1 -- start start 2023-11-06T01:37:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.426+0000 7f3362816b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f07ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.426+0000 7f3362816b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f08900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.426+0000 7f3362816b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117f09360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.426+0000 7f3362816b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117e4e9b0 con 0x563117f08ee0 2023-11-06T01:37:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.426+0000 7f3362816b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117e4c370 con 0x563117f07690 2023-11-06T01:37:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.426+0000 7f3362816b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117e47c60 con 0x563117f07ff0 2023-11-06T01:37:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.426+0000 7f334d7fa700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f33380a0520 0x7f33380a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.426+0000 7f334d7fa700 1 -- 172.21.15.53:0/2943103521 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f33480ccfb0 con 0x7f3350101f90 2023-11-06T01:37:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.426+0000 7f3354c23700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f07ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.426+0000 7f334ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f08900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.427+0000 7f334ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f08900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55210/0 (socket says 172.21.15.53:55210) 2023-11-06T01:37:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.427+0000 7f334ffff700 1 -- 172.21.15.53:0/2368301543 learned_addr learned my addr 172.21.15.53:0/2368301543 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.427+0000 7f334ffff700 1 -- 172.21.15.53:0/2368301543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 msgr2=0x563117f07ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.427+0000 7f334ffff700 1 --2- 172.21.15.53:0/2368301543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f07ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.427+0000 7f334ffff700 1 -- 172.21.15.53:0/2368301543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 msgr2=0x563117f09360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:37:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.427+0000 7f334ffff700 1 --2- 172.21.15.53:0/2368301543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117f09360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.427+0000 7f334ffff700 1 -- 172.21.15.53:0/2368301543 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563117c75d80 con 0x563117f07ff0 2023-11-06T01:37:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.427+0000 7f3354c23700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f33380a0520 0x7f33380a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.427+0000 7f334ffff700 1 --2- 172.21.15.53:0/2368301543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f08900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33480a2330 tx=0x7f334800b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=160229a177ee0569 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.427+0000 7f33367fc700 1 -- 172.21.15.53:0/2368301543 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334809af80 con 0x563117f07ff0 2023-11-06T01:37:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.428+0000 7f33367fc700 1 -- 172.21.15.53:0/2368301543 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334809a880 con 0x563117f07ff0 2023-11-06T01:37:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.428+0000 7f33367fc700 1 -- 172.21.15.53:0/2368301543 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33480a23d0 con 0x563117f07ff0 2023-11-06T01:37:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.428+0000 7f3354c23700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f33380a0520 0x7f33380a29e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3340005d90 tx=0x7f33400047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.428+0000 7f3362816b80 1 -- 172.21.15.53:0/2368301543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 msgr2=0x563117f08900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.428+0000 7f3362816b80 1 --2- 172.21.15.53:0/2368301543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f08900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33480a2330 tx=0x7f334800b3e0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.429+0000 7f3362816b80 1 -- 172.21.15.53:0/2368301543 shutdown_connections 2023-11-06T01:37:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.429+0000 7f3362816b80 1 --2- 172.21.15.53:0/2368301543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f07ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.429+0000 7f3362816b80 1 --2- 172.21.15.53:0/2368301543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117f09360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.429+0000 7f3362816b80 1 --2- 172.21.15.53:0/2368301543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f08900 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.429+0000 7f3362816b80 1 -- 172.21.15.53:0/2368301543 >> 172.21.15.53:0/2368301543 conn(0x563117e56070 msgr2=0x563117ef3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.431+0000 7f3362816b80 1 -- 172.21.15.53:0/2368301543 shutdown_connections 2023-11-06T01:37:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.431+0000 7f3362816b80 1 -- 172.21.15.53:0/2368301543 wait complete. 2023-11-06T01:37:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.432+0000 7f3362816b80 1 -- start start 2023-11-06T01:37:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.432+0000 7f3362816b80 1 Processor -- start 2023-11-06T01:37:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f3362816b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f11140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f3362816b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f07ff0 0x563117f11680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f3362816b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f08ee0 0x563117f186c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f3362816b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117de44b0 con 0x563117f07ff0 2023-11-06T01:37:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f3362816b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117de4150 con 0x563117f07690 2023-11-06T01:37:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f3362816b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117ccb8c0 con 0x563117f08ee0 2023-11-06T01:37:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f3362816b80 10 client.0 _ll_register_callbacks cb 0x563117d31a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f334ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f11140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f07ff0 0x563117f11680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f3354c23700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f08ee0 0x563117f186c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f334ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f11140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52412/0 (socket says 172.21.15.53:52412) 2023-11-06T01:37:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f334ffff700 1 -- 172.21.15.53:0/2452105919 learned_addr learned my addr 172.21.15.53:0/2452105919 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f334ffff700 1 -- 172.21.15.53:0/2452105919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f08ee0 msgr2=0x563117f186c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f334ffff700 1 --2- 172.21.15.53:0/2452105919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f08ee0 0x563117f186c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f334ffff700 1 -- 172.21.15.53:0/2452105919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f07ff0 msgr2=0x563117f11680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f334ffff700 1 --2- 172.21.15.53:0/2452105919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f07ff0 0x563117f11680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.433+0000 7f334ffff700 1 -- 172.21.15.53:0/2452105919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33501a5f70 con 0x563117f07690 2023-11-06T01:37:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.434+0000 7f334ffff700 1 --2- 172.21.15.53:0/2452105919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f11140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f334809a810 tx=0x7f334809a310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fc22161c5339107 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.434+0000 7f332f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:37:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.434+0000 7f332f7fe700 1 -- 172.21.15.53:0/2452105919 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334809af80 con 0x563117f07690 2023-11-06T01:37:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.434+0000 7f332f7fe700 1 -- 172.21.15.53:0/2452105919 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f334809e6b0 con 0x563117f07690 2023-11-06T01:37:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.434+0000 7f3362816b80 1 -- 172.21.15.53:0/2452105919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563117e72cb0 con 0x563117f07690 2023-11-06T01:37:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.434+0000 7f332f7fe700 1 -- 172.21.15.53:0/2452105919 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33480a2860 con 0x563117f07690 2023-11-06T01:37:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.435+0000 7f3362816b80 10 client.25478 fetch_fsmap learned FSMap version 39 2023-11-06T01:37:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.435+0000 7f3362816b80 1 -- 172.21.15.53:0/2452105919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x563117c5ec00 con 0x563117f07690 2023-11-06T01:37:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.435+0000 7f332f7fe700 1 -- 172.21.15.53:0/2452105919 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563117e72cb0 con 0x563117f07690 2023-11-06T01:37:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.435+0000 7f332f7fe700 1 -- 172.21.15.53:0/2452105919 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f33480af070 con 0x563117f07690 2023-11-06T01:37:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.435+0000 7f3362816b80 10 client.25478 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T01:37:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.435+0000 7f3362816b80 10 client.25478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.436+0000 7f3362816b80 20 client.25478 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.436+0000 7f3362816b80 1 --2- 172.21.15.53:0/2452105919 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563117f1c980 0x563117f1ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.436+0000 7f3362816b80 4 client.25478 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:37:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.436+0000 7f3362816b80 1 -- 172.21.15.53:0/2452105919 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563117ec5db0 con 0x563117f1c980 2023-11-06T01:37:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.436+0000 7f334f7fe700 1 --2- 172.21.15.53:0/2452105919 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563117f1c980 0x563117f1ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.436+0000 7f332f7fe700 10 client.25478 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.436+0000 7f334f7fe700 1 --2- 172.21.15.53:0/2452105919 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563117f1c980 0x563117f1ed70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caab94f862d8e20 server_cookie=6811b52024cf6e3e in_seq=0 out_seq=0 2023-11-06T01:37:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.437+0000 7f332f7fe700 1 -- 172.21.15.53:0/2452105919 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563117ec5db0 con 0x563117f1c980 2023-11-06T01:37:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.437+0000 7f332f7fe700 10 client.25478 handle_command_reply: tid=0 2023-11-06T01:37:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.437+0000 7f3362816b80 1 client.25478 shutdown 2023-11-06T01:37:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.439+0000 7f3362816b80 1 -- 172.21.15.53:0/2452105919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 msgr2=0x563117f11140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.439+0000 7f3362816b80 1 --2- 172.21.15.53:0/2452105919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f11140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f334809a810 tx=0x7f334809a310 comp rx=0 tx=0).stop 2023-11-06T01:37:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.439+0000 7f3362816b80 1 -- 172.21.15.53:0/2452105919 shutdown_connections 2023-11-06T01:37:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.439+0000 7f3362816b80 1 --2- 172.21.15.53:0/2452105919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f11140 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.439+0000 7f3362816b80 1 --2- 172.21.15.53:0/2452105919 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563117f1c980 0x563117f1ed70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.439+0000 7f3362816b80 1 --2- 172.21.15.53:0/2452105919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f08ee0 0x563117f186c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.439+0000 7f3362816b80 1 --2- 172.21.15.53:0/2452105919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f07ff0 0x563117f11680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.439+0000 7f3362816b80 1 -- 172.21.15.53:0/2452105919 >> 172.21.15.53:0/2452105919 conn(0x563117e56070 msgr2=0x563117ef13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.439+0000 7f332f7fe700 0 client.25478 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.439+0000 7f3362816b80 1 -- 172.21.15.53:0/2452105919 shutdown_connections 2023-11-06T01:37:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.440+0000 7f3362816b80 1 -- 172.21.15.53:0/2452105919 wait complete. 2023-11-06T01:37:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.440+0000 7f3362816b80 20 client.25478 trim_cache size 0 max 16384 2023-11-06T01:37:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.463+0000 7f3362816b80 1 Processor -- start 2023-11-06T01:37:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f3362816b80 1 -- start start 2023-11-06T01:37:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f3362816b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07690 0x563117f16670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f3362816b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07ff0 0x563117f0c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f3362816b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117ef2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f3362816b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117f10cb0 con 0x563117f08ee0 2023-11-06T01:37:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f3362816b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117f12150 con 0x563117f07ff0 2023-11-06T01:37:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f3362816b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117f1c770 con 0x563117f07690 2023-11-06T01:37:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f3354c23700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117ef2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07ff0 0x563117f0c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07ff0 0x563117f0c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52424/0 (socket says 172.21.15.53:52424) 2023-11-06T01:37:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f334f7fe700 1 -- 172.21.15.53:0/915450437 learned_addr learned my addr 172.21.15.53:0/915450437 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.464+0000 7f334ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07690 0x563117f16670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.465+0000 7f334f7fe700 1 -- 172.21.15.53:0/915450437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07690 msgr2=0x563117f16670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.465+0000 7f334f7fe700 1 --2- 172.21.15.53:0/915450437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07690 0x563117f16670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.465+0000 7f334f7fe700 1 -- 172.21.15.53:0/915450437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 msgr2=0x563117ef2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.465+0000 7f334f7fe700 1 --2- 172.21.15.53:0/915450437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117ef2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.465+0000 7f334f7fe700 1 -- 172.21.15.53:0/915450437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563117c9c1f0 con 0x563117f07ff0 2023-11-06T01:37:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.465+0000 7f3354c23700 1 --2- 172.21.15.53:0/915450437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117ef2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:37:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.465+0000 7f334f7fe700 1 --2- 172.21.15.53:0/915450437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07ff0 0x563117f0c0b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x563117f086c0 tx=0x7f33440058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf110c48f439b947 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.465+0000 7f3336ffd700 1 -- 172.21.15.53:0/915450437 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33440183b0 con 0x563117f07ff0 2023-11-06T01:37:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.465+0000 7f3336ffd700 1 -- 172.21.15.53:0/915450437 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3344018040 con 0x563117f07ff0 2023-11-06T01:37:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.465+0000 7f3336ffd700 1 -- 172.21.15.53:0/915450437 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344003680 con 0x563117f07ff0 2023-11-06T01:37:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.466+0000 7f3362816b80 1 -- 172.21.15.53:0/915450437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07ff0 msgr2=0x563117f0c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.466+0000 7f3362816b80 1 --2- 172.21.15.53:0/915450437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07ff0 0x563117f0c0b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x563117f086c0 tx=0x7f33440058e0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.466+0000 7f3362816b80 1 -- 172.21.15.53:0/915450437 shutdown_connections 2023-11-06T01:37:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.466+0000 7f3362816b80 1 --2- 172.21.15.53:0/915450437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117ef2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.466+0000 7f3362816b80 1 --2- 172.21.15.53:0/915450437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07690 0x563117f16670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.466+0000 7f3362816b80 1 --2- 172.21.15.53:0/915450437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07ff0 0x563117f0c0b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.466+0000 7f3362816b80 1 -- 172.21.15.53:0/915450437 >> 172.21.15.53:0/915450437 conn(0x563117e56070 msgr2=0x563117eedb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.466+0000 7f3362816b80 1 -- 172.21.15.53:0/915450437 shutdown_connections 2023-11-06T01:37:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.466+0000 7f3362816b80 1 -- 172.21.15.53:0/915450437 wait complete. 2023-11-06T01:37:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.467+0000 7f3362816b80 1 -- start start 2023-11-06T01:37:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.467+0000 7f3362816b80 1 Processor -- start 2023-11-06T01:37:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.467+0000 7f3362816b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f66630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.467+0000 7f3362816b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f66b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.467+0000 7f3362816b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117f6d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.467+0000 7f3362816b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117f6dbb0 con 0x563117f08ee0 2023-11-06T01:37:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.467+0000 7f3362816b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117f6dcf0 con 0x563117f07690 2023-11-06T01:37:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.467+0000 7f3362816b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563117f6de80 con 0x563117f07ff0 2023-11-06T01:37:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.467+0000 7f3362816b80 10 client.0 _ll_register_callbacks cb 0x7f33480964a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f334ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f66630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f66b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f66b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55230/0 (socket says 172.21.15.53:55230) 2023-11-06T01:37:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f334ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f66630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52426/0 (socket says 172.21.15.53:52426) 2023-11-06T01:37:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f334f7fe700 1 -- 172.21.15.53:0/1342053504 learned_addr learned my addr 172.21.15.53:0/1342053504 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f3354c23700 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117f6d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f334f7fe700 1 -- 172.21.15.53:0/1342053504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 msgr2=0x563117f66630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f334f7fe700 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f66630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f334f7fe700 1 -- 172.21.15.53:0/1342053504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 msgr2=0x563117f6d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f334f7fe700 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117f6d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f334f7fe700 1 -- 172.21.15.53:0/1342053504 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3340013c40 con 0x563117f07ff0 2023-11-06T01:37:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f334f7fe700 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f66b70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f33440057d0 tx=0x7f3344005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96f243259a9ce08 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f332f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:37:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f332f7fe700 1 -- 172.21.15.53:0/1342053504 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344026070 con 0x563117f07ff0 2023-11-06T01:37:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f332f7fe700 1 -- 172.21.15.53:0/1342053504 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3344004c50 con 0x563117f07ff0 2023-11-06T01:37:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.468+0000 7f3362816b80 1 -- 172.21.15.53:0/1342053504 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f334809a880 con 0x563117f07ff0 2023-11-06T01:37:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.470+0000 7f332f7fe700 1 -- 172.21.15.53:0/1342053504 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3344004e10 con 0x563117f07ff0 2023-11-06T01:37:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.474+0000 7f3362816b80 10 client.25558 fetch_fsmap learned FSMap version 39 2023-11-06T01:37:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.474+0000 7f3362816b80 1 -- 172.21.15.53:0/1342053504 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f334400cca0 con 0x563117f07ff0 2023-11-06T01:37:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.474+0000 7f332f7fe700 1 -- 172.21.15.53:0/1342053504 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334809a880 con 0x563117f07ff0 2023-11-06T01:37:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.474+0000 7f332f7fe700 1 -- 172.21.15.53:0/1342053504 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f3344033020 con 0x563117f07ff0 2023-11-06T01:37:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.474+0000 7f3362816b80 10 client.25558 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T01:37:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.474+0000 7f3362816b80 10 client.25558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.474+0000 7f3362816b80 20 client.25558 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.474+0000 7f3362816b80 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563117f6f7b0 0x563117f71bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.474+0000 7f3362816b80 4 client.25558 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:37:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.474+0000 7f3362816b80 1 -- 172.21.15.53:0/1342053504 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563117e40ce0 con 0x563117f6f7b0 2023-11-06T01:37:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.475+0000 7f334ffff700 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563117f6f7b0 0x563117f71bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.475+0000 7f332f7fe700 10 client.25558 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.475+0000 7f334ffff700 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563117f6f7b0 0x563117f71bd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff62a155cb157cf5 server_cookie=9701d5ed9702c597 in_seq=0 out_seq=0 2023-11-06T01:37:31.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:31 smithi163 ceph-mon[104416]: pgmap v406: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.15k op/s 2023-11-06T01:37:31.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:31 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.492+0000 7f332f7fe700 1 -- 172.21.15.53:0/1342053504 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563117e40ce0 con 0x563117f6f7b0 2023-11-06T01:37:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.492+0000 7f332f7fe700 10 client.25558 handle_command_reply: tid=0 2023-11-06T01:37:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.492+0000 7f3362816b80 1 client.25558 shutdown 2023-11-06T01:37:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.492+0000 7f3362816b80 1 -- 172.21.15.53:0/1342053504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 msgr2=0x563117f66b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.492+0000 7f3362816b80 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f66b70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f33440057d0 tx=0x7f3344005800 comp rx=0 tx=0).stop 2023-11-06T01:37:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.493+0000 7f3362816b80 1 -- 172.21.15.53:0/1342053504 shutdown_connections 2023-11-06T01:37:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.493+0000 7f3362816b80 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563117f07690 0x563117f66630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.493+0000 7f3362816b80 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563117f6f7b0 0x563117f71bd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.493+0000 7f3362816b80 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563117f08ee0 0x563117f6d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.493+0000 7f3362816b80 1 --2- 172.21.15.53:0/1342053504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563117f07ff0 0x563117f66b70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.493+0000 7f3362816b80 1 -- 172.21.15.53:0/1342053504 >> 172.21.15.53:0/1342053504 conn(0x563117e56070 msgr2=0x563117f0c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.494+0000 7f332f7fe700 0 client.25558 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.494+0000 7f3362816b80 1 -- 172.21.15.53:0/1342053504 shutdown_connections 2023-11-06T01:37:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.494+0000 7f3362816b80 1 -- 172.21.15.53:0/1342053504 wait complete. 2023-11-06T01:37:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.496+0000 7f3362816b80 20 client.25558 trim_cache size 0 max 16384 2023-11-06T01:37:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:37:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:37:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:37:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:37:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.498+0000 7f3356686700 1 -- 172.21.15.53:0/2943103521 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f33380a0520 msgr2=0x7f33380a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.498+0000 7f3356686700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f33380a0520 0x7f33380a29e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3340005d90 tx=0x7f33400047c0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.499+0000 7f3356686700 1 -- 172.21.15.53:0/2943103521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3350101f90 msgr2=0x7f3350073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.499+0000 7f3356686700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3350101f90 0x7f3350073180 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3348000f80 tx=0x7f3348009410 comp rx=0 tx=0).stop 2023-11-06T01:37:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.499+0000 7f3356686700 1 -- 172.21.15.53:0/2943103521 shutdown_connections 2023-11-06T01:37:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.499+0000 7f3356686700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f33380a0520 0x7f33380a29e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.499+0000 7f3356686700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350074cd0 0x7f3350073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.499+0000 7f3356686700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3350101f90 0x7f3350073180 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.499+0000 7f3356686700 1 --2- 172.21.15.53:0/2943103521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3350102930 0x7f33500736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.499+0000 7f3356686700 1 -- 172.21.15.53:0/2943103521 >> 172.21.15.53:0/2943103521 conn(0x7f33500fc310 msgr2=0x7f3350105640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.499+0000 7f3356686700 1 -- 172.21.15.53:0/2943103521 shutdown_connections 2023-11-06T01:37:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:31.499+0000 7f3356686700 1 -- 172.21.15.53:0/2943103521 wait complete. 2023-11-06T01:37:31.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:31 smithi053 ceph-mon[99472]: pgmap v406: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.15k op/s 2023-11-06T01:37:31.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:31 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:31.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:31 smithi062 ceph-mon[103924]: pgmap v406: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.15k op/s 2023-11-06T01:37:31.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:31 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 195.0 s, 1924.4 tps, lat 51.735 ms stddev 87.456 2023-11-06T01:37:33.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:33 smithi163 ceph-mon[104416]: pgmap v407: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 MiB/s wr, 2.13k op/s 2023-11-06T01:37:33.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:33 smithi053 ceph-mon[99472]: pgmap v407: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 MiB/s wr, 2.13k op/s 2023-11-06T01:37:33.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:33 smithi062 ceph-mon[103924]: pgmap v407: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 MiB/s wr, 2.13k op/s 2023-11-06T01:37:35.454 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:37:35.489 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:35 smithi053 ceph-mon[99472]: pgmap v408: 97 pgs: 97 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 49 MiB/s wr, 2.14k op/s 2023-11-06T01:37:35.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:35 smithi163 ceph-mon[104416]: pgmap v408: 97 pgs: 97 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 49 MiB/s wr, 2.14k op/s 2023-11-06T01:37:35.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:35 smithi062 ceph-mon[103924]: pgmap v408: 97 pgs: 97 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 49 MiB/s wr, 2.14k op/s 2023-11-06T01:37:35.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.655+0000 7f89be688700 1 -- 172.21.15.53:0/3451878861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b810d350 msgr2=0x7f89b810f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:35.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.655+0000 7f89be688700 1 --2- 172.21.15.53:0/3451878861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b810d350 0x7f89b810f740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f89b000b0a0 tx=0x7f89b000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.655+0000 7f89be688700 1 -- 172.21.15.53:0/3451878861 shutdown_connections 2023-11-06T01:37:35.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.655+0000 7f89be688700 1 --2- 172.21.15.53:0/3451878861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b810d350 0x7f89b810f740 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.655+0000 7f89be688700 1 --2- 172.21.15.53:0/3451878861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89b8074dd0 0x7f89b8073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.655+0000 7f89be688700 1 --2- 172.21.15.53:0/3451878861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80737f0 0x7f89b8073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.655+0000 7f89be688700 1 -- 172.21.15.53:0/3451878861 >> 172.21.15.53:0/3451878861 conn(0x7f89b80fc310 msgr2=0x7f89b80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:35.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.655+0000 7f89be688700 1 -- 172.21.15.53:0/3451878861 shutdown_connections 2023-11-06T01:37:35.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.655+0000 7f89be688700 1 -- 172.21.15.53:0/3451878861 wait complete. 2023-11-06T01:37:35.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.656+0000 7f89be688700 1 Processor -- start 2023-11-06T01:37:35.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.658+0000 7f89be688700 1 -- start start 2023-11-06T01:37:35.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89be688700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b80737f0 0x7f89b806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89be688700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b8074dd0 0x7f89b806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89be688700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89b810d350 0x7f89b81ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89be688700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b8072c00 con 0x7f89b8074dd0 2023-11-06T01:37:35.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89be688700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b8072d70 con 0x7f89b80737f0 2023-11-06T01:37:35.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89be688700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b81ad780 con 0x7f89b810d350 2023-11-06T01:37:35.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b80737f0 0x7f89b806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b80737f0 0x7f89b806f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52440/0 (socket says 172.21.15.53:52440) 2023-11-06T01:37:35.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89b7fff700 1 -- 172.21.15.53:0/1406369998 learned_addr learned my addr 172.21.15.53:0/1406369998 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:35.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89b77fe700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b8074dd0 0x7f89b806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89bcc25700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89b810d350 0x7f89b81ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89b7fff700 1 -- 172.21.15.53:0/1406369998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89b810d350 msgr2=0x7f89b81ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:35.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89b7fff700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89b810d350 0x7f89b81ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89b7fff700 1 -- 172.21.15.53:0/1406369998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b8074dd0 msgr2=0x7f89b806fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:35.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89b7fff700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b8074dd0 0x7f89b806fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89b7fff700 1 -- 172.21.15.53:0/1406369998 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b0009d00 con 0x7f89b80737f0 2023-11-06T01:37:35.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89b77fe700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b8074dd0 0x7f89b806fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:37:35.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.659+0000 7f89bcc25700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89b810d350 0x7f89b81ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:37:35.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.660+0000 7f89b7fff700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b80737f0 0x7f89b806f5e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00d950 tx=0x7f89ac00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:35.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.660+0000 7f89b57fa700 1 -- 172.21.15.53:0/1406369998 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0098e0 con 0x7f89b80737f0 2023-11-06T01:37:35.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.660+0000 7f89b57fa700 1 -- 172.21.15.53:0/1406369998 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89ac010460 con 0x7f89b80737f0 2023-11-06T01:37:35.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.660+0000 7f89be688700 1 -- 172.21.15.53:0/1406369998 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b81ad980 con 0x7f89b80737f0 2023-11-06T01:37:35.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.660+0000 7f89b57fa700 1 -- 172.21.15.53:0/1406369998 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac00f620 con 0x7f89b80737f0 2023-11-06T01:37:35.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.660+0000 7f89be688700 1 -- 172.21.15.53:0/1406369998 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b81adee0 con 0x7f89b80737f0 2023-11-06T01:37:35.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.663+0000 7f89b57fa700 1 -- 172.21.15.53:0/1406369998 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f89ac0108d0 con 0x7f89b80737f0 2023-11-06T01:37:35.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.664+0000 7f89b57fa700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89a00a05a0 0x7f89a00a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.664+0000 7f89b57fa700 1 -- 172.21.15.53:0/1406369998 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f89ac0cb750 con 0x7f89b80737f0 2023-11-06T01:37:35.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.664+0000 7f89b77fe700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89a00a05a0 0x7f89a00a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.665+0000 7f89ca818b80 1 Processor -- start 2023-11-06T01:37:35.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89ca818b80 1 -- start start 2023-11-06T01:37:35.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f3c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3ca70 0x562829f3ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3b3f0 0x562829f3b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89ca818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829e92a10 con 0x562829f3ca70 2023-11-06T01:37:35.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89ca818b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829e903d0 con 0x562829f3c0f0 2023-11-06T01:37:35.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89ca818b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829e8bcc0 con 0x562829f3b3f0 2023-11-06T01:37:35.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89bcc25700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3b3f0 0x562829f3b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f3c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f3c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52448/0 (socket says 172.21.15.53:52448) 2023-11-06T01:37:35.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89bcc25700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3b3f0 0x562829f3b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55252/0 (socket says 172.21.15.53:55252) 2023-11-06T01:37:35.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89b7fff700 1 -- 172.21.15.53:0/1292221272 learned_addr learned my addr 172.21.15.53:0/1292221272 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:35.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.668+0000 7f89b77fe700 1 --2- 172.21.15.53:0/1292221272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3ca70 0x562829f3ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.669+0000 7f89b7fff700 1 -- 172.21.15.53:0/1292221272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3b3f0 msgr2=0x562829f3b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:35.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.669+0000 7f89b7fff700 1 --2- 172.21.15.53:0/1292221272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3b3f0 0x562829f3b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.669+0000 7f89b7fff700 1 -- 172.21.15.53:0/1292221272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3ca70 msgr2=0x562829f3ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:35.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.669+0000 7f89b7fff700 1 --2- 172.21.15.53:0/1292221272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3ca70 0x562829f3ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.669+0000 7f89b7fff700 1 -- 172.21.15.53:0/1292221272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562829cb9de0 con 0x562829f3c0f0 2023-11-06T01:37:35.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.669+0000 7f89b7fff700 1 --2- 172.21.15.53:0/1292221272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f3c530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89ac095490 tx=0x7f89ac098be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f1492a3a2fdc7f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:35.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.669+0000 7f89b77fe700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89a00a05a0 0x7f89a00a2a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f89b8074af0 tx=0x7f89a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:35.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.669+0000 7f899e7fc700 1 -- 172.21.15.53:0/1292221272 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0a1000 con 0x562829f3c0f0 2023-11-06T01:37:35.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.669+0000 7f899e7fc700 1 -- 172.21.15.53:0/1292221272 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89ac097330 con 0x562829f3c0f0 2023-11-06T01:37:35.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.669+0000 7f899e7fc700 1 -- 172.21.15.53:0/1292221272 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0a0b70 con 0x562829f3c0f0 2023-11-06T01:37:35.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.670+0000 7f89ca818b80 1 -- 172.21.15.53:0/1292221272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 msgr2=0x562829f3c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:35.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.670+0000 7f89ca818b80 1 --2- 172.21.15.53:0/1292221272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f3c530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89ac095490 tx=0x7f89ac098be0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.670+0000 7f89ca818b80 1 -- 172.21.15.53:0/1292221272 shutdown_connections 2023-11-06T01:37:35.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.670+0000 7f89ca818b80 1 --2- 172.21.15.53:0/1292221272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f3c530 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.670+0000 7f89ca818b80 1 --2- 172.21.15.53:0/1292221272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3b3f0 0x562829f3b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.670+0000 7f89ca818b80 1 --2- 172.21.15.53:0/1292221272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3ca70 0x562829f3ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.670+0000 7f89ca818b80 1 -- 172.21.15.53:0/1292221272 >> 172.21.15.53:0/1292221272 conn(0x562829e9a0d0 msgr2=0x562829f35940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:35.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.670+0000 7f89ca818b80 1 -- 172.21.15.53:0/1292221272 shutdown_connections 2023-11-06T01:37:35.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.670+0000 7f89ca818b80 1 -- 172.21.15.53:0/1292221272 wait complete. 2023-11-06T01:37:35.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.670+0000 7f89ca818b80 1 -- start start 2023-11-06T01:37:35.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89ca818b80 1 Processor -- start 2023-11-06T01:37:35.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3b3f0 0x562829f44950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f44e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3ca70 0x562829f4bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89ca818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829e28510 con 0x562829f3ca70 2023-11-06T01:37:35.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89ca818b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829e281b0 con 0x562829f3c0f0 2023-11-06T01:37:35.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89ca818b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829d0f920 con 0x562829f3b3f0 2023-11-06T01:37:35.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89ca818b80 10 client.0 _ll_register_callbacks cb 0x562829d75a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:35.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f44e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f44e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52452/0 (socket says 172.21.15.53:52452) 2023-11-06T01:37:35.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89b77fe700 1 -- 172.21.15.53:0/2192113623 learned_addr learned my addr 172.21.15.53:0/2192113623 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:35.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89bcc25700 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3ca70 0x562829f4bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.671+0000 7f89b7fff700 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3b3f0 0x562829f44950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89bcc25700 1 -- 172.21.15.53:0/2192113623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3b3f0 msgr2=0x562829f44950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:35.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89bcc25700 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3b3f0 0x562829f44950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89bcc25700 1 -- 172.21.15.53:0/2192113623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 msgr2=0x562829f44e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:35.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89bcc25700 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f44e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89bcc25700 1 -- 172.21.15.53:0/2192113623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b81adee0 con 0x562829f3ca70 2023-11-06T01:37:35.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89bcc25700 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3ca70 0x562829f4bed0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f89b000b070 tx=0x7f89b0004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704129a74b287c4f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:35.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89977fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:37:35.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89977fe700 1 -- 172.21.15.53:0/2192113623 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b000e040 con 0x562829f3ca70 2023-11-06T01:37:35.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89977fe700 1 -- 172.21.15.53:0/2192113623 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f89b00096a0 con 0x562829f3ca70 2023-11-06T01:37:35.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89ca818b80 1 -- 172.21.15.53:0/2192113623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562829eb6d10 con 0x562829f3ca70 2023-11-06T01:37:35.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89977fe700 1 -- 172.21.15.53:0/2192113623 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89b0004e80 con 0x562829f3ca70 2023-11-06T01:37:35.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89ca818b80 10 client.15918 fetch_fsmap learned FSMap version 39 2023-11-06T01:37:35.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.672+0000 7f89ca818b80 1 -- 172.21.15.53:0/2192113623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562829ca2c60 con 0x562829f3ca70 2023-11-06T01:37:35.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.673+0000 7f89977fe700 1 -- 172.21.15.53:0/2192113623 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562829eb6d10 con 0x562829f3ca70 2023-11-06T01:37:35.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.673+0000 7f89977fe700 1 -- 172.21.15.53:0/2192113623 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f89b0019070 con 0x562829f3ca70 2023-11-06T01:37:35.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.673+0000 7f89ca818b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T01:37:35.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.673+0000 7f89ca818b80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:35.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.673+0000 7f89ca818b80 20 client.15918 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:35.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.673+0000 7f89ca818b80 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562829f61680 0x562829f63aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.673+0000 7f89ca818b80 4 client.15918 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:37:35.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.673+0000 7f89ca818b80 1 -- 172.21.15.53:0/2192113623 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562829f09e10 con 0x562829f61680 2023-11-06T01:37:35.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.673+0000 7f89b7fff700 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562829f61680 0x562829f63aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.673+0000 7f89977fe700 10 client.15918 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:35.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.673+0000 7f89b7fff700 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562829f61680 0x562829f63aa0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d2a60965dadb10b server_cookie=7dad5645874249bf in_seq=0 out_seq=0 2023-11-06T01:37:35.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89977fe700 1 -- 172.21.15.53:0/2192113623 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562829f09e10 con 0x562829f61680 2023-11-06T01:37:35.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89977fe700 10 client.15918 handle_command_reply: tid=0 2023-11-06T01:37:35.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89ca818b80 1 client.15918 shutdown 2023-11-06T01:37:35.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89ca818b80 1 -- 172.21.15.53:0/2192113623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3ca70 msgr2=0x562829f4bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:35.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89ca818b80 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3ca70 0x562829f4bed0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f89b000b070 tx=0x7f89b0004c60 comp rx=0 tx=0).stop 2023-11-06T01:37:35.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89ca818b80 1 -- 172.21.15.53:0/2192113623 shutdown_connections 2023-11-06T01:37:35.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89ca818b80 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562829f61680 0x562829f63aa0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89ca818b80 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3ca70 0x562829f4bed0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89977fe700 0 client.15918 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:35.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89ca818b80 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3b3f0 0x562829f44950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89ca818b80 1 --2- 172.21.15.53:0/2192113623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f44e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89ca818b80 1 -- 172.21.15.53:0/2192113623 >> 172.21.15.53:0/2192113623 conn(0x562829e9a0d0 msgr2=0x562829f350b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:35.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89ca818b80 1 -- 172.21.15.53:0/2192113623 shutdown_connections 2023-11-06T01:37:35.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.674+0000 7f89ca818b80 1 -- 172.21.15.53:0/2192113623 wait complete. 2023-11-06T01:37:35.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.675+0000 7f89ca818b80 20 client.15918 trim_cache size 0 max 16384 2023-11-06T01:37:35.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89ca818b80 1 Processor -- start 2023-11-06T01:37:35.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89ca818b80 1 -- start start 2023-11-06T01:37:35.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3b3f0 0x562829f3ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f4be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3ca70 0x562829f44f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89ca818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829f43e00 con 0x562829f3b3f0 2023-11-06T01:37:35.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89ca818b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829f36820 con 0x562829f3c0f0 2023-11-06T01:37:35.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89ca818b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829f444c0 con 0x562829f3ca70 2023-11-06T01:37:35.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f4be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f4be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52462/0 (socket says 172.21.15.53:52462) 2023-11-06T01:37:35.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89b77fe700 1 -- 172.21.15.53:0/341131656 learned_addr learned my addr 172.21.15.53:0/341131656 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:35.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.703+0000 7f89b7fff700 1 --2- 172.21.15.53:0/341131656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3b3f0 0x562829f3ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.704+0000 7f89b77fe700 1 -- 172.21.15.53:0/341131656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3ca70 msgr2=0x562829f44f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:37:35.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.704+0000 7f89b77fe700 1 --2- 172.21.15.53:0/341131656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3ca70 0x562829f44f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.704+0000 7f89b77fe700 1 -- 172.21.15.53:0/341131656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3b3f0 msgr2=0x562829f3ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:35.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.704+0000 7f89b77fe700 1 --2- 172.21.15.53:0/341131656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3b3f0 0x562829f3ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.704+0000 7f89b77fe700 1 -- 172.21.15.53:0/341131656 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562829ce0250 con 0x562829f3c0f0 2023-11-06T01:37:35.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.704+0000 7f89b7fff700 1 --2- 172.21.15.53:0/341131656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3b3f0 0x562829f3ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:37:35.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.704+0000 7f89b77fe700 1 --2- 172.21.15.53:0/341131656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f4be90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f89a8004490 tx=0x7f89a8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceb447bc80d59681 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:35.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.704+0000 7f899effd700 1 -- 172.21.15.53:0/341131656 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8024070 con 0x562829f3c0f0 2023-11-06T01:37:35.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.704+0000 7f899effd700 1 -- 172.21.15.53:0/341131656 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89a80044c0 con 0x562829f3c0f0 2023-11-06T01:37:35.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.704+0000 7f899effd700 1 -- 172.21.15.53:0/341131656 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8005720 con 0x562829f3c0f0 2023-11-06T01:37:35.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.705+0000 7f89ca818b80 1 -- 172.21.15.53:0/341131656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 msgr2=0x562829f4be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:35.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.705+0000 7f89ca818b80 1 --2- 172.21.15.53:0/341131656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f4be90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f89a8004490 tx=0x7f89a8014000 comp rx=0 tx=0).stop 2023-11-06T01:37:35.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.705+0000 7f89ca818b80 1 -- 172.21.15.53:0/341131656 shutdown_connections 2023-11-06T01:37:35.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.705+0000 7f89ca818b80 1 --2- 172.21.15.53:0/341131656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3ca70 0x562829f44f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.705+0000 7f89ca818b80 1 --2- 172.21.15.53:0/341131656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f3b3f0 0x562829f3ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.705+0000 7f89ca818b80 1 --2- 172.21.15.53:0/341131656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3c0f0 0x562829f4be90 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f89a8004490 tx=0x7f89a8014000 comp rx=0 tx=0).stop 2023-11-06T01:37:35.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.705+0000 7f89ca818b80 1 -- 172.21.15.53:0/341131656 >> 172.21.15.53:0/341131656 conn(0x562829e9a0d0 msgr2=0x562829f5d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:35.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.706+0000 7f89ca818b80 1 -- 172.21.15.53:0/341131656 shutdown_connections 2023-11-06T01:37:35.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.706+0000 7f89ca818b80 1 -- 172.21.15.53:0/341131656 wait complete. 2023-11-06T01:37:35.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.707+0000 7f89ca818b80 1 -- start start 2023-11-06T01:37:35.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.707+0000 7f89ca818b80 1 Processor -- start 2023-11-06T01:37:35.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.707+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3b3f0 0x562829f4a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.707+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3ca70 0x562829f4aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.707+0000 7f89ca818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f4b030 0x562829fb3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.707+0000 7f89ca818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829f4c0d0 con 0x562829f4b030 2023-11-06T01:37:35.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.707+0000 7f89ca818b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829f4c240 con 0x562829f3b3f0 2023-11-06T01:37:35.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.707+0000 7f89ca818b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562829fb3700 con 0x562829f3ca70 2023-11-06T01:37:35.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.707+0000 7f89ca818b80 10 client.0 _ll_register_callbacks cb 0x7f89b000bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:35.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.708+0000 7f89b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3b3f0 0x562829f4a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.708+0000 7f89b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3b3f0 0x562829f4a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52466/0 (socket says 172.21.15.53:52466) 2023-11-06T01:37:35.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.708+0000 7f89b7fff700 1 -- 172.21.15.53:0/497330974 learned_addr learned my addr 172.21.15.53:0/497330974 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:35.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.708+0000 7f89b77fe700 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3ca70 0x562829f4aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.708+0000 7f89b7fff700 1 -- 172.21.15.53:0/497330974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3ca70 msgr2=0x562829f4aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:35.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.708+0000 7f89b7fff700 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3ca70 0x562829f4aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.708+0000 7f89b7fff700 1 -- 172.21.15.53:0/497330974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f4b030 msgr2=0x562829fb3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:35.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.708+0000 7f89b7fff700 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f4b030 0x562829fb3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.708+0000 7f89b7fff700 1 -- 172.21.15.53:0/497330974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a8017070 con 0x562829f3b3f0 2023-11-06T01:37:35.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.708+0000 7f89b7fff700 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3b3f0 0x562829f4a5b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f89ac095a00 tx=0x7f89ac099110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637c954b026a36f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:35.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.709+0000 7f89977fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:37:35.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.709+0000 7f89977fe700 1 -- 172.21.15.53:0/497330974 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0993f0 con 0x562829f3b3f0 2023-11-06T01:37:35.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.709+0000 7f89977fe700 1 -- 172.21.15.53:0/497330974 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f89ac098480 con 0x562829f3b3f0 2023-11-06T01:37:35.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.709+0000 7f89ca818b80 1 -- 172.21.15.53:0/497330974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89ac097330 con 0x562829f3b3f0 2023-11-06T01:37:35.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.709+0000 7f89977fe700 1 -- 172.21.15.53:0/497330974 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ac098640 con 0x562829f3b3f0 2023-11-06T01:37:35.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.709+0000 7f89ca818b80 10 client.25502 fetch_fsmap learned FSMap version 39 2023-11-06T01:37:35.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.709+0000 7f89ca818b80 1 -- 172.21.15.53:0/497330974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562829cb82b0 con 0x562829f3b3f0 2023-11-06T01:37:35.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.709+0000 7f89977fe700 1 -- 172.21.15.53:0/497330974 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89ac097330 con 0x562829f3b3f0 2023-11-06T01:37:35.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.710+0000 7f89977fe700 1 -- 172.21.15.53:0/497330974 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f89ac093ba0 con 0x562829f3b3f0 2023-11-06T01:37:35.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.710+0000 7f89ca818b80 10 client.25502 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T01:37:35.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.710+0000 7f89ca818b80 10 client.25502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:35.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.710+0000 7f89ca818b80 20 client.25502 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:35.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.710+0000 7f89ca818b80 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562829fb4dc0 0x562829fb71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:35.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.710+0000 7f89ca818b80 4 client.25502 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:37:35.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.710+0000 7f89ca818b80 1 -- 172.21.15.53:0/497330974 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562829e84d40 con 0x562829fb4dc0 2023-11-06T01:37:35.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.710+0000 7f89b77fe700 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562829fb4dc0 0x562829fb71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:35.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.710+0000 7f89b77fe700 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562829fb4dc0 0x562829fb71e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7be5799a3dc3c3f server_cookie=a54db4568bc3ed7 in_seq=0 out_seq=0 2023-11-06T01:37:35.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.710+0000 7f89977fe700 10 client.25502 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:35.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.711+0000 7f89977fe700 1 -- 172.21.15.53:0/497330974 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562829e84d40 con 0x562829fb4dc0 2023-11-06T01:37:35.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.711+0000 7f89977fe700 10 client.25502 handle_command_reply: tid=0 2023-11-06T01:37:35.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.711+0000 7f89ca818b80 1 client.25502 shutdown 2023-11-06T01:37:35.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89ca818b80 1 -- 172.21.15.53:0/497330974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3b3f0 msgr2=0x562829f4a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:35.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89ca818b80 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3b3f0 0x562829f4a5b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f89ac095a00 tx=0x7f89ac099110 comp rx=0 tx=0).stop 2023-11-06T01:37:35.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89ca818b80 1 -- 172.21.15.53:0/497330974 shutdown_connections 2023-11-06T01:37:35.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89ca818b80 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562829f3b3f0 0x562829f4a5b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89ca818b80 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562829fb4dc0 0x562829fb71e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89ca818b80 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562829f4b030 0x562829fb3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89ca818b80 1 --2- 172.21.15.53:0/497330974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562829f3ca70 0x562829f4aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89ca818b80 1 -- 172.21.15.53:0/497330974 >> 172.21.15.53:0/497330974 conn(0x562829e9a0d0 msgr2=0x562829f44700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:35.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89977fe700 0 client.25502 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:35.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89ca818b80 1 -- 172.21.15.53:0/497330974 shutdown_connections 2023-11-06T01:37:35.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89ca818b80 1 -- 172.21.15.53:0/497330974 wait complete. 2023-11-06T01:37:35.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.713+0000 7f89ca818b80 20 client.25502 trim_cache size 0 max 16384 2023-11-06T01:37:35.724 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:37:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:37:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:37:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:37:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:37:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:37:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:37:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:37:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:37:35.726 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:37:35.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.714+0000 7f89be688700 1 -- 172.21.15.53:0/1406369998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89a00a05a0 msgr2=0x7f89a00a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:35.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.714+0000 7f89be688700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89a00a05a0 0x7f89a00a2a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f89b8074af0 tx=0x7f89a8009260 comp rx=0 tx=0).stop 2023-11-06T01:37:35.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.714+0000 7f89be688700 1 -- 172.21.15.53:0/1406369998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b80737f0 msgr2=0x7f89b806f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:35.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.714+0000 7f89be688700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b80737f0 0x7f89b806f5e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00d950 tx=0x7f89ac00dc60 comp rx=0 tx=0).stop 2023-11-06T01:37:35.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.715+0000 7f89be688700 1 -- 172.21.15.53:0/1406369998 shutdown_connections 2023-11-06T01:37:35.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.715+0000 7f89be688700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89a00a05a0 0x7f89a00a2a60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.715+0000 7f89be688700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b80737f0 0x7f89b806f5e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.716+0000 7f89be688700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89b810d350 0x7f89b81ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.716+0000 7f89be688700 1 --2- 172.21.15.53:0/1406369998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b8074dd0 0x7f89b806fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:35.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.716+0000 7f89be688700 1 -- 172.21.15.53:0/1406369998 >> 172.21.15.53:0/1406369998 conn(0x7f89b80fc310 msgr2=0x7f89b80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:35.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.716+0000 7f89be688700 1 -- 172.21.15.53:0/1406369998 shutdown_connections 2023-11-06T01:37:35.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:37:35.716+0000 7f89be688700 1 -- 172.21.15.53:0/1406369998 wait complete. 2023-11-06T01:37:35.753 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:37:37.185 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 200.0 s, 2674.2 tps, lat 37.127 ms stddev 39.569 2023-11-06T01:37:37.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:37 smithi163 ceph-mon[104416]: pgmap v409: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 51 MiB/s wr, 2.26k op/s 2023-11-06T01:37:37.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:37 smithi053 ceph-mon[99472]: pgmap v409: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 51 MiB/s wr, 2.26k op/s 2023-11-06T01:37:37.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:37 smithi062 ceph-mon[103924]: pgmap v409: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 51 MiB/s wr, 2.26k op/s 2023-11-06T01:37:39.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:39 smithi163 ceph-mon[104416]: pgmap v410: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 MiB/s wr, 1.97k op/s 2023-11-06T01:37:39.607 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:39 smithi053 ceph-mon[99472]: pgmap v410: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 MiB/s wr, 1.97k op/s 2023-11-06T01:37:39.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:39 smithi062 ceph-mon[103924]: pgmap v410: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 MiB/s wr, 1.97k op/s 2023-11-06T01:37:41.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:41 smithi163 ceph-mon[104416]: pgmap v411: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 45 MiB/s wr, 1.99k op/s 2023-11-06T01:37:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:37:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:41.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:41.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:41 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:41 smithi053 ceph-mon[99472]: pgmap v411: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 45 MiB/s wr, 1.99k op/s 2023-11-06T01:37:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:37:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:41.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:41.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:41.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:41 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:41.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:41 smithi062 ceph-mon[103924]: pgmap v411: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 45 MiB/s wr, 1.99k op/s 2023-11-06T01:37:41.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:37:41.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:41.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:41.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:41.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:41 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:42.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 205.0 s, 2633.3 tps, lat 38.438 ms stddev 44.211 2023-11-06T01:37:42.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:42 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:37:42.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:42 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:37:42.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:42 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:42.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:42 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:37:42.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:42 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:37:42.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:42 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:42.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:42 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:37:42.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:42 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:37:42.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:42 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:37:43.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:43 smithi053 ceph-mon[99472]: pgmap v412: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 1.97k op/s 2023-11-06T01:37:43.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:43 smithi062 ceph-mon[103924]: pgmap v412: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 1.97k op/s 2023-11-06T01:37:43.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:43 smithi163 ceph-mon[104416]: pgmap v412: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 1.97k op/s 2023-11-06T01:37:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.676+0000 7f4e77d4e700 1 -- 172.21.15.53:0/1318803673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e70100df0 msgr2=0x7f4e70101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.676+0000 7f4e77d4e700 1 --2- 172.21.15.53:0/1318803673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e70100df0 0x7f4e70101210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009990 tx=0x7f4e60009ca0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.676+0000 7f4e77d4e700 1 -- 172.21.15.53:0/1318803673 shutdown_connections 2023-11-06T01:37:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.676+0000 7f4e77d4e700 1 --2- 172.21.15.53:0/1318803673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e70102980 0x7f4e70109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.676+0000 7f4e77d4e700 1 --2- 172.21.15.53:0/1318803673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e70100df0 0x7f4e70101210 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.676+0000 7f4e77d4e700 1 --2- 172.21.15.53:0/1318803673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e70101fe0 0x7f4e70102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.676+0000 7f4e77d4e700 1 -- 172.21.15.53:0/1318803673 >> 172.21.15.53:0/1318803673 conn(0x7f4e700fc340 msgr2=0x7f4e700fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.677+0000 7f4e77d4e700 1 -- 172.21.15.53:0/1318803673 shutdown_connections 2023-11-06T01:37:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.677+0000 7f4e77d4e700 1 -- 172.21.15.53:0/1318803673 wait complete. 2023-11-06T01:37:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.678+0000 7f4e77d4e700 1 Processor -- start 2023-11-06T01:37:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.678+0000 7f4e77d4e700 1 -- start start 2023-11-06T01:37:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e77d4e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e70100df0 0x7f4e701a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e77d4e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e70102980 0x7f4e701a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e77d4e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701a6f90 0x7f4e701a7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e77d4e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e701a7a70 con 0x7f4e70100df0 2023-11-06T01:37:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e77d4e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e701a7be0 con 0x7f4e701a6f90 2023-11-06T01:37:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e77d4e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e701a7da0 con 0x7f4e70102980 2023-11-06T01:37:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e75aea700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e70100df0 0x7f4e701a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e752e9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e70102980 0x7f4e701a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e75aea700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e70100df0 0x7f4e701a5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41044/0 (socket says 172.21.15.53:41044) 2023-11-06T01:37:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e75aea700 1 -- 172.21.15.53:0/963598137 learned_addr learned my addr 172.21.15.53:0/963598137 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e762eb700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701a6f90 0x7f4e701a7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e75aea700 1 -- 172.21.15.53:0/963598137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e70102980 msgr2=0x7f4e701a6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e75aea700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e70102980 0x7f4e701a6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e75aea700 1 -- 172.21.15.53:0/963598137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701a6f90 msgr2=0x7f4e701a7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e75aea700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701a6f90 0x7f4e701a7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e75aea700 1 -- 172.21.15.53:0/963598137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e600095f0 con 0x7f4e70100df0 2023-11-06T01:37:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e762eb700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701a6f90 0x7f4e701a7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:37:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.679+0000 7f4e75aea700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e70100df0 0x7f4e701a5d10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4e6001a040 tx=0x7f4e6000bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.680+0000 7f4e66ffd700 1 -- 172.21.15.53:0/963598137 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60005470 con 0x7f4e70100df0 2023-11-06T01:37:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.680+0000 7f4e66ffd700 1 -- 172.21.15.53:0/963598137 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e60010e90 con 0x7f4e70100df0 2023-11-06T01:37:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.680+0000 7f4e66ffd700 1 -- 172.21.15.53:0/963598137 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60018740 con 0x7f4e70100df0 2023-11-06T01:37:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.680+0000 7f4e77d4e700 1 -- 172.21.15.53:0/963598137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e701ae2a0 con 0x7f4e70100df0 2023-11-06T01:37:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.680+0000 7f4e77d4e700 1 -- 172.21.15.53:0/963598137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e701ae720 con 0x7f4e70100df0 2023-11-06T01:37:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.683+0000 7f4e66ffd700 1 -- 172.21.15.53:0/963598137 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4e60003680 con 0x7f4e70100df0 2023-11-06T01:37:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.685+0000 7f4e83edeb80 1 Processor -- start 2023-11-06T01:37:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.686+0000 7f4e83edeb80 1 -- start start 2023-11-06T01:37:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.686+0000 7f4e66ffd700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4e5c0a0600 0x7f4e5c0a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.686+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9f680 0x55f274b9faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.686+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9ffe0 0x55f274ba08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.686+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 0x55f274ba1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.686+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f274af79b0 con 0x55f274b9f680 2023-11-06T01:37:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.686+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f274af5370 con 0x55f274b9ffe0 2023-11-06T01:37:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.686+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f274af0c60 con 0x55f274ba0ed0 2023-11-06T01:37:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.686+0000 7f4e66ffd700 1 -- 172.21.15.53:0/963598137 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4e600cd030 con 0x7f4e70100df0 2023-11-06T01:37:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.686+0000 7f4e762eb700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4e5c0a0600 0x7f4e5c0a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.687+0000 7f4e762eb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 0x55f274ba1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.687+0000 7f4e762eb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 0x55f274ba1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55290/0 (socket says 172.21.15.53:55290) 2023-11-06T01:37:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.687+0000 7f4e762eb700 1 -- 172.21.15.53:0/4213200541 learned_addr learned my addr 172.21.15.53:0/4213200541 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.687+0000 7f4e75aea700 1 --2- 172.21.15.53:0/4213200541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9f680 0x55f274b9faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.687+0000 7f4e762eb700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4e5c0a0600 0x7f4e5c0a2ac0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c980 tx=0x7f4e68009b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.687+0000 7f4e762eb700 1 -- 172.21.15.53:0/4213200541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9ffe0 msgr2=0x55f274ba08f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:37:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.687+0000 7f4e762eb700 1 --2- 172.21.15.53:0/4213200541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9ffe0 0x55f274ba08f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.687+0000 7f4e762eb700 1 -- 172.21.15.53:0/4213200541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9f680 msgr2=0x55f274b9faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.687+0000 7f4e762eb700 1 --2- 172.21.15.53:0/4213200541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9f680 0x55f274b9faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.687+0000 7f4e762eb700 1 -- 172.21.15.53:0/4213200541 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f27491ed80 con 0x55f274ba0ed0 2023-11-06T01:37:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.690+0000 7f4e762eb700 1 --2- 172.21.15.53:0/4213200541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 0x55f274ba1350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009fd0 tx=0x7f4e6801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c34d950827b38522 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.692+0000 7f4e5bfff700 1 -- 172.21.15.53:0/4213200541 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68025670 con 0x55f274ba0ed0 2023-11-06T01:37:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.692+0000 7f4e5bfff700 1 -- 172.21.15.53:0/4213200541 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e68025c90 con 0x55f274ba0ed0 2023-11-06T01:37:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.692+0000 7f4e5bfff700 1 -- 172.21.15.53:0/4213200541 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6801acd0 con 0x55f274ba0ed0 2023-11-06T01:37:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.693+0000 7f4e83edeb80 1 -- 172.21.15.53:0/4213200541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 msgr2=0x55f274ba1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.693+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/4213200541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 0x55f274ba1350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009fd0 tx=0x7f4e6801d000 comp rx=0 tx=0).stop 2023-11-06T01:37:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.693+0000 7f4e83edeb80 1 -- 172.21.15.53:0/4213200541 shutdown_connections 2023-11-06T01:37:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.693+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/4213200541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 0x55f274ba1350 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.693+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/4213200541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9f680 0x55f274b9faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.693+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/4213200541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9ffe0 0x55f274ba08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.693+0000 7f4e83edeb80 1 -- 172.21.15.53:0/4213200541 >> 172.21.15.53:0/4213200541 conn(0x55f274aff070 msgr2=0x55f274b9c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.695+0000 7f4e83edeb80 1 -- 172.21.15.53:0/4213200541 shutdown_connections 2023-11-06T01:37:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.695+0000 7f4e83edeb80 1 -- 172.21.15.53:0/4213200541 wait complete. 2023-11-06T01:37:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.695+0000 7f4e83edeb80 1 -- start start 2023-11-06T01:37:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.696+0000 7f4e83edeb80 1 Processor -- start 2023-11-06T01:37:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.696+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9f680 0x55f274ba94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.696+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9ffe0 0x55f274ba9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.696+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274ba0ed0 0x55f274bb0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.696+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f274a8d4b0 con 0x55f274b9ffe0 2023-11-06T01:37:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.696+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f274a8d150 con 0x55f274ba0ed0 2023-11-06T01:37:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.696+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2749748c0 con 0x55f274b9f680 2023-11-06T01:37:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.696+0000 7f4e83edeb80 10 client.0 _ll_register_callbacks cb 0x55f2749daa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.697+0000 7f4e762eb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274ba0ed0 0x55f274bb0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.697+0000 7f4e75aea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9f680 0x55f274ba94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.697+0000 7f4e762eb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274ba0ed0 0x55f274bb0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52492/0 (socket says 172.21.15.53:52492) 2023-11-06T01:37:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.697+0000 7f4e75aea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9f680 0x55f274ba94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55296/0 (socket says 172.21.15.53:55296) 2023-11-06T01:37:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.697+0000 7f4e762eb700 1 -- 172.21.15.53:0/2597919896 learned_addr learned my addr 172.21.15.53:0/2597919896 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.697+0000 7f4e752e9700 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9ffe0 0x55f274ba9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.697+0000 7f4e75aea700 1 -- 172.21.15.53:0/2597919896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274ba0ed0 msgr2=0x55f274bb0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.697+0000 7f4e75aea700 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274ba0ed0 0x55f274bb0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.697+0000 7f4e75aea700 1 -- 172.21.15.53:0/2597919896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9ffe0 msgr2=0x55f274ba9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.697+0000 7f4e75aea700 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9ffe0 0x55f274ba9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.697+0000 7f4e75aea700 1 -- 172.21.15.53:0/2597919896 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6801c480 con 0x55f274b9f680 2023-11-06T01:37:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.698+0000 7f4e75aea700 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9f680 0x55f274ba94c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4e6009a830 tx=0x7f4e60005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c0d36f05ef15ea2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.698+0000 7f4e597fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:37:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.699+0000 7f4e597fa700 1 -- 172.21.15.53:0/2597919896 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6009e6d0 con 0x55f274b9f680 2023-11-06T01:37:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.699+0000 7f4e83edeb80 1 -- 172.21.15.53:0/2597919896 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f274b1bcb0 con 0x55f274b9f680 2023-11-06T01:37:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.699+0000 7f4e597fa700 1 -- 172.21.15.53:0/2597919896 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4e600a2880 con 0x55f274b9f680 2023-11-06T01:37:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.699+0000 7f4e597fa700 1 -- 172.21.15.53:0/2597919896 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e60099d90 con 0x55f274b9f680 2023-11-06T01:37:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.699+0000 7f4e83edeb80 10 client.25582 fetch_fsmap learned FSMap version 39 2023-11-06T01:37:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.699+0000 7f4e83edeb80 1 -- 172.21.15.53:0/2597919896 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f274907c00 con 0x55f274b9f680 2023-11-06T01:37:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.699+0000 7f4e597fa700 1 -- 172.21.15.53:0/2597919896 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e6007d080 con 0x55f274b9f680 2023-11-06T01:37:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.700+0000 7f4e597fa700 1 -- 172.21.15.53:0/2597919896 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f4e60003930 con 0x55f274b9f680 2023-11-06T01:37:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.700+0000 7f4e83edeb80 10 client.25582 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T01:37:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.700+0000 7f4e83edeb80 10 client.25582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.700+0000 7f4e83edeb80 20 client.25582 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.701+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f274bb5010 0x55f274bb7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.701+0000 7f4e83edeb80 4 client.25582 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:37:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.701+0000 7f4e83edeb80 1 -- 172.21.15.53:0/2597919896 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f274b6edb0 con 0x55f274bb5010 2023-11-06T01:37:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.701+0000 7f4e752e9700 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f274bb5010 0x55f274bb7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.701+0000 7f4e597fa700 10 client.25582 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.701+0000 7f4e752e9700 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f274bb5010 0x55f274bb7430 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1012b4c537e96215 server_cookie=8c195e0afa772b6e in_seq=0 out_seq=0 2023-11-06T01:37:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e597fa700 1 -- 172.21.15.53:0/2597919896 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f274b6edb0 con 0x55f274bb5010 2023-11-06T01:37:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e597fa700 10 client.25582 handle_command_reply: tid=0 2023-11-06T01:37:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e83edeb80 1 client.25582 shutdown 2023-11-06T01:37:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e83edeb80 1 -- 172.21.15.53:0/2597919896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9f680 msgr2=0x55f274ba94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9f680 0x55f274ba94c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4e6009a830 tx=0x7f4e60005fd0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e83edeb80 1 -- 172.21.15.53:0/2597919896 shutdown_connections 2023-11-06T01:37:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274ba0ed0 0x55f274bb0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f274bb5010 0x55f274bb7430 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9f680 0x55f274ba94c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/2597919896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9ffe0 0x55f274ba9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e83edeb80 1 -- 172.21.15.53:0/2597919896 >> 172.21.15.53:0/2597919896 conn(0x55f274aff070 msgr2=0x55f274b9a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.702+0000 7f4e597fa700 0 client.25582 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.703+0000 7f4e83edeb80 1 -- 172.21.15.53:0/2597919896 shutdown_connections 2023-11-06T01:37:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.704+0000 7f4e83edeb80 1 -- 172.21.15.53:0/2597919896 wait complete. 2023-11-06T01:37:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.704+0000 7f4e83edeb80 20 client.25582 trim_cache size 0 max 16384 2023-11-06T01:37:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e83edeb80 1 Processor -- start 2023-11-06T01:37:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e83edeb80 1 -- start start 2023-11-06T01:37:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 0x55f274bae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9ffe0 0x55f274b9bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274ba0ed0 0x55f274ba40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f274b927a0 con 0x55f274ba0ed0 2023-11-06T01:37:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f274ba8970 con 0x55f274b9f680 2023-11-06T01:37:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f274b95500 con 0x55f274b9ffe0 2023-11-06T01:37:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e762eb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274ba0ed0 0x55f274ba40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e762eb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274ba0ed0 0x55f274ba40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41064/0 (socket says 172.21.15.53:41064) 2023-11-06T01:37:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e75aea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 0x55f274bae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e762eb700 1 -- 172.21.15.53:0/1165560858 learned_addr learned my addr 172.21.15.53:0/1165560858 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.725+0000 7f4e752e9700 1 --2- 172.21.15.53:0/1165560858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9ffe0 0x55f274b9bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.726+0000 7f4e75aea700 1 -- 172.21.15.53:0/1165560858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9ffe0 msgr2=0x55f274b9bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.726+0000 7f4e75aea700 1 --2- 172.21.15.53:0/1165560858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9ffe0 0x55f274b9bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.726+0000 7f4e75aea700 1 -- 172.21.15.53:0/1165560858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274ba0ed0 msgr2=0x55f274ba40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.726+0000 7f4e75aea700 1 --2- 172.21.15.53:0/1165560858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274ba0ed0 0x55f274ba40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.726+0000 7f4e75aea700 1 -- 172.21.15.53:0/1165560858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2749451f0 con 0x55f274b9f680 2023-11-06T01:37:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.726+0000 7f4e75aea700 1 --2- 172.21.15.53:0/1165560858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 0x55f274bae9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4e600a2350 tx=0x7f4e6009a9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4701f5fc07725e47 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.727+0000 7f4e64ff9700 1 -- 172.21.15.53:0/1165560858 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6009e6d0 con 0x55f274b9f680 2023-11-06T01:37:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.727+0000 7f4e64ff9700 1 -- 172.21.15.53:0/1165560858 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f274b1bcb0 con 0x55f274b9f680 2023-11-06T01:37:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.727+0000 7f4e64ff9700 1 -- 172.21.15.53:0/1165560858 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60099800 con 0x55f274b9f680 2023-11-06T01:37:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.728+0000 7f4e83edeb80 1 -- 172.21.15.53:0/1165560858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 msgr2=0x55f274bae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.728+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/1165560858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 0x55f274bae9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4e600a2350 tx=0x7f4e6009a9e0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.728+0000 7f4e83edeb80 1 -- 172.21.15.53:0/1165560858 shutdown_connections 2023-11-06T01:37:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.728+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/1165560858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 0x55f274bae9f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.728+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/1165560858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274ba0ed0 0x55f274ba40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.728+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/1165560858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274b9ffe0 0x55f274b9bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.728+0000 7f4e83edeb80 1 -- 172.21.15.53:0/1165560858 >> 172.21.15.53:0/1165560858 conn(0x55f274aff070 msgr2=0x55f274ba8140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.728+0000 7f4e83edeb80 1 -- 172.21.15.53:0/1165560858 shutdown_connections 2023-11-06T01:37:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.728+0000 7f4e83edeb80 1 -- 172.21.15.53:0/1165560858 wait complete. 2023-11-06T01:37:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e83edeb80 1 -- start start 2023-11-06T01:37:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e83edeb80 1 Processor -- start 2023-11-06T01:37:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 0x55f274c00af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9ffe0 0x55f274c03040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e83edeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 0x55f274c05640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f274c05d50 con 0x55f274b9ffe0 2023-11-06T01:37:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f274c05ec0 con 0x55f274b9f680 2023-11-06T01:37:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e83edeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f274c06080 con 0x55f274ba0ed0 2023-11-06T01:37:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e83edeb80 10 client.0 _ll_register_callbacks cb 0x55f2749daa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e75aea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 0x55f274c00af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e752e9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9ffe0 0x55f274c03040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e75aea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 0x55f274c00af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52506/0 (socket says 172.21.15.53:52506) 2023-11-06T01:37:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e75aea700 1 -- 172.21.15.53:0/291625560 learned_addr learned my addr 172.21.15.53:0/291625560 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.729+0000 7f4e762eb700 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 0x55f274c05640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.730+0000 7f4e75aea700 1 -- 172.21.15.53:0/291625560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 msgr2=0x55f274c05640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.730+0000 7f4e75aea700 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 0x55f274c05640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.730+0000 7f4e75aea700 1 -- 172.21.15.53:0/291625560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9ffe0 msgr2=0x55f274c03040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.730+0000 7f4e75aea700 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9ffe0 0x55f274c03040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.730+0000 7f4e75aea700 1 -- 172.21.15.53:0/291625560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f274907c00 con 0x55f274b9f680 2023-11-06T01:37:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.730+0000 7f4e752e9700 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9ffe0 0x55f274c03040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:37:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.733+0000 7f4e75aea700 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 0x55f274c00af0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e600107c0 tx=0x7f4e60000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c082b5af08c4678b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.733+0000 7f4e597fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:37:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.738+0000 7f4e597fa700 1 -- 172.21.15.53:0/291625560 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60096890 con 0x55f274b9f680 2023-11-06T01:37:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.738+0000 7f4e597fa700 1 -- 172.21.15.53:0/291625560 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4e600969f0 con 0x55f274b9f680 2023-11-06T01:37:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.738+0000 7f4e83edeb80 1 -- 172.21.15.53:0/291625560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e68025c90 con 0x55f274b9f680 2023-11-06T01:37:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.738+0000 7f4e597fa700 1 -- 172.21.15.53:0/291625560 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e6009e6d0 con 0x55f274b9f680 2023-11-06T01:37:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.738+0000 7f4e83edeb80 10 client.25514 fetch_fsmap learned FSMap version 39 2023-11-06T01:37:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.738+0000 7f4e83edeb80 1 -- 172.21.15.53:0/291625560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f27491d250 con 0x55f274b9f680 2023-11-06T01:37:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.739+0000 7f4e597fa700 1 -- 172.21.15.53:0/291625560 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e68025c90 con 0x55f274b9f680 2023-11-06T01:37:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.739+0000 7f4e597fa700 1 -- 172.21.15.53:0/291625560 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f4e600852c0 con 0x55f274b9f680 2023-11-06T01:37:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.739+0000 7f4e83edeb80 10 client.25514 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T01:37:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.739+0000 7f4e83edeb80 10 client.25514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.739+0000 7f4e83edeb80 20 client.25514 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.739+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f274c07380 0x55f274c09770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.739+0000 7f4e83edeb80 4 client.25514 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:37:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.739+0000 7f4e83edeb80 1 -- 172.21.15.53:0/291625560 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f274ae9ce0 con 0x55f274c07380 2023-11-06T01:37:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.740+0000 7f4e752e9700 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f274c07380 0x55f274c09770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.741+0000 7f4e752e9700 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f274c07380 0x55f274c09770 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31398d7eae0b999 server_cookie=18d6fe6032514487 in_seq=0 out_seq=0 2023-11-06T01:37:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.741+0000 7f4e597fa700 10 client.25514 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.755+0000 7f4e597fa700 1 -- 172.21.15.53:0/291625560 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f274ae9ce0 con 0x55f274c07380 2023-11-06T01:37:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.755+0000 7f4e597fa700 10 client.25514 handle_command_reply: tid=0 2023-11-06T01:37:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.755+0000 7f4e83edeb80 1 client.25514 shutdown 2023-11-06T01:37:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e83edeb80 1 -- 172.21.15.53:0/291625560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 msgr2=0x55f274c00af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 0x55f274c00af0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e600107c0 tx=0x7f4e60000bc0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e83edeb80 1 -- 172.21.15.53:0/291625560 shutdown_connections 2023-11-06T01:37:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f274b9f680 0x55f274c00af0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f274c07380 0x55f274c09770 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f274ba0ed0 0x55f274c05640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e597fa700 0 client.25514 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e83edeb80 1 --2- 172.21.15.53:0/291625560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f274b9ffe0 0x55f274c03040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e83edeb80 1 -- 172.21.15.53:0/291625560 >> 172.21.15.53:0/291625560 conn(0x55f274aff070 msgr2=0x55f274b9bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e83edeb80 1 -- 172.21.15.53:0/291625560 shutdown_connections 2023-11-06T01:37:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e83edeb80 1 -- 172.21.15.53:0/291625560 wait complete. 2023-11-06T01:37:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e83edeb80 20 client.25514 trim_cache size 0 max 16384 2023-11-06T01:37:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:37:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:37:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:37:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:37:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e77d4e700 1 -- 172.21.15.53:0/963598137 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4e5c0a0600 msgr2=0x7f4e5c0a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e77d4e700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4e5c0a0600 0x7f4e5c0a2ac0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c980 tx=0x7f4e68009b00 comp rx=0 tx=0).stop 2023-11-06T01:37:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e77d4e700 1 -- 172.21.15.53:0/963598137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e70100df0 msgr2=0x7f4e701a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e77d4e700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e70100df0 0x7f4e701a5d10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4e6001a040 tx=0x7f4e6000bbc0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e77d4e700 1 -- 172.21.15.53:0/963598137 shutdown_connections 2023-11-06T01:37:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e77d4e700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4e5c0a0600 0x7f4e5c0a2ac0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e77d4e700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701a6f90 0x7f4e701a7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e77d4e700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e70100df0 0x7f4e701a5d10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e77d4e700 1 --2- 172.21.15.53:0/963598137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e70102980 0x7f4e701a6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.756+0000 7f4e77d4e700 1 -- 172.21.15.53:0/963598137 >> 172.21.15.53:0/963598137 conn(0x7f4e700fc340 msgr2=0x7f4e701055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.757+0000 7f4e77d4e700 1 -- 172.21.15.53:0/963598137 shutdown_connections 2023-11-06T01:37:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:44.757+0000 7f4e77d4e700 1 -- 172.21.15.53:0/963598137 wait complete. 2023-11-06T01:37:45.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:45 smithi053 ceph-mon[99472]: pgmap v413: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 908 op/s 2023-11-06T01:37:45.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:45 smithi062 ceph-mon[103924]: pgmap v413: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 908 op/s 2023-11-06T01:37:45.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:45 smithi163 ceph-mon[104416]: pgmap v413: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 908 op/s 2023-11-06T01:37:47.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 210.0 s, 2620.2 tps, lat 37.615 ms stddev 38.378 2023-11-06T01:37:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:47 smithi053 ceph-mon[99472]: pgmap v414: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 846 op/s 2023-11-06T01:37:47.654 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:47 smithi062 ceph-mon[103924]: pgmap v414: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 846 op/s 2023-11-06T01:37:47.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:47 smithi163 ceph-mon[104416]: pgmap v414: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 846 op/s 2023-11-06T01:37:49.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:49 smithi053 ceph-mon[99472]: pgmap v415: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 16 MiB/s wr, 659 op/s 2023-11-06T01:37:49.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:49 smithi062 ceph-mon[103924]: pgmap v415: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 16 MiB/s wr, 659 op/s 2023-11-06T01:37:49.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:49 smithi163 ceph-mon[104416]: pgmap v415: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 16 MiB/s wr, 659 op/s 2023-11-06T01:37:51.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:51 smithi053 ceph-mon[99472]: pgmap v416: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 698 op/s 2023-11-06T01:37:51.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:51 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:51 smithi062 ceph-mon[103924]: pgmap v416: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 698 op/s 2023-11-06T01:37:51.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:51 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:51.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:51 smithi163 ceph-mon[104416]: pgmap v416: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 698 op/s 2023-11-06T01:37:51.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:51 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:37:52.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 215.0 s, 1757.4 tps, lat 57.280 ms stddev 64.482 2023-11-06T01:37:53.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:53 smithi053 ceph-mon[99472]: pgmap v417: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 1.00k op/s 2023-11-06T01:37:53.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:53 smithi062 ceph-mon[103924]: pgmap v417: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 1.00k op/s 2023-11-06T01:37:53.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:53 smithi163 ceph-mon[104416]: pgmap v417: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 1.00k op/s 2023-11-06T01:37:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:55 smithi062 ceph-mon[103924]: pgmap v418: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 910 op/s 2023-11-06T01:37:55.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:55 smithi163 ceph-mon[104416]: pgmap v418: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 910 op/s 2023-11-06T01:37:55.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:55 smithi053 ceph-mon[99472]: pgmap v418: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 910 op/s 2023-11-06T01:37:57.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 220.0 s, 2228.4 tps, lat 44.868 ms stddev 52.203 2023-11-06T01:37:57.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:57 smithi062 ceph-mon[103924]: pgmap v419: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 1.15k op/s 2023-11-06T01:37:57.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:57 smithi163 ceph-mon[104416]: pgmap v419: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 1.15k op/s 2023-11-06T01:37:57.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:57 smithi053 ceph-mon[99472]: pgmap v419: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 1.15k op/s 2023-11-06T01:37:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.919+0000 7f365a26d700 1 -- 172.21.15.53:0/2122818766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365410a9c0 msgr2=0x7f365410cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.919+0000 7f365a26d700 1 --2- 172.21.15.53:0/2122818766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365410a9c0 0x7f365410cdd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f364400b970 tx=0x7f3644009260 comp rx=0 tx=0).stop 2023-11-06T01:37:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.920+0000 7f365a26d700 1 -- 172.21.15.53:0/2122818766 shutdown_connections 2023-11-06T01:37:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.920+0000 7f365a26d700 1 --2- 172.21.15.53:0/2122818766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f365410d460 0x7f365410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.920+0000 7f365a26d700 1 --2- 172.21.15.53:0/2122818766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3654108000 0x7f365410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.920+0000 7f365a26d700 1 --2- 172.21.15.53:0/2122818766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365410a9c0 0x7f365410cdd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.920+0000 7f365a26d700 1 -- 172.21.15.53:0/2122818766 >> 172.21.15.53:0/2122818766 conn(0x7f365406dbc0 msgr2=0x7f3654070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.920+0000 7f365a26d700 1 -- 172.21.15.53:0/2122818766 shutdown_connections 2023-11-06T01:37:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.920+0000 7f365a26d700 1 -- 172.21.15.53:0/2122818766 wait complete. 2023-11-06T01:37:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.921+0000 7f365a26d700 1 Processor -- start 2023-11-06T01:37:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.921+0000 7f365a26d700 1 -- start start 2023-11-06T01:37:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.921+0000 7f365a26d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3654108000 0x7f365419d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.921+0000 7f365a26d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f365410a9c0 0x7f365419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.921+0000 7f365a26d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365410d460 0x7f365419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.921+0000 7f365a26d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365419eb50 con 0x7f365410a9c0 2023-11-06T01:37:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.921+0000 7f365a26d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365419ecc0 con 0x7f365410d460 2023-11-06T01:37:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.921+0000 7f365a26d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365419ee80 con 0x7f3654108000 2023-11-06T01:37:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.921+0000 7f36537fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f365410a9c0 0x7f365419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.921+0000 7f36537fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f365410a9c0 0x7f365419d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41080/0 (socket says 172.21.15.53:41080) 2023-11-06T01:37:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.921+0000 7f36537fe700 1 -- 172.21.15.53:0/239360494 learned_addr learned my addr 172.21.15.53:0/239360494 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.922+0000 7f365880a700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365410d460 0x7f365419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.922+0000 7f36537fe700 1 -- 172.21.15.53:0/239360494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3654108000 msgr2=0x7f365419d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.922+0000 7f36537fe700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3654108000 0x7f365419d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.922+0000 7f36537fe700 1 -- 172.21.15.53:0/239360494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365410d460 msgr2=0x7f365419e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.922+0000 7f36537fe700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365410d460 0x7f365419e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.922+0000 7f36537fe700 1 -- 172.21.15.53:0/239360494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364400b5a0 con 0x7f365410a9c0 2023-11-06T01:37:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.922+0000 7f36537fe700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f365410a9c0 0x7f365419d670 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3644009810 tx=0x7f3644009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:57.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.922+0000 7f36517fa700 1 -- 172.21.15.53:0/239360494 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644009ca0 con 0x7f365410a9c0 2023-11-06T01:37:57.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.922+0000 7f36517fa700 1 -- 172.21.15.53:0/239360494 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3644003680 con 0x7f365410a9c0 2023-11-06T01:37:57.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.922+0000 7f36517fa700 1 -- 172.21.15.53:0/239360494 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364401e680 con 0x7f365410a9c0 2023-11-06T01:37:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.923+0000 7f365a26d700 1 -- 172.21.15.53:0/239360494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365419f100 con 0x7f365410a9c0 2023-11-06T01:37:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.923+0000 7f365a26d700 1 -- 172.21.15.53:0/239360494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365419f350 con 0x7f365410a9c0 2023-11-06T01:37:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.932+0000 7f36663fdb80 1 Processor -- start 2023-11-06T01:37:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.935+0000 7f36663fdb80 1 -- start start 2023-11-06T01:37:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.936+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c63e90 0x562009c642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.936+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c65100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.936+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c656e0 0x562009c65b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.936+0000 7f36663fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009bb39b0 con 0x562009c647f0 2023-11-06T01:37:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.936+0000 7f36663fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009bb1370 con 0x562009c656e0 2023-11-06T01:37:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.936+0000 7f36663fdb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009bacc60 con 0x562009c63e90 2023-11-06T01:37:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.938+0000 7f36537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c656e0 0x562009c65b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.938+0000 7f36537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c656e0 0x562009c65b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52526/0 (socket says 172.21.15.53:52526) 2023-11-06T01:37:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.938+0000 7f36537fe700 1 -- 172.21.15.53:0/2709871380 learned_addr learned my addr 172.21.15.53:0/2709871380 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.939+0000 7f36537fe700 1 -- 172.21.15.53:0/2709871380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c63e90 msgr2=0x562009c642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.939+0000 7f36537fe700 1 --2- 172.21.15.53:0/2709871380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c63e90 0x562009c642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.939+0000 7f36537fe700 1 -- 172.21.15.53:0/2709871380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 msgr2=0x562009c65100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:37:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.939+0000 7f36537fe700 1 --2- 172.21.15.53:0/2709871380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c65100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.939+0000 7f36537fe700 1 -- 172.21.15.53:0/2709871380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620099dad80 con 0x562009c656e0 2023-11-06T01:37:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.939+0000 7f36537fe700 1 --2- 172.21.15.53:0/2709871380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c656e0 0x562009c65b60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f36440d35c0 tx=0x7f36440d35f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d56536b87a46ee8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.940+0000 7f36517fa700 1 -- 172.21.15.53:0/239360494 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f36440052c0 con 0x7f365410a9c0 2023-11-06T01:37:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.940+0000 7f36517fa700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f36400a0670 0x7f36400a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.941+0000 7f36517fa700 1 -- 172.21.15.53:0/239360494 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f36440cd650 con 0x7f365410a9c0 2023-11-06T01:37:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.945+0000 7f365880a700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f36400a0670 0x7f36400a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.945+0000 7f365880a700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f36400a0670 0x7f36400a2b30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f364c00dfd0 tx=0x7f364c00dd30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.945+0000 7f364a7fc700 1 -- 172.21.15.53:0/2709871380 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36440cda40 con 0x562009c656e0 2023-11-06T01:37:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.946+0000 7f364a7fc700 1 -- 172.21.15.53:0/2709871380 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36440cdba0 con 0x562009c656e0 2023-11-06T01:37:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.946+0000 7f364a7fc700 1 -- 172.21.15.53:0/2709871380 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36440d9560 con 0x562009c656e0 2023-11-06T01:37:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.947+0000 7f36663fdb80 1 -- 172.21.15.53:0/2709871380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c656e0 msgr2=0x562009c65b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.947+0000 7f36663fdb80 1 --2- 172.21.15.53:0/2709871380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c656e0 0x562009c65b60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f36440d35c0 tx=0x7f36440d35f0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.951+0000 7f36663fdb80 1 -- 172.21.15.53:0/2709871380 shutdown_connections 2023-11-06T01:37:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.951+0000 7f36663fdb80 1 --2- 172.21.15.53:0/2709871380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c656e0 0x562009c65b60 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f36440d35c0 tx=0x7f36440d35f0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.951+0000 7f36663fdb80 1 --2- 172.21.15.53:0/2709871380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c63e90 0x562009c642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.951+0000 7f36663fdb80 1 --2- 172.21.15.53:0/2709871380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c65100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.951+0000 7f36663fdb80 1 -- 172.21.15.53:0/2709871380 >> 172.21.15.53:0/2709871380 conn(0x562009bbb070 msgr2=0x562009c58040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.956+0000 7f36663fdb80 1 -- 172.21.15.53:0/2709871380 shutdown_connections 2023-11-06T01:37:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.962+0000 7f36663fdb80 1 -- 172.21.15.53:0/2709871380 wait complete. 2023-11-06T01:37:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.962+0000 7f36663fdb80 1 -- start start 2023-11-06T01:37:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f36663fdb80 1 Processor -- start 2023-11-06T01:37:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c63e90 0x562009c5cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c5d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c62050 0x562009c624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f36663fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009b494b0 con 0x562009c647f0 2023-11-06T01:37:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f36663fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009b49150 con 0x562009c62050 2023-11-06T01:37:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f36663fdb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009a308c0 con 0x562009c63e90 2023-11-06T01:37:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f36663fdb80 10 client.0 _ll_register_callbacks cb 0x562009a96a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f365880a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c62050 0x562009c624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f3653fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c63e90 0x562009c5cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f3653fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c63e90 0x562009c5cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55332/0 (socket says 172.21.15.53:55332) 2023-11-06T01:37:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f365880a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c62050 0x562009c624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52532/0 (socket says 172.21.15.53:52532) 2023-11-06T01:37:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.963+0000 7f3653fff700 1 -- 172.21.15.53:0/1890517435 learned_addr learned my addr 172.21.15.53:0/1890517435 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f36537fe700 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c5d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f3653fff700 1 -- 172.21.15.53:0/1890517435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c62050 msgr2=0x562009c624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f3653fff700 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c62050 0x562009c624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f3653fff700 1 -- 172.21.15.53:0/1890517435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 msgr2=0x562009c5d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f3653fff700 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c5d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f3653fff700 1 -- 172.21.15.53:0/1890517435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365419f350 con 0x562009c63e90 2023-11-06T01:37:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f36537fe700 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c5d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:37:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f3653fff700 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c63e90 0x562009c5cae0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f363c00a8b0 tx=0x7f363c003980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f544e6f5281def8a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f36337fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:37:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f36337fe700 1 -- 172.21.15.53:0/1890517435 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c00f040 con 0x562009c63e90 2023-11-06T01:37:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f36337fe700 1 -- 172.21.15.53:0/1890517435 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f363c004830 con 0x562009c63e90 2023-11-06T01:37:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.964+0000 7f36663fdb80 1 -- 172.21.15.53:0/1890517435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562009bd7cb0 con 0x562009c63e90 2023-11-06T01:37:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.965+0000 7f36337fe700 1 -- 172.21.15.53:0/1890517435 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f363c0049f0 con 0x562009c63e90 2023-11-06T01:37:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.965+0000 7f36663fdb80 10 client.25594 fetch_fsmap learned FSMap version 40 2023-11-06T01:37:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.965+0000 7f36663fdb80 1 -- 172.21.15.53:0/1890517435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5620099c3c00 con 0x562009c63e90 2023-11-06T01:37:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.965+0000 7f36337fe700 1 -- 172.21.15.53:0/1890517435 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562009bd7cb0 con 0x562009c63e90 2023-11-06T01:37:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.965+0000 7f36337fe700 1 -- 172.21.15.53:0/1890517435 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f363c012070 con 0x562009c63e90 2023-11-06T01:37:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.965+0000 7f36663fdb80 10 client.25594 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:37:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.965+0000 7f36663fdb80 10 client.25594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.965+0000 7f36663fdb80 20 client.25594 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.965+0000 7f36663fdb80 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562009c825e0 0x562009c84a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.965+0000 7f36663fdb80 4 client.25594 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:37:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.965+0000 7f36663fdb80 1 -- 172.21.15.53:0/1890517435 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562009c2adb0 con 0x562009c825e0 2023-11-06T01:37:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.968+0000 7f36537fe700 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562009c825e0 0x562009c84a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.968+0000 7f36537fe700 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562009c825e0 0x562009c84a00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba8128858114d396 server_cookie=4b42afe9e14ceb70 in_seq=0 out_seq=0 2023-11-06T01:37:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.968+0000 7f36337fe700 10 client.25594 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.969+0000 7f36337fe700 1 -- 172.21.15.53:0/1890517435 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562009c2adb0 con 0x562009c825e0 2023-11-06T01:37:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.969+0000 7f36337fe700 10 client.25594 handle_command_reply: tid=0 2023-11-06T01:37:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.969+0000 7f36663fdb80 1 client.25594 shutdown 2023-11-06T01:37:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36663fdb80 1 -- 172.21.15.53:0/1890517435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c63e90 msgr2=0x562009c5cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36663fdb80 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c63e90 0x562009c5cae0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f363c00a8b0 tx=0x7f363c003980 comp rx=0 tx=0).stop 2023-11-06T01:37:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36663fdb80 1 -- 172.21.15.53:0/1890517435 shutdown_connections 2023-11-06T01:37:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36663fdb80 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c62050 0x562009c624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36663fdb80 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562009c825e0 0x562009c84a00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36663fdb80 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c63e90 0x562009c5cae0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36663fdb80 1 --2- 172.21.15.53:0/1890517435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c5d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36663fdb80 1 -- 172.21.15.53:0/1890517435 >> 172.21.15.53:0/1890517435 conn(0x562009bbb070 msgr2=0x562009c563c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36337fe700 0 client.25594 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36663fdb80 1 -- 172.21.15.53:0/1890517435 shutdown_connections 2023-11-06T01:37:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36663fdb80 1 -- 172.21.15.53:0/1890517435 wait complete. 2023-11-06T01:37:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.970+0000 7f36663fdb80 20 client.25594 trim_cache size 0 max 16384 2023-11-06T01:37:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.994+0000 7f36663fdb80 1 Processor -- start 2023-11-06T01:37:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f36663fdb80 1 -- start start 2023-11-06T01:37:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c62050 0x562009c91ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c65060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f36663fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009c2adb0 con 0x562009c647f0 2023-11-06T01:37:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f36663fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009bd7cb0 con 0x562009c63e90 2023-11-06T01:37:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f36663fdb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009c63b30 con 0x562009c62050 2023-11-06T01:37:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f3653fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c62050 0x562009c91ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f36537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c65060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f36537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c65060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52542/0 (socket says 172.21.15.53:52542) 2023-11-06T01:37:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f36537fe700 1 -- 172.21.15.53:0/3577168467 learned_addr learned my addr 172.21.15.53:0/3577168467 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.995+0000 7f365880a700 1 --2- 172.21.15.53:0/3577168467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.996+0000 7f3653fff700 1 -- 172.21.15.53:0/3577168467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 msgr2=0x562009c65060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.996+0000 7f3653fff700 1 --2- 172.21.15.53:0/3577168467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c65060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.996+0000 7f3653fff700 1 -- 172.21.15.53:0/3577168467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 msgr2=0x562009c688b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.996+0000 7f3653fff700 1 --2- 172.21.15.53:0/3577168467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c688b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.996+0000 7f3653fff700 1 -- 172.21.15.53:0/3577168467 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562009a011f0 con 0x562009c62050 2023-11-06T01:37:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.996+0000 7f36537fe700 1 --2- 172.21.15.53:0/3577168467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c65060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:37:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.997+0000 7f3653fff700 1 --2- 172.21.15.53:0/3577168467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c62050 0x562009c91ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f363c013780 tx=0x7f363c00aaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5018f9abeefc887a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.997+0000 7f364affd700 1 -- 172.21.15.53:0/3577168467 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c0138a0 con 0x562009c62050 2023-11-06T01:37:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.997+0000 7f364affd700 1 -- 172.21.15.53:0/3577168467 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f363c013a00 con 0x562009c62050 2023-11-06T01:37:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.997+0000 7f364affd700 1 -- 172.21.15.53:0/3577168467 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c005690 con 0x562009c62050 2023-11-06T01:37:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.998+0000 7f36663fdb80 1 -- 172.21.15.53:0/3577168467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c62050 msgr2=0x562009c91ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.998+0000 7f36663fdb80 1 --2- 172.21.15.53:0/3577168467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c62050 0x562009c91ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f363c013780 tx=0x7f363c00aaf0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.998+0000 7f36663fdb80 1 -- 172.21.15.53:0/3577168467 shutdown_connections 2023-11-06T01:37:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.998+0000 7f36663fdb80 1 --2- 172.21.15.53:0/3577168467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009c688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.998+0000 7f36663fdb80 1 --2- 172.21.15.53:0/3577168467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c62050 0x562009c91ee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.998+0000 7f36663fdb80 1 --2- 172.21.15.53:0/3577168467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c65060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.998+0000 7f36663fdb80 1 -- 172.21.15.53:0/3577168467 >> 172.21.15.53:0/3577168467 conn(0x562009bbb070 msgr2=0x562009c62c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.998+0000 7f36663fdb80 1 -- 172.21.15.53:0/3577168467 shutdown_connections 2023-11-06T01:37:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.999+0000 7f36663fdb80 1 -- 172.21.15.53:0/3577168467 wait complete. 2023-11-06T01:37:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.999+0000 7f36663fdb80 1 -- start start 2023-11-06T01:37:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:57.999+0000 7f36663fdb80 1 Processor -- start 2023-11-06T01:37:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c62050 0x562009c915e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c91b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36663fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009cd2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36663fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009cd3290 con 0x562009c647f0 2023-11-06T01:37:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36663fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009cd33d0 con 0x562009c63e90 2023-11-06T01:37:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36663fdb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009cd3560 con 0x562009c62050 2023-11-06T01:37:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36663fdb80 10 client.0 _ll_register_callbacks cb 0x7f363c004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:37:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c91b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c91b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52544/0 (socket says 172.21.15.53:52544) 2023-11-06T01:37:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36537fe700 1 -- 172.21.15.53:0/3740464565 learned_addr learned my addr 172.21.15.53:0/3740464565 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:37:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f365880a700 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009cd2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36537fe700 1 -- 172.21.15.53:0/3740464565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c62050 msgr2=0x562009c915e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:37:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36537fe700 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c62050 0x562009c915e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36537fe700 1 -- 172.21.15.53:0/3740464565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 msgr2=0x562009cd2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:37:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36537fe700 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009cd2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.000+0000 7f36537fe700 1 -- 172.21.15.53:0/3740464565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620099dad80 con 0x562009c63e90 2023-11-06T01:37:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.001+0000 7f36537fe700 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c91b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3644097930 tx=0x7f3644005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19913accb544c6f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:37:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.001+0000 7f36337fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:37:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.001+0000 7f36337fe700 1 -- 172.21.15.53:0/3740464565 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644097200 con 0x562009c63e90 2023-11-06T01:37:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.001+0000 7f36337fe700 1 -- 172.21.15.53:0/3740464565 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f364409b5c0 con 0x562009c63e90 2023-11-06T01:37:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.001+0000 7f36663fdb80 1 -- 172.21.15.53:0/3740464565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36440cdba0 con 0x562009c63e90 2023-11-06T01:37:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.001+0000 7f36337fe700 1 -- 172.21.15.53:0/3740464565 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f364409ac90 con 0x562009c63e90 2023-11-06T01:37:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.001+0000 7f36663fdb80 10 client.25532 fetch_fsmap learned FSMap version 40 2023-11-06T01:37:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.001+0000 7f36663fdb80 1 -- 172.21.15.53:0/3740464565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5620099c3c00 con 0x562009c63e90 2023-11-06T01:37:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.002+0000 7f36337fe700 1 -- 172.21.15.53:0/3740464565 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36440cdba0 con 0x562009c63e90 2023-11-06T01:37:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.003+0000 7f36337fe700 1 -- 172.21.15.53:0/3740464565 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f36440975d0 con 0x562009c63e90 2023-11-06T01:37:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.003+0000 7f36663fdb80 10 client.25532 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:37:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.003+0000 7f36663fdb80 10 client.25532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:37:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.003+0000 7f36663fdb80 20 client.25532 populate_metadata read hostname 'smithi053' 2023-11-06T01:37:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.003+0000 7f36663fdb80 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562009cd4960 0x562009cd6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:37:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.003+0000 7f36663fdb80 4 client.25532 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:37:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.003+0000 7f36663fdb80 1 -- 172.21.15.53:0/3740464565 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562009ba5ce0 con 0x562009cd4960 2023-11-06T01:37:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.004+0000 7f3653fff700 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562009cd4960 0x562009cd6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:37:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.004+0000 7f36337fe700 10 client.25532 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.004+0000 7f3653fff700 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562009cd4960 0x562009cd6d50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d437983dce96cb server_cookie=544bb1f399f267f3 in_seq=0 out_seq=0 2023-11-06T01:37:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.020+0000 7f36337fe700 1 -- 172.21.15.53:0/3740464565 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562009ba5ce0 con 0x562009cd4960 2023-11-06T01:37:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.020+0000 7f36337fe700 10 client.25532 handle_command_reply: tid=0 2023-11-06T01:37:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.020+0000 7f36663fdb80 1 client.25532 shutdown 2023-11-06T01:37:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.021+0000 7f36663fdb80 1 -- 172.21.15.53:0/3740464565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 msgr2=0x562009c91b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.021+0000 7f36663fdb80 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c91b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3644097930 tx=0x7f3644005b40 comp rx=0 tx=0).stop 2023-11-06T01:37:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.022+0000 7f36663fdb80 1 -- 172.21.15.53:0/3740464565 shutdown_connections 2023-11-06T01:37:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.022+0000 7f36663fdb80 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x562009cd4960 0x562009cd6d50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.022+0000 7f36663fdb80 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562009c647f0 0x562009cd2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.022+0000 7f36663fdb80 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562009c62050 0x562009c915e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.022+0000 7f36663fdb80 1 --2- 172.21.15.53:0/3740464565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562009c63e90 0x562009c91b20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.022+0000 7f36337fe700 0 client.25532 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:37:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.022+0000 7f36663fdb80 1 -- 172.21.15.53:0/3740464565 >> 172.21.15.53:0/3740464565 conn(0x562009bbb070 msgr2=0x562009c62c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.022+0000 7f36663fdb80 1 -- 172.21.15.53:0/3740464565 shutdown_connections 2023-11-06T01:37:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.022+0000 7f36663fdb80 1 -- 172.21.15.53:0/3740464565 wait complete. 2023-11-06T01:37:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.022+0000 7f36663fdb80 20 client.25532 trim_cache size 0 max 16384 2023-11-06T01:37:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:37:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:37:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:37:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:37:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.024+0000 7f365a26d700 1 -- 172.21.15.53:0/239360494 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f36400a0670 msgr2=0x7f36400a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.024+0000 7f365a26d700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f36400a0670 0x7f36400a2b30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f364c00dfd0 tx=0x7f364c00dd30 comp rx=0 tx=0).stop 2023-11-06T01:37:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.024+0000 7f365a26d700 1 -- 172.21.15.53:0/239360494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f365410a9c0 msgr2=0x7f365419d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:37:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.024+0000 7f365a26d700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f365410a9c0 0x7f365419d670 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3644009810 tx=0x7f3644009840 comp rx=0 tx=0).stop 2023-11-06T01:37:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.025+0000 7f365a26d700 1 -- 172.21.15.53:0/239360494 shutdown_connections 2023-11-06T01:37:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.025+0000 7f365a26d700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f36400a0670 0x7f36400a2b30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.025+0000 7f365a26d700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365410d460 0x7f365419e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.025+0000 7f365a26d700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3654108000 0x7f365419d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.025+0000 7f365a26d700 1 --2- 172.21.15.53:0/239360494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f365410a9c0 0x7f365419d670 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:37:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.025+0000 7f365a26d700 1 -- 172.21.15.53:0/239360494 >> 172.21.15.53:0/239360494 conn(0x7f365406dbc0 msgr2=0x7f36541098c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:37:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.025+0000 7f365a26d700 1 -- 172.21.15.53:0/239360494 shutdown_connections 2023-11-06T01:37:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:37:58.025+0000 7f365a26d700 1 -- 172.21.15.53:0/239360494 wait complete. 2023-11-06T01:37:59.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:37:59 smithi062 ceph-mon[103924]: pgmap v420: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-11-06T01:37:59.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:37:59 smithi163 ceph-mon[104416]: pgmap v420: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-11-06T01:37:59.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:37:59 smithi053 ceph-mon[99472]: pgmap v420: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-11-06T01:38:01.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:01 smithi062 ceph-mon[103924]: pgmap v421: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.19k op/s 2023-11-06T01:38:01.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:01 smithi163 ceph-mon[104416]: pgmap v421: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.19k op/s 2023-11-06T01:38:01.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:01 smithi053 ceph-mon[99472]: pgmap v421: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.19k op/s 2023-11-06T01:38:02.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 225.0 s, 2241.4 tps, lat 44.854 ms stddev 85.551 2023-11-06T01:38:03.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:03 smithi062 ceph-mon[103924]: pgmap v422: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 50 MiB/s wr, 2.21k op/s 2023-11-06T01:38:03.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:03 smithi163 ceph-mon[104416]: pgmap v422: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 50 MiB/s wr, 2.21k op/s 2023-11-06T01:38:03.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:03 smithi053 ceph-mon[99472]: pgmap v422: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 50 MiB/s wr, 2.21k op/s 2023-11-06T01:38:05.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:05 smithi062 ceph-mon[103924]: pgmap v423: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 MiB/s wr, 2.07k op/s 2023-11-06T01:38:05.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:05 smithi163 ceph-mon[104416]: pgmap v423: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 MiB/s wr, 2.07k op/s 2023-11-06T01:38:05.754 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:38:05.792 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:05 smithi053 ceph-mon[99472]: pgmap v423: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 MiB/s wr, 2.07k op/s 2023-11-06T01:38:05.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.939+0000 7fa48b4f9700 1 -- 172.21.15.53:0/2218298737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa484101f50 msgr2=0x7fa4841023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:05.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.939+0000 7fa48b4f9700 1 --2- 172.21.15.53:0/2218298737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa484101f50 0x7fa4841023d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa474009990 tx=0x7fa474009ca0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.939+0000 7fa48b4f9700 1 -- 172.21.15.53:0/2218298737 shutdown_connections 2023-11-06T01:38:05.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.939+0000 7fa48b4f9700 1 --2- 172.21.15.53:0/2218298737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa484102910 0x7fa48410b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.939+0000 7fa48b4f9700 1 --2- 172.21.15.53:0/2218298737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4840ff0a0 0x7fa4840ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.939+0000 7fa48b4f9700 1 --2- 172.21.15.53:0/2218298737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa484101f50 0x7fa4841023d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.939+0000 7fa48b4f9700 1 -- 172.21.15.53:0/2218298737 >> 172.21.15.53:0/2218298737 conn(0x7fa4840fa9b0 msgr2=0x7fa4840fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:05.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.939+0000 7fa48b4f9700 1 -- 172.21.15.53:0/2218298737 shutdown_connections 2023-11-06T01:38:05.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.939+0000 7fa48b4f9700 1 -- 172.21.15.53:0/2218298737 wait complete. 2023-11-06T01:38:05.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.940+0000 7fa48b4f9700 1 Processor -- start 2023-11-06T01:38:05.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.941+0000 7fa48b4f9700 1 -- start start 2023-11-06T01:38:05.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.941+0000 7fa48b4f9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4840ff0a0 0x7fa48406f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.941+0000 7fa48b4f9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa484101f50 0x7fa48406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.941+0000 7fa48b4f9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa484102910 0x7fa484197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.941+0000 7fa48b4f9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa484072bf0 con 0x7fa484101f50 2023-11-06T01:38:05.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.941+0000 7fa48b4f9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa484072d60 con 0x7fa484102910 2023-11-06T01:38:05.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.941+0000 7fa48b4f9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4841984c0 con 0x7fa4840ff0a0 2023-11-06T01:38:05.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.941+0000 7fa489295700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4840ff0a0 0x7fa48406f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.941+0000 7fa489295700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4840ff0a0 0x7fa48406f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55356/0 (socket says 172.21.15.53:55356) 2023-11-06T01:38:05.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.941+0000 7fa489295700 1 -- 172.21.15.53:0/729429444 learned_addr learned my addr 172.21.15.53:0/729429444 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:05.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.942+0000 7fa488a94700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa484101f50 0x7fa48406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.942+0000 7fa489295700 1 -- 172.21.15.53:0/729429444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa484102910 msgr2=0x7fa484197f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:38:05.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.942+0000 7fa489295700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa484102910 0x7fa484197f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.942+0000 7fa489295700 1 -- 172.21.15.53:0/729429444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa484101f50 msgr2=0x7fa48406faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:05.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.942+0000 7fa489295700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa484101f50 0x7fa48406faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.942+0000 7fa489295700 1 -- 172.21.15.53:0/729429444 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740095f0 con 0x7fa4840ff0a0 2023-11-06T01:38:05.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.942+0000 7fa488a94700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa484101f50 0x7fa48406faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:38:05.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.942+0000 7fa489295700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4840ff0a0 0x7fa48406f560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e970 tx=0x7fa48000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:05.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.942+0000 7fa47a7fc700 1 -- 172.21.15.53:0/729429444 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48000cbd0 con 0x7fa4840ff0a0 2023-11-06T01:38:05.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.943+0000 7fa47a7fc700 1 -- 172.21.15.53:0/729429444 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4800044e0 con 0x7fa4840ff0a0 2023-11-06T01:38:05.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.943+0000 7fa47a7fc700 1 -- 172.21.15.53:0/729429444 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480003df0 con 0x7fa4840ff0a0 2023-11-06T01:38:05.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.943+0000 7fa48b4f9700 1 -- 172.21.15.53:0/729429444 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4841987a0 con 0x7fa4840ff0a0 2023-11-06T01:38:05.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.943+0000 7fa48b4f9700 1 -- 172.21.15.53:0/729429444 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa484198ce0 con 0x7fa4840ff0a0 2023-11-06T01:38:05.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.946+0000 7fa47a7fc700 1 -- 172.21.15.53:0/729429444 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa480004020 con 0x7fa4840ff0a0 2023-11-06T01:38:05.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.946+0000 7fa47a7fc700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4700a05f0 0x7fa4700a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.947+0000 7fa47a7fc700 1 -- 172.21.15.53:0/729429444 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa480014070 con 0x7fa4840ff0a0 2023-11-06T01:38:05.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.947+0000 7fa488a94700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4700a05f0 0x7fa4700a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.947+0000 7fa497689b80 1 Processor -- start 2023-11-06T01:38:05.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.947+0000 7fa497689b80 1 -- start start 2023-11-06T01:38:05.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.948+0000 7fa497689b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5417ded80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.948+0000 7fa497689b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.948+0000 7fa497689b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b5417e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.948+0000 7fa497689b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54172ca10 con 0x55b5417dd670 2023-11-06T01:38:05.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.948+0000 7fa497689b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54172a3d0 con 0x55b5417de030 2023-11-06T01:38:05.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.948+0000 7fa497689b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b541725cc0 con 0x55b5417de960 2023-11-06T01:38:05.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.949+0000 7fa489a96700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b5417e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.949+0000 7fa489a96700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b5417e58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52560/0 (socket says 172.21.15.53:52560) 2023-11-06T01:38:05.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.949+0000 7fa489a96700 1 -- 172.21.15.53:0/602491106 learned_addr learned my addr 172.21.15.53:0/602491106 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:05.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.949+0000 7fa489295700 1 --2- 172.21.15.53:0/602491106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5417ded80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.949+0000 7fa488a94700 1 --2- 172.21.15.53:0/602491106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.949+0000 7fa489a96700 1 -- 172.21.15.53:0/602491106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 msgr2=0x55b5417ded80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:05.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.949+0000 7fa489a96700 1 --2- 172.21.15.53:0/602491106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5417ded80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.949+0000 7fa489a96700 1 -- 172.21.15.53:0/602491106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 msgr2=0x55b5417ddaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:05.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.949+0000 7fa489a96700 1 --2- 172.21.15.53:0/602491106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417ddaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.949+0000 7fa489a96700 1 -- 172.21.15.53:0/602491106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b541553de0 con 0x55b5417de030 2023-11-06T01:38:05.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.949+0000 7fa488a94700 1 --2- 172.21.15.53:0/602491106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417ddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:38:05.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.950+0000 7fa488a94700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4700a05f0 0x7fa4700a2ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa484070680 tx=0x7fa47400bb10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:05.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.950+0000 7fa489a96700 1 --2- 172.21.15.53:0/602491106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b5417e58d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009d00 tx=0x7fa47c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6fa5111dbdb229e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:05.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.950+0000 7fa46f7fe700 1 -- 172.21.15.53:0/602491106 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c00e070 con 0x55b5417de030 2023-11-06T01:38:05.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.950+0000 7fa46f7fe700 1 -- 172.21.15.53:0/602491106 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa47c009830 con 0x55b5417de030 2023-11-06T01:38:05.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.950+0000 7fa46f7fe700 1 -- 172.21.15.53:0/602491106 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c011390 con 0x55b5417de030 2023-11-06T01:38:05.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.951+0000 7fa497689b80 1 -- 172.21.15.53:0/602491106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 msgr2=0x55b5417e58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:05.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.951+0000 7fa497689b80 1 --2- 172.21.15.53:0/602491106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b5417e58d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009d00 tx=0x7fa47c00b330 comp rx=0 tx=0).stop 2023-11-06T01:38:05.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.951+0000 7fa497689b80 1 -- 172.21.15.53:0/602491106 shutdown_connections 2023-11-06T01:38:05.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.951+0000 7fa497689b80 1 --2- 172.21.15.53:0/602491106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b5417e58d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.951+0000 7fa497689b80 1 --2- 172.21.15.53:0/602491106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5417ded80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.951+0000 7fa497689b80 1 --2- 172.21.15.53:0/602491106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417ddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.951+0000 7fa497689b80 1 -- 172.21.15.53:0/602491106 >> 172.21.15.53:0/602491106 conn(0x55b5417340d0 msgr2=0x55b5417cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:05.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.951+0000 7fa497689b80 1 -- 172.21.15.53:0/602491106 shutdown_connections 2023-11-06T01:38:05.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.951+0000 7fa497689b80 1 -- 172.21.15.53:0/602491106 wait complete. 2023-11-06T01:38:05.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.951+0000 7fa497689b80 1 -- start start 2023-11-06T01:38:05.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.951+0000 7fa497689b80 1 Processor -- start 2023-11-06T01:38:05.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.953+0000 7fa497689b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417d6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.953+0000 7fa497689b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de030 0x55b5417d66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.953+0000 7fa497689b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de960 0x55b5417db700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.953+0000 7fa497689b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5416c2510 con 0x55b5417dd670 2023-11-06T01:38:05.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.953+0000 7fa497689b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5416c21b0 con 0x55b5417de960 2023-11-06T01:38:05.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.953+0000 7fa497689b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5415a9920 con 0x55b5417de030 2023-11-06T01:38:05.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.953+0000 7fa497689b80 10 client.0 _ll_register_callbacks cb 0x55b54160fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:05.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.953+0000 7fa488a94700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de030 0x55b5417d66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.953+0000 7fa488a94700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de030 0x55b5417d66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55368/0 (socket says 172.21.15.53:55368) 2023-11-06T01:38:05.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.953+0000 7fa488a94700 1 -- 172.21.15.53:0/3726184754 learned_addr learned my addr 172.21.15.53:0/3726184754 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:05.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.953+0000 7fa489a96700 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de960 0x55b5417db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.954+0000 7fa489295700 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417d6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.954+0000 7fa489a96700 1 -- 172.21.15.53:0/3726184754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de030 msgr2=0x55b5417d66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:05.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.954+0000 7fa489a96700 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de030 0x55b5417d66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.954+0000 7fa489a96700 1 -- 172.21.15.53:0/3726184754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 msgr2=0x55b5417d6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:05.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.954+0000 7fa489a96700 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417d6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.954+0000 7fa489a96700 1 -- 172.21.15.53:0/3726184754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c0099b0 con 0x55b5417de960 2023-11-06T01:38:05.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.954+0000 7fa488a94700 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de030 0x55b5417d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:38:05.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.954+0000 7fa489295700 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417d6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:38:05.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.954+0000 7fa489a96700 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de960 0x55b5417db700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00b700 tx=0x7fa47c00be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d996c02509e9e960 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:05.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.954+0000 7fa46cff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:38:05.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.954+0000 7fa46cff9700 1 -- 172.21.15.53:0/3726184754 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c00e040 con 0x55b5417de960 2023-11-06T01:38:05.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.955+0000 7fa46cff9700 1 -- 172.21.15.53:0/3726184754 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa47c01b8f0 con 0x55b5417de960 2023-11-06T01:38:05.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.955+0000 7fa497689b80 1 -- 172.21.15.53:0/3726184754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5417a3e10 con 0x55b5417de960 2023-11-06T01:38:05.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.955+0000 7fa46cff9700 1 -- 172.21.15.53:0/3726184754 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa47c01bab0 con 0x55b5417de960 2023-11-06T01:38:05.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.955+0000 7fa497689b80 10 client.25538 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:05.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.955+0000 7fa497689b80 1 -- 172.21.15.53:0/3726184754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b54153cc60 con 0x55b5417de960 2023-11-06T01:38:05.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.955+0000 7fa46cff9700 1 -- 172.21.15.53:0/3726184754 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa47c02a030 con 0x55b5417de960 2023-11-06T01:38:05.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.956+0000 7fa46cff9700 1 -- 172.21.15.53:0/3726184754 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa47c024020 con 0x55b5417de960 2023-11-06T01:38:05.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.956+0000 7fa497689b80 10 client.25538 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:05.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.956+0000 7fa497689b80 10 client.25538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:05.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.956+0000 7fa497689b80 20 client.25538 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:05.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.956+0000 7fa497689b80 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b5417fb640 0x55b5417fda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.956+0000 7fa497689b80 4 client.25538 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:38:05.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.956+0000 7fa497689b80 1 -- 172.21.15.53:0/3726184754 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b54171ed40 con 0x55b5417fb640 2023-11-06T01:38:05.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.956+0000 7fa489295700 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b5417fb640 0x55b5417fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.956+0000 7fa46cff9700 10 client.25538 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:05.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.956+0000 7fa489295700 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b5417fb640 0x55b5417fda60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a772a8221330b8c7 server_cookie=d5eb06989a535d7f in_seq=0 out_seq=0 2023-11-06T01:38:05.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa46cff9700 1 -- 172.21.15.53:0/3726184754 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b54171ed40 con 0x55b5417fb640 2023-11-06T01:38:05.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa46cff9700 10 client.25538 handle_command_reply: tid=0 2023-11-06T01:38:05.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa497689b80 1 client.25538 shutdown 2023-11-06T01:38:05.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa497689b80 1 -- 172.21.15.53:0/3726184754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de960 msgr2=0x55b5417db700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:05.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa497689b80 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de960 0x55b5417db700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00b700 tx=0x7fa47c00be80 comp rx=0 tx=0).stop 2023-11-06T01:38:05.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa497689b80 1 -- 172.21.15.53:0/3726184754 shutdown_connections 2023-11-06T01:38:05.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa497689b80 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de960 0x55b5417db700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa497689b80 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b5417fb640 0x55b5417fda60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa497689b80 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417d6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa497689b80 1 --2- 172.21.15.53:0/3726184754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de030 0x55b5417d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa46cff9700 0 client.25538 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:05.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa497689b80 1 -- 172.21.15.53:0/3726184754 >> 172.21.15.53:0/3726184754 conn(0x55b5417340d0 msgr2=0x55b5417cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:05.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa497689b80 1 -- 172.21.15.53:0/3726184754 shutdown_connections 2023-11-06T01:38:05.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.957+0000 7fa497689b80 1 -- 172.21.15.53:0/3726184754 wait complete. 2023-11-06T01:38:05.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.958+0000 7fa497689b80 20 client.25538 trim_cache size 0 max 16384 2023-11-06T01:38:05.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.982+0000 7fa497689b80 1 Processor -- start 2023-11-06T01:38:05.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.982+0000 7fa497689b80 1 -- start start 2023-11-06T01:38:05.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.982+0000 7fa497689b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417db6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.982+0000 7fa497689b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b5417e1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.982+0000 7fa497689b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5417e2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.982+0000 7fa497689b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5417d7380 con 0x55b5417dd670 2023-11-06T01:38:05.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.982+0000 7fa497689b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5417d5640 con 0x55b5417de030 2023-11-06T01:38:05.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.982+0000 7fa497689b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5417d5d00 con 0x55b5417de960 2023-11-06T01:38:05.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.982+0000 7fa489295700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417db6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.982+0000 7fa489295700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417db6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41136/0 (socket says 172.21.15.53:41136) 2023-11-06T01:38:05.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.982+0000 7fa489295700 1 -- 172.21.15.53:0/55654434 learned_addr learned my addr 172.21.15.53:0/55654434 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:05.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.983+0000 7fa489a96700 1 --2- 172.21.15.53:0/55654434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5417e2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.983+0000 7fa489295700 1 -- 172.21.15.53:0/55654434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 msgr2=0x55b5417e2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:05.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.983+0000 7fa489295700 1 --2- 172.21.15.53:0/55654434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5417e2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.983+0000 7fa489295700 1 -- 172.21.15.53:0/55654434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 msgr2=0x55b5417e1ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:38:05.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.983+0000 7fa489295700 1 --2- 172.21.15.53:0/55654434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b5417e1ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.983+0000 7fa489295700 1 -- 172.21.15.53:0/55654434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b54157a250 con 0x55b5417dd670 2023-11-06T01:38:05.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.984+0000 7fa489295700 1 --2- 172.21.15.53:0/55654434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417db6c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa480005950 tx=0x7fa480084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b251c607e3fc1a6a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:05.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.985+0000 7fa46ffff700 1 -- 172.21.15.53:0/55654434 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480098e40 con 0x55b5417dd670 2023-11-06T01:38:05.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.985+0000 7fa46ffff700 1 -- 172.21.15.53:0/55654434 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa480099be0 con 0x55b5417dd670 2023-11-06T01:38:05.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.985+0000 7fa46ffff700 1 -- 172.21.15.53:0/55654434 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480097880 con 0x55b5417dd670 2023-11-06T01:38:05.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.986+0000 7fa497689b80 1 -- 172.21.15.53:0/55654434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 msgr2=0x55b5417db6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:05.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.986+0000 7fa497689b80 1 --2- 172.21.15.53:0/55654434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417db6c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa480005950 tx=0x7fa480084040 comp rx=0 tx=0).stop 2023-11-06T01:38:05.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.986+0000 7fa497689b80 1 -- 172.21.15.53:0/55654434 shutdown_connections 2023-11-06T01:38:05.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.986+0000 7fa497689b80 1 --2- 172.21.15.53:0/55654434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5417e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.986+0000 7fa497689b80 1 --2- 172.21.15.53:0/55654434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5417dd670 0x55b5417db6c0 secure :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa480005950 tx=0x7fa480084040 comp rx=0 tx=0).stop 2023-11-06T01:38:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.986+0000 7fa497689b80 1 --2- 172.21.15.53:0/55654434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b5417e1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.986+0000 7fa497689b80 1 -- 172.21.15.53:0/55654434 >> 172.21.15.53:0/55654434 conn(0x55b5417340d0 msgr2=0x55b5417cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.987+0000 7fa497689b80 1 -- 172.21.15.53:0/55654434 shutdown_connections 2023-11-06T01:38:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.987+0000 7fa497689b80 1 -- 172.21.15.53:0/55654434 wait complete. 2023-11-06T01:38:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.987+0000 7fa497689b80 1 -- start start 2023-11-06T01:38:05.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.987+0000 7fa497689b80 1 Processor -- start 2023-11-06T01:38:05.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa497689b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b541845360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa497689b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5418458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa497689b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b54184a6f0 0x55b54184cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:05.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa497689b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54184d020 con 0x55b54184a6f0 2023-11-06T01:38:05.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa497689b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54184d160 con 0x55b5417de030 2023-11-06T01:38:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa497689b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54184d320 con 0x55b5417de960 2023-11-06T01:38:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa497689b80 10 client.0 _ll_register_callbacks cb 0x7fa480003790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa489a96700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b54184a6f0 0x55b54184cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa488a94700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5418458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa488a94700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5418458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55380/0 (socket says 172.21.15.53:55380) 2023-11-06T01:38:05.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa489a96700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b54184a6f0 0x55b54184cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41142/0 (socket says 172.21.15.53:41142) 2023-11-06T01:38:05.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa488a94700 1 -- 172.21.15.53:0/3819400376 learned_addr learned my addr 172.21.15.53:0/3819400376 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:05.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa489295700 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b541845360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:05.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.988+0000 7fa488a94700 1 -- 172.21.15.53:0/3819400376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 msgr2=0x55b541845360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa488a94700 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b541845360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa488a94700 1 -- 172.21.15.53:0/3819400376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b54184a6f0 msgr2=0x55b54184cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa488a94700 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b54184a6f0 0x55b54184cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa488a94700 1 -- 172.21.15.53:0/3819400376 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa484198ce0 con 0x55b5417de960 2023-11-06T01:38:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa489a96700 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b54184a6f0 0x55b54184cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:38:05.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa488a94700 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5418458a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa4740098e0 tx=0x7fa474009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ec9bbda01a682c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:05.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa46cff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:38:05.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa46cff9700 1 -- 172.21.15.53:0/3819400376 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474005680 con 0x55b5417de960 2023-11-06T01:38:05.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa46cff9700 1 -- 172.21.15.53:0/3819400376 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa474004b70 con 0x55b5417de960 2023-11-06T01:38:05.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa497689b80 1 -- 172.21.15.53:0/3819400376 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b541750d10 con 0x55b5417de960 2023-11-06T01:38:05.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa46cff9700 1 -- 172.21.15.53:0/3819400376 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa474004d30 con 0x55b5417de960 2023-11-06T01:38:05.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa497689b80 10 client.25615 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:05.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa497689b80 1 -- 172.21.15.53:0/3819400376 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b5415522b0 con 0x55b5417de960 2023-11-06T01:38:05.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.989+0000 7fa46cff9700 1 -- 172.21.15.53:0/3819400376 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b541750d10 con 0x55b5417de960 2023-11-06T01:38:05.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.990+0000 7fa46cff9700 1 -- 172.21.15.53:0/3819400376 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa474029430 con 0x55b5417de960 2023-11-06T01:38:06.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.990+0000 7fa497689b80 10 client.25615 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:06.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.990+0000 7fa497689b80 10 client.25615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:06.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.990+0000 7fa497689b80 20 client.25615 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:06.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.990+0000 7fa497689b80 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b54184ec80 0x55b5418510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:06.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.990+0000 7fa497689b80 4 client.25615 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:38:06.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.990+0000 7fa497689b80 1 -- 172.21.15.53:0/3819400376 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b54171ea60 con 0x55b54184ec80 2023-11-06T01:38:06.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.990+0000 7fa489295700 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b54184ec80 0x55b5418510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:06.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.991+0000 7fa489295700 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b54184ec80 0x55b5418510a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40d467976e9e536 server_cookie=a8e42955931d6da5 in_seq=0 out_seq=0 2023-11-06T01:38:06.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.991+0000 7fa46cff9700 10 client.25615 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:06.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa46cff9700 1 -- 172.21.15.53:0/3819400376 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b54171ea60 con 0x55b54184ec80 2023-11-06T01:38:06.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa46cff9700 10 client.25615 handle_command_reply: tid=0 2023-11-06T01:38:06.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 1 client.25615 shutdown 2023-11-06T01:38:06.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 1 -- 172.21.15.53:0/3819400376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 msgr2=0x55b5418458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:06.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5418458a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa4740098e0 tx=0x7fa474009ca0 comp rx=0 tx=0).stop 2023-11-06T01:38:06.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 1 -- 172.21.15.53:0/3819400376 shutdown_connections 2023-11-06T01:38:06.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5417de030 0x55b541845360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:06.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b54184ec80 0x55b5418510a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:06.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b54184a6f0 0x55b54184cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:06.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 1 --2- 172.21.15.53:0/3819400376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5417de960 0x55b5418458a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:06.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 1 -- 172.21.15.53:0/3819400376 >> 172.21.15.53:0/3819400376 conn(0x55b5417340d0 msgr2=0x55b5417e1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:06.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa46cff9700 0 client.25615 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:06.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 1 -- 172.21.15.53:0/3819400376 shutdown_connections 2023-11-06T01:38:06.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 1 -- 172.21.15.53:0/3819400376 wait complete. 2023-11-06T01:38:06.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.993+0000 7fa497689b80 20 client.25615 trim_cache size 0 max 16384 2023-11-06T01:38:06.004 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:38:06.004 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:38:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:38:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:38:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:38:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:38:06.005 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:38:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:38:06.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:38:06.006 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:38:06.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.994+0000 7fa48b4f9700 1 -- 172.21.15.53:0/729429444 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4700a05f0 msgr2=0x7fa4700a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:06.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.994+0000 7fa48b4f9700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4700a05f0 0x7fa4700a2ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa484070680 tx=0x7fa47400bb10 comp rx=0 tx=0).stop 2023-11-06T01:38:06.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.994+0000 7fa48b4f9700 1 -- 172.21.15.53:0/729429444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4840ff0a0 msgr2=0x7fa48406f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:06.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.994+0000 7fa48b4f9700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4840ff0a0 0x7fa48406f560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e970 tx=0x7fa48000ed30 comp rx=0 tx=0).stop 2023-11-06T01:38:06.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.995+0000 7fa48b4f9700 1 -- 172.21.15.53:0/729429444 shutdown_connections 2023-11-06T01:38:06.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.995+0000 7fa48b4f9700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4700a05f0 0x7fa4700a2ab0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:06.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.995+0000 7fa48b4f9700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa484102910 0x7fa484197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:06.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.995+0000 7fa48b4f9700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4840ff0a0 0x7fa48406f560 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:06.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.995+0000 7fa48b4f9700 1 --2- 172.21.15.53:0/729429444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa484101f50 0x7fa48406faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:06.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.995+0000 7fa48b4f9700 1 -- 172.21.15.53:0/729429444 >> 172.21.15.53:0/729429444 conn(0x7fa4840fa9b0 msgr2=0x7fa4840fb620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:06.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.995+0000 7fa48b4f9700 1 -- 172.21.15.53:0/729429444 shutdown_connections 2023-11-06T01:38:06.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:05.995+0000 7fa48b4f9700 1 -- 172.21.15.53:0/729429444 wait complete. 2023-11-06T01:38:06.015 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:38:07.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 230.0 s, 2830.5 tps, lat 35.277 ms stddev 39.379 2023-11-06T01:38:07.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:07 smithi062 ceph-mon[103924]: pgmap v424: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 MiB/s wr, 2.21k op/s 2023-11-06T01:38:07.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:07 smithi163 ceph-mon[104416]: pgmap v424: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 MiB/s wr, 2.21k op/s 2023-11-06T01:38:07.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:07 smithi053 ceph-mon[99472]: pgmap v424: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 MiB/s wr, 2.21k op/s 2023-11-06T01:38:09.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:09 smithi062 ceph-mon[103924]: pgmap v425: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 MiB/s wr, 1.97k op/s 2023-11-06T01:38:09.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:09 smithi163 ceph-mon[104416]: pgmap v425: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 MiB/s wr, 1.97k op/s 2023-11-06T01:38:09.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:09 smithi053 ceph-mon[99472]: pgmap v425: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 MiB/s wr, 1.97k op/s 2023-11-06T01:38:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.191+0000 7ff61c40d700 1 -- 172.21.15.53:0/3450159656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6140ffcc0 msgr2=0x7ff6141000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.191+0000 7ff61c40d700 1 --2- 172.21.15.53:0/3450159656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6140ffcc0 0x7ff6141000e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff610009990 tx=0x7ff610009ca0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.191+0000 7ff61c40d700 1 -- 172.21.15.53:0/3450159656 shutdown_connections 2023-11-06T01:38:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.191+0000 7ff61c40d700 1 --2- 172.21.15.53:0/3450159656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff614101850 0x7ff614068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.191+0000 7ff61c40d700 1 --2- 172.21.15.53:0/3450159656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6140ffcc0 0x7ff6141000e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.191+0000 7ff61c40d700 1 --2- 172.21.15.53:0/3450159656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff614100eb0 0x7ff614101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.191+0000 7ff61c40d700 1 -- 172.21.15.53:0/3450159656 >> 172.21.15.53:0/3450159656 conn(0x7ff6140fb210 msgr2=0x7ff6140fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.191+0000 7ff61c40d700 1 -- 172.21.15.53:0/3450159656 shutdown_connections 2023-11-06T01:38:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.191+0000 7ff61c40d700 1 -- 172.21.15.53:0/3450159656 wait complete. 2023-11-06T01:38:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61c40d700 1 Processor -- start 2023-11-06T01:38:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61c40d700 1 -- start start 2023-11-06T01:38:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61c40d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140ffcc0 0x7ff614198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61c40d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff614100eb0 0x7ff614199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61c40d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff614101850 0x7ff614199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61c40d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61419a6c0 con 0x7ff614100eb0 2023-11-06T01:38:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61c40d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61419a830 con 0x7ff6140ffcc0 2023-11-06T01:38:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61c40d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61419a9f0 con 0x7ff614101850 2023-11-06T01:38:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff6199a8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff614100eb0 0x7ff614199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61a1a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140ffcc0 0x7ff614198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61a1a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140ffcc0 0x7ff614198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52592/0 (socket says 172.21.15.53:52592) 2023-11-06T01:38:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61a1a9700 1 -- 172.21.15.53:0/3164883481 learned_addr learned my addr 172.21.15.53:0/3164883481 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff6199a8700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff614100eb0 0x7ff614199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41156/0 (socket says 172.21.15.53:41156) 2023-11-06T01:38:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.192+0000 7ff61a9aa700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff614101850 0x7ff614199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.193+0000 7ff6199a8700 1 -- 172.21.15.53:0/3164883481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff614101850 msgr2=0x7ff614199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.193+0000 7ff6199a8700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff614101850 0x7ff614199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.193+0000 7ff6199a8700 1 -- 172.21.15.53:0/3164883481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140ffcc0 msgr2=0x7ff614198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.193+0000 7ff6199a8700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140ffcc0 0x7ff614198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.193+0000 7ff6199a8700 1 -- 172.21.15.53:0/3164883481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6100095f0 con 0x7ff614100eb0 2023-11-06T01:38:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.193+0000 7ff61a9aa700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff614101850 0x7ff614199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.193+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140ffcc0 0x7ff614198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.193+0000 7ff6199a8700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff614100eb0 0x7ff614199280 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff60400c970 tx=0x7ff60400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.193+0000 7ff60b7fe700 1 -- 172.21.15.53:0/3164883481 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604004440 con 0x7ff614100eb0 2023-11-06T01:38:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.194+0000 7ff60b7fe700 1 -- 172.21.15.53:0/3164883481 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff604003b40 con 0x7ff614100eb0 2023-11-06T01:38:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.194+0000 7ff60b7fe700 1 -- 172.21.15.53:0/3164883481 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604007b60 con 0x7ff614100eb0 2023-11-06T01:38:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.194+0000 7ff61c40d700 1 -- 172.21.15.53:0/3164883481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61419acd0 con 0x7ff614100eb0 2023-11-06T01:38:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.194+0000 7ff61c40d700 1 -- 172.21.15.53:0/3164883481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6141a13f0 con 0x7ff614100eb0 2023-11-06T01:38:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.197+0000 7ff60b7fe700 1 -- 172.21.15.53:0/3164883481 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff604003cb0 con 0x7ff614100eb0 2023-11-06T01:38:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.198+0000 7ff60b7fe700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6000a0570 0x7ff6000a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.198+0000 7ff60b7fe700 1 -- 172.21.15.53:0/3164883481 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff6040cb650 con 0x7ff614100eb0 2023-11-06T01:38:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.198+0000 7ff62859db80 1 Processor -- start 2023-11-06T01:38:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.199+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6000a0570 0x7ff6000a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.199+0000 7ff62859db80 1 -- start start 2023-11-06T01:38:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.199+0000 7ff62859db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2d95aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.199+0000 7ff62859db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2d968f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.199+0000 7ff62859db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2d97350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.199+0000 7ff62859db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2ced9b0 con 0x563fa2d95fe0 2023-11-06T01:38:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.199+0000 7ff62859db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2ceb370 con 0x563fa2d96ed0 2023-11-06T01:38:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.199+0000 7ff62859db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2ce6c60 con 0x563fa2d95680 2023-11-06T01:38:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a9aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2d97350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a9aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2d97350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52600/0 (socket says 172.21.15.53:52600) 2023-11-06T01:38:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a9aa700 1 -- 172.21.15.53:0/2877052700 learned_addr learned my addr 172.21.15.53:0/2877052700 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a9aa700 1 -- 172.21.15.53:0/2877052700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 msgr2=0x563fa2d95aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/2877052700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2d95aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a9aa700 1 --2- 172.21.15.53:0/2877052700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2d95aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a9aa700 1 -- 172.21.15.53:0/2877052700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 msgr2=0x563fa2d968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a9aa700 1 --2- 172.21.15.53:0/2877052700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2d968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a9aa700 1 -- 172.21.15.53:0/2877052700 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fa2b14d80 con 0x563fa2d96ed0 2023-11-06T01:38:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/2877052700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2d95aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:38:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6000a0570 0x7ff6000a2a30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b2e0 tx=0x7ff61000bc00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff61a9aa700 1 --2- 172.21.15.53:0/2877052700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2d97350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff60c003e00 tx=0x7ff60c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8393caa8a9992f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.200+0000 7ff608daa700 1 -- 172.21.15.53:0/2877052700 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0092d0 con 0x563fa2d96ed0 2023-11-06T01:38:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.201+0000 7ff608daa700 1 -- 172.21.15.53:0/2877052700 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff60c00f040 con 0x563fa2d96ed0 2023-11-06T01:38:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.201+0000 7ff608daa700 1 -- 172.21.15.53:0/2877052700 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c00e710 con 0x563fa2d96ed0 2023-11-06T01:38:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.201+0000 7ff62859db80 1 -- 172.21.15.53:0/2877052700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 msgr2=0x563fa2d97350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.201+0000 7ff62859db80 1 --2- 172.21.15.53:0/2877052700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2d97350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff60c003e00 tx=0x7ff60c009920 comp rx=0 tx=0).stop 2023-11-06T01:38:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.201+0000 7ff62859db80 1 -- 172.21.15.53:0/2877052700 shutdown_connections 2023-11-06T01:38:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.201+0000 7ff62859db80 1 --2- 172.21.15.53:0/2877052700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2d97350 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.201+0000 7ff62859db80 1 --2- 172.21.15.53:0/2877052700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2d95aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.201+0000 7ff62859db80 1 --2- 172.21.15.53:0/2877052700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2d968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.201+0000 7ff62859db80 1 -- 172.21.15.53:0/2877052700 >> 172.21.15.53:0/2877052700 conn(0x563fa2cf5070 msgr2=0x563fa2d92040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.202+0000 7ff62859db80 1 -- 172.21.15.53:0/2877052700 shutdown_connections 2023-11-06T01:38:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.202+0000 7ff62859db80 1 -- 172.21.15.53:0/2877052700 wait complete. 2023-11-06T01:38:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.202+0000 7ff62859db80 1 -- start start 2023-11-06T01:38:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.202+0000 7ff62859db80 1 Processor -- start 2023-11-06T01:38:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.202+0000 7ff62859db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2db0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.202+0000 7ff62859db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2db0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.202+0000 7ff62859db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2db7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.202+0000 7ff62859db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2c834b0 con 0x563fa2d95fe0 2023-11-06T01:38:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.202+0000 7ff62859db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2c83150 con 0x563fa2d96ed0 2023-11-06T01:38:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.202+0000 7ff62859db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2b6a8c0 con 0x563fa2d95680 2023-11-06T01:38:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.202+0000 7ff62859db80 10 client.0 _ll_register_callbacks cb 0x563fa2bd0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.203+0000 7ff6199a8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2db0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.203+0000 7ff61a1a9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2db0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.203+0000 7ff61a1a9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2db0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55408/0 (socket says 172.21.15.53:55408) 2023-11-06T01:38:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.203+0000 7ff61a1a9700 1 -- 172.21.15.53:0/3912625756 learned_addr learned my addr 172.21.15.53:0/3912625756 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.203+0000 7ff61a9aa700 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2db7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.203+0000 7ff61a1a9700 1 -- 172.21.15.53:0/3912625756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 msgr2=0x563fa2db7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.203+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2db7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.203+0000 7ff61a1a9700 1 -- 172.21.15.53:0/3912625756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 msgr2=0x563fa2db0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.203+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2db0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.203+0000 7ff61a1a9700 1 -- 172.21.15.53:0/3912625756 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c003ae0 con 0x563fa2d95680 2023-11-06T01:38:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.204+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2db0560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff610004850 tx=0x7ff610009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d811a7577dbf93ea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.204+0000 7ff5fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:38:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.204+0000 7ff5fdffb700 1 -- 172.21.15.53:0/3912625756 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000f930 con 0x563fa2d95680 2023-11-06T01:38:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.204+0000 7ff5fdffb700 1 -- 172.21.15.53:0/3912625756 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff61000fa90 con 0x563fa2d95680 2023-11-06T01:38:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.204+0000 7ff62859db80 1 -- 172.21.15.53:0/3912625756 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fa2d11cb0 con 0x563fa2d95680 2023-11-06T01:38:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.204+0000 7ff5fdffb700 1 -- 172.21.15.53:0/3912625756 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff61000fc50 con 0x563fa2d95680 2023-11-06T01:38:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.204+0000 7ff62859db80 10 client.25624 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.204+0000 7ff62859db80 1 -- 172.21.15.53:0/3912625756 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563fa2afdc00 con 0x563fa2d95680 2023-11-06T01:38:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.205+0000 7ff5fdffb700 1 -- 172.21.15.53:0/3912625756 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563fa2d11cb0 con 0x563fa2d95680 2023-11-06T01:38:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.205+0000 7ff5fdffb700 1 -- 172.21.15.53:0/3912625756 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff610038030 con 0x563fa2d95680 2023-11-06T01:38:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.205+0000 7ff62859db80 10 client.25624 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.205+0000 7ff62859db80 10 client.25624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.205+0000 7ff62859db80 20 client.25624 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.205+0000 7ff62859db80 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563fa2dbc280 0x563fa2dbe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.205+0000 7ff62859db80 4 client.25624 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:38:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.205+0000 7ff62859db80 1 -- 172.21.15.53:0/3912625756 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fa2d64db0 con 0x563fa2dbc280 2023-11-06T01:38:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.205+0000 7ff6199a8700 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563fa2dbc280 0x563fa2dbe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.205+0000 7ff6199a8700 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563fa2dbc280 0x563fa2dbe6a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28dc978c31511fae server_cookie=d3c7281f211cdfae in_seq=0 out_seq=0 2023-11-06T01:38:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.205+0000 7ff5fdffb700 10 client.25624 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff5fdffb700 1 -- 172.21.15.53:0/3912625756 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563fa2d64db0 con 0x563fa2dbc280 2023-11-06T01:38:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff5fdffb700 10 client.25624 handle_command_reply: tid=0 2023-11-06T01:38:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 1 client.25624 shutdown 2023-11-06T01:38:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 1 -- 172.21.15.53:0/3912625756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 msgr2=0x563fa2db0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2db0560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff610004850 tx=0x7ff610009ca0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 1 -- 172.21.15.53:0/3912625756 shutdown_connections 2023-11-06T01:38:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2db7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563fa2dbc280 0x563fa2dbe6a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2db0560 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff5fdffb700 0 client.25624 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 1 --2- 172.21.15.53:0/3912625756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2db0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 1 -- 172.21.15.53:0/3912625756 >> 172.21.15.53:0/3912625756 conn(0x563fa2cf5070 msgr2=0x563fa2d903c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 1 -- 172.21.15.53:0/3912625756 shutdown_connections 2023-11-06T01:38:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 1 -- 172.21.15.53:0/3912625756 wait complete. 2023-11-06T01:38:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.206+0000 7ff62859db80 20 client.25624 trim_cache size 0 max 16384 2023-11-06T01:38:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.226+0000 7ff62859db80 1 Processor -- start 2023-11-06T01:38:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.226+0000 7ff62859db80 1 -- start start 2023-11-06T01:38:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.226+0000 7ff62859db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2d91c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.226+0000 7ff62859db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2db7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.226+0000 7ff62859db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2db8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.226+0000 7ff62859db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2d96d50 con 0x563fa2d95fe0 2023-11-06T01:38:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.226+0000 7ff62859db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2d887a0 con 0x563fa2d96ed0 2023-11-06T01:38:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.226+0000 7ff62859db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2d889f0 con 0x563fa2d95680 2023-11-06T01:38:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.227+0000 7ff6199a8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2db7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.227+0000 7ff6199a8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2db7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41176/0 (socket says 172.21.15.53:41176) 2023-11-06T01:38:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.227+0000 7ff6199a8700 1 -- 172.21.15.53:0/1935484719 learned_addr learned my addr 172.21.15.53:0/1935484719 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.227+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/1935484719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2d91c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.227+0000 7ff61a1a9700 1 -- 172.21.15.53:0/1935484719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 msgr2=0x563fa2db8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.227+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/1935484719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2db8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.227+0000 7ff61a1a9700 1 -- 172.21.15.53:0/1935484719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 msgr2=0x563fa2db7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.227+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/1935484719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2db7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.227+0000 7ff61a1a9700 1 -- 172.21.15.53:0/1935484719 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fa2b3b1f0 con 0x563fa2d95680 2023-11-06T01:38:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.228+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/1935484719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2d91c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b680 tx=0x7ff6100057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44a4104acbc4220a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.228+0000 7ff6095ab700 1 -- 172.21.15.53:0/1935484719 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61002f6f0 con 0x563fa2d95680 2023-11-06T01:38:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.228+0000 7ff6095ab700 1 -- 172.21.15.53:0/1935484719 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61001e040 con 0x563fa2d95680 2023-11-06T01:38:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.228+0000 7ff6095ab700 1 -- 172.21.15.53:0/1935484719 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000f990 con 0x563fa2d95680 2023-11-06T01:38:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.229+0000 7ff62859db80 1 -- 172.21.15.53:0/1935484719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 msgr2=0x563fa2d91c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.229+0000 7ff62859db80 1 --2- 172.21.15.53:0/1935484719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2d91c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b680 tx=0x7ff6100057d0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.229+0000 7ff62859db80 1 -- 172.21.15.53:0/1935484719 shutdown_connections 2023-11-06T01:38:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.229+0000 7ff62859db80 1 --2- 172.21.15.53:0/1935484719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d96ed0 0x563fa2db8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.229+0000 7ff62859db80 1 --2- 172.21.15.53:0/1935484719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d95680 0x563fa2d91c20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.229+0000 7ff62859db80 1 --2- 172.21.15.53:0/1935484719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95fe0 0x563fa2db7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.229+0000 7ff62859db80 1 -- 172.21.15.53:0/1935484719 >> 172.21.15.53:0/1935484719 conn(0x563fa2cf5070 msgr2=0x563fa2daf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.229+0000 7ff62859db80 1 -- 172.21.15.53:0/1935484719 shutdown_connections 2023-11-06T01:38:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.229+0000 7ff62859db80 1 -- 172.21.15.53:0/1935484719 wait complete. 2023-11-06T01:38:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.229+0000 7ff62859db80 1 -- start start 2023-11-06T01:38:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff62859db80 1 Processor -- start 2023-11-06T01:38:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff62859db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95680 0x563fa2db3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff62859db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d95fe0 0x563fa2db3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff62859db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d96ed0 0x563fa2db6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff62859db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2db74e0 con 0x563fa2d95680 2023-11-06T01:38:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff62859db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2db7650 con 0x563fa2d95fe0 2023-11-06T01:38:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff62859db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fa2db7810 con 0x563fa2d96ed0 2023-11-06T01:38:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff62859db80 10 client.0 _ll_register_callbacks cb 0x7ff610005720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff61a1a9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95680 0x563fa2db3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff61a9aa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d96ed0 0x563fa2db6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff6199a8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d95fe0 0x563fa2db3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff61a1a9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95680 0x563fa2db3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41180/0 (socket says 172.21.15.53:41180) 2023-11-06T01:38:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff61a9aa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d96ed0 0x563fa2db6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55424/0 (socket says 172.21.15.53:55424) 2023-11-06T01:38:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff61a1a9700 1 -- 172.21.15.53:0/937897129 learned_addr learned my addr 172.21.15.53:0/937897129 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff61a9aa700 1 -- 172.21.15.53:0/937897129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d95fe0 msgr2=0x563fa2db3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff61a9aa700 1 --2- 172.21.15.53:0/937897129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d95fe0 0x563fa2db3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff61a9aa700 1 -- 172.21.15.53:0/937897129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95680 msgr2=0x563fa2db3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff61a9aa700 1 --2- 172.21.15.53:0/937897129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95680 0x563fa2db3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.230+0000 7ff61a9aa700 1 -- 172.21.15.53:0/937897129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fa2afdc00 con 0x563fa2d96ed0 2023-11-06T01:38:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.231+0000 7ff61a9aa700 1 --2- 172.21.15.53:0/937897129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d96ed0 0x563fa2db6fa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009ec0 tx=0x7ff60c004cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eee6e09933aefe6a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.231+0000 7ff5fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:38:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.231+0000 7ff5fdffb700 1 -- 172.21.15.53:0/937897129 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c007830 con 0x563fa2d96ed0 2023-11-06T01:38:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.231+0000 7ff5fdffb700 1 -- 172.21.15.53:0/937897129 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff60c00e4f0 con 0x563fa2d96ed0 2023-11-06T01:38:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.231+0000 7ff62859db80 1 -- 172.21.15.53:0/937897129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff60c00f040 con 0x563fa2d96ed0 2023-11-06T01:38:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.231+0000 7ff5fdffb700 1 -- 172.21.15.53:0/937897129 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff60c004e60 con 0x563fa2d96ed0 2023-11-06T01:38:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.231+0000 7ff62859db80 10 client.25630 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.231+0000 7ff62859db80 1 -- 172.21.15.53:0/937897129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563fa2b13250 con 0x563fa2d96ed0 2023-11-06T01:38:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.232+0000 7ff5fdffb700 1 -- 172.21.15.53:0/937897129 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff60c00f040 con 0x563fa2d96ed0 2023-11-06T01:38:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.233+0000 7ff5fdffb700 1 -- 172.21.15.53:0/937897129 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff60c00bd50 con 0x563fa2d96ed0 2023-11-06T01:38:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.233+0000 7ff62859db80 10 client.25630 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.233+0000 7ff62859db80 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.233+0000 7ff62859db80 20 client.25630 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.233+0000 7ff62859db80 1 --2- 172.21.15.53:0/937897129 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563fa2e1fee0 0x563fa2e22300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.233+0000 7ff62859db80 4 client.25630 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:38:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.233+0000 7ff62859db80 1 -- 172.21.15.53:0/937897129 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563fa2cdfce0 con 0x563fa2e1fee0 2023-11-06T01:38:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.233+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/937897129 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563fa2e1fee0 0x563fa2e22300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.234+0000 7ff5fdffb700 10 client.25630 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.234+0000 7ff61a1a9700 1 --2- 172.21.15.53:0/937897129 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563fa2e1fee0 0x563fa2e22300 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15626a3b0acf3da4 server_cookie=e48a794f6c1e3a65 in_seq=0 out_seq=0 2023-11-06T01:38:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff5fdffb700 1 -- 172.21.15.53:0/937897129 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563fa2cdfce0 con 0x563fa2e1fee0 2023-11-06T01:38:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff5fdffb700 10 client.25630 handle_command_reply: tid=0 2023-11-06T01:38:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff62859db80 1 client.25630 shutdown 2023-11-06T01:38:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff62859db80 1 -- 172.21.15.53:0/937897129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d96ed0 msgr2=0x563fa2db6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff62859db80 1 --2- 172.21.15.53:0/937897129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d96ed0 0x563fa2db6fa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009ec0 tx=0x7ff60c004cc0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff62859db80 1 -- 172.21.15.53:0/937897129 shutdown_connections 2023-11-06T01:38:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff62859db80 1 --2- 172.21.15.53:0/937897129 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563fa2e1fee0 0x563fa2e22300 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff62859db80 1 --2- 172.21.15.53:0/937897129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fa2d96ed0 0x563fa2db6fa0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff62859db80 1 --2- 172.21.15.53:0/937897129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fa2d95680 0x563fa2db3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff62859db80 1 --2- 172.21.15.53:0/937897129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fa2d95fe0 0x563fa2db3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff5fdffb700 0 client.25630 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff62859db80 1 -- 172.21.15.53:0/937897129 >> 172.21.15.53:0/937897129 conn(0x563fa2cf5070 msgr2=0x563fa2d9d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.250+0000 7ff62859db80 1 -- 172.21.15.53:0/937897129 shutdown_connections 2023-11-06T01:38:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.251+0000 7ff62859db80 1 -- 172.21.15.53:0/937897129 wait complete. 2023-11-06T01:38:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.251+0000 7ff62859db80 20 client.25630 trim_cache size 0 max 16384 2023-11-06T01:38:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:38:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:38:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:38:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:38:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 -- 172.21.15.53:0/3164883481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6000a0570 msgr2=0x7ff6000a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6000a0570 0x7ff6000a2a30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b2e0 tx=0x7ff61000bc00 comp rx=0 tx=0).stop 2023-11-06T01:38:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 -- 172.21.15.53:0/3164883481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff614100eb0 msgr2=0x7ff614199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff614100eb0 0x7ff614199280 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff60400c970 tx=0x7ff60400cc80 comp rx=0 tx=0).stop 2023-11-06T01:38:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 -- 172.21.15.53:0/3164883481 shutdown_connections 2023-11-06T01:38:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff6000a0570 0x7ff6000a2a30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140ffcc0 0x7ff614198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff614101850 0x7ff614199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 --2- 172.21.15.53:0/3164883481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff614100eb0 0x7ff614199280 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 -- 172.21.15.53:0/3164883481 >> 172.21.15.53:0/3164883481 conn(0x7ff6140fb210 msgr2=0x7ff6140fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 -- 172.21.15.53:0/3164883481 shutdown_connections 2023-11-06T01:38:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:11.252+0000 7ff61c40d700 1 -- 172.21.15.53:0/3164883481 wait complete. 2023-11-06T01:38:11.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:11 smithi062 ceph-mon[103924]: pgmap v426: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 MiB/s wr, 1.96k op/s 2023-11-06T01:38:11.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:11 smithi163 ceph-mon[104416]: pgmap v426: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 MiB/s wr, 1.96k op/s 2023-11-06T01:38:11.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:11 smithi053 ceph-mon[99472]: pgmap v426: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 MiB/s wr, 1.96k op/s 2023-11-06T01:38:12.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 235.0 s, 2698.7 tps, lat 36.735 ms stddev 42.263 2023-11-06T01:38:13.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:13 smithi062 ceph-mon[103924]: pgmap v427: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 1.96k op/s 2023-11-06T01:38:13.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:13 smithi163 ceph-mon[104416]: pgmap v427: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 1.96k op/s 2023-11-06T01:38:13.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:13 smithi053 ceph-mon[99472]: pgmap v427: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 1.96k op/s 2023-11-06T01:38:15.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:15 smithi062 ceph-mon[103924]: pgmap v428: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 916 op/s 2023-11-06T01:38:15.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:15 smithi163 ceph-mon[104416]: pgmap v428: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 916 op/s 2023-11-06T01:38:15.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:15 smithi053 ceph-mon[99472]: pgmap v428: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 916 op/s 2023-11-06T01:38:17.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 240.0 s, 2974.0 tps, lat 33.899 ms stddev 38.761 2023-11-06T01:38:17.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:17 smithi163 ceph-mon[104416]: pgmap v429: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 844 op/s 2023-11-06T01:38:17.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:17 smithi053 ceph-mon[99472]: pgmap v429: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 844 op/s 2023-11-06T01:38:17.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:17 smithi062 ceph-mon[103924]: pgmap v429: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 844 op/s 2023-11-06T01:38:19.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:19 smithi163 ceph-mon[104416]: pgmap v430: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 656 op/s 2023-11-06T01:38:19.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:19 smithi053 ceph-mon[99472]: pgmap v430: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 656 op/s 2023-11-06T01:38:19.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:19 smithi062 ceph-mon[103924]: pgmap v430: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 656 op/s 2023-11-06T01:38:21.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:21 smithi163 ceph-mon[104416]: pgmap v431: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 677 op/s 2023-11-06T01:38:21.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:21 smithi053 ceph-mon[99472]: pgmap v431: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 677 op/s 2023-11-06T01:38:21.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:21 smithi062 ceph-mon[103924]: pgmap v431: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 677 op/s 2023-11-06T01:38:22.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 245.0 s, 2075.6 tps, lat 48.304 ms stddev 55.030 2023-11-06T01:38:23.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:23 smithi163 ceph-mon[104416]: pgmap v432: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 819 op/s 2023-11-06T01:38:23.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:23 smithi053 ceph-mon[99472]: pgmap v432: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 819 op/s 2023-11-06T01:38:23.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:23 smithi062 ceph-mon[103924]: pgmap v432: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 819 op/s 2023-11-06T01:38:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.412+0000 7fe77f9b0700 1 -- 172.21.15.53:0/461090232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780ffca0 msgr2=0x7fe7781000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.413+0000 7fe77f9b0700 1 --2- 172.21.15.53:0/461090232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780ffca0 0x7fe7781000c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe7740098f0 tx=0x7fe774009c00 comp rx=0 tx=0).stop 2023-11-06T01:38:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.413+0000 7fe77f9b0700 1 -- 172.21.15.53:0/461090232 shutdown_connections 2023-11-06T01:38:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.413+0000 7fe77f9b0700 1 --2- 172.21.15.53:0/461090232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778101830 0x7fe7781080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.413+0000 7fe77f9b0700 1 --2- 172.21.15.53:0/461090232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780ffca0 0x7fe7781000c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.413+0000 7fe77f9b0700 1 --2- 172.21.15.53:0/461090232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe778100e90 0x7fe7781012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.413+0000 7fe77f9b0700 1 -- 172.21.15.53:0/461090232 >> 172.21.15.53:0/461090232 conn(0x7fe7780fb230 msgr2=0x7fe7780fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.413+0000 7fe77f9b0700 1 -- 172.21.15.53:0/461090232 shutdown_connections 2023-11-06T01:38:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.413+0000 7fe77f9b0700 1 -- 172.21.15.53:0/461090232 wait complete. 2023-11-06T01:38:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.413+0000 7fe77f9b0700 1 Processor -- start 2023-11-06T01:38:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.413+0000 7fe77f9b0700 1 -- start start 2023-11-06T01:38:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77f9b0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780ffca0 0x7fe778194a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77f9b0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe778100e90 0x7fe778194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77f9b0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778101830 0x7fe778195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77f9b0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe778196470 con 0x7fe778100e90 2023-11-06T01:38:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77f9b0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7781965e0 con 0x7fe778101830 2023-11-06T01:38:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77f9b0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7781967a0 con 0x7fe7780ffca0 2023-11-06T01:38:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77cf4b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe778100e90 0x7fe778194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77df4d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778101830 0x7fe778195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77cf4b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe778100e90 0x7fe778194f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41196/0 (socket says 172.21.15.53:41196) 2023-11-06T01:38:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77cf4b700 1 -- 172.21.15.53:0/642138699 learned_addr learned my addr 172.21.15.53:0/642138699 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77d74c700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780ffca0 0x7fe778194a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77cf4b700 1 -- 172.21.15.53:0/642138699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780ffca0 msgr2=0x7fe778194a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780ffca0 0x7fe778194a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77cf4b700 1 -- 172.21.15.53:0/642138699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778101830 msgr2=0x7fe778195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778101830 0x7fe778195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77cf4b700 1 -- 172.21.15.53:0/642138699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe774009520 con 0x7fe778100e90 2023-11-06T01:38:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77df4d700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778101830 0x7fe778195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.414+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe778100e90 0x7fe778194f90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe76800e9c0 tx=0x7fe76800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.415+0000 7fe76e7fc700 1 -- 172.21.15.53:0/642138699 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768004a30 con 0x7fe778100e90 2023-11-06T01:38:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.415+0000 7fe77f9b0700 1 -- 172.21.15.53:0/642138699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe778196a80 con 0x7fe778100e90 2023-11-06T01:38:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.415+0000 7fe76e7fc700 1 -- 172.21.15.53:0/642138699 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe76800cc80 con 0x7fe778100e90 2023-11-06T01:38:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.415+0000 7fe76e7fc700 1 -- 172.21.15.53:0/642138699 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680054e0 con 0x7fe778100e90 2023-11-06T01:38:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.415+0000 7fe77f9b0700 1 -- 172.21.15.53:0/642138699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77819d240 con 0x7fe778100e90 2023-11-06T01:38:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.420+0000 7fe78bb40b80 1 Processor -- start 2023-11-06T01:38:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.430+0000 7fe78bb40b80 1 -- start start 2023-11-06T01:38:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.430+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 0x55c0681ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.430+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681dc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.430+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681df1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.430+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0681269b0 con 0x55c0681ce530 2023-11-06T01:38:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.430+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c068124370 con 0x55c0681cee90 2023-11-06T01:38:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.430+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06811fc60 con 0x55c0681dce00 2023-11-06T01:38:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.432+0000 7fe77d74c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681dc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.432+0000 7fe77d74c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681dc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52640/0 (socket says 172.21.15.53:52640) 2023-11-06T01:38:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.432+0000 7fe77d74c700 1 -- 172.21.15.53:0/3142133454 learned_addr learned my addr 172.21.15.53:0/3142133454 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.437+0000 7fe76e7fc700 1 -- 172.21.15.53:0/642138699 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe768004b90 con 0x7fe778100e90 2023-11-06T01:38:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.437+0000 7fe76e7fc700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7640a05e0 0x7fe7640a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.438+0000 7fe76e7fc700 1 -- 172.21.15.53:0/642138699 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe768014070 con 0x7fe778100e90 2023-11-06T01:38:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.441+0000 7fe77d74c700 1 -- 172.21.15.53:0/3142133454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 msgr2=0x55c0681df1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.441+0000 7fe77d74c700 1 --2- 172.21.15.53:0/3142133454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681df1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.441+0000 7fe77d74c700 1 -- 172.21.15.53:0/3142133454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 msgr2=0x55c0681ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.441+0000 7fe77d74c700 1 --2- 172.21.15.53:0/3142133454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 0x55c0681ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.441+0000 7fe77d74c700 1 -- 172.21.15.53:0/3142133454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c067f4dd80 con 0x55c0681cee90 2023-11-06T01:38:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.448+0000 7fe77d74c700 1 --2- 172.21.15.53:0/3142133454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681dc8c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe7740098f0 tx=0x7fe774009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4aaedc5911b18d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.448+0000 7fe77df4d700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7640a05e0 0x7fe7640a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.463+0000 7fe77df4d700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7640a05e0 0x7fe7640a2aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe770009ec0 tx=0x7fe770009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.467+0000 7fe7637fe700 1 -- 172.21.15.53:0/3142133454 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77400f720 con 0x55c0681cee90 2023-11-06T01:38:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.467+0000 7fe7637fe700 1 -- 172.21.15.53:0/3142133454 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe77401d040 con 0x55c0681cee90 2023-11-06T01:38:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.467+0000 7fe7637fe700 1 -- 172.21.15.53:0/3142133454 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7740054b0 con 0x55c0681cee90 2023-11-06T01:38:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.469+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3142133454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 msgr2=0x55c0681dc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.470+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/3142133454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681dc8c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe7740098f0 tx=0x7fe774009f50 comp rx=0 tx=0).stop 2023-11-06T01:38:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.471+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3142133454 shutdown_connections 2023-11-06T01:38:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.471+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/3142133454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681df1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.471+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/3142133454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 0x55c0681ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.471+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/3142133454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681dc8c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.471+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3142133454 >> 172.21.15.53:0/3142133454 conn(0x55c06812e070 msgr2=0x55c0681cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.472+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3142133454 shutdown_connections 2023-11-06T01:38:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.473+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3142133454 wait complete. 2023-11-06T01:38:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.474+0000 7fe78bb40b80 1 -- start start 2023-11-06T01:38:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.475+0000 7fe78bb40b80 1 Processor -- start 2023-11-06T01:38:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.475+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 0x55c0681f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.475+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.475+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681f9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.475+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0680bc4b0 con 0x55c0681ce530 2023-11-06T01:38:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.475+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0680bc150 con 0x55c0681cee90 2023-11-06T01:38:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.475+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067fa38c0 con 0x55c0681dce00 2023-11-06T01:38:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.475+0000 7fe78bb40b80 10 client.0 _ll_register_callbacks cb 0x55c068009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.478+0000 7fe77cf4b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.478+0000 7fe77cf4b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52650/0 (socket says 172.21.15.53:52650) 2023-11-06T01:38:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.478+0000 7fe77cf4b700 1 -- 172.21.15.53:0/2959773581 learned_addr learned my addr 172.21.15.53:0/2959773581 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.478+0000 7fe77cf4b700 1 -- 172.21.15.53:0/2959773581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 msgr2=0x55c0681f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.478+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/2959773581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.478+0000 7fe77cf4b700 1 -- 172.21.15.53:0/2959773581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 msgr2=0x55c0681f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.478+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/2959773581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 0x55c0681f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.478+0000 7fe77cf4b700 1 -- 172.21.15.53:0/2959773581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe778108a30 con 0x55c0681cee90 2023-11-06T01:38:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.478+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/2959773581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f2230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe768099250 tx=0x7fe768098cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ef27b05a70d1a23 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.478+0000 7fe760ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:38:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe760ff9700 1 -- 172.21.15.53:0/2959773581 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680999c0 con 0x55c0681cee90 2023-11-06T01:38:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe760ff9700 1 -- 172.21.15.53:0/2959773581 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe7680992c0 con 0x55c0681cee90 2023-11-06T01:38:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2959773581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c06814acb0 con 0x55c0681cee90 2023-11-06T01:38:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe760ff9700 1 -- 172.21.15.53:0/2959773581 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe76809d0f0 con 0x55c0681cee90 2023-11-06T01:38:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe78bb40b80 10 client.25565 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2959773581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c067f36c00 con 0x55c0681cee90 2023-11-06T01:38:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe760ff9700 1 -- 172.21.15.53:0/2959773581 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe768063080 con 0x55c0681cee90 2023-11-06T01:38:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe760ff9700 1 -- 172.21.15.53:0/2959773581 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe768064f70 con 0x55c0681cee90 2023-11-06T01:38:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe78bb40b80 10 client.25565 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe78bb40b80 10 client.25565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe78bb40b80 20 client.25565 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/2959773581 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c0681fdad0 0x55c0681ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe78bb40b80 4 client.25565 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:38:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.479+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2959773581 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c06819ddb0 con 0x55c0681fdad0 2023-11-06T01:38:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.480+0000 7fe77d74c700 1 --2- 172.21.15.53:0/2959773581 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c0681fdad0 0x55c0681ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.481+0000 7fe760ff9700 10 client.25565 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.481+0000 7fe77d74c700 1 --2- 172.21.15.53:0/2959773581 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c0681fdad0 0x55c0681ffef0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5923a87d03b4ac0 server_cookie=5afcec3539fd0be5 in_seq=0 out_seq=0 2023-11-06T01:38:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe760ff9700 1 -- 172.21.15.53:0/2959773581 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c06819ddb0 con 0x55c0681fdad0 2023-11-06T01:38:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe760ff9700 10 client.25565 handle_command_reply: tid=0 2023-11-06T01:38:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe78bb40b80 1 client.25565 shutdown 2023-11-06T01:38:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2959773581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 msgr2=0x55c0681f2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/2959773581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f2230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe768099250 tx=0x7fe768098cd0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2959773581 shutdown_connections 2023-11-06T01:38:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/2959773581 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c0681fdad0 0x55c0681ffef0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/2959773581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe760ff9700 0 client.25565 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/2959773581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 0x55c0681f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/2959773581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f2230 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2959773581 >> 172.21.15.53:0/2959773581 conn(0x55c06812e070 msgr2=0x55c0681c93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2959773581 shutdown_connections 2023-11-06T01:38:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.487+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2959773581 wait complete. 2023-11-06T01:38:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.488+0000 7fe78bb40b80 20 client.25565 trim_cache size 0 max 16384 2023-11-06T01:38:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe78bb40b80 1 Processor -- start 2023-11-06T01:38:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe78bb40b80 1 -- start start 2023-11-06T01:38:24.515 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 0x55c0681f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.515 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.515 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0681f11a0 con 0x55c0681ce530 2023-11-06T01:38:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0681c4500 con 0x55c0681cee90 2023-11-06T01:38:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0681c91f0 con 0x55c0681dce00 2023-11-06T01:38:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe77cf4b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe77cf4b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52654/0 (socket says 172.21.15.53:52654) 2023-11-06T01:38:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe77cf4b700 1 -- 172.21.15.53:0/2113539957 learned_addr learned my addr 172.21.15.53:0/2113539957 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.513+0000 7fe77df4d700 1 --2- 172.21.15.53:0/2113539957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.514+0000 7fe77d74c700 1 --2- 172.21.15.53:0/2113539957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 0x55c0681f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.514+0000 7fe77cf4b700 1 -- 172.21.15.53:0/2113539957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 msgr2=0x55c0681f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.514+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/2113539957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681f1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.514+0000 7fe77cf4b700 1 -- 172.21.15.53:0/2113539957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 msgr2=0x55c0681f9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.514+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/2113539957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 0x55c0681f9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.514+0000 7fe77cf4b700 1 -- 172.21.15.53:0/2113539957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c067f741f0 con 0x55c0681cee90 2023-11-06T01:38:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.514+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/2113539957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f9230 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe768098fc0 tx=0x7fe768009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44afea27dfad32f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.514+0000 7fe763fff700 1 -- 172.21.15.53:0/2113539957 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680992c0 con 0x55c0681cee90 2023-11-06T01:38:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.514+0000 7fe763fff700 1 -- 172.21.15.53:0/2113539957 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c06814acb0 con 0x55c0681cee90 2023-11-06T01:38:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.514+0000 7fe763fff700 1 -- 172.21.15.53:0/2113539957 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680a0fc0 con 0x55c0681cee90 2023-11-06T01:38:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.515+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2113539957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 msgr2=0x55c0681f9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.515+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/2113539957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f9230 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe768098fc0 tx=0x7fe768009550 comp rx=0 tx=0).stop 2023-11-06T01:38:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.515+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2113539957 shutdown_connections 2023-11-06T01:38:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.515+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/2113539957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.515+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/2113539957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681ce530 0x55c0681f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.515+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/2113539957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681cee90 0x55c0681f9230 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.515+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2113539957 >> 172.21.15.53:0/2113539957 conn(0x55c06812e070 msgr2=0x55c0681f0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.516+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2113539957 shutdown_connections 2023-11-06T01:38:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.516+0000 7fe78bb40b80 1 -- 172.21.15.53:0/2113539957 wait complete. 2023-11-06T01:38:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.517+0000 7fe78bb40b80 1 -- start start 2023-11-06T01:38:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.517+0000 7fe78bb40b80 1 Processor -- start 2023-11-06T01:38:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.517+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681ce530 0x55c0681f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.517+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681cee90 0x55c0681f9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.517+0000 7fe78bb40b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.517+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0681f79b0 con 0x55c0681cee90 2023-11-06T01:38:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.517+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0681f7b20 con 0x55c0681ce530 2023-11-06T01:38:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.517+0000 7fe78bb40b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0681f7ce0 con 0x55c0681dce00 2023-11-06T01:38:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.517+0000 7fe78bb40b80 10 client.0 _ll_register_callbacks cb 0x7fe768099090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.517+0000 7fe77d74c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681ce530 0x55c0681f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe77d74c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681ce530 0x55c0681f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52660/0 (socket says 172.21.15.53:52660) 2023-11-06T01:38:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe77d74c700 1 -- 172.21.15.53:0/3441529734 learned_addr learned my addr 172.21.15.53:0/3441529734 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681cee90 0x55c0681f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe77d74c700 1 -- 172.21.15.53:0/3441529734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 msgr2=0x55c0681f72a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:38:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe77d74c700 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681f72a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe77d74c700 1 -- 172.21.15.53:0/3441529734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681cee90 msgr2=0x55c0681f9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe77d74c700 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681cee90 0x55c0681f9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe77d74c700 1 -- 172.21.15.53:0/3441529734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c067f36c00 con 0x55c0681ce530 2023-11-06T01:38:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe77d74c700 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681ce530 0x55c0681f8c30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7740098c0 tx=0x7fe77400b370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3722ab60adb8e8e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe760ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:38:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe760ff9700 1 -- 172.21.15.53:0/3441529734 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77402fd60 con 0x55c0681ce530 2023-11-06T01:38:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.518+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3441529734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe77401d040 con 0x55c0681ce530 2023-11-06T01:38:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe760ff9700 1 -- 172.21.15.53:0/3441529734 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe77402f030 con 0x55c0681ce530 2023-11-06T01:38:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe760ff9700 1 -- 172.21.15.53:0/3441529734 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe77402f1f0 con 0x55c0681ce530 2023-11-06T01:38:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe78bb40b80 10 client.25571 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3441529734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c067f4c250 con 0x55c0681ce530 2023-11-06T01:38:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe760ff9700 1 -- 172.21.15.53:0/3441529734 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe77403f080 con 0x55c0681ce530 2023-11-06T01:38:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe760ff9700 1 -- 172.21.15.53:0/3441529734 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe774039030 con 0x55c0681ce530 2023-11-06T01:38:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe78bb40b80 10 client.25571 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe78bb40b80 10 client.25571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe78bb40b80 20 client.25571 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c0682505b0 0x55c0682529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe78bb40b80 4 client.25571 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:38:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.519+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3441529734 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c068118ce0 con 0x55c0682505b0 2023-11-06T01:38:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.520+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c0682505b0 0x55c0682529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.521+0000 7fe77cf4b700 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c0682505b0 0x55c0682529d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd970467b50e3e6 server_cookie=ff8646a592fb363a in_seq=0 out_seq=0 2023-11-06T01:38:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.521+0000 7fe760ff9700 10 client.25571 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.535+0000 7fe760ff9700 1 -- 172.21.15.53:0/3441529734 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c068118ce0 con 0x55c0682505b0 2023-11-06T01:38:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.535+0000 7fe760ff9700 10 client.25571 handle_command_reply: tid=0 2023-11-06T01:38:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe78bb40b80 1 client.25571 shutdown 2023-11-06T01:38:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3441529734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681ce530 msgr2=0x55c0681f8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681ce530 0x55c0681f8c30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7740098c0 tx=0x7fe77400b370 comp rx=0 tx=0).stop 2023-11-06T01:38:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3441529734 shutdown_connections 2023-11-06T01:38:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0681ce530 0x55c0681f8c30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c0682505b0 0x55c0682529d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c0681dce00 0x55c0681f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe760ff9700 0 client.25571 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe78bb40b80 1 --2- 172.21.15.53:0/3441529734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0681cee90 0x55c0681f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3441529734 >> 172.21.15.53:0/3441529734 conn(0x55c06812e070 msgr2=0x55c0681f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3441529734 shutdown_connections 2023-11-06T01:38:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.536+0000 7fe78bb40b80 1 -- 172.21.15.53:0/3441529734 wait complete. 2023-11-06T01:38:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.537+0000 7fe78bb40b80 20 client.25571 trim_cache size 0 max 16384 2023-11-06T01:38:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:38:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:38:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:38:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:38:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.537+0000 7fe77f9b0700 1 -- 172.21.15.53:0/642138699 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7640a05e0 msgr2=0x7fe7640a2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.537+0000 7fe77f9b0700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7640a05e0 0x7fe7640a2aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe770009ec0 tx=0x7fe770009920 comp rx=0 tx=0).stop 2023-11-06T01:38:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.537+0000 7fe77f9b0700 1 -- 172.21.15.53:0/642138699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe778100e90 msgr2=0x7fe778194f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.537+0000 7fe77f9b0700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe778100e90 0x7fe778194f90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe76800e9c0 tx=0x7fe76800ecd0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.538+0000 7fe77f9b0700 1 -- 172.21.15.53:0/642138699 shutdown_connections 2023-11-06T01:38:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.538+0000 7fe77f9b0700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7640a05e0 0x7fe7640a2aa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.538+0000 7fe77f9b0700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778101830 0x7fe778195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.538+0000 7fe77f9b0700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780ffca0 0x7fe778194a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.538+0000 7fe77f9b0700 1 --2- 172.21.15.53:0/642138699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe778100e90 0x7fe778194f90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.538+0000 7fe77f9b0700 1 -- 172.21.15.53:0/642138699 >> 172.21.15.53:0/642138699 conn(0x7fe7780fb230 msgr2=0x7fe7780fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.538+0000 7fe77f9b0700 1 -- 172.21.15.53:0/642138699 shutdown_connections 2023-11-06T01:38:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:24.538+0000 7fe77f9b0700 1 -- 172.21.15.53:0/642138699 wait complete. 2023-11-06T01:38:25.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:25 smithi163 ceph-mon[104416]: pgmap v433: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 759 op/s 2023-11-06T01:38:25.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:25 smithi053 ceph-mon[99472]: pgmap v433: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 759 op/s 2023-11-06T01:38:25.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:25 smithi062 ceph-mon[103924]: pgmap v433: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 759 op/s 2023-11-06T01:38:27.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 250.0 s, 2300.6 tps, lat 43.599 ms stddev 48.529 2023-11-06T01:38:27.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:27 smithi163 ceph-mon[104416]: pgmap v434: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 981 op/s 2023-11-06T01:38:27.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:27 smithi053 ceph-mon[99472]: pgmap v434: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 981 op/s 2023-11-06T01:38:27.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:27 smithi062 ceph-mon[103924]: pgmap v434: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 981 op/s 2023-11-06T01:38:29.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:29 smithi053 ceph-mon[99472]: pgmap v435: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 875 op/s 2023-11-06T01:38:29.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:29 smithi062 ceph-mon[103924]: pgmap v435: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 875 op/s 2023-11-06T01:38:29.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:29 smithi163 ceph-mon[104416]: pgmap v435: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 875 op/s 2023-11-06T01:38:31.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:31 smithi053 ceph-mon[99472]: pgmap v436: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 928 op/s 2023-11-06T01:38:31.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:31 smithi062 ceph-mon[103924]: pgmap v436: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 928 op/s 2023-11-06T01:38:31.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:31 smithi163 ceph-mon[104416]: pgmap v436: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 928 op/s 2023-11-06T01:38:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 255.0 s, 2669.0 tps, lat 37.388 ms stddev 41.336 2023-11-06T01:38:32.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:32 smithi053 ceph-mon[99472]: pgmap v437: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 25 MiB/s wr, 1.14k op/s 2023-11-06T01:38:32.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:32 smithi062 ceph-mon[103924]: pgmap v437: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 25 MiB/s wr, 1.14k op/s 2023-11-06T01:38:32.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:32 smithi163 ceph-mon[104416]: pgmap v437: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 25 MiB/s wr, 1.14k op/s 2023-11-06T01:38:35.003 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:34 smithi163 ceph-mon[104416]: pgmap v438: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 MiB/s wr, 1.00k op/s 2023-11-06T01:38:35.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:34 smithi053 ceph-mon[99472]: pgmap v438: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 MiB/s wr, 1.00k op/s 2023-11-06T01:38:35.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:34 smithi062 ceph-mon[103924]: pgmap v438: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 MiB/s wr, 1.00k op/s 2023-11-06T01:38:36.015 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:38:36.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.220+0000 7f405490c700 1 -- 172.21.15.53:0/2645312247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40501097e0 msgr2=0x7f4050110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:36.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.220+0000 7f405490c700 1 --2- 172.21.15.53:0/2645312247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40501097e0 0x7f4050110090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f404800c210 tx=0x7f404800c520 comp rx=0 tx=0).stop 2023-11-06T01:38:36.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.221+0000 7f405490c700 1 -- 172.21.15.53:0/2645312247 shutdown_connections 2023-11-06T01:38:36.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.221+0000 7f405490c700 1 --2- 172.21.15.53:0/2645312247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40501097e0 0x7f4050110090 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.221+0000 7f405490c700 1 --2- 172.21.15.53:0/2645312247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4050108130 0x7f4050108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.221+0000 7f405490c700 1 --2- 172.21.15.53:0/2645312247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4050108e40 0x7f40501092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.221+0000 7f405490c700 1 -- 172.21.15.53:0/2645312247 >> 172.21.15.53:0/2645312247 conn(0x7f405006db40 msgr2=0x7f405006ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.221+0000 7f405490c700 1 -- 172.21.15.53:0/2645312247 shutdown_connections 2023-11-06T01:38:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.221+0000 7f405490c700 1 -- 172.21.15.53:0/2645312247 wait complete. 2023-11-06T01:38:36.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.221+0000 7f405490c700 1 Processor -- start 2023-11-06T01:38:36.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.222+0000 7f405490c700 1 -- start start 2023-11-06T01:38:36.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.222+0000 7f405490c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4050108130 0x7f405019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.222+0000 7f405490c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4050108e40 0x7f405019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.222+0000 7f405490c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40501097e0 0x7f40501a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.222+0000 7f405490c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501a4ed0 con 0x7f40501097e0 2023-11-06T01:38:36.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.222+0000 7f405490c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501a5010 con 0x7f4050108130 2023-11-06T01:38:36.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.222+0000 7f405490c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501a51a0 con 0x7f4050108e40 2023-11-06T01:38:36.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.222+0000 7f404ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40501097e0 0x7f40501a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.222+0000 7f404ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40501097e0 0x7f40501a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41236/0 (socket says 172.21.15.53:41236) 2023-11-06T01:38:36.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.222+0000 7f404ed9d700 1 -- 172.21.15.53:0/3152562911 learned_addr learned my addr 172.21.15.53:0/3152562911 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:36.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.223+0000 7f404ed9d700 1 -- 172.21.15.53:0/3152562911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4050108e40 msgr2=0x7f405019d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:36.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.223+0000 7f404ed9d700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4050108e40 0x7f405019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.223+0000 7f404ed9d700 1 -- 172.21.15.53:0/3152562911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4050108130 msgr2=0x7f405019d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:36.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.223+0000 7f404ed9d700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4050108130 0x7f405019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.223+0000 7f404ed9d700 1 -- 172.21.15.53:0/3152562911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404800ae10 con 0x7f40501097e0 2023-11-06T01:38:36.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.223+0000 7f404ed9d700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40501097e0 0x7f40501a4900 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f404800c9a0 tx=0x7f4048009ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.225+0000 7f403f7fe700 1 -- 172.21.15.53:0/3152562911 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404800f040 con 0x7f40501097e0 2023-11-06T01:38:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.225+0000 7f403f7fe700 1 -- 172.21.15.53:0/3152562911 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404800a2d0 con 0x7f40501097e0 2023-11-06T01:38:36.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.225+0000 7f405490c700 1 -- 172.21.15.53:0/3152562911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40501a5420 con 0x7f40501097e0 2023-11-06T01:38:36.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.225+0000 7f403f7fe700 1 -- 172.21.15.53:0/3152562911 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404801c6b0 con 0x7f40501097e0 2023-11-06T01:38:36.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.225+0000 7f405490c700 1 -- 172.21.15.53:0/3152562911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40501a5970 con 0x7f40501097e0 2023-11-06T01:38:36.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.228+0000 7f403f7fe700 1 -- 172.21.15.53:0/3152562911 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f40480074a0 con 0x7f40501097e0 2023-11-06T01:38:36.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.229+0000 7f403f7fe700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a49b0 0x7f40380a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.229+0000 7f403f7fe700 1 -- 172.21.15.53:0/3152562911 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f40480cccb0 con 0x7f40501097e0 2023-11-06T01:38:36.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.229+0000 7f404e59c700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a49b0 0x7f40380a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.230+0000 7f404e59c700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a49b0 0x7f40380a6e70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4050072150 tx=0x7f4040006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:36.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.230+0000 7f4060a9cb80 1 Processor -- start 2023-11-06T01:38:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.230+0000 7f4060a9cb80 1 -- start start 2023-11-06T01:38:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931bc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931bae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93112a10 con 0x55cd931bc0f0 2023-11-06T01:38:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd931103d0 con 0x55cd931bb3f0 2023-11-06T01:38:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9310bcc0 con 0x55cd931bca70 2023-11-06T01:38:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f404e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f404e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931bc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41240/0 (socket says 172.21.15.53:41240) 2023-11-06T01:38:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f404e59c700 1 -- 172.21.15.53:0/2214650249 learned_addr learned my addr 172.21.15.53:0/2214650249 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f404ed9d700 1 --2- 172.21.15.53:0/2214650249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f404dd9b700 1 --2- 172.21.15.53:0/2214650249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931bae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f404e59c700 1 -- 172.21.15.53:0/2214650249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 msgr2=0x55cd931bae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f404e59c700 1 --2- 172.21.15.53:0/2214650249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931bae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f404e59c700 1 -- 172.21.15.53:0/2214650249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 msgr2=0x55cd931bb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f404e59c700 1 --2- 172.21.15.53:0/2214650249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931bb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f404e59c700 1 -- 172.21.15.53:0/2214650249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd92f39de0 con 0x55cd931bc0f0 2023-11-06T01:38:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f404e59c700 1 --2- 172.21.15.53:0/2214650249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931bc530 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4040015950 tx=0x7f4040006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c64e97a38deec9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.231+0000 7f403cff9700 1 -- 172.21.15.53:0/2214650249 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404001a070 con 0x55cd931bc0f0 2023-11-06T01:38:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.232+0000 7f403cff9700 1 -- 172.21.15.53:0/2214650249 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4040012950 con 0x55cd931bc0f0 2023-11-06T01:38:36.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.232+0000 7f403cff9700 1 -- 172.21.15.53:0/2214650249 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040013920 con 0x55cd931bc0f0 2023-11-06T01:38:36.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.233+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2214650249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 msgr2=0x55cd931bc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:36.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.233+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2214650249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931bc530 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4040015950 tx=0x7f4040006f90 comp rx=0 tx=0).stop 2023-11-06T01:38:36.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.233+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2214650249 shutdown_connections 2023-11-06T01:38:36.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.233+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2214650249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931bb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.233+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2214650249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931bc530 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.233+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2214650249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931bae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.233+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2214650249 >> 172.21.15.53:0/2214650249 conn(0x55cd9311a0d0 msgr2=0x55cd931b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:36.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.233+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2214650249 shutdown_connections 2023-11-06T01:38:36.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.234+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2214650249 wait complete. 2023-11-06T01:38:36.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.234+0000 7f4060a9cb80 1 -- start start 2023-11-06T01:38:36.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.234+0000 7f4060a9cb80 1 Processor -- start 2023-11-06T01:38:36.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd930a8510 con 0x55cd931bc0f0 2023-11-06T01:38:36.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd930a81b0 con 0x55cd931bb3f0 2023-11-06T01:38:36.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd92f8f920 con 0x55cd931bca70 2023-11-06T01:38:36.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f4060a9cb80 10 client.0 _ll_register_callbacks cb 0x55cd92ff5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:36.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41246/0 (socket says 172.21.15.53:41246) 2023-11-06T01:38:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52684/0 (socket says 172.21.15.53:52684) 2023-11-06T01:38:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404dd9b700 1 -- 172.21.15.53:0/3603777351 learned_addr learned my addr 172.21.15.53:0/3603777351 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404ed9d700 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404dd9b700 1 -- 172.21.15.53:0/3603777351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 msgr2=0x55cd931d47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:36.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404dd9b700 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931d47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404dd9b700 1 -- 172.21.15.53:0/3603777351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 msgr2=0x55cd931cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:36.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404dd9b700 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404dd9b700 1 -- 172.21.15.53:0/3603777351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4040015630 con 0x55cd931bc0f0 2023-11-06T01:38:36.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.235+0000 7f404ed9d700 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931d47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:36.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.236+0000 7f404dd9b700 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931cd780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f404400e450 tx=0x7f404400e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd6f5fab402edde server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:36.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.236+0000 7f4035ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:38:36.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.236+0000 7f4035ffb700 1 -- 172.21.15.53:0/3603777351 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044009550 con 0x55cd931bc0f0 2023-11-06T01:38:36.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.236+0000 7f4035ffb700 1 -- 172.21.15.53:0/3603777351 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4044013470 con 0x55cd931bc0f0 2023-11-06T01:38:36.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.236+0000 7f4060a9cb80 1 -- 172.21.15.53:0/3603777351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd93136d10 con 0x55cd931bc0f0 2023-11-06T01:38:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.236+0000 7f4035ffb700 1 -- 172.21.15.53:0/3603777351 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4044013670 con 0x55cd931bc0f0 2023-11-06T01:38:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.237+0000 7f4060a9cb80 10 client.16002 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.237+0000 7f4060a9cb80 1 -- 172.21.15.53:0/3603777351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55cd92f22c60 con 0x55cd931bc0f0 2023-11-06T01:38:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.237+0000 7f4035ffb700 1 -- 172.21.15.53:0/3603777351 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd93136d10 con 0x55cd931bc0f0 2023-11-06T01:38:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.237+0000 7f4035ffb700 1 -- 172.21.15.53:0/3603777351 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f4044011070 con 0x55cd931bc0f0 2023-11-06T01:38:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.237+0000 7f4060a9cb80 10 client.16002 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.237+0000 7f4060a9cb80 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.237+0000 7f4060a9cb80 20 client.16002 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.237+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cd931d9120 0x55cd931db540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.237+0000 7f4060a9cb80 4 client.16002 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:38:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.237+0000 7f4060a9cb80 1 -- 172.21.15.53:0/3603777351 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd93189e10 con 0x55cd931d9120 2023-11-06T01:38:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.237+0000 7f404e59c700 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cd931d9120 0x55cd931db540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.238+0000 7f4035ffb700 10 client.16002 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.238+0000 7f404e59c700 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cd931d9120 0x55cd931db540 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d0419a99c51d8f8 server_cookie=a5f9016365b2c70d in_seq=0 out_seq=0 2023-11-06T01:38:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.238+0000 7f4035ffb700 1 -- 172.21.15.53:0/3603777351 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cd93189e10 con 0x55cd931d9120 2023-11-06T01:38:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.238+0000 7f4035ffb700 10 client.16002 handle_command_reply: tid=0 2023-11-06T01:38:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.238+0000 7f4060a9cb80 1 client.16002 shutdown 2023-11-06T01:38:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.238+0000 7f4060a9cb80 1 -- 172.21.15.53:0/3603777351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 msgr2=0x55cd931cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.238+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931cd780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f404400e450 tx=0x7f404400e760 comp rx=0 tx=0).stop 2023-11-06T01:38:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.239+0000 7f4060a9cb80 1 -- 172.21.15.53:0/3603777351 shutdown_connections 2023-11-06T01:38:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.239+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.239+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cd931d9120 0x55cd931db540 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.239+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931d47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.239+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/3603777351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931cd780 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.239+0000 7f4035ffb700 0 client.16002 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:36.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.239+0000 7f4060a9cb80 1 -- 172.21.15.53:0/3603777351 >> 172.21.15.53:0/3603777351 conn(0x55cd9311a0d0 msgr2=0x55cd931b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.239+0000 7f4060a9cb80 1 -- 172.21.15.53:0/3603777351 shutdown_connections 2023-11-06T01:38:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.239+0000 7f4060a9cb80 1 -- 172.21.15.53:0/3603777351 wait complete. 2023-11-06T01:38:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.239+0000 7f4060a9cb80 20 client.16002 trim_cache size 0 max 16384 2023-11-06T01:38:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.260+0000 7f4060a9cb80 1 Processor -- start 2023-11-06T01:38:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.260+0000 7f4060a9cb80 1 -- start start 2023-11-06T01:38:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931bae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93136d10 con 0x55cd931bc0f0 2023-11-06T01:38:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd931b6440 con 0x55cd931bb3f0 2023-11-06T01:38:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd931b6610 con 0x55cd931bca70 2023-11-06T01:38:36.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f404e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931bae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f404e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931bae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52690/0 (socket says 172.21.15.53:52690) 2023-11-06T01:38:36.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f404e59c700 1 -- 172.21.15.53:0/2605204582 learned_addr learned my addr 172.21.15.53:0/2605204582 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:36.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f404ed9d700 1 --2- 172.21.15.53:0/2605204582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931bf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f404dd9b700 1 --2- 172.21.15.53:0/2605204582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f404e59c700 1 -- 172.21.15.53:0/2605204582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 msgr2=0x55cd931bf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:36.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f404e59c700 1 --2- 172.21.15.53:0/2605204582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931bf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f404e59c700 1 -- 172.21.15.53:0/2605204582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 msgr2=0x55cd931d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:36.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f404e59c700 1 --2- 172.21.15.53:0/2605204582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.261+0000 7f404e59c700 1 -- 172.21.15.53:0/2605204582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd92f60250 con 0x55cd931bb3f0 2023-11-06T01:38:36.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.262+0000 7f404e59c700 1 --2- 172.21.15.53:0/2605204582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931bae60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4040015950 tx=0x7f4040004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e98cf03aa38d23 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:36.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.262+0000 7f403d7fa700 1 -- 172.21.15.53:0/2605204582 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404001a040 con 0x55cd931bb3f0 2023-11-06T01:38:36.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.262+0000 7f403d7fa700 1 -- 172.21.15.53:0/2605204582 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4040012ac0 con 0x55cd931bb3f0 2023-11-06T01:38:36.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.262+0000 7f403d7fa700 1 -- 172.21.15.53:0/2605204582 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404002c950 con 0x55cd931bb3f0 2023-11-06T01:38:36.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.263+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2605204582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 msgr2=0x55cd931bae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:36.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.263+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2605204582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931bae60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4040015950 tx=0x7f4040004860 comp rx=0 tx=0).stop 2023-11-06T01:38:36.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.263+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2605204582 shutdown_connections 2023-11-06T01:38:36.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.263+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2605204582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931bae60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.263+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2605204582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bca70 0x55cd931bf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.263+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2605204582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bc0f0 0x55cd931d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.263+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2605204582 >> 172.21.15.53:0/2605204582 conn(0x55cd9311a0d0 msgr2=0x55cd931cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:36.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.263+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2605204582 shutdown_connections 2023-11-06T01:38:36.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.263+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2605204582 wait complete. 2023-11-06T01:38:36.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.263+0000 7f4060a9cb80 1 -- start start 2023-11-06T01:38:36.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f4060a9cb80 1 Processor -- start 2023-11-06T01:38:36.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931c3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bc0f0 0x55cd931c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f4060a9cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bca70 0x55cd931ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd931caf20 con 0x55cd931bca70 2023-11-06T01:38:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd931cb090 con 0x55cd931bb3f0 2023-11-06T01:38:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f4060a9cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd931cb250 con 0x55cd931bc0f0 2023-11-06T01:38:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f4060a9cb80 10 client.0 _ll_register_callbacks cb 0x7f4044009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bc0f0 0x55cd931c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bca70 0x55cd931ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bc0f0 0x55cd931c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55500/0 (socket says 172.21.15.53:55500) 2023-11-06T01:38:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404dd9b700 1 -- 172.21.15.53:0/2693864299 learned_addr learned my addr 172.21.15.53:0/2693864299 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:36.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931c3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52696/0 (socket says 172.21.15.53:52696) 2023-11-06T01:38:36.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404ed9d700 1 -- 172.21.15.53:0/2693864299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bc0f0 msgr2=0x55cd931c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:36.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404ed9d700 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bc0f0 0x55cd931c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404ed9d700 1 -- 172.21.15.53:0/2693864299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 msgr2=0x55cd931c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:36.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404ed9d700 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404ed9d700 1 -- 172.21.15.53:0/2693864299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd92f39de0 con 0x55cd931bca70 2023-11-06T01:38:36.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.264+0000 7f404dd9b700 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bc0f0 0x55cd931c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:36.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.265+0000 7f404ed9d700 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bca70 0x55cd931ca9e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f40480cab00 tx=0x7f404800a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997da45e297047ce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:36.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.265+0000 7f4035ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:38:36.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.266+0000 7f4035ffb700 1 -- 172.21.15.53:0/2693864299 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404809e3b0 con 0x55cd931bca70 2023-11-06T01:38:36.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.266+0000 7f4035ffb700 1 -- 172.21.15.53:0/2693864299 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4048096040 con 0x55cd931bca70 2023-11-06T01:38:36.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.266+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2693864299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd93104d40 con 0x55cd931bca70 2023-11-06T01:38:36.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.266+0000 7f4035ffb700 1 -- 172.21.15.53:0/2693864299 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40480a2560 con 0x55cd931bca70 2023-11-06T01:38:36.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.266+0000 7f4060a9cb80 10 client.16008 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:36.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.266+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2693864299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55cd92f382b0 con 0x55cd931bca70 2023-11-06T01:38:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.266+0000 7f4035ffb700 1 -- 172.21.15.53:0/2693864299 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd93104d40 con 0x55cd931bca70 2023-11-06T01:38:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.267+0000 7f4035ffb700 1 -- 172.21.15.53:0/2693864299 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f404809a7e0 con 0x55cd931bca70 2023-11-06T01:38:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.267+0000 7f4060a9cb80 10 client.16008 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.267+0000 7f4060a9cb80 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.267+0000 7f4060a9cb80 20 client.16008 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.267+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cd93234520 0x55cd93236940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.267+0000 7f4060a9cb80 4 client.16008 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:38:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.267+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2693864299 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd93104a60 con 0x55cd93234520 2023-11-06T01:38:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.267+0000 7f404e59c700 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cd93234520 0x55cd93236940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:36.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.268+0000 7f404e59c700 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cd93234520 0x55cd93236940 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63584a06656825e1 server_cookie=9b0d4f7821860cec in_seq=0 out_seq=0 2023-11-06T01:38:36.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.268+0000 7f4035ffb700 10 client.16008 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:36.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.268+0000 7f4035ffb700 1 -- 172.21.15.53:0/2693864299 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55cd93104a60 con 0x55cd93234520 2023-11-06T01:38:36.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.268+0000 7f4035ffb700 10 client.16008 handle_command_reply: tid=0 2023-11-06T01:38:36.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.268+0000 7f4060a9cb80 1 client.16008 shutdown 2023-11-06T01:38:36.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2693864299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bca70 msgr2=0x55cd931ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:36.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bca70 0x55cd931ca9e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f40480cab00 tx=0x7f404800a8f0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2693864299 shutdown_connections 2023-11-06T01:38:36.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd931bb3f0 0x55cd931c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55cd93234520 0x55cd93236940 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd931bca70 0x55cd931ca9e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4035ffb700 0 client.16008 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:36.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4060a9cb80 1 --2- 172.21.15.53:0/2693864299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cd931bc0f0 0x55cd931c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2693864299 >> 172.21.15.53:0/2693864299 conn(0x55cd9311a0d0 msgr2=0x55cd931cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:36.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2693864299 shutdown_connections 2023-11-06T01:38:36.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4060a9cb80 1 -- 172.21.15.53:0/2693864299 wait complete. 2023-11-06T01:38:36.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f4060a9cb80 20 client.16008 trim_cache size 0 max 16384 2023-11-06T01:38:36.286 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:38:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:38:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:38:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:38:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:38:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:38:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:38:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:38:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:38:36.288 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:38:36.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 -- 172.21.15.53:0/3152562911 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a49b0 msgr2=0x7f40380a6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:36.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a49b0 0x7f40380a6e70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4050072150 tx=0x7f4040006d20 comp rx=0 tx=0).stop 2023-11-06T01:38:36.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 -- 172.21.15.53:0/3152562911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40501097e0 msgr2=0x7f40501a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:36.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40501097e0 0x7f40501a4900 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f404800c9a0 tx=0x7f4048009ff0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 -- 172.21.15.53:0/3152562911 shutdown_connections 2023-11-06T01:38:36.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a49b0 0x7f40380a6e70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4050108130 0x7f405019d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40501097e0 0x7f40501a4900 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 --2- 172.21.15.53:0/3152562911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4050108e40 0x7f405019d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:36.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 -- 172.21.15.53:0/3152562911 >> 172.21.15.53:0/3152562911 conn(0x7f405006db40 msgr2=0x7f405006fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:36.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 -- 172.21.15.53:0/3152562911 shutdown_connections 2023-11-06T01:38:36.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:38:36.269+0000 7f405490c700 1 -- 172.21.15.53:0/3152562911 wait complete. 2023-11-06T01:38:36.293 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:38:37.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 260.0 s, 1828.8 tps, lat 54.709 ms stddev 107.282 2023-11-06T01:38:37.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:37 smithi062 ceph-mon[103924]: pgmap v439: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 MiB/s wr, 1.81k op/s 2023-11-06T01:38:37.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:37 smithi163 ceph-mon[104416]: pgmap v439: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 MiB/s wr, 1.81k op/s 2023-11-06T01:38:37.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:37 smithi053 ceph-mon[99472]: pgmap v439: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 MiB/s wr, 1.81k op/s 2023-11-06T01:38:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.695+0000 7fa45b3f8700 1 -- 172.21.15.53:0/3967798661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4540ffc90 msgr2=0x7fa4541000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.695+0000 7fa45b3f8700 1 --2- 172.21.15.53:0/3967798661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4540ffc90 0x7fa4541000b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa4500099c0 tx=0x7fa450009cd0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.695+0000 7fa45b3f8700 1 -- 172.21.15.53:0/3967798661 shutdown_connections 2023-11-06T01:38:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.695+0000 7fa45b3f8700 1 --2- 172.21.15.53:0/3967798661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa454101820 0x7fa4541080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.695+0000 7fa45b3f8700 1 --2- 172.21.15.53:0/3967798661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4540ffc90 0x7fa4541000b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.695+0000 7fa45b3f8700 1 --2- 172.21.15.53:0/3967798661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa454100e80 0x7fa4541012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.695+0000 7fa45b3f8700 1 -- 172.21.15.53:0/3967798661 >> 172.21.15.53:0/3967798661 conn(0x7fa4540fb1e0 msgr2=0x7fa4540fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.696+0000 7fa45b3f8700 1 -- 172.21.15.53:0/3967798661 shutdown_connections 2023-11-06T01:38:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.696+0000 7fa45b3f8700 1 -- 172.21.15.53:0/3967798661 wait complete. 2023-11-06T01:38:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.696+0000 7fa45b3f8700 1 Processor -- start 2023-11-06T01:38:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.696+0000 7fa45b3f8700 1 -- start start 2023-11-06T01:38:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.696+0000 7fa45b3f8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4540ffc90 0x7fa45419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.696+0000 7fa45b3f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa454100e80 0x7fa45419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.696+0000 7fa45b3f8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa454101820 0x7fa45419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.696+0000 7fa45b3f8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45419dc20 con 0x7fa454101820 2023-11-06T01:38:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa45b3f8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45419dd90 con 0x7fa4540ffc90 2023-11-06T01:38:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa45b3f8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45419df50 con 0x7fa454100e80 2023-11-06T01:38:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa459995700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa454101820 0x7fa45419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa459194700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4540ffc90 0x7fa45419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa458993700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa454100e80 0x7fa45419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa459995700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa454101820 0x7fa45419d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41276/0 (socket says 172.21.15.53:41276) 2023-11-06T01:38:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa459995700 1 -- 172.21.15.53:0/790777631 learned_addr learned my addr 172.21.15.53:0/790777631 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa459194700 1 -- 172.21.15.53:0/790777631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa454100e80 msgr2=0x7fa45419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa459194700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa454100e80 0x7fa45419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa459194700 1 -- 172.21.15.53:0/790777631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa454101820 msgr2=0x7fa45419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa459194700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa454101820 0x7fa45419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa459194700 1 -- 172.21.15.53:0/790777631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500095f0 con 0x7fa4540ffc90 2023-11-06T01:38:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa458993700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa454100e80 0x7fa45419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa459995700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa454101820 0x7fa45419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa459194700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4540ffc90 0x7fa45419e850 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa4500058e0 tx=0x7fa45000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa44a7fc700 1 -- 172.21.15.53:0/790777631 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45001d070 con 0x7fa4540ffc90 2023-11-06T01:38:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa44a7fc700 1 -- 172.21.15.53:0/790777631 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa450005020 con 0x7fa4540ffc90 2023-11-06T01:38:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa45b3f8700 1 -- 172.21.15.53:0/790777631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45419e1d0 con 0x7fa4540ffc90 2023-11-06T01:38:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.697+0000 7fa44a7fc700 1 -- 172.21.15.53:0/790777631 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa450004a90 con 0x7fa4540ffc90 2023-11-06T01:38:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.698+0000 7fa45b3f8700 1 -- 172.21.15.53:0/790777631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4541a57f0 con 0x7fa4540ffc90 2023-11-06T01:38:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.702+0000 7fa44a7fc700 1 -- 172.21.15.53:0/790777631 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa450005400 con 0x7fa4540ffc90 2023-11-06T01:38:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.702+0000 7fa467588b80 1 Processor -- start 2023-11-06T01:38:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.702+0000 7fa467588b80 1 -- start start 2023-11-06T01:38:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.702+0000 7fa467588b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686e86aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.702+0000 7fa467588b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e86fe0 0x560686e878f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.702+0000 7fa467588b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e87ed0 0x560686e88350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa467588b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686dde9b0 con 0x560686e86680 2023-11-06T01:38:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa467588b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686ddc370 con 0x560686e86fe0 2023-11-06T01:38:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa467588b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686dd7c60 con 0x560686e87ed0 2023-11-06T01:38:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa44a7fc700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4400a0520 0x7fa4400a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa459194700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e86fe0 0x560686e878f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa459995700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686e86aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa458993700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e87ed0 0x560686e88350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa459995700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686e86aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41278/0 (socket says 172.21.15.53:41278) 2023-11-06T01:38:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa459995700 1 -- 172.21.15.53:0/3471969241 learned_addr learned my addr 172.21.15.53:0/3471969241 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa459995700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4400a0520 0x7fa4400a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa44a7fc700 1 -- 172.21.15.53:0/790777631 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa4500cc5d0 con 0x7fa4540ffc90 2023-11-06T01:38:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa459995700 1 -- 172.21.15.53:0/3471969241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e87ed0 msgr2=0x560686e88350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa459995700 1 --2- 172.21.15.53:0/3471969241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e87ed0 0x560686e88350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa459995700 1 -- 172.21.15.53:0/3471969241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e86fe0 msgr2=0x560686e878f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa459995700 1 --2- 172.21.15.53:0/3471969241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e86fe0 0x560686e878f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.703+0000 7fa459995700 1 -- 172.21.15.53:0/3471969241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560686c05d80 con 0x560686e86680 2023-11-06T01:38:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.704+0000 7fa459995700 1 --2- 172.21.15.53:0/3471969241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686e86aa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009fd0 tx=0x7fa44c00e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a9973fb96ce240 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.704+0000 7fa459995700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4400a0520 0x7fa4400a29e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009d30 tx=0x7fa44c01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.704+0000 7fa43f7fe700 1 -- 172.21.15.53:0/3471969241 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c009370 con 0x560686e86680 2023-11-06T01:38:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.704+0000 7fa43f7fe700 1 -- 172.21.15.53:0/3471969241 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa44c00f040 con 0x560686e86680 2023-11-06T01:38:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.704+0000 7fa43f7fe700 1 -- 172.21.15.53:0/3471969241 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c023940 con 0x560686e86680 2023-11-06T01:38:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.705+0000 7fa467588b80 1 -- 172.21.15.53:0/3471969241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 msgr2=0x560686e86aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.705+0000 7fa467588b80 1 --2- 172.21.15.53:0/3471969241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686e86aa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009fd0 tx=0x7fa44c00e8a0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.705+0000 7fa467588b80 1 -- 172.21.15.53:0/3471969241 shutdown_connections 2023-11-06T01:38:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.705+0000 7fa467588b80 1 --2- 172.21.15.53:0/3471969241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e87ed0 0x560686e88350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.705+0000 7fa467588b80 1 --2- 172.21.15.53:0/3471969241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686e86aa0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.705+0000 7fa467588b80 1 --2- 172.21.15.53:0/3471969241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e86fe0 0x560686e878f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.705+0000 7fa467588b80 1 -- 172.21.15.53:0/3471969241 >> 172.21.15.53:0/3471969241 conn(0x560686de6070 msgr2=0x560686e83040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.705+0000 7fa467588b80 1 -- 172.21.15.53:0/3471969241 shutdown_connections 2023-11-06T01:38:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.705+0000 7fa467588b80 1 -- 172.21.15.53:0/3471969241 wait complete. 2023-11-06T01:38:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.706+0000 7fa467588b80 1 -- start start 2023-11-06T01:38:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.706+0000 7fa467588b80 1 Processor -- start 2023-11-06T01:38:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.706+0000 7fa467588b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86680 0x560686e904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.706+0000 7fa467588b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e86fe0 0x560686e90a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.706+0000 7fa467588b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e87ed0 0x560686e97a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.706+0000 7fa467588b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686d744b0 con 0x560686e87ed0 2023-11-06T01:38:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.706+0000 7fa467588b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686d74150 con 0x560686e86fe0 2023-11-06T01:38:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.706+0000 7fa467588b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686c5b8c0 con 0x560686e86680 2023-11-06T01:38:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.706+0000 7fa467588b80 10 client.0 _ll_register_callbacks cb 0x560686cc1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.706+0000 7fa459995700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e87ed0 0x560686e97a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa459194700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86680 0x560686e904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa459995700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e87ed0 0x560686e97a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41290/0 (socket says 172.21.15.53:41290) 2023-11-06T01:38:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa459995700 1 -- 172.21.15.53:0/1568165578 learned_addr learned my addr 172.21.15.53:0/1568165578 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa458993700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e86fe0 0x560686e90a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa459995700 1 -- 172.21.15.53:0/1568165578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86680 msgr2=0x560686e904c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa459995700 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86680 0x560686e904c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa459995700 1 -- 172.21.15.53:0/1568165578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e86fe0 msgr2=0x560686e90a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa459995700 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e86fe0 0x560686e90a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa459995700 1 -- 172.21.15.53:0/1568165578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c008c60 con 0x560686e87ed0 2023-11-06T01:38:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa458993700 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e86fe0 0x560686e90a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa459995700 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e87ed0 0x560686e97a40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009990 tx=0x7fa44c01bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802a5d12b9e2b48 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa43cff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:38:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa43cff9700 1 -- 172.21.15.53:0/1568165578 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c004c90 con 0x560686e87ed0 2023-11-06T01:38:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa43cff9700 1 -- 172.21.15.53:0/1568165578 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa44c004df0 con 0x560686e87ed0 2023-11-06T01:38:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa467588b80 1 -- 172.21.15.53:0/1568165578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560686e02cb0 con 0x560686e87ed0 2023-11-06T01:38:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa43cff9700 1 -- 172.21.15.53:0/1568165578 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa44c0236c0 con 0x560686e87ed0 2023-11-06T01:38:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa467588b80 10 client.16020 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.707+0000 7fa467588b80 1 -- 172.21.15.53:0/1568165578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560686beec00 con 0x560686e87ed0 2023-11-06T01:38:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.708+0000 7fa43cff9700 1 -- 172.21.15.53:0/1568165578 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa44c03d020 con 0x560686e87ed0 2023-11-06T01:38:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.708+0000 7fa43cff9700 1 -- 172.21.15.53:0/1568165578 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa44c02e880 con 0x560686e87ed0 2023-11-06T01:38:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.708+0000 7fa467588b80 10 client.16020 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.708+0000 7fa467588b80 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.708+0000 7fa467588b80 20 client.16020 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.708+0000 7fa467588b80 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x560686e9c2b0 0x560686e9e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.709+0000 7fa467588b80 4 client.16020 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:38:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.709+0000 7fa467588b80 1 -- 172.21.15.53:0/1568165578 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560686e55db0 con 0x560686e9c2b0 2023-11-06T01:38:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.709+0000 7fa459194700 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x560686e9c2b0 0x560686e9e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.710+0000 7fa459194700 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x560686e9c2b0 0x560686e9e6d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=953513a54a673fb server_cookie=267c2a212973bb50 in_seq=0 out_seq=0 2023-11-06T01:38:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.710+0000 7fa43cff9700 10 client.16020 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.710+0000 7fa43cff9700 1 -- 172.21.15.53:0/1568165578 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560686e55db0 con 0x560686e9c2b0 2023-11-06T01:38:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.710+0000 7fa43cff9700 10 client.16020 handle_command_reply: tid=0 2023-11-06T01:38:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.710+0000 7fa467588b80 1 client.16020 shutdown 2023-11-06T01:38:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa467588b80 1 -- 172.21.15.53:0/1568165578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e87ed0 msgr2=0x560686e97a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa467588b80 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e87ed0 0x560686e97a40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009990 tx=0x7fa44c01bac0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa467588b80 1 -- 172.21.15.53:0/1568165578 shutdown_connections 2023-11-06T01:38:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa467588b80 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x560686e9c2b0 0x560686e9e6d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa467588b80 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e87ed0 0x560686e97a40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa43cff9700 0 client.16020 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa467588b80 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86680 0x560686e904c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa467588b80 1 --2- 172.21.15.53:0/1568165578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e86fe0 0x560686e90a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa467588b80 1 -- 172.21.15.53:0/1568165578 >> 172.21.15.53:0/1568165578 conn(0x560686de6070 msgr2=0x560686e813c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa467588b80 1 -- 172.21.15.53:0/1568165578 shutdown_connections 2023-11-06T01:38:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa467588b80 1 -- 172.21.15.53:0/1568165578 wait complete. 2023-11-06T01:38:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.711+0000 7fa467588b80 20 client.16020 trim_cache size 0 max 16384 2023-11-06T01:38:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa467588b80 1 Processor -- start 2023-11-06T01:38:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa467588b80 1 -- start start 2023-11-06T01:38:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa467588b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86680 0x560686e959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa467588b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86fe0 0x560686e882b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa467588b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e87ed0 0x560686e8f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa467588b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686e797a0 con 0x560686e86fe0 2023-11-06T01:38:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa467588b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686e8f970 con 0x560686e87ed0 2023-11-06T01:38:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa467588b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686e7c500 con 0x560686e86680 2023-11-06T01:38:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa458993700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86fe0 0x560686e882b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa459194700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86680 0x560686e959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa458993700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86fe0 0x560686e882b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41294/0 (socket says 172.21.15.53:41294) 2023-11-06T01:38:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa458993700 1 -- 172.21.15.53:0/662799077 learned_addr learned my addr 172.21.15.53:0/662799077 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.731+0000 7fa459995700 1 --2- 172.21.15.53:0/662799077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e87ed0 0x560686e8f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.732+0000 7fa459995700 1 -- 172.21.15.53:0/662799077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86680 msgr2=0x560686e959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.732+0000 7fa459995700 1 --2- 172.21.15.53:0/662799077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86680 0x560686e959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.732+0000 7fa459995700 1 -- 172.21.15.53:0/662799077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86fe0 msgr2=0x560686e882b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.732+0000 7fa459995700 1 --2- 172.21.15.53:0/662799077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86fe0 0x560686e882b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.732+0000 7fa459995700 1 -- 172.21.15.53:0/662799077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560686c2c1f0 con 0x560686e87ed0 2023-11-06T01:38:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.732+0000 7fa458993700 1 --2- 172.21.15.53:0/662799077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86fe0 0x560686e882b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.732+0000 7fa459194700 1 --2- 172.21.15.53:0/662799077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86680 0x560686e959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.732+0000 7fa459995700 1 --2- 172.21.15.53:0/662799077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e87ed0 0x560686e8f110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa44c02ec60 tx=0x7fa44c00e8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91eff4bdf99c5a29 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.732+0000 7fa43ffff700 1 -- 172.21.15.53:0/662799077 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c008640 con 0x560686e87ed0 2023-11-06T01:38:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.732+0000 7fa43ffff700 1 -- 172.21.15.53:0/662799077 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa44c03d030 con 0x560686e87ed0 2023-11-06T01:38:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.732+0000 7fa43ffff700 1 -- 172.21.15.53:0/662799077 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c0234a0 con 0x560686e87ed0 2023-11-06T01:38:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.733+0000 7fa467588b80 1 -- 172.21.15.53:0/662799077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e87ed0 msgr2=0x560686e8f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.733+0000 7fa467588b80 1 --2- 172.21.15.53:0/662799077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e87ed0 0x560686e8f110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa44c02ec60 tx=0x7fa44c00e8a0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.733+0000 7fa467588b80 1 -- 172.21.15.53:0/662799077 shutdown_connections 2023-11-06T01:38:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.733+0000 7fa467588b80 1 --2- 172.21.15.53:0/662799077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e87ed0 0x560686e8f110 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.733+0000 7fa467588b80 1 --2- 172.21.15.53:0/662799077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86680 0x560686e959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.733+0000 7fa467588b80 1 --2- 172.21.15.53:0/662799077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86fe0 0x560686e882b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.733+0000 7fa467588b80 1 -- 172.21.15.53:0/662799077 >> 172.21.15.53:0/662799077 conn(0x560686de6070 msgr2=0x560686e82c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.734+0000 7fa467588b80 1 -- 172.21.15.53:0/662799077 shutdown_connections 2023-11-06T01:38:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.734+0000 7fa467588b80 1 -- 172.21.15.53:0/662799077 wait complete. 2023-11-06T01:38:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.734+0000 7fa467588b80 1 -- start start 2023-11-06T01:38:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.734+0000 7fa467588b80 1 Processor -- start 2023-11-06T01:38:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa467588b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686ee7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa467588b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86fe0 0x560686eea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa467588b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e87ed0 0x560686eec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa467588b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686eece50 con 0x560686e86680 2023-11-06T01:38:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa467588b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686eecfc0 con 0x560686e87ed0 2023-11-06T01:38:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa467588b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560686eed180 con 0x560686e86fe0 2023-11-06T01:38:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa467588b80 10 client.0 _ll_register_callbacks cb 0x7fa4500961e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa459194700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686ee7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa459194700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686ee7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41298/0 (socket says 172.21.15.53:41298) 2023-11-06T01:38:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa458993700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86fe0 0x560686eea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa459194700 1 -- 172.21.15.53:0/1178283749 learned_addr learned my addr 172.21.15.53:0/1178283749 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa459995700 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e87ed0 0x560686eec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa459194700 1 -- 172.21.15.53:0/1178283749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86fe0 msgr2=0x560686eea2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa459194700 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86fe0 0x560686eea2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa459194700 1 -- 172.21.15.53:0/1178283749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e87ed0 msgr2=0x560686eec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa459194700 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e87ed0 0x560686eec830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa459194700 1 -- 172.21.15.53:0/1178283749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4541a57f0 con 0x560686e86680 2023-11-06T01:38:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa459194700 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686ee7d90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa450009fd0 tx=0x7fa4500c9f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cd813ed88d5dcf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa43cff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:38:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa43cff9700 1 -- 172.21.15.53:0/1178283749 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45009a5a0 con 0x560686e86680 2023-11-06T01:38:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa467588b80 1 -- 172.21.15.53:0/1178283749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa44c00f040 con 0x560686e86680 2023-11-06T01:38:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.735+0000 7fa43cff9700 1 -- 172.21.15.53:0/1178283749 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa450099ea0 con 0x560686e86680 2023-11-06T01:38:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa43cff9700 1 -- 172.21.15.53:0/1178283749 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa45009dcd0 con 0x560686e86680 2023-11-06T01:38:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa467588b80 10 client.16026 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa467588b80 1 -- 172.21.15.53:0/1178283749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fa4440040a0 con 0x560686e86680 2023-11-06T01:38:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa43cff9700 1 -- 172.21.15.53:0/1178283749 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa44c00f040 con 0x560686e86680 2023-11-06T01:38:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa43cff9700 1 -- 172.21.15.53:0/1178283749 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa450099400 con 0x560686e86680 2023-11-06T01:38:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa467588b80 10 client.16026 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa467588b80 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa467588b80 20 client.16026 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa467588b80 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x560686eee770 0x560686ef0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa467588b80 4 client.16026 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:38:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa467588b80 1 -- 172.21.15.53:0/1178283749 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560686dd0ce0 con 0x560686eee770 2023-11-06T01:38:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.736+0000 7fa458993700 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x560686eee770 0x560686ef0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.737+0000 7fa43cff9700 10 client.16026 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.737+0000 7fa458993700 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x560686eee770 0x560686ef0b60 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a47ac8cfe042f062 server_cookie=fa2e9c532b7bbd66 in_seq=0 out_seq=0 2023-11-06T01:38:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.754+0000 7fa43cff9700 1 -- 172.21.15.53:0/1178283749 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560686dd0ce0 con 0x560686eee770 2023-11-06T01:38:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.754+0000 7fa43cff9700 10 client.16026 handle_command_reply: tid=0 2023-11-06T01:38:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa467588b80 1 client.16026 shutdown 2023-11-06T01:38:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa467588b80 1 -- 172.21.15.53:0/1178283749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 msgr2=0x560686ee7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa467588b80 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686ee7d90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa450009fd0 tx=0x7fa4500c9f80 comp rx=0 tx=0).stop 2023-11-06T01:38:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa467588b80 1 -- 172.21.15.53:0/1178283749 shutdown_connections 2023-11-06T01:38:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa467588b80 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560686e87ed0 0x560686eec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa467588b80 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x560686eee770 0x560686ef0b60 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa467588b80 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560686e86680 0x560686ee7d90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa43cff9700 0 client.16026 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa467588b80 1 --2- 172.21.15.53:0/1178283749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560686e86fe0 0x560686eea2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa467588b80 1 -- 172.21.15.53:0/1178283749 >> 172.21.15.53:0/1178283749 conn(0x560686de6070 msgr2=0x560686e8f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa467588b80 1 -- 172.21.15.53:0/1178283749 shutdown_connections 2023-11-06T01:38:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.755+0000 7fa467588b80 1 -- 172.21.15.53:0/1178283749 wait complete. 2023-11-06T01:38:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa467588b80 20 client.16026 trim_cache size 0 max 16384 2023-11-06T01:38:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:38:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:38:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:38:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:38:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 -- 172.21.15.53:0/790777631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4400a0520 msgr2=0x7fa4400a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4400a0520 0x7fa4400a29e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009d30 tx=0x7fa44c01e000 comp rx=0 tx=0).stop 2023-11-06T01:38:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 -- 172.21.15.53:0/790777631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4540ffc90 msgr2=0x7fa45419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4540ffc90 0x7fa45419e850 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa4500058e0 tx=0x7fa45000f690 comp rx=0 tx=0).stop 2023-11-06T01:38:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 -- 172.21.15.53:0/790777631 shutdown_connections 2023-11-06T01:38:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa4400a0520 0x7fa4400a29e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4540ffc90 0x7fa45419e850 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa454101820 0x7fa45419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 --2- 172.21.15.53:0/790777631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa454100e80 0x7fa45419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 -- 172.21.15.53:0/790777631 >> 172.21.15.53:0/790777631 conn(0x7fa4540fb1e0 msgr2=0x7fa4540fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 -- 172.21.15.53:0/790777631 shutdown_connections 2023-11-06T01:38:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:37.756+0000 7fa45b3f8700 1 -- 172.21.15.53:0/790777631 wait complete. 2023-11-06T01:38:39.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:39 smithi062 ceph-mon[103924]: pgmap v440: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 1.76k op/s 2023-11-06T01:38:39.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:39 smithi163 ceph-mon[104416]: pgmap v440: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 1.76k op/s 2023-11-06T01:38:39.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:39 smithi053 ceph-mon[99472]: pgmap v440: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 1.76k op/s 2023-11-06T01:38:41.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:41 smithi062 ceph-mon[103924]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 1.95k op/s 2023-11-06T01:38:41.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:41 smithi163 ceph-mon[104416]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 1.95k op/s 2023-11-06T01:38:41.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:41 smithi053 ceph-mon[99472]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 1.95k op/s 2023-11-06T01:38:42.184 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 265.0 s, 2675.9 tps, lat 37.394 ms stddev 42.599 2023-11-06T01:38:42.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:42 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:38:42.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:42 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:38:42.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:42 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:38:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:43 smithi062 ceph-mon[103924]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 MiB/s wr, 2.03k op/s 2023-11-06T01:38:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:43 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:38:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:43 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:38:43.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:43 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:38:43.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:43 smithi163 ceph-mon[104416]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 MiB/s wr, 2.03k op/s 2023-11-06T01:38:43.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:43 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:38:43.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:43 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:38:43.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:43 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:38:43.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:43 smithi053 ceph-mon[99472]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 MiB/s wr, 2.03k op/s 2023-11-06T01:38:43.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:43 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:38:43.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:43 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:38:43.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:43 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:38:45.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:45 smithi062 ceph-mon[103924]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 MiB/s wr, 1.85k op/s 2023-11-06T01:38:45.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:45 smithi163 ceph-mon[104416]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 MiB/s wr, 1.85k op/s 2023-11-06T01:38:45.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:45 smithi053 ceph-mon[99472]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 MiB/s wr, 1.85k op/s 2023-11-06T01:38:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 270.0 s, 2759.1 tps, lat 36.105 ms stddev 42.577 2023-11-06T01:38:47.497 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:47 smithi163 ceph-mon[104416]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 1.98k op/s 2023-11-06T01:38:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:47 smithi053 ceph-mon[99472]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 1.98k op/s 2023-11-06T01:38:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:47 smithi062 ceph-mon[103924]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 1.98k op/s 2023-11-06T01:38:49.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:49 smithi163 ceph-mon[104416]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2023-11-06T01:38:49.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:49 smithi053 ceph-mon[99472]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2023-11-06T01:38:49.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:49 smithi062 ceph-mon[103924]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2023-11-06T01:38:50.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.880+0000 7f291c881700 1 -- 172.21.15.53:0/2005300771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2918104150 msgr2=0x7f2918106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:50.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.880+0000 7f291c881700 1 --2- 172.21.15.53:0/2005300771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2918104150 0x7f2918106540 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f290c009990 tx=0x7f290c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.880+0000 7f291c881700 1 -- 172.21.15.53:0/2005300771 shutdown_connections 2023-11-06T01:38:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.880+0000 7f291c881700 1 --2- 172.21.15.53:0/2005300771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2918106a80 0x7f2918108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.880+0000 7f291c881700 1 --2- 172.21.15.53:0/2005300771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2918101820 0x7f2918103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.880+0000 7f291c881700 1 --2- 172.21.15.53:0/2005300771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2918104150 0x7f2918106540 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.880+0000 7f291c881700 1 -- 172.21.15.53:0/2005300771 >> 172.21.15.53:0/2005300771 conn(0x7f29180fb210 msgr2=0x7f29180fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.881+0000 7f291c881700 1 -- 172.21.15.53:0/2005300771 shutdown_connections 2023-11-06T01:38:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.881+0000 7f291c881700 1 -- 172.21.15.53:0/2005300771 wait complete. 2023-11-06T01:38:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.882+0000 7f291c881700 1 Processor -- start 2023-11-06T01:38:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.882+0000 7f291c881700 1 -- start start 2023-11-06T01:38:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291c881700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2918101820 0x7f2918196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291c881700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2918104150 0x7f2918197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291c881700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2918106a80 0x7f2918197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291c881700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2918198550 con 0x7f2918106a80 2023-11-06T01:38:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291c881700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29181986c0 con 0x7f2918101820 2023-11-06T01:38:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291c881700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2918198880 con 0x7f2918104150 2023-11-06T01:38:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2918101820 0x7f2918196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2918101820 0x7f2918196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52750/0 (socket says 172.21.15.53:52750) 2023-11-06T01:38:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f2915d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2918104150 0x7f2918197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f2916d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2918106a80 0x7f2918197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291659c700 1 -- 172.21.15.53:0/4085775723 learned_addr learned my addr 172.21.15.53:0/4085775723 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291659c700 1 -- 172.21.15.53:0/4085775723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2918104150 msgr2=0x7f2918197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291659c700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2918104150 0x7f2918197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291659c700 1 -- 172.21.15.53:0/4085775723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2918106a80 msgr2=0x7f2918197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291659c700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2918106a80 0x7f2918197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f291659c700 1 -- 172.21.15.53:0/4085775723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c0095f0 con 0x7f2918101820 2023-11-06T01:38:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f2916d9d700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2918106a80 0x7f2918197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.883+0000 7f2915d9b700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2918104150 0x7f2918197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.884+0000 7f291659c700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2918101820 0x7f2918196b30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f290800e9a0 tx=0x7f290800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.884+0000 7f29077fe700 1 -- 172.21.15.53:0/4085775723 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908009790 con 0x7f2918101820 2023-11-06T01:38:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.884+0000 7f291c881700 1 -- 172.21.15.53:0/4085775723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2918198b60 con 0x7f2918101820 2023-11-06T01:38:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.884+0000 7f29077fe700 1 -- 172.21.15.53:0/4085775723 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2908010430 con 0x7f2918101820 2023-11-06T01:38:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.884+0000 7f29077fe700 1 -- 172.21.15.53:0/4085775723 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908018680 con 0x7f2918101820 2023-11-06T01:38:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.884+0000 7f291c881700 1 -- 172.21.15.53:0/4085775723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291819f320 con 0x7f2918101820 2023-11-06T01:38:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.887+0000 7f29077fe700 1 -- 172.21.15.53:0/4085775723 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f29080105a0 con 0x7f2918101820 2023-11-06T01:38:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.887+0000 7f29077fe700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f29000a0520 0x7f29000a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.888+0000 7f29077fe700 1 -- 172.21.15.53:0/4085775723 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2908014070 con 0x7f2918101820 2023-11-06T01:38:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.888+0000 7f2915d9b700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f29000a0520 0x7f29000a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.888+0000 7f2928a11b80 1 Processor -- start 2023-11-06T01:38:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.888+0000 7f2915d9b700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f29000a0520 0x7f29000a29e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f290c00b2e0 tx=0x7f290c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.888+0000 7f2928a11b80 1 -- start start 2023-11-06T01:38:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950680 0x55a7c2950aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950fe0 0x55a7c29518f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c2952350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2928a11b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c28a89b0 con 0x55a7c2951ed0 2023-11-06T01:38:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2928a11b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c28a6370 con 0x55a7c2950fe0 2023-11-06T01:38:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2928a11b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c28a1c60 con 0x55a7c2950680 2023-11-06T01:38:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2916d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c2952350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2915d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950fe0 0x55a7c29518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f291659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950680 0x55a7c2950aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2915d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950fe0 0x55a7c29518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52760/0 (socket says 172.21.15.53:52760) 2023-11-06T01:38:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f291659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950680 0x55a7c2950aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55560/0 (socket says 172.21.15.53:55560) 2023-11-06T01:38:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2916d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c2952350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41324/0 (socket says 172.21.15.53:41324) 2023-11-06T01:38:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2915d9b700 1 -- 172.21.15.53:0/1497574568 learned_addr learned my addr 172.21.15.53:0/1497574568 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2915d9b700 1 -- 172.21.15.53:0/1497574568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950680 msgr2=0x55a7c2950aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2915d9b700 1 --2- 172.21.15.53:0/1497574568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950680 0x55a7c2950aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2915d9b700 1 -- 172.21.15.53:0/1497574568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 msgr2=0x55a7c2952350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2915d9b700 1 --2- 172.21.15.53:0/1497574568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c2952350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2915d9b700 1 -- 172.21.15.53:0/1497574568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c26cfd80 con 0x55a7c2950fe0 2023-11-06T01:38:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.889+0000 7f2915d9b700 1 --2- 172.21.15.53:0/1497574568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950fe0 0x55a7c29518f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f290c0057e0 tx=0x7f290c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1e621276f8033b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.890+0000 7f2904ff9700 1 -- 172.21.15.53:0/1497574568 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c02b070 con 0x55a7c2950fe0 2023-11-06T01:38:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.890+0000 7f2904ff9700 1 -- 172.21.15.53:0/1497574568 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f290c01e040 con 0x55a7c2950fe0 2023-11-06T01:38:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.890+0000 7f2904ff9700 1 -- 172.21.15.53:0/1497574568 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c02f400 con 0x55a7c2950fe0 2023-11-06T01:38:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.890+0000 7f2928a11b80 1 -- 172.21.15.53:0/1497574568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950fe0 msgr2=0x55a7c29518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.890+0000 7f2928a11b80 1 --2- 172.21.15.53:0/1497574568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950fe0 0x55a7c29518f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f290c0057e0 tx=0x7f290c020000 comp rx=0 tx=0).stop 2023-11-06T01:38:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.891+0000 7f2928a11b80 1 -- 172.21.15.53:0/1497574568 shutdown_connections 2023-11-06T01:38:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.891+0000 7f2928a11b80 1 --2- 172.21.15.53:0/1497574568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c2952350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.891+0000 7f2928a11b80 1 --2- 172.21.15.53:0/1497574568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950680 0x55a7c2950aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.891+0000 7f2928a11b80 1 --2- 172.21.15.53:0/1497574568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950fe0 0x55a7c29518f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.891+0000 7f2928a11b80 1 -- 172.21.15.53:0/1497574568 >> 172.21.15.53:0/1497574568 conn(0x55a7c28b0070 msgr2=0x55a7c294d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.892+0000 7f2928a11b80 1 -- 172.21.15.53:0/1497574568 shutdown_connections 2023-11-06T01:38:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.892+0000 7f2928a11b80 1 -- 172.21.15.53:0/1497574568 wait complete. 2023-11-06T01:38:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.892+0000 7f2928a11b80 1 -- start start 2023-11-06T01:38:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.892+0000 7f2928a11b80 1 Processor -- start 2023-11-06T01:38:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.892+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950680 0x55a7c295a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.892+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950fe0 0x55a7c295a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.892+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c2961940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.892+0000 7f2928a11b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c283e4b0 con 0x55a7c2951ed0 2023-11-06T01:38:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.892+0000 7f2928a11b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c283e150 con 0x55a7c2950680 2023-11-06T01:38:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.892+0000 7f2928a11b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c27258c0 con 0x55a7c2950fe0 2023-11-06T01:38:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.892+0000 7f2928a11b80 10 client.0 _ll_register_callbacks cb 0x55a7c278ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f291659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950680 0x55a7c295a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f2915d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950fe0 0x55a7c295a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f291659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950680 0x55a7c295a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52764/0 (socket says 172.21.15.53:52764) 2023-11-06T01:38:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f2915d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950fe0 0x55a7c295a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55568/0 (socket says 172.21.15.53:55568) 2023-11-06T01:38:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f291659c700 1 -- 172.21.15.53:0/3501789625 learned_addr learned my addr 172.21.15.53:0/3501789625 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f2916d9d700 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c2961940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f291659c700 1 -- 172.21.15.53:0/3501789625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950fe0 msgr2=0x55a7c295a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f291659c700 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950fe0 0x55a7c295a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f291659c700 1 -- 172.21.15.53:0/3501789625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 msgr2=0x55a7c2961940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f291659c700 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c2961940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f291659c700 1 -- 172.21.15.53:0/3501789625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2918109ae0 con 0x55a7c2950680 2023-11-06T01:38:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f2916d9d700 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c2961940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:38:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f291659c700 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950680 0x55a7c295a3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f29080c96a0 tx=0x7f2908099020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4abed4b04200de24 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f28fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:38:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f28fdffb700 1 -- 172.21.15.53:0/3501789625 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290809d1e0 con 0x55a7c2950680 2023-11-06T01:38:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f2928a11b80 1 -- 172.21.15.53:0/3501789625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7c28cccb0 con 0x55a7c2950680 2023-11-06T01:38:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.893+0000 7f28fdffb700 1 -- 172.21.15.53:0/3501789625 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f29080a1420 con 0x55a7c2950680 2023-11-06T01:38:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f28fdffb700 1 -- 172.21.15.53:0/3501789625 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2908010880 con 0x55a7c2950680 2023-11-06T01:38:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f2928a11b80 10 client.25616 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f2928a11b80 1 -- 172.21.15.53:0/3501789625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a7c26b8c00 con 0x55a7c2950680 2023-11-06T01:38:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f28fdffb700 1 -- 172.21.15.53:0/3501789625 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7c28cccb0 con 0x55a7c2950680 2023-11-06T01:38:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f28fdffb700 1 -- 172.21.15.53:0/3501789625 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f29080b1b50 con 0x55a7c2950680 2023-11-06T01:38:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f2928a11b80 10 client.25616 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f2928a11b80 10 client.25616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f2928a11b80 20 client.25616 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f2928a11b80 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a7c2966170 0x55a7c2968590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f2928a11b80 4 client.25616 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:38:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f2928a11b80 1 -- 172.21.15.53:0/3501789625 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7c291fdb0 con 0x55a7c2966170 2023-11-06T01:38:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.894+0000 7f2915d9b700 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a7c2966170 0x55a7c2968590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.895+0000 7f2915d9b700 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a7c2966170 0x55a7c2968590 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeef951c2a923551 server_cookie=b3c1f3c135cf4003 in_seq=0 out_seq=0 2023-11-06T01:38:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.895+0000 7f28fdffb700 10 client.25616 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.895+0000 7f28fdffb700 1 -- 172.21.15.53:0/3501789625 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a7c291fdb0 con 0x55a7c2966170 2023-11-06T01:38:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.895+0000 7f28fdffb700 10 client.25616 handle_command_reply: tid=0 2023-11-06T01:38:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.895+0000 7f2928a11b80 1 client.25616 shutdown 2023-11-06T01:38:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.895+0000 7f2928a11b80 1 -- 172.21.15.53:0/3501789625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950680 msgr2=0x55a7c295a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.895+0000 7f2928a11b80 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950680 0x55a7c295a3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f29080c96a0 tx=0x7f2908099020 comp rx=0 tx=0).stop 2023-11-06T01:38:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.896+0000 7f2928a11b80 1 -- 172.21.15.53:0/3501789625 shutdown_connections 2023-11-06T01:38:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.896+0000 7f2928a11b80 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950680 0x55a7c295a3c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.896+0000 7f2928a11b80 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a7c2966170 0x55a7c2968590 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.896+0000 7f2928a11b80 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c2961940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.896+0000 7f2928a11b80 1 --2- 172.21.15.53:0/3501789625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950fe0 0x55a7c295a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.896+0000 7f28fdffb700 0 client.25616 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.896+0000 7f2928a11b80 1 -- 172.21.15.53:0/3501789625 >> 172.21.15.53:0/3501789625 conn(0x55a7c28b0070 msgr2=0x55a7c294b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.896+0000 7f2928a11b80 1 -- 172.21.15.53:0/3501789625 shutdown_connections 2023-11-06T01:38:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.896+0000 7f2928a11b80 1 -- 172.21.15.53:0/3501789625 wait complete. 2023-11-06T01:38:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.896+0000 7f2928a11b80 20 client.25616 trim_cache size 0 max 16384 2023-11-06T01:38:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.924+0000 7f2928a11b80 1 Processor -- start 2023-11-06T01:38:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.924+0000 7f2928a11b80 1 -- start start 2023-11-06T01:38:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.924+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950680 0x55a7c295f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.924+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950fe0 0x55a7c29522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.924+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c29582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.924+0000 7f2928a11b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2946530 con 0x55a7c2951ed0 2023-11-06T01:38:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.924+0000 7f2928a11b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2959f60 con 0x55a7c2950fe0 2023-11-06T01:38:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.924+0000 7f2928a11b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2951410 con 0x55a7c2950680 2023-11-06T01:38:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.925+0000 7f2916d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c29582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.925+0000 7f2916d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c29582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41338/0 (socket says 172.21.15.53:41338) 2023-11-06T01:38:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.925+0000 7f2916d9d700 1 -- 172.21.15.53:0/4105010161 learned_addr learned my addr 172.21.15.53:0/4105010161 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.925+0000 7f2916d9d700 1 -- 172.21.15.53:0/4105010161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950680 msgr2=0x55a7c295f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.925+0000 7f2916d9d700 1 --2- 172.21.15.53:0/4105010161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950680 0x55a7c295f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.925+0000 7f2916d9d700 1 -- 172.21.15.53:0/4105010161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950fe0 msgr2=0x55a7c29522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.925+0000 7f2916d9d700 1 --2- 172.21.15.53:0/4105010161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950fe0 0x55a7c29522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.925+0000 7f2916d9d700 1 -- 172.21.15.53:0/4105010161 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c26f61f0 con 0x55a7c2951ed0 2023-11-06T01:38:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.925+0000 7f2916d9d700 1 --2- 172.21.15.53:0/4105010161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c29582f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f291000e8f0 tx=0x7f2910009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3fc435c9da2128 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.925+0000 7f29057fa700 1 -- 172.21.15.53:0/4105010161 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910004db0 con 0x55a7c2951ed0 2023-11-06T01:38:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.926+0000 7f29057fa700 1 -- 172.21.15.53:0/4105010161 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f291000f040 con 0x55a7c2951ed0 2023-11-06T01:38:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.926+0000 7f29057fa700 1 -- 172.21.15.53:0/4105010161 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910005880 con 0x55a7c2951ed0 2023-11-06T01:38:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.926+0000 7f2928a11b80 1 -- 172.21.15.53:0/4105010161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 msgr2=0x55a7c29582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.926+0000 7f2928a11b80 1 --2- 172.21.15.53:0/4105010161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c29582f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f291000e8f0 tx=0x7f2910009920 comp rx=0 tx=0).stop 2023-11-06T01:38:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.926+0000 7f2928a11b80 1 -- 172.21.15.53:0/4105010161 shutdown_connections 2023-11-06T01:38:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.926+0000 7f2928a11b80 1 --2- 172.21.15.53:0/4105010161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2951ed0 0x55a7c29582f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.926+0000 7f2928a11b80 1 --2- 172.21.15.53:0/4105010161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2950680 0x55a7c295f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.926+0000 7f2928a11b80 1 --2- 172.21.15.53:0/4105010161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950fe0 0x55a7c29522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.926+0000 7f2928a11b80 1 -- 172.21.15.53:0/4105010161 >> 172.21.15.53:0/4105010161 conn(0x55a7c28b0070 msgr2=0x55a7c29590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2928a11b80 1 -- 172.21.15.53:0/4105010161 shutdown_connections 2023-11-06T01:38:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2928a11b80 1 -- 172.21.15.53:0/4105010161 wait complete. 2023-11-06T01:38:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2928a11b80 1 -- start start 2023-11-06T01:38:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2928a11b80 1 Processor -- start 2023-11-06T01:38:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950680 0x55a7c295da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2950fe0 0x55a7c295dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2928a11b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2951ed0 0x55a7c295e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2928a11b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c295ec00 con 0x55a7c2950fe0 2023-11-06T01:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2928a11b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c295ed70 con 0x55a7c2950680 2023-11-06T01:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2928a11b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c295ef30 con 0x55a7c2951ed0 2023-11-06T01:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2928a11b80 10 client.0 _ll_register_callbacks cb 0x7f2908098f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:38:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.927+0000 7f2915d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2950fe0 0x55a7c295dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f2915d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2950fe0 0x55a7c295dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41342/0 (socket says 172.21.15.53:41342) 2023-11-06T01:38:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f2915d9b700 1 -- 172.21.15.53:0/2730413034 learned_addr learned my addr 172.21.15.53:0/2730413034 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:38:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f2916d9d700 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2951ed0 0x55a7c295e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f2915d9b700 1 -- 172.21.15.53:0/2730413034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2951ed0 msgr2=0x55a7c295e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f2915d9b700 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2951ed0 0x55a7c295e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f2915d9b700 1 -- 172.21.15.53:0/2730413034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950680 msgr2=0x55a7c295da90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f2915d9b700 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950680 0x55a7c295da90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f2915d9b700 1 -- 172.21.15.53:0/2730413034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291000e5d0 con 0x55a7c2950fe0 2023-11-06T01:38:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f2915d9b700 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2950fe0 0x55a7c295dfd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f290c031040 tx=0x7f290c009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d453d6bc47212d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:38:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f28fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:38:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f28fdffb700 1 -- 172.21.15.53:0/2730413034 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c02b070 con 0x55a7c2950fe0 2023-11-06T01:38:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f2928a11b80 1 -- 172.21.15.53:0/2730413034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f290c01e040 con 0x55a7c2950fe0 2023-11-06T01:38:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f28fdffb700 1 -- 172.21.15.53:0/2730413034 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f290c03c070 con 0x55a7c2950fe0 2023-11-06T01:38:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.928+0000 7f28fdffb700 1 -- 172.21.15.53:0/2730413034 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f290c03c230 con 0x55a7c2950fe0 2023-11-06T01:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.929+0000 7f2928a11b80 10 client.16044 fetch_fsmap learned FSMap version 40 2023-11-06T01:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.929+0000 7f2928a11b80 1 -- 172.21.15.53:0/2730413034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a7c26ce250 con 0x55a7c2950fe0 2023-11-06T01:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.929+0000 7f28fdffb700 1 -- 172.21.15.53:0/2730413034 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f290c01e040 con 0x55a7c2950fe0 2023-11-06T01:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.929+0000 7f28fdffb700 1 -- 172.21.15.53:0/2730413034 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f290c00f740 con 0x55a7c2950fe0 2023-11-06T01:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.929+0000 7f2928a11b80 10 client.16044 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:38:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.929+0000 7f2928a11b80 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:38:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.929+0000 7f2928a11b80 20 client.16044 populate_metadata read hostname 'smithi053' 2023-11-06T01:38:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.929+0000 7f2928a11b80 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a7c29c0e80 0x55a7c29c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:38:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.929+0000 7f2928a11b80 4 client.16044 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:38:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.929+0000 7f2928a11b80 1 -- 172.21.15.53:0/2730413034 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a7c289ace0 con 0x55a7c29c0e80 2023-11-06T01:38:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.929+0000 7f291659c700 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a7c29c0e80 0x55a7c29c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:38:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.930+0000 7f28fdffb700 10 client.16044 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.930+0000 7f291659c700 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a7c29c0e80 0x55a7c29c3270 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566244284149cc6a server_cookie=89323d41dde0d35a in_seq=0 out_seq=0 2023-11-06T01:38:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.953+0000 7f28fdffb700 1 -- 172.21.15.53:0/2730413034 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a7c289ace0 con 0x55a7c29c0e80 2023-11-06T01:38:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.953+0000 7f28fdffb700 10 client.16044 handle_command_reply: tid=0 2023-11-06T01:38:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.953+0000 7f2928a11b80 1 client.16044 shutdown 2023-11-06T01:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.954+0000 7f2928a11b80 1 -- 172.21.15.53:0/2730413034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2950fe0 msgr2=0x55a7c295dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.954+0000 7f2928a11b80 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2950fe0 0x55a7c295dfd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f290c031040 tx=0x7f290c009290 comp rx=0 tx=0).stop 2023-11-06T01:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.954+0000 7f2928a11b80 1 -- 172.21.15.53:0/2730413034 shutdown_connections 2023-11-06T01:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.954+0000 7f2928a11b80 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7c2950680 0x55a7c295da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.954+0000 7f2928a11b80 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a7c29c0e80 0x55a7c29c3270 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.954+0000 7f2928a11b80 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a7c2951ed0 0x55a7c295e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.954+0000 7f2928a11b80 1 --2- 172.21.15.53:0/2730413034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c2950fe0 0x55a7c295dfd0 secure :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f290c031040 tx=0x7f290c009290 comp rx=0 tx=0).stop 2023-11-06T01:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.954+0000 7f2928a11b80 1 -- 172.21.15.53:0/2730413034 >> 172.21.15.53:0/2730413034 conn(0x55a7c28b0070 msgr2=0x55a7c2962170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.954+0000 7f28fdffb700 0 client.16044 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.954+0000 7f2928a11b80 1 -- 172.21.15.53:0/2730413034 shutdown_connections 2023-11-06T01:38:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.954+0000 7f2928a11b80 1 -- 172.21.15.53:0/2730413034 wait complete. 2023-11-06T01:38:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f2928a11b80 20 client.16044 trim_cache size 0 max 16384 2023-11-06T01:38:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:38:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:38:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:38:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 -- 172.21.15.53:0/4085775723 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f29000a0520 msgr2=0x7f29000a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f29000a0520 0x7f29000a29e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f290c00b2e0 tx=0x7f290c005fb0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 -- 172.21.15.53:0/4085775723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2918101820 msgr2=0x7f2918196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2918101820 0x7f2918196b30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f290800e9a0 tx=0x7f290800ecb0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 -- 172.21.15.53:0/4085775723 shutdown_connections 2023-11-06T01:38:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f29000a0520 0x7f29000a29e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2918101820 0x7f2918196b30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2918106a80 0x7f2918197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 --2- 172.21.15.53:0/4085775723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2918104150 0x7f2918197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:38:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 -- 172.21.15.53:0/4085775723 >> 172.21.15.53:0/4085775723 conn(0x7f29180fb210 msgr2=0x7f29181024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:38:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 -- 172.21.15.53:0/4085775723 shutdown_connections 2023-11-06T01:38:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:38:50.955+0000 7f291c881700 1 -- 172.21.15.53:0/4085775723 wait complete. 2023-11-06T01:38:51.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:51 smithi163 ceph-mon[104416]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 907 op/s 2023-11-06T01:38:51.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:51 smithi053 ceph-mon[99472]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 907 op/s 2023-11-06T01:38:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:51 smithi062 ceph-mon[103924]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 907 op/s 2023-11-06T01:38:52.184 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 275.0 s, 2334.6 tps, lat 42.780 ms stddev 47.031 2023-11-06T01:38:53.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:53 smithi163 ceph-mon[104416]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 810 op/s 2023-11-06T01:38:53.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:53 smithi053 ceph-mon[99472]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 810 op/s 2023-11-06T01:38:53.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:53 smithi062 ceph-mon[103924]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 810 op/s 2023-11-06T01:38:55.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:55 smithi163 ceph-mon[104416]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 752 op/s 2023-11-06T01:38:55.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:55 smithi053 ceph-mon[99472]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 752 op/s 2023-11-06T01:38:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:55 smithi062 ceph-mon[103924]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 752 op/s 2023-11-06T01:38:57.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 280.0 s, 2434.0 tps, lat 41.298 ms stddev 46.783 2023-11-06T01:38:57.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:57 smithi163 ceph-mon[104416]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 830 op/s 2023-11-06T01:38:57.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:57 smithi053 ceph-mon[99472]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 830 op/s 2023-11-06T01:38:57.653 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:57 smithi062 ceph-mon[103924]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 830 op/s 2023-11-06T01:38:59.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:38:59 smithi163 ceph-mon[104416]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 693 op/s 2023-11-06T01:38:59.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:38:59 smithi053 ceph-mon[99472]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 693 op/s 2023-11-06T01:38:59.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:38:59 smithi062 ceph-mon[103924]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 693 op/s 2023-11-06T01:39:01.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:01 smithi163 ceph-mon[104416]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 724 op/s 2023-11-06T01:39:01.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:01 smithi053 ceph-mon[99472]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 724 op/s 2023-11-06T01:39:01.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:01 smithi062 ceph-mon[103924]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 724 op/s 2023-11-06T01:39:02.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 285.0 s, 2928.0 tps, lat 34.054 ms stddev 38.765 2023-11-06T01:39:03.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:03 smithi163 ceph-mon[104416]: pgmap v452: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 880 op/s 2023-11-06T01:39:03.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:03 smithi053 ceph-mon[99472]: pgmap v452: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 880 op/s 2023-11-06T01:39:03.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:03 smithi062 ceph-mon[103924]: pgmap v452: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 880 op/s 2023-11-06T01:39:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.125+0000 7febdf022700 1 -- 172.21.15.53:0/3248392563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd810a700 msgr2=0x7febd810cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.125+0000 7febdf022700 1 --2- 172.21.15.53:0/3248392563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd810a700 0x7febd810cb90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7febd0009f90 tx=0x7febd000b330 comp rx=0 tx=0).stop 2023-11-06T01:39:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.126+0000 7febdf022700 1 -- 172.21.15.53:0/3248392563 shutdown_connections 2023-11-06T01:39:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.126+0000 7febdf022700 1 --2- 172.21.15.53:0/3248392563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd810d0d0 0x7febd810d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.126+0000 7febdf022700 1 --2- 172.21.15.53:0/3248392563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd8107d90 0x7febd810a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.126+0000 7febdf022700 1 --2- 172.21.15.53:0/3248392563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd810a700 0x7febd810cb90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.126+0000 7febdf022700 1 -- 172.21.15.53:0/3248392563 >> 172.21.15.53:0/3248392563 conn(0x7febd806dbc0 msgr2=0x7febd8070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.126+0000 7febdf022700 1 -- 172.21.15.53:0/3248392563 shutdown_connections 2023-11-06T01:39:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.126+0000 7febdf022700 1 -- 172.21.15.53:0/3248392563 wait complete. 2023-11-06T01:39:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.127+0000 7febdf022700 1 Processor -- start 2023-11-06T01:39:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.127+0000 7febdf022700 1 -- start start 2023-11-06T01:39:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.127+0000 7febdf022700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd8107d90 0x7febd81a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.127+0000 7febdf022700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd810a700 0x7febd81a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.127+0000 7febdf022700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd810d0d0 0x7febd81a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.127+0000 7febdf022700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd81a7540 con 0x7febd810a700 2023-11-06T01:39:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.127+0000 7febdf022700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd81a76b0 con 0x7febd810d0d0 2023-11-06T01:39:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.127+0000 7febdf022700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd81a7870 con 0x7febd8107d90 2023-11-06T01:39:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.129+0000 7febdcdbe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd8107d90 0x7febd81a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.129+0000 7febdcdbe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd8107d90 0x7febd81a5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55596/0 (socket says 172.21.15.53:55596) 2023-11-06T01:39:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.129+0000 7febdcdbe700 1 -- 172.21.15.53:0/1343275824 learned_addr learned my addr 172.21.15.53:0/1343275824 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.129+0000 7febdd5bf700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd810d0d0 0x7febd81a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.129+0000 7febd7fff700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd810a700 0x7febd81a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.130+0000 7febd7fff700 1 -- 172.21.15.53:0/1343275824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd8107d90 msgr2=0x7febd81a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.130+0000 7febd7fff700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd8107d90 0x7febd81a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.130+0000 7febd7fff700 1 -- 172.21.15.53:0/1343275824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd810d0d0 msgr2=0x7febd81a6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.130+0000 7febd7fff700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd810d0d0 0x7febd81a6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.130+0000 7febd7fff700 1 -- 172.21.15.53:0/1343275824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd0009c40 con 0x7febd810a700 2023-11-06T01:39:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.130+0000 7febd7fff700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd810a700 0x7febd81a6060 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7febd000b770 tx=0x7febd000bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.130+0000 7febd5ffb700 1 -- 172.21.15.53:0/1343275824 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd000e040 con 0x7febd810a700 2023-11-06T01:39:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.130+0000 7febdf022700 1 -- 172.21.15.53:0/1343275824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd81a7af0 con 0x7febd810a700 2023-11-06T01:39:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.130+0000 7febd5ffb700 1 -- 172.21.15.53:0/1343275824 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd0004e40 con 0x7febd810a700 2023-11-06T01:39:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.130+0000 7febd5ffb700 1 -- 172.21.15.53:0/1343275824 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd001b7e0 con 0x7febd810a700 2023-11-06T01:39:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.131+0000 7febdf022700 1 -- 172.21.15.53:0/1343275824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd810f6a0 con 0x7febd810a700 2023-11-06T01:39:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.135+0000 7febd5ffb700 1 -- 172.21.15.53:0/1343275824 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7febd001daa0 con 0x7febd810a700 2023-11-06T01:39:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.142+0000 7febeb1b2b80 1 Processor -- start 2023-11-06T01:39:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.142+0000 7febd5ffb700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0570 0x7febc00a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.142+0000 7febd5ffb700 1 -- 172.21.15.53:0/1343275824 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7febd00cc660 con 0x7febd810a700 2023-11-06T01:39:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.148+0000 7febeb1b2b80 1 -- start start 2023-11-06T01:39:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.148+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d90680 0x558d44d90aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.148+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90fe0 0x558d44d918f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.148+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d91ed0 0x558d44d92350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.148+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44ce89b0 con 0x558d44d91ed0 2023-11-06T01:39:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.148+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44ce6370 con 0x558d44d90fe0 2023-11-06T01:39:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.148+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44ce1c60 con 0x558d44d90680 2023-11-06T01:39:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.149+0000 7febdd5bf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d91ed0 0x558d44d92350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.149+0000 7febdd5bf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d91ed0 0x558d44d92350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41366/0 (socket says 172.21.15.53:41366) 2023-11-06T01:39:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.149+0000 7febdd5bf700 1 -- 172.21.15.53:0/1352060752 learned_addr learned my addr 172.21.15.53:0/1352060752 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.149+0000 7febdd5bf700 1 -- 172.21.15.53:0/1352060752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d90680 msgr2=0x558d44d90aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.149+0000 7febdd5bf700 1 --2- 172.21.15.53:0/1352060752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d90680 0x558d44d90aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.149+0000 7febdd5bf700 1 -- 172.21.15.53:0/1352060752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90fe0 msgr2=0x558d44d918f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.149+0000 7febdd5bf700 1 --2- 172.21.15.53:0/1352060752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90fe0 0x558d44d918f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.149+0000 7febdd5bf700 1 -- 172.21.15.53:0/1352060752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d44b0fd80 con 0x558d44d91ed0 2023-11-06T01:39:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.150+0000 7febdd5bf700 1 --2- 172.21.15.53:0/1352060752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d91ed0 0x558d44d92350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7febc800a9a0 tx=0x7febc800acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e1704707a2314c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.150+0000 7febbeffd700 1 -- 172.21.15.53:0/1352060752 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc800f950 con 0x558d44d91ed0 2023-11-06T01:39:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.150+0000 7febbeffd700 1 -- 172.21.15.53:0/1352060752 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc800c030 con 0x558d44d91ed0 2023-11-06T01:39:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.150+0000 7febbeffd700 1 -- 172.21.15.53:0/1352060752 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8013d10 con 0x558d44d91ed0 2023-11-06T01:39:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.150+0000 7febdd5bf700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0570 0x7febc00a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.151+0000 7febdd5bf700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0570 0x7febc00a2a30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7febc800a730 tx=0x7febc8013e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.151+0000 7febeb1b2b80 1 -- 172.21.15.53:0/1352060752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d91ed0 msgr2=0x558d44d92350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.151+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/1352060752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d91ed0 0x558d44d92350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7febc800a9a0 tx=0x7febc800acb0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.151+0000 7febeb1b2b80 1 -- 172.21.15.53:0/1352060752 shutdown_connections 2023-11-06T01:39:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.151+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/1352060752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d91ed0 0x558d44d92350 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.151+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/1352060752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d90680 0x558d44d90aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.151+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/1352060752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90fe0 0x558d44d918f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.152+0000 7febeb1b2b80 1 -- 172.21.15.53:0/1352060752 >> 172.21.15.53:0/1352060752 conn(0x558d44cf0070 msgr2=0x558d44d8d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.152+0000 7febeb1b2b80 1 -- 172.21.15.53:0/1352060752 shutdown_connections 2023-11-06T01:39:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.152+0000 7febeb1b2b80 1 -- 172.21.15.53:0/1352060752 wait complete. 2023-11-06T01:39:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.152+0000 7febeb1b2b80 1 -- start start 2023-11-06T01:39:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.152+0000 7febeb1b2b80 1 Processor -- start 2023-11-06T01:39:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90680 0x558d44d9a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 0x558d44d9aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44da1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44c7e4b0 con 0x558d44d90fe0 2023-11-06T01:39:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44c7e150 con 0x558d44d90680 2023-11-06T01:39:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44b658c0 con 0x558d44d91ed0 2023-11-06T01:39:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febeb1b2b80 10 client.0 _ll_register_callbacks cb 0x558d44bcba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febdd5bf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44da1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febdd5bf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44da1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55612/0 (socket says 172.21.15.53:55612) 2023-11-06T01:39:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febdd5bf700 1 -- 172.21.15.53:0/3931098478 learned_addr learned my addr 172.21.15.53:0/3931098478 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febd7fff700 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 0x558d44d9aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febd7fff700 1 -- 172.21.15.53:0/3931098478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 msgr2=0x558d44da1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febd7fff700 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44da1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febd7fff700 1 -- 172.21.15.53:0/3931098478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90680 msgr2=0x558d44d9a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.153+0000 7febd7fff700 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90680 0x558d44d9a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.154+0000 7febd7fff700 1 -- 172.21.15.53:0/3931098478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc800e040 con 0x558d44d90fe0 2023-11-06T01:39:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.154+0000 7febdd5bf700 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44da1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.154+0000 7febd7fff700 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 0x558d44d9aa30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7febd00ca4b0 tx=0x7febd0099b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c3b6333cd11671 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.154+0000 7febb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:39:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.154+0000 7febb7fff700 1 -- 172.21.15.53:0/3931098478 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd009a390 con 0x558d44d90fe0 2023-11-06T01:39:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.154+0000 7febb7fff700 1 -- 172.21.15.53:0/3931098478 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7febd0099f30 con 0x558d44d90fe0 2023-11-06T01:39:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.154+0000 7febeb1b2b80 1 -- 172.21.15.53:0/3931098478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d44d0ccb0 con 0x558d44d90fe0 2023-11-06T01:39:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.154+0000 7febb7fff700 1 -- 172.21.15.53:0/3931098478 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7febd009dd60 con 0x558d44d90fe0 2023-11-06T01:39:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.154+0000 7febeb1b2b80 10 client.16056 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.154+0000 7febeb1b2b80 1 -- 172.21.15.53:0/3931098478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558d44af8c00 con 0x558d44d90fe0 2023-11-06T01:39:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.154+0000 7febb7fff700 1 -- 172.21.15.53:0/3931098478 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd00b6080 con 0x558d44d90fe0 2023-11-06T01:39:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.155+0000 7febb7fff700 1 -- 172.21.15.53:0/3931098478 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7febd00994b0 con 0x558d44d90fe0 2023-11-06T01:39:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.155+0000 7febeb1b2b80 10 client.16056 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.155+0000 7febeb1b2b80 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.155+0000 7febeb1b2b80 20 client.16056 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.155+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d44da6090 0x558d44da8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.155+0000 7febeb1b2b80 4 client.16056 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:39:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.155+0000 7febeb1b2b80 1 -- 172.21.15.53:0/3931098478 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d44d5fdb0 con 0x558d44da6090 2023-11-06T01:39:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.155+0000 7febdcdbe700 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d44da6090 0x558d44da8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.156+0000 7febdcdbe700 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d44da6090 0x558d44da8480 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e0a060fd0025e47 server_cookie=d119aeef95f486b4 in_seq=0 out_seq=0 2023-11-06T01:39:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.156+0000 7febb7fff700 10 client.16056 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.156+0000 7febb7fff700 1 -- 172.21.15.53:0/3931098478 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558d44d5fdb0 con 0x558d44da6090 2023-11-06T01:39:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.156+0000 7febb7fff700 10 client.16056 handle_command_reply: tid=0 2023-11-06T01:39:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.156+0000 7febeb1b2b80 1 client.16056 shutdown 2023-11-06T01:39:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febeb1b2b80 1 -- 172.21.15.53:0/3931098478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 msgr2=0x558d44d9aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 0x558d44d9aa30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7febd00ca4b0 tx=0x7febd0099b60 comp rx=0 tx=0).stop 2023-11-06T01:39:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febeb1b2b80 1 -- 172.21.15.53:0/3931098478 shutdown_connections 2023-11-06T01:39:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90680 0x558d44d9a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d44da6090 0x558d44da8480 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44da1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/3931098478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 0x558d44d9aa30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febeb1b2b80 1 -- 172.21.15.53:0/3931098478 >> 172.21.15.53:0/3931098478 conn(0x558d44cf0070 msgr2=0x558d44d8b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febb7fff700 0 client.16056 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febeb1b2b80 1 -- 172.21.15.53:0/3931098478 shutdown_connections 2023-11-06T01:39:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febeb1b2b80 1 -- 172.21.15.53:0/3931098478 wait complete. 2023-11-06T01:39:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.157+0000 7febeb1b2b80 20 client.16056 trim_cache size 0 max 16384 2023-11-06T01:39:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febeb1b2b80 1 Processor -- start 2023-11-06T01:39:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febeb1b2b80 1 -- start start 2023-11-06T01:39:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90680 0x558d44d922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90fe0 0x558d44da1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44d99140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44d86500 con 0x558d44d90680 2023-11-06T01:39:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44d8c6e0 con 0x558d44d90fe0 2023-11-06T01:39:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44d90280 con 0x558d44d91ed0 2023-11-06T01:39:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febdcdbe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90680 0x558d44d922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febd7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90fe0 0x558d44da1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febdcdbe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90680 0x558d44d922d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41376/0 (socket says 172.21.15.53:41376) 2023-11-06T01:39:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.178+0000 7febdcdbe700 1 -- 172.21.15.53:0/274307795 learned_addr learned my addr 172.21.15.53:0/274307795 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.179+0000 7febdd5bf700 1 --2- 172.21.15.53:0/274307795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44d99140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.179+0000 7febdcdbe700 1 -- 172.21.15.53:0/274307795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 msgr2=0x558d44d99140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.179+0000 7febdcdbe700 1 --2- 172.21.15.53:0/274307795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44d99140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.179+0000 7febdcdbe700 1 -- 172.21.15.53:0/274307795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90fe0 msgr2=0x558d44da1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.179+0000 7febdcdbe700 1 --2- 172.21.15.53:0/274307795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90fe0 0x558d44da1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.179+0000 7febdcdbe700 1 -- 172.21.15.53:0/274307795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d44b361f0 con 0x558d44d90680 2023-11-06T01:39:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.179+0000 7febdcdbe700 1 --2- 172.21.15.53:0/274307795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90680 0x558d44d922d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7febcc00b700 tx=0x7febcc0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1102255d058462 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.179+0000 7febbf7fe700 1 -- 172.21.15.53:0/274307795 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc017070 con 0x558d44d90680 2023-11-06T01:39:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.179+0000 7febbf7fe700 1 -- 172.21.15.53:0/274307795 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febcc0171d0 con 0x558d44d90680 2023-11-06T01:39:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.179+0000 7febbf7fe700 1 -- 172.21.15.53:0/274307795 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc010690 con 0x558d44d90680 2023-11-06T01:39:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.180+0000 7febeb1b2b80 1 -- 172.21.15.53:0/274307795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90680 msgr2=0x558d44d922d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.180+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/274307795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90680 0x558d44d922d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7febcc00b700 tx=0x7febcc0058e0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.180+0000 7febeb1b2b80 1 -- 172.21.15.53:0/274307795 shutdown_connections 2023-11-06T01:39:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.180+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/274307795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44d99140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.180+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/274307795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90680 0x558d44d922d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.180+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/274307795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90fe0 0x558d44da1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.180+0000 7febeb1b2b80 1 -- 172.21.15.53:0/274307795 >> 172.21.15.53:0/274307795 conn(0x558d44cf0070 msgr2=0x558d44d8cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.180+0000 7febeb1b2b80 1 -- 172.21.15.53:0/274307795 shutdown_connections 2023-11-06T01:39:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.180+0000 7febeb1b2b80 1 -- 172.21.15.53:0/274307795 wait complete. 2023-11-06T01:39:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.180+0000 7febeb1b2b80 1 -- start start 2023-11-06T01:39:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.180+0000 7febeb1b2b80 1 Processor -- start 2023-11-06T01:39:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90680 0x558d44d9f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 0x558d44d9f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febeb1b2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44da09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44da0ef0 con 0x558d44d90fe0 2023-11-06T01:39:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44da1060 con 0x558d44d90680 2023-11-06T01:39:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febeb1b2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d44da1220 con 0x558d44d91ed0 2023-11-06T01:39:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febeb1b2b80 10 client.0 _ll_register_callbacks cb 0x7febd00b9040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febd7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 0x558d44d9f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febd7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 0x558d44d9f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41384/0 (socket says 172.21.15.53:41384) 2023-11-06T01:39:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febd7fff700 1 -- 172.21.15.53:0/141688401 learned_addr learned my addr 172.21.15.53:0/141688401 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febdcdbe700 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90680 0x558d44d9f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febdd5bf700 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44da09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febd7fff700 1 -- 172.21.15.53:0/141688401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 msgr2=0x558d44da09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febd7fff700 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44da09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febd7fff700 1 -- 172.21.15.53:0/141688401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90680 msgr2=0x558d44d9f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febd7fff700 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90680 0x558d44d9f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.181+0000 7febd7fff700 1 -- 172.21.15.53:0/141688401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc00bd00 con 0x558d44d90fe0 2023-11-06T01:39:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.182+0000 7febd7fff700 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 0x558d44d9f9a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7febd001a390 tx=0x7febd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea066354eacc69 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.182+0000 7febb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:39:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.182+0000 7febb7fff700 1 -- 172.21.15.53:0/141688401 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0099f30 con 0x558d44d90fe0 2023-11-06T01:39:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.182+0000 7febeb1b2b80 1 -- 172.21.15.53:0/141688401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febc800c030 con 0x558d44d90fe0 2023-11-06T01:39:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.182+0000 7febb7fff700 1 -- 172.21.15.53:0/141688401 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7febd009dd60 con 0x558d44d90fe0 2023-11-06T01:39:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.182+0000 7febb7fff700 1 -- 172.21.15.53:0/141688401 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7febd00a1f10 con 0x558d44d90fe0 2023-11-06T01:39:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.182+0000 7febeb1b2b80 10 client.16062 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.182+0000 7febeb1b2b80 1 -- 172.21.15.53:0/141688401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558d44b0e250 con 0x558d44d90fe0 2023-11-06T01:39:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.182+0000 7febb7fff700 1 -- 172.21.15.53:0/141688401 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc800c030 con 0x558d44d90fe0 2023-11-06T01:39:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.182+0000 7febb7fff700 1 -- 172.21.15.53:0/141688401 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7febd0095e40 con 0x558d44d90fe0 2023-11-06T01:39:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.183+0000 7febeb1b2b80 10 client.16062 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.183+0000 7febeb1b2b80 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.183+0000 7febeb1b2b80 20 client.16062 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.183+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d44df82a0 0x558d44dfa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.183+0000 7febeb1b2b80 4 client.16062 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:39:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.183+0000 7febeb1b2b80 1 -- 172.21.15.53:0/141688401 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558d44cdace0 con 0x558d44df82a0 2023-11-06T01:39:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.183+0000 7febdcdbe700 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d44df82a0 0x558d44dfa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.184+0000 7febdcdbe700 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d44df82a0 0x558d44dfa6c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4925cee57e3a87 server_cookie=5f18638b77fe90f7 in_seq=0 out_seq=0 2023-11-06T01:39:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.184+0000 7febb7fff700 10 client.16062 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.239+0000 7febb7fff700 1 -- 172.21.15.53:0/141688401 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558d44cdace0 con 0x558d44df82a0 2023-11-06T01:39:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.239+0000 7febb7fff700 10 client.16062 handle_command_reply: tid=0 2023-11-06T01:39:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.241+0000 7febeb1b2b80 1 client.16062 shutdown 2023-11-06T01:39:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febeb1b2b80 1 -- 172.21.15.53:0/141688401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 msgr2=0x558d44d9f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 0x558d44d9f9a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7febd001a390 tx=0x7febd000b330 comp rx=0 tx=0).stop 2023-11-06T01:39:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febeb1b2b80 1 -- 172.21.15.53:0/141688401 shutdown_connections 2023-11-06T01:39:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d44d90680 0x558d44d9f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d44df82a0 0x558d44dfa6c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d44d91ed0 0x558d44da09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febeb1b2b80 1 --2- 172.21.15.53:0/141688401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d44d90fe0 0x558d44d9f9a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febeb1b2b80 1 -- 172.21.15.53:0/141688401 >> 172.21.15.53:0/141688401 conn(0x558d44cf0070 msgr2=0x558d44d99140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febb7fff700 0 client.16062 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febeb1b2b80 1 -- 172.21.15.53:0/141688401 shutdown_connections 2023-11-06T01:39:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febeb1b2b80 1 -- 172.21.15.53:0/141688401 wait complete. 2023-11-06T01:39:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.242+0000 7febeb1b2b80 20 client.16062 trim_cache size 0 max 16384 2023-11-06T01:39:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:39:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:39:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:39:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:39:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 -- 172.21.15.53:0/1343275824 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0570 msgr2=0x7febc00a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0570 0x7febc00a2a30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7febc800a730 tx=0x7febc8013e70 comp rx=0 tx=0).stop 2023-11-06T01:39:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 -- 172.21.15.53:0/1343275824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd810a700 msgr2=0x7febd81a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd810a700 0x7febd81a6060 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7febd000b770 tx=0x7febd000bcd0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 -- 172.21.15.53:0/1343275824 shutdown_connections 2023-11-06T01:39:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0570 0x7febc00a2a30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd810d0d0 0x7febd81a6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd8107d90 0x7febd81a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 --2- 172.21.15.53:0/1343275824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd810a700 0x7febd81a6060 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 -- 172.21.15.53:0/1343275824 >> 172.21.15.53:0/1343275824 conn(0x7febd806dbc0 msgr2=0x7febd81095d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 -- 172.21.15.53:0/1343275824 shutdown_connections 2023-11-06T01:39:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:04.243+0000 7febdf022700 1 -- 172.21.15.53:0/1343275824 wait complete. 2023-11-06T01:39:05.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:05 smithi163 ceph-mon[104416]: pgmap v453: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 798 op/s 2023-11-06T01:39:05.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:05 smithi053 ceph-mon[99472]: pgmap v453: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 798 op/s 2023-11-06T01:39:05.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:05 smithi062 ceph-mon[103924]: pgmap v453: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 798 op/s 2023-11-06T01:39:06.294 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:39:06.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.482+0000 7fa028a4c700 1 -- 172.21.15.53:0/4085740222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa024100e70 msgr2=0x7fa0241012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:06.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.482+0000 7fa028a4c700 1 --2- 172.21.15.53:0/4085740222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa024100e70 0x7fa0241012d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa0180099c0 tx=0x7fa018009cd0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.483+0000 7fa028a4c700 1 -- 172.21.15.53:0/4085740222 shutdown_connections 2023-11-06T01:39:06.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.483+0000 7fa028a4c700 1 --2- 172.21.15.53:0/4085740222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa024101810 0x7fa0241080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.483+0000 7fa028a4c700 1 --2- 172.21.15.53:0/4085740222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0240ffc80 0x7fa0241000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.483+0000 7fa028a4c700 1 --2- 172.21.15.53:0/4085740222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa024100e70 0x7fa0241012d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.483+0000 7fa028a4c700 1 -- 172.21.15.53:0/4085740222 >> 172.21.15.53:0/4085740222 conn(0x7fa0240fb1f0 msgr2=0x7fa0240fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:06.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.483+0000 7fa028a4c700 1 -- 172.21.15.53:0/4085740222 shutdown_connections 2023-11-06T01:39:06.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.483+0000 7fa028a4c700 1 -- 172.21.15.53:0/4085740222 wait complete. 2023-11-06T01:39:06.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.483+0000 7fa028a4c700 1 Processor -- start 2023-11-06T01:39:06.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.484+0000 7fa028a4c700 1 -- start start 2023-11-06T01:39:06.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.484+0000 7fa028a4c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0240ffc80 0x7fa024068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.484+0000 7fa028a4c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa024100e70 0x7fa02419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.484+0000 7fa028a4c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa024101810 0x7fa02419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.484+0000 7fa028a4c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02419e5c0 con 0x7fa024101810 2023-11-06T01:39:06.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.484+0000 7fa028a4c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02419e730 con 0x7fa0240ffc80 2023-11-06T01:39:06.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.484+0000 7fa028a4c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02419e8f0 con 0x7fa024100e70 2023-11-06T01:39:06.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.484+0000 7fa022d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa024101810 0x7fa02419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.484+0000 7fa022d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa024101810 0x7fa02419ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41396/0 (socket says 172.21.15.53:41396) 2023-11-06T01:39:06.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.484+0000 7fa022d9d700 1 -- 172.21.15.53:0/3350674360 learned_addr learned my addr 172.21.15.53:0/3350674360 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:06.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.484+0000 7fa021d9b700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa024100e70 0x7fa02419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.485+0000 7fa022d9d700 1 -- 172.21.15.53:0/3350674360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa024100e70 msgr2=0x7fa02419b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:06.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.485+0000 7fa022d9d700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa024100e70 0x7fa02419b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.485+0000 7fa022d9d700 1 -- 172.21.15.53:0/3350674360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0240ffc80 msgr2=0x7fa024068a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:39:06.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.485+0000 7fa022d9d700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0240ffc80 0x7fa024068a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.485+0000 7fa022d9d700 1 -- 172.21.15.53:0/3350674360 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180095f0 con 0x7fa024101810 2023-11-06T01:39:06.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.485+0000 7fa022d9d700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa024101810 0x7fa02419ddf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00ef30 tx=0x7fa01c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:06.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.486+0000 7fa0137fe700 1 -- 172.21.15.53:0/3350674360 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c018070 con 0x7fa024101810 2023-11-06T01:39:06.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.486+0000 7fa0137fe700 1 -- 172.21.15.53:0/3350674360 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01c00f040 con 0x7fa024101810 2023-11-06T01:39:06.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.486+0000 7fa0137fe700 1 -- 172.21.15.53:0/3350674360 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c00cd60 con 0x7fa024101810 2023-11-06T01:39:06.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.486+0000 7fa028a4c700 1 -- 172.21.15.53:0/3350674360 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa024195570 con 0x7fa024101810 2023-11-06T01:39:06.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.486+0000 7fa028a4c700 1 -- 172.21.15.53:0/3350674360 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa024195a00 con 0x7fa024101810 2023-11-06T01:39:06.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.488+0000 7fa0137fe700 1 -- 172.21.15.53:0/3350674360 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa01c003a00 con 0x7fa024101810 2023-11-06T01:39:06.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.489+0000 7fa0137fe700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa00c0a0670 0x7fa00c0a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.489+0000 7fa0137fe700 1 -- 172.21.15.53:0/3350674360 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa01c0cb430 con 0x7fa024101810 2023-11-06T01:39:06.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.489+0000 7fa02259c700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa00c0a0670 0x7fa00c0a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.490+0000 7fa02259c700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa00c0a0670 0x7fa00c0a2b30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa024100cd0 tx=0x7fa014009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:06.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.490+0000 7fa034bdcb80 1 Processor -- start 2023-11-06T01:39:06.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.490+0000 7fa034bdcb80 1 -- start start 2023-11-06T01:39:06.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.490+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8320f0 0x55d83f832530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 0x55d83f830e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 0x55d83f831870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f788a10 con 0x55d83f8313f0 2023-11-06T01:39:06.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f7863d0 con 0x55d83f832a70 2023-11-06T01:39:06.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f781cc0 con 0x55d83f8320f0 2023-11-06T01:39:06.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa021d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 0x55d83f830e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa022d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 0x55d83f831870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa02259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8320f0 0x55d83f832530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa021d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 0x55d83f830e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52842/0 (socket says 172.21.15.53:52842) 2023-11-06T01:39:06.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa021d9b700 1 -- 172.21.15.53:0/1301292674 learned_addr learned my addr 172.21.15.53:0/1301292674 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:06.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa021d9b700 1 -- 172.21.15.53:0/1301292674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8320f0 msgr2=0x55d83f832530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:06.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa021d9b700 1 --2- 172.21.15.53:0/1301292674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8320f0 0x55d83f832530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa021d9b700 1 -- 172.21.15.53:0/1301292674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 msgr2=0x55d83f831870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:06.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa021d9b700 1 --2- 172.21.15.53:0/1301292674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 0x55d83f831870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa021d9b700 1 -- 172.21.15.53:0/1301292674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d83f5afde0 con 0x55d83f832a70 2023-11-06T01:39:06.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa022d9d700 1 --2- 172.21.15.53:0/1301292674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 0x55d83f831870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:06.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa021d9b700 1 --2- 172.21.15.53:0/1301292674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 0x55d83f830e60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa0180099c0 tx=0x7fa018005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fe6d91759d229ab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:06.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.491+0000 7fa010ff9700 1 -- 172.21.15.53:0/1301292674 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01800ba10 con 0x55d83f832a70 2023-11-06T01:39:06.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.492+0000 7fa010ff9700 1 -- 172.21.15.53:0/1301292674 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa018022040 con 0x55d83f832a70 2023-11-06T01:39:06.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.492+0000 7fa010ff9700 1 -- 172.21.15.53:0/1301292674 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180053a0 con 0x55d83f832a70 2023-11-06T01:39:06.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.492+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1301292674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 msgr2=0x55d83f830e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:06.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.492+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/1301292674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 0x55d83f830e60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa0180099c0 tx=0x7fa018005ee0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.492+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1301292674 shutdown_connections 2023-11-06T01:39:06.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.492+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/1301292674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 0x55d83f831870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.492+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/1301292674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8320f0 0x55d83f832530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.492+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/1301292674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 0x55d83f830e60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.492+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1301292674 >> 172.21.15.53:0/1301292674 conn(0x55d83f7900d0 msgr2=0x55d83f82b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:06.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.493+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1301292674 shutdown_connections 2023-11-06T01:39:06.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.493+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1301292674 wait complete. 2023-11-06T01:39:06.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa034bdcb80 1 -- start start 2023-11-06T01:39:06.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa034bdcb80 1 Processor -- start 2023-11-06T01:39:06.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8313f0 0x55d83f83a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8320f0 0x55d83f83ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 0x55d83f841ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f71e510 con 0x55d83f8320f0 2023-11-06T01:39:06.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f71e1b0 con 0x55d83f832a70 2023-11-06T01:39:06.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f605920 con 0x55d83f8313f0 2023-11-06T01:39:06.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa034bdcb80 10 client.0 _ll_register_callbacks cb 0x55d83f66ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:06.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa021d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8320f0 0x55d83f83ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa021d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8320f0 0x55d83f83ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41410/0 (socket says 172.21.15.53:41410) 2023-11-06T01:39:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa021d9b700 1 -- 172.21.15.53:0/1924462735 learned_addr learned my addr 172.21.15.53:0/1924462735 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa02259c700 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8313f0 0x55d83f83a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.494+0000 7fa022d9d700 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 0x55d83f841ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa021d9b700 1 -- 172.21.15.53:0/1924462735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8313f0 msgr2=0x55d83f83a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:06.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa021d9b700 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8313f0 0x55d83f83a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa021d9b700 1 -- 172.21.15.53:0/1924462735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 msgr2=0x55d83f841ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:06.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa021d9b700 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 0x55d83f841ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa021d9b700 1 -- 172.21.15.53:0/1924462735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa024195a00 con 0x55d83f8320f0 2023-11-06T01:39:06.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa02259c700 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8313f0 0x55d83f83a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:06.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa021d9b700 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8320f0 0x55d83f83ae60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa01800a010 tx=0x7fa018005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccd6d29723e29b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:06.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa009ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:39:06.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa009ffb700 1 -- 172.21.15.53:0/1924462735 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01801ba00 con 0x55d83f8320f0 2023-11-06T01:39:06.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa009ffb700 1 -- 172.21.15.53:0/1924462735 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa01801bb60 con 0x55d83f8320f0 2023-11-06T01:39:06.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1924462735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d83f7acd10 con 0x55d83f8320f0 2023-11-06T01:39:06.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.495+0000 7fa009ffb700 1 -- 172.21.15.53:0/1924462735 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa01801bd20 con 0x55d83f8320f0 2023-11-06T01:39:06.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.497+0000 7fa034bdcb80 10 client.16074 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:06.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.497+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1924462735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fa024108a10 con 0x55d83f8320f0 2023-11-06T01:39:06.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.505+0000 7fa009ffb700 1 -- 172.21.15.53:0/1924462735 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d83f7acd10 con 0x55d83f8320f0 2023-11-06T01:39:06.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.505+0000 7fa009ffb700 1 -- 172.21.15.53:0/1924462735 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa01800f4f0 con 0x55d83f8320f0 2023-11-06T01:39:06.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.505+0000 7fa034bdcb80 10 client.16074 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:06.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.505+0000 7fa034bdcb80 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:06.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.505+0000 7fa034bdcb80 20 client.16074 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:06.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.505+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d83f8461b0 0x55d83f8485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.505+0000 7fa034bdcb80 4 client.16074 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:39:06.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.505+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1924462735 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d83f7ffe10 con 0x55d83f8461b0 2023-11-06T01:39:06.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.510+0000 7fa02259c700 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d83f8461b0 0x55d83f8485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.510+0000 7fa02259c700 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d83f8461b0 0x55d83f8485d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9118c3f99da6797 server_cookie=c090b4de3665382f in_seq=0 out_seq=0 2023-11-06T01:39:06.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.511+0000 7fa009ffb700 10 client.16074 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:06.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.513+0000 7fa009ffb700 1 -- 172.21.15.53:0/1924462735 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d83f7ffe10 con 0x55d83f8461b0 2023-11-06T01:39:06.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.513+0000 7fa009ffb700 10 client.16074 handle_command_reply: tid=0 2023-11-06T01:39:06.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.513+0000 7fa034bdcb80 1 client.16074 shutdown 2023-11-06T01:39:06.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.513+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1924462735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8320f0 msgr2=0x55d83f83ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:06.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.513+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8320f0 0x55d83f83ae60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa01800a010 tx=0x7fa018005390 comp rx=0 tx=0).stop 2023-11-06T01:39:06.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.514+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1924462735 shutdown_connections 2023-11-06T01:39:06.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.514+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f832a70 0x55d83f841ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.514+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d83f8461b0 0x55d83f8485d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.514+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8313f0 0x55d83f83a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.514+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/1924462735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8320f0 0x55d83f83ae60 secure :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa01800a010 tx=0x7fa018005390 comp rx=0 tx=0).stop 2023-11-06T01:39:06.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.514+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1924462735 >> 172.21.15.53:0/1924462735 conn(0x55d83f7900d0 msgr2=0x55d83f82b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:06.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.514+0000 7fa009ffb700 0 client.16074 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:06.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.525+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1924462735 shutdown_connections 2023-11-06T01:39:06.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.530+0000 7fa034bdcb80 1 -- 172.21.15.53:0/1924462735 wait complete. 2023-11-06T01:39:06.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.531+0000 7fa034bdcb80 20 client.16074 trim_cache size 0 max 16384 2023-11-06T01:39:06.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa034bdcb80 1 Processor -- start 2023-11-06T01:39:06.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa034bdcb80 1 -- start start 2023-11-06T01:39:06.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 0x55d83f830e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f832a70 0x55d83f83fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f827d20 0x55d83f8353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f83b5e0 con 0x55d83f8313f0 2023-11-06T01:39:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f839280 con 0x55d83f827d20 2023-11-06T01:39:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f8390b0 con 0x55d83f832a70 2023-11-06T01:39:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa02259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 0x55d83f830e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa021d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f832a70 0x55d83f83fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa02259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 0x55d83f830e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41416/0 (socket says 172.21.15.53:41416) 2023-11-06T01:39:06.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.565+0000 7fa02259c700 1 -- 172.21.15.53:0/3575976632 learned_addr learned my addr 172.21.15.53:0/3575976632 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:06.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.566+0000 7fa021d9b700 1 -- 172.21.15.53:0/3575976632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f827d20 msgr2=0x55d83f8353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:06.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.566+0000 7fa021d9b700 1 --2- 172.21.15.53:0/3575976632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f827d20 0x55d83f8353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.566+0000 7fa021d9b700 1 -- 172.21.15.53:0/3575976632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 msgr2=0x55d83f830e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:06.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.566+0000 7fa021d9b700 1 --2- 172.21.15.53:0/3575976632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 0x55d83f830e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.566+0000 7fa021d9b700 1 -- 172.21.15.53:0/3575976632 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d83f598c60 con 0x55d83f832a70 2023-11-06T01:39:06.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.566+0000 7fa021d9b700 1 --2- 172.21.15.53:0/3575976632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f832a70 0x55d83f83fe50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55d83f8263f0 tx=0x7fa018009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cbd878a4b267343 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:06.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.566+0000 7fa0117fa700 1 -- 172.21.15.53:0/3575976632 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01800bc80 con 0x55d83f832a70 2023-11-06T01:39:06.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.567+0000 7fa0117fa700 1 -- 172.21.15.53:0/3575976632 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01800fb10 con 0x55d83f832a70 2023-11-06T01:39:06.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.567+0000 7fa0117fa700 1 -- 172.21.15.53:0/3575976632 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01800f460 con 0x55d83f832a70 2023-11-06T01:39:06.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.568+0000 7fa034bdcb80 1 -- 172.21.15.53:0/3575976632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f832a70 msgr2=0x55d83f83fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:06.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.568+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/3575976632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f832a70 0x55d83f83fe50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55d83f8263f0 tx=0x7fa018009290 comp rx=0 tx=0).stop 2023-11-06T01:39:06.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.570+0000 7fa034bdcb80 1 -- 172.21.15.53:0/3575976632 shutdown_connections 2023-11-06T01:39:06.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.570+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/3575976632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f827d20 0x55d83f8353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.570+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/3575976632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f8313f0 0x55d83f830e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.570+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/3575976632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f832a70 0x55d83f83fe50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.570+0000 7fa034bdcb80 1 -- 172.21.15.53:0/3575976632 >> 172.21.15.53:0/3575976632 conn(0x55d83f7900d0 msgr2=0x55d83f82b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.573+0000 7fa034bdcb80 1 -- 172.21.15.53:0/3575976632 shutdown_connections 2023-11-06T01:39:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.573+0000 7fa034bdcb80 1 -- 172.21.15.53:0/3575976632 wait complete. 2023-11-06T01:39:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.573+0000 7fa034bdcb80 1 -- start start 2023-11-06T01:39:06.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.573+0000 7fa034bdcb80 1 Processor -- start 2023-11-06T01:39:06.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f827d20 0x55d83f8986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8313f0 0x55d83f898be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa034bdcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f832a70 0x55d83f89fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f89ff80 con 0x55d83f832a70 2023-11-06T01:39:06.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f8a00f0 con 0x55d83f827d20 2023-11-06T01:39:06.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa034bdcb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83f8a02b0 con 0x55d83f8313f0 2023-11-06T01:39:06.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa034bdcb80 10 client.0 _ll_register_callbacks cb 0x7fa01c009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:06.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa02259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f827d20 0x55d83f8986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa02259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f827d20 0x55d83f8986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52860/0 (socket says 172.21.15.53:52860) 2023-11-06T01:39:06.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa02259c700 1 -- 172.21.15.53:0/4117700683 learned_addr learned my addr 172.21.15.53:0/4117700683 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:06.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa021d9b700 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8313f0 0x55d83f898be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa02259c700 1 -- 172.21.15.53:0/4117700683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8313f0 msgr2=0x55d83f898be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:06.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa02259c700 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8313f0 0x55d83f898be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa02259c700 1 -- 172.21.15.53:0/4117700683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f832a70 msgr2=0x55d83f89fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:06.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa022d9d700 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f832a70 0x55d83f89fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa02259c700 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f832a70 0x55d83f89fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa02259c700 1 -- 172.21.15.53:0/4117700683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa024108a10 con 0x55d83f827d20 2023-11-06T01:39:06.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.574+0000 7fa022d9d700 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f832a70 0x55d83f89fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:39:06.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.575+0000 7fa02259c700 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f827d20 0x55d83f8986a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa01400e630 tx=0x7fa0140047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f395363bc61ba1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:06.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.575+0000 7fa009ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:39:06.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.575+0000 7fa009ffb700 1 -- 172.21.15.53:0/4117700683 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014022040 con 0x55d83f827d20 2023-11-06T01:39:06.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.575+0000 7fa009ffb700 1 -- 172.21.15.53:0/4117700683 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa014027e10 con 0x55d83f827d20 2023-11-06T01:39:06.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.575+0000 7fa034bdcb80 1 -- 172.21.15.53:0/4117700683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa018022040 con 0x55d83f827d20 2023-11-06T01:39:06.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.575+0000 7fa009ffb700 1 -- 172.21.15.53:0/4117700683 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa014010630 con 0x55d83f827d20 2023-11-06T01:39:06.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.575+0000 7fa034bdcb80 10 client.25640 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:06.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.575+0000 7fa034bdcb80 1 -- 172.21.15.53:0/4117700683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d83f5d6250 con 0x55d83f827d20 2023-11-06T01:39:06.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.576+0000 7fa009ffb700 1 -- 172.21.15.53:0/4117700683 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa018022040 con 0x55d83f827d20 2023-11-06T01:39:06.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.576+0000 7fa009ffb700 1 -- 172.21.15.53:0/4117700683 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa014037070 con 0x55d83f827d20 2023-11-06T01:39:06.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.576+0000 7fa034bdcb80 10 client.25640 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:06.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.576+0000 7fa034bdcb80 10 client.25640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:06.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.576+0000 7fa034bdcb80 20 client.25640 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:06.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.576+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d83f8a1d00 0x55d83f8a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:06.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.576+0000 7fa034bdcb80 4 client.25640 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:39:06.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.576+0000 7fa034bdcb80 1 -- 172.21.15.53:0/4117700683 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d83f77ad40 con 0x55d83f8a1d00 2023-11-06T01:39:06.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.576+0000 7fa021d9b700 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d83f8a1d00 0x55d83f8a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:06.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.577+0000 7fa021d9b700 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d83f8a1d00 0x55d83f8a4120 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3918f7cf48078e59 server_cookie=73f7b2fe4633b844 in_seq=0 out_seq=0 2023-11-06T01:39:06.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.577+0000 7fa009ffb700 10 client.25640 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:06.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.578+0000 7fa009ffb700 1 -- 172.21.15.53:0/4117700683 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d83f77ad40 con 0x55d83f8a1d00 2023-11-06T01:39:06.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.578+0000 7fa009ffb700 10 client.25640 handle_command_reply: tid=0 2023-11-06T01:39:06.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.578+0000 7fa034bdcb80 1 client.25640 shutdown 2023-11-06T01:39:06.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa034bdcb80 1 -- 172.21.15.53:0/4117700683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f827d20 msgr2=0x55d83f8986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:06.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f827d20 0x55d83f8986a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa01400e630 tx=0x7fa0140047c0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa034bdcb80 1 -- 172.21.15.53:0/4117700683 shutdown_connections 2023-11-06T01:39:06.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d83f827d20 0x55d83f8986a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d83f8a1d00 0x55d83f8a4120 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d83f832a70 0x55d83f89fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa034bdcb80 1 --2- 172.21.15.53:0/4117700683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d83f8313f0 0x55d83f898be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa009ffb700 0 client.25640 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:06.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa034bdcb80 1 -- 172.21.15.53:0/4117700683 >> 172.21.15.53:0/4117700683 conn(0x55d83f7900d0 msgr2=0x55d83f83a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:06.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa034bdcb80 1 -- 172.21.15.53:0/4117700683 shutdown_connections 2023-11-06T01:39:06.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa034bdcb80 1 -- 172.21.15.53:0/4117700683 wait complete. 2023-11-06T01:39:06.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.579+0000 7fa034bdcb80 20 client.25640 trim_cache size 0 max 16384 2023-11-06T01:39:06.589 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:39:06.590 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:39:06.590 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:39:06.590 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:39:06.590 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:39:06.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:39:06.591 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:39:06.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:39:06.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:39:06.591 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:39:06.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.580+0000 7fa028a4c700 1 -- 172.21.15.53:0/3350674360 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa00c0a0670 msgr2=0x7fa00c0a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:06.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.580+0000 7fa028a4c700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa00c0a0670 0x7fa00c0a2b30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa024100cd0 tx=0x7fa014009260 comp rx=0 tx=0).stop 2023-11-06T01:39:06.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.580+0000 7fa028a4c700 1 -- 172.21.15.53:0/3350674360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa024101810 msgr2=0x7fa02419ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:06.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.580+0000 7fa028a4c700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa024101810 0x7fa02419ddf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00ef30 tx=0x7fa01c00c3a0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.580+0000 7fa028a4c700 1 -- 172.21.15.53:0/3350674360 shutdown_connections 2023-11-06T01:39:06.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.580+0000 7fa028a4c700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa00c0a0670 0x7fa00c0a2b30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.580+0000 7fa028a4c700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0240ffc80 0x7fa024068a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.580+0000 7fa028a4c700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa024101810 0x7fa02419ddf0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.580+0000 7fa028a4c700 1 --2- 172.21.15.53:0/3350674360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa024100e70 0x7fa02419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:06.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.580+0000 7fa028a4c700 1 -- 172.21.15.53:0/3350674360 >> 172.21.15.53:0/3350674360 conn(0x7fa0240fb1f0 msgr2=0x7fa0240fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:06.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.582+0000 7fa028a4c700 1 -- 172.21.15.53:0/3350674360 shutdown_connections 2023-11-06T01:39:06.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:06.582+0000 7fa028a4c700 1 -- 172.21.15.53:0/3350674360 wait complete. 2023-11-06T01:39:06.599 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:39:07.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 290.0 s, 1775.0 tps, lat 56.179 ms stddev 60.873 2023-11-06T01:39:07.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:07 smithi163 ceph-mon[104416]: pgmap v454: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 943 op/s 2023-11-06T01:39:07.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:07 smithi053 ceph-mon[99472]: pgmap v454: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 943 op/s 2023-11-06T01:39:07.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:07 smithi062 ceph-mon[103924]: pgmap v454: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 943 op/s 2023-11-06T01:39:09.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:09 smithi163 ceph-mon[104416]: pgmap v455: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 832 op/s 2023-11-06T01:39:09.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:09 smithi053 ceph-mon[99472]: pgmap v455: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 832 op/s 2023-11-06T01:39:09.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:09 smithi062 ceph-mon[103924]: pgmap v455: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 832 op/s 2023-11-06T01:39:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:11 smithi163 ceph-mon[104416]: pgmap v456: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 867 op/s 2023-11-06T01:39:11.598 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:11 smithi053 ceph-mon[99472]: pgmap v456: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 867 op/s 2023-11-06T01:39:11.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:11 smithi062 ceph-mon[103924]: pgmap v456: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 867 op/s 2023-11-06T01:39:12.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 295.0 s, 2544.4 tps, lat 39.120 ms stddev 43.905 2023-11-06T01:39:13.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:13 smithi163 ceph-mon[104416]: pgmap v457: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 958 op/s 2023-11-06T01:39:13.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:13 smithi053 ceph-mon[99472]: pgmap v457: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 958 op/s 2023-11-06T01:39:13.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:13 smithi062 ceph-mon[103924]: pgmap v457: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 958 op/s 2023-11-06T01:39:15.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:15 smithi163 ceph-mon[104416]: pgmap v458: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 844 op/s 2023-11-06T01:39:15.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:15 smithi053 ceph-mon[99472]: pgmap v458: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 844 op/s 2023-11-06T01:39:15.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:15 smithi062 ceph-mon[103924]: pgmap v458: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 844 op/s 2023-11-06T01:39:17.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 300.0 s, 2048.4 tps, lat 46.143 ms stddev 54.987 2023-11-06T01:39:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.432+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3743811353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0c07adc0 msgr2=0x7f8e0c07b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.432+0000 7f8e11bd7700 1 --2- 172.21.15.53:0/3743811353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0c07adc0 0x7f8e0c07b240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009990 tx=0x7f8dfc009ca0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.433+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3743811353 shutdown_connections 2023-11-06T01:39:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.433+0000 7f8e11bd7700 1 --2- 172.21.15.53:0/3743811353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0c07b780 0x7f8e0c080020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.433+0000 7f8e11bd7700 1 --2- 172.21.15.53:0/3743811353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0c079b80 0x7f8e0c079fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.433+0000 7f8e11bd7700 1 --2- 172.21.15.53:0/3743811353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0c07adc0 0x7f8e0c07b240 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.433+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3743811353 >> 172.21.15.53:0/3743811353 conn(0x7f8e0c076370 msgr2=0x7f8e0c0787d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.433+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3743811353 shutdown_connections 2023-11-06T01:39:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.433+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3743811353 wait complete. 2023-11-06T01:39:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.433+0000 7f8e11bd7700 1 Processor -- start 2023-11-06T01:39:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.434+0000 7f8e11bd7700 1 -- start start 2023-11-06T01:39:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.434+0000 7f8e11bd7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0c079b80 0x7f8e0c10bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.434+0000 7f8e11bd7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0c07adc0 0x7f8e0c10c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.434+0000 7f8e11bd7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0c07b780 0x7f8e0c10d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.434+0000 7f8e11bd7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c10d730 con 0x7f8e0c07b780 2023-11-06T01:39:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.434+0000 7f8e11bd7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c10d8a0 con 0x7f8e0c07adc0 2023-11-06T01:39:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.434+0000 7f8e11bd7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c10da60 con 0x7f8e0c079b80 2023-11-06T01:39:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0c07adc0 0x7f8e0c10c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0c079b80 0x7f8e0c10bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0c07adc0 0x7f8e0c10c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52876/0 (socket says 172.21.15.53:52876) 2023-11-06T01:39:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0c079b80 0x7f8e0c10bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55676/0 (socket says 172.21.15.53:55676) 2023-11-06T01:39:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0affd700 1 -- 172.21.15.53:0/3336597774 learned_addr learned my addr 172.21.15.53:0/3336597774 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0c07b780 0x7f8e0c10d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/3336597774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0c07adc0 msgr2=0x7f8e0c10c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0b7fe700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0c07adc0 0x7f8e0c10c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/3336597774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0c07b780 msgr2=0x7f8e0c10d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0b7fe700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0c07b780 0x7f8e0c10d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/3336597774 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc0095f0 con 0x7f8e0c079b80 2023-11-06T01:39:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.435+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0c07b780 0x7f8e0c10d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.436+0000 7f8e0b7fe700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0c079b80 0x7f8e0c10bcc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8df400b450 tx=0x7f8df400b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.436+0000 7f8e08ff9700 1 -- 172.21.15.53:0/3336597774 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4011830 con 0x7f8e0c079b80 2023-11-06T01:39:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.436+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3336597774 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0c10dd40 con 0x7f8e0c079b80 2023-11-06T01:39:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.436+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3336597774 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0c1145a0 con 0x7f8e0c079b80 2023-11-06T01:39:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.438+0000 7f8e08ff9700 1 -- 172.21.15.53:0/3336597774 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8df400be60 con 0x7f8e0c079b80 2023-11-06T01:39:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.438+0000 7f8e08ff9700 1 -- 172.21.15.53:0/3336597774 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4010d20 con 0x7f8e0c079b80 2023-11-06T01:39:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e1dd67b80 1 Processor -- start 2023-11-06T01:39:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e1dd67b80 1 -- start start 2023-11-06T01:39:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b3ff690 0x55c65b3ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ffff0 0x55c65b400900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b400ee0 0x55c65b401360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b3469b0 con 0x55c65b400ee0 2023-11-06T01:39:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b344370 con 0x55c65b3ff690 2023-11-06T01:39:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b33fc60 con 0x55c65b3ffff0 2023-11-06T01:39:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ffff0 0x55c65b400900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ffff0 0x55c65b400900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55682/0 (socket says 172.21.15.53:55682) 2023-11-06T01:39:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/2524213124 learned_addr learned my addr 172.21.15.53:0/2524213124 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.446+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/2524213124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b3ff690 0x55c65b3ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.447+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/2524213124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b3ff690 msgr2=0x55c65b3ffab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.447+0000 7f8e0b7fe700 1 --2- 172.21.15.53:0/2524213124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b3ff690 0x55c65b3ffab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.447+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/2524213124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b400ee0 msgr2=0x55c65b401360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.447+0000 7f8e0b7fe700 1 --2- 172.21.15.53:0/2524213124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b400ee0 0x55c65b401360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.447+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/2524213124 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c65b16dd80 con 0x55c65b3ffff0 2023-11-06T01:39:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.448+0000 7f8e0b7fe700 1 --2- 172.21.15.53:0/2524213124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ffff0 0x55c65b400900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8df40f52a0 tx=0x7f8df40f55b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9eece7b32b4523c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.448+0000 7f8e08ff9700 1 -- 172.21.15.53:0/3336597774 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f8df401a400 con 0x7f8e0c079b80 2023-11-06T01:39:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.449+0000 7f8e08ff9700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8df80a0620 0x7f8df80a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.449+0000 7f8e08ff9700 1 -- 172.21.15.53:0/3336597774 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8df40cb210 con 0x7f8e0c079b80 2023-11-06T01:39:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.450+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8df80a0620 0x7f8df80a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.451+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8df80a0620 0x7f8df80a2ae0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009fd0 tx=0x7f8e0400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.451+0000 7f8e01ffb700 1 -- 172.21.15.53:0/2524213124 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df40cedf0 con 0x55c65b3ffff0 2023-11-06T01:39:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.451+0000 7f8e01ffb700 1 -- 172.21.15.53:0/2524213124 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8df4095440 con 0x55c65b3ffff0 2023-11-06T01:39:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.451+0000 7f8e01ffb700 1 -- 172.21.15.53:0/2524213124 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df40a0a60 con 0x55c65b3ffff0 2023-11-06T01:39:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.452+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2524213124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ffff0 msgr2=0x55c65b400900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.452+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/2524213124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ffff0 0x55c65b400900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8df40f52a0 tx=0x7f8df40f55b0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.455+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2524213124 shutdown_connections 2023-11-06T01:39:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.455+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/2524213124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b3ff690 0x55c65b3ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.455+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/2524213124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b400ee0 0x55c65b401360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.455+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/2524213124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ffff0 0x55c65b400900 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.455+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2524213124 >> 172.21.15.53:0/2524213124 conn(0x55c65b34e070 msgr2=0x55c65b3eb040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.455+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2524213124 shutdown_connections 2023-11-06T01:39:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.455+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2524213124 wait complete. 2023-11-06T01:39:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.455+0000 7f8e1dd67b80 1 -- start start 2023-11-06T01:39:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.456+0000 7f8e1dd67b80 1 Processor -- start 2023-11-06T01:39:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.456+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3efa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.456+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b3ffff0 0x55c65b3effa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.456+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b400ee0 0x55c65b3f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.456+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b2dc4b0 con 0x55c65b400ee0 2023-11-06T01:39:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.456+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b2dc150 con 0x55c65b3ffff0 2023-11-06T01:39:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.456+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b1c38c0 con 0x55c65b3ff690 2023-11-06T01:39:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.456+0000 7f8e1dd67b80 10 client.0 _ll_register_callbacks cb 0x55c65b229a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.457+0000 7f8e0bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b400ee0 0x55c65b3f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.457+0000 7f8e0bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b400ee0 0x55c65b3f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41452/0 (socket says 172.21.15.53:41452) 2023-11-06T01:39:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.457+0000 7f8e0bfff700 1 -- 172.21.15.53:0/827530621 learned_addr learned my addr 172.21.15.53:0/827530621 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.457+0000 7f8e0bfff700 1 -- 172.21.15.53:0/827530621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 msgr2=0x55c65b3efa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.457+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/827530621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.457+0000 7f8e0bfff700 1 -- 172.21.15.53:0/827530621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b3ffff0 msgr2=0x55c65b3effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.457+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/827530621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b3ffff0 0x55c65b3effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.457+0000 7f8e0bfff700 1 -- 172.21.15.53:0/827530621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0c1145a0 con 0x55c65b400ee0 2023-11-06T01:39:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.457+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/827530621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b400ee0 0x55c65b3f6fe0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8e04018e80 tx=0x7f8e04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d17ed11674fc17 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.457+0000 7f8deaffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:39:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.459+0000 7f8deaffd700 1 -- 172.21.15.53:0/827530621 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04010040 con 0x55c65b400ee0 2023-11-06T01:39:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.459+0000 7f8deaffd700 1 -- 172.21.15.53:0/827530621 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8e04006e40 con 0x55c65b400ee0 2023-11-06T01:39:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.459+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/827530621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c65b36acb0 con 0x55c65b400ee0 2023-11-06T01:39:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.462+0000 7f8deaffd700 1 -- 172.21.15.53:0/827530621 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e04016c50 con 0x55c65b400ee0 2023-11-06T01:39:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.462+0000 7f8e1dd67b80 10 client.16083 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.462+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/827530621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c65b156c00 con 0x55c65b400ee0 2023-11-06T01:39:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.463+0000 7f8deaffd700 1 -- 172.21.15.53:0/827530621 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e0402f030 con 0x55c65b400ee0 2023-11-06T01:39:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.463+0000 7f8deaffd700 1 -- 172.21.15.53:0/827530621 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8e040173c0 con 0x55c65b400ee0 2023-11-06T01:39:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.463+0000 7f8e1dd67b80 10 client.16083 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.463+0000 7f8e1dd67b80 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.463+0000 7f8e1dd67b80 20 client.16083 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.463+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/827530621 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c65b3fb9a0 0x55c65b3fddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.463+0000 7f8e1dd67b80 4 client.16083 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:39:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.463+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/827530621 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c65b3bddb0 con 0x55c65b3fb9a0 2023-11-06T01:39:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.468+0000 7f8e0b7fe700 1 --2- 172.21.15.53:0/827530621 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c65b3fb9a0 0x55c65b3fddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.468+0000 7f8e0b7fe700 1 --2- 172.21.15.53:0/827530621 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c65b3fb9a0 0x55c65b3fddc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b2b582e68ae40f server_cookie=669296d2a98f01ce in_seq=0 out_seq=0 2023-11-06T01:39:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.468+0000 7f8deaffd700 10 client.16083 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.469+0000 7f8deaffd700 1 -- 172.21.15.53:0/827530621 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c65b3bddb0 con 0x55c65b3fb9a0 2023-11-06T01:39:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.469+0000 7f8deaffd700 10 client.16083 handle_command_reply: tid=0 2023-11-06T01:39:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.469+0000 7f8e1dd67b80 1 client.16083 shutdown 2023-11-06T01:39:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.470+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/827530621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b400ee0 msgr2=0x55c65b3f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.470+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/827530621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b400ee0 0x55c65b3f6fe0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8e04018e80 tx=0x7f8e04004ad0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.474+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/827530621 shutdown_connections 2023-11-06T01:39:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.474+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/827530621 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c65b3fb9a0 0x55c65b3fddc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.474+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/827530621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b400ee0 0x55c65b3f6fe0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.474+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/827530621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.474+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/827530621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b3ffff0 0x55c65b3effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.474+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/827530621 >> 172.21.15.53:0/827530621 conn(0x55c65b34e070 msgr2=0x55c65b3e93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.474+0000 7f8deaffd700 0 client.16083 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.475+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/827530621 shutdown_connections 2023-11-06T01:39:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.475+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/827530621 wait complete. 2023-11-06T01:39:17.483 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.482+0000 7f8e1dd67b80 20 client.16083 trim_cache size 0 max 16384 2023-11-06T01:39:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.510+0000 7f8e1dd67b80 1 Processor -- start 2023-11-06T01:39:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.510+0000 7f8e1dd67b80 1 -- start start 2023-11-06T01:39:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.510+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.510+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b3ffff0 0x55c65b3ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.510+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b400ee0 0x55c65b3efa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.510+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b3ee440 con 0x55c65b3ffff0 2023-11-06T01:39:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.510+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b3f8ca0 con 0x55c65b400ee0 2023-11-06T01:39:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.510+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b4003d0 con 0x55c65b3ff690 2023-11-06T01:39:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.510+0000 7f8e0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.510+0000 7f8e0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3f4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55696/0 (socket says 172.21.15.53:55696) 2023-11-06T01:39:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.510+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/4189542402 learned_addr learned my addr 172.21.15.53:0/4189542402 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.511+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/4189542402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b400ee0 0x55c65b3efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.511+0000 7f8e0affd700 1 --2- 172.21.15.53:0/4189542402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b3ffff0 0x55c65b3ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.511+0000 7f8e0bfff700 1 -- 172.21.15.53:0/4189542402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 msgr2=0x55c65b3f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.511+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/4189542402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.511+0000 7f8e0bfff700 1 -- 172.21.15.53:0/4189542402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b3ffff0 msgr2=0x55c65b3ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.511+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/4189542402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b3ffff0 0x55c65b3ee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.511+0000 7f8e0bfff700 1 -- 172.21.15.53:0/4189542402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c65b1941f0 con 0x55c65b400ee0 2023-11-06T01:39:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.511+0000 7f8e0affd700 1 --2- 172.21.15.53:0/4189542402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b3ffff0 0x55c65b3ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.511+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/4189542402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b400ee0 0x55c65b3efa60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8e04016aa0 tx=0x7f8e040111b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1d74a7d735c3254 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.512+0000 7f8e027fc700 1 -- 172.21.15.53:0/4189542402 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04010040 con 0x55c65b400ee0 2023-11-06T01:39:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.512+0000 7f8e027fc700 1 -- 172.21.15.53:0/4189542402 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e0402f040 con 0x55c65b400ee0 2023-11-06T01:39:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.512+0000 7f8e027fc700 1 -- 172.21.15.53:0/4189542402 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04016bd0 con 0x55c65b400ee0 2023-11-06T01:39:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.513+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/4189542402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b400ee0 msgr2=0x55c65b3efa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.513+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/4189542402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b400ee0 0x55c65b3efa60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8e04016aa0 tx=0x7f8e040111b0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.513+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/4189542402 shutdown_connections 2023-11-06T01:39:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.513+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/4189542402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b400ee0 0x55c65b3efa60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.513+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/4189542402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.513+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/4189542402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b3ffff0 0x55c65b3ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.513+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/4189542402 >> 172.21.15.53:0/4189542402 conn(0x55c65b34e070 msgr2=0x55c65b3fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.514+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/4189542402 shutdown_connections 2023-11-06T01:39:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.514+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/4189542402 wait complete. 2023-11-06T01:39:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.514+0000 7f8e1dd67b80 1 -- start start 2023-11-06T01:39:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.514+0000 7f8e1dd67b80 1 Processor -- start 2023-11-06T01:39:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b3ffff0 0x55c65b3f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e1dd67b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b400ee0 0x55c65b454df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b455330 con 0x55c65b3ffff0 2023-11-06T01:39:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b4554a0 con 0x55c65b400ee0 2023-11-06T01:39:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e1dd67b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b455660 con 0x55c65b3ff690 2023-11-06T01:39:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e1dd67b80 10 client.0 _ll_register_callbacks cb 0x7f8e04018820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3f42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55702/0 (socket says 172.21.15.53:55702) 2023-11-06T01:39:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/2676081279 learned_addr learned my addr 172.21.15.53:0/2676081279 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e0bfff700 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b400ee0 0x55c65b454df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/2676081279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b400ee0 msgr2=0x55c65b454df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e0b7fe700 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b400ee0 0x55c65b454df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/2676081279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b3ffff0 msgr2=0x55c65b3f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e0b7fe700 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b3ffff0 0x55c65b3f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.515+0000 7f8e0b7fe700 1 -- 172.21.15.53:0/2676081279 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0c080a50 con 0x55c65b3ff690 2023-11-06T01:39:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.516+0000 7f8e0b7fe700 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3f42a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8df40955b0 tx=0x7f8df4010e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d739093be639cc99 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.516+0000 7f8deaffd700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:39:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.516+0000 7f8deaffd700 1 -- 172.21.15.53:0/2676081279 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4009280 con 0x55c65b3ff690 2023-11-06T01:39:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.516+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2676081279 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8df4095440 con 0x55c65b3ff690 2023-11-06T01:39:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.516+0000 7f8deaffd700 1 -- 172.21.15.53:0/2676081279 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8df40b65b0 con 0x55c65b3ff690 2023-11-06T01:39:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.516+0000 7f8deaffd700 1 -- 172.21.15.53:0/2676081279 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8df40cedf0 con 0x55c65b3ff690 2023-11-06T01:39:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.516+0000 7f8e1dd67b80 10 client.25723 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.516+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2676081279 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c65b156c00 con 0x55c65b3ff690 2023-11-06T01:39:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.517+0000 7f8deaffd700 1 -- 172.21.15.53:0/2676081279 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8df4095440 con 0x55c65b3ff690 2023-11-06T01:39:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.517+0000 7f8deaffd700 1 -- 172.21.15.53:0/2676081279 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8df4095810 con 0x55c65b3ff690 2023-11-06T01:39:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.517+0000 7f8e1dd67b80 10 client.25723 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.517+0000 7f8e1dd67b80 10 client.25723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.517+0000 7f8e1dd67b80 20 client.25723 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.517+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c65b456c60 0x55c65b459080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.517+0000 7f8e1dd67b80 4 client.25723 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:39:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.517+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2676081279 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c65b338ce0 con 0x55c65b456c60 2023-11-06T01:39:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.517+0000 7f8e0affd700 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c65b456c60 0x55c65b459080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.517+0000 7f8deaffd700 10 client.25723 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.517+0000 7f8e0affd700 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c65b456c60 0x55c65b459080 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6130a41cfc0bf352 server_cookie=9ad92769e1f4555 in_seq=0 out_seq=0 2023-11-06T01:39:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8deaffd700 1 -- 172.21.15.53:0/2676081279 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c65b338ce0 con 0x55c65b456c60 2023-11-06T01:39:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8deaffd700 10 client.25723 handle_command_reply: tid=0 2023-11-06T01:39:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 1 client.25723 shutdown 2023-11-06T01:39:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2676081279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 msgr2=0x55c65b3f42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3f42a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8df40955b0 tx=0x7f8df4010e80 comp rx=0 tx=0).stop 2023-11-06T01:39:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2676081279 shutdown_connections 2023-11-06T01:39:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c65b400ee0 0x55c65b454df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c65b456c60 0x55c65b459080 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c65b3ff690 0x55c65b3f42a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8deaffd700 0 client.25723 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 1 --2- 172.21.15.53:0/2676081279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c65b3ffff0 0x55c65b3f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2676081279 >> 172.21.15.53:0/2676081279 conn(0x55c65b34e070 msgr2=0x55c65b3ee680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2676081279 shutdown_connections 2023-11-06T01:39:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 1 -- 172.21.15.53:0/2676081279 wait complete. 2023-11-06T01:39:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.549+0000 7f8e1dd67b80 20 client.25723 trim_cache size 0 max 16384 2023-11-06T01:39:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:39:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:39:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:39:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:39:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3336597774 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8df80a0620 msgr2=0x7f8df80a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8df80a0620 0x7f8df80a2ae0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009fd0 tx=0x7f8e0400d040 comp rx=0 tx=0).stop 2023-11-06T01:39:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3336597774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0c079b80 msgr2=0x7f8e0c10bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0c079b80 0x7f8e0c10bcc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8df400b450 tx=0x7f8df400b760 comp rx=0 tx=0).stop 2023-11-06T01:39:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3336597774 shutdown_connections 2023-11-06T01:39:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8df80a0620 0x7f8df80a2ae0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0c07b780 0x7f8e0c10d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0c079b80 0x7f8e0c10bcc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 --2- 172.21.15.53:0/3336597774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0c07adc0 0x7f8e0c10c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3336597774 >> 172.21.15.53:0/3336597774 conn(0x7f8e0c076370 msgr2=0x7f8e0c076fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3336597774 shutdown_connections 2023-11-06T01:39:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:17.550+0000 7f8e11bd7700 1 -- 172.21.15.53:0/3336597774 wait complete. 2023-11-06T01:39:17.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:17 smithi053 ceph-mon[99472]: pgmap v459: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 938 op/s 2023-11-06T01:39:17.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:17 smithi062 ceph-mon[103924]: pgmap v459: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 938 op/s 2023-11-06T01:39:17.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:17 smithi163 ceph-mon[104416]: pgmap v459: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 938 op/s 2023-11-06T01:39:19.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:19 smithi053 ceph-mon[99472]: pgmap v460: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 826 op/s 2023-11-06T01:39:19.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:19 smithi062 ceph-mon[103924]: pgmap v460: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 826 op/s 2023-11-06T01:39:19.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:19 smithi163 ceph-mon[104416]: pgmap v460: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 826 op/s 2023-11-06T01:39:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:21 smithi053 ceph-mon[99472]: pgmap v461: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 1.16k op/s 2023-11-06T01:39:21.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:21 smithi062 ceph-mon[103924]: pgmap v461: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 1.16k op/s 2023-11-06T01:39:21.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:21 smithi163 ceph-mon[104416]: pgmap v461: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 1.16k op/s 2023-11-06T01:39:22.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 305.0 s, 2401.0 tps, lat 43.348 ms stddev 81.196 2023-11-06T01:39:23.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:23 smithi053 ceph-mon[99472]: pgmap v462: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 MiB/s wr, 1.94k op/s 2023-11-06T01:39:23.658 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:23 smithi062 ceph-mon[103924]: pgmap v462: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 MiB/s wr, 1.94k op/s 2023-11-06T01:39:23.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:23 smithi163 ceph-mon[104416]: pgmap v462: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 MiB/s wr, 1.94k op/s 2023-11-06T01:39:25.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:25 smithi053 ceph-mon[99472]: pgmap v463: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 MiB/s wr, 1.99k op/s 2023-11-06T01:39:25.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:25 smithi062 ceph-mon[103924]: pgmap v463: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 MiB/s wr, 1.99k op/s 2023-11-06T01:39:25.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:25 smithi163 ceph-mon[104416]: pgmap v463: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 MiB/s wr, 1.99k op/s 2023-11-06T01:39:27.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 310.0 s, 2475.2 tps, lat 40.855 ms stddev 50.968 2023-11-06T01:39:27.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:27 smithi053 ceph-mon[99472]: pgmap v464: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 MiB/s wr, 2.19k op/s 2023-11-06T01:39:27.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:27 smithi062 ceph-mon[103924]: pgmap v464: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 MiB/s wr, 2.19k op/s 2023-11-06T01:39:27.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:27 smithi163 ceph-mon[104416]: pgmap v464: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 MiB/s wr, 2.19k op/s 2023-11-06T01:39:29.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:29 smithi053 ceph-mon[99472]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.05k op/s 2023-11-06T01:39:29.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:29 smithi062 ceph-mon[103924]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.05k op/s 2023-11-06T01:39:29.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:29 smithi163 ceph-mon[104416]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.05k op/s 2023-11-06T01:39:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.724+0000 7fc7abf80700 1 -- 172.21.15.53:0/2859464795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a410b7d0 msgr2=0x7fc7a410dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.724+0000 7fc7abf80700 1 --2- 172.21.15.53:0/2859464795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a410b7d0 0x7fc7a410dbc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7940098f0 tx=0x7fc794009c00 comp rx=0 tx=0).stop 2023-11-06T01:39:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.724+0000 7fc7abf80700 1 -- 172.21.15.53:0/2859464795 shutdown_connections 2023-11-06T01:39:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.724+0000 7fc7abf80700 1 --2- 172.21.15.53:0/2859464795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a410e190 0x7fc7a4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.724+0000 7fc7abf80700 1 --2- 172.21.15.53:0/2859464795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a4108ea0 0x7fc7a410b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.724+0000 7fc7abf80700 1 --2- 172.21.15.53:0/2859464795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a410b7d0 0x7fc7a410dbc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.724+0000 7fc7abf80700 1 -- 172.21.15.53:0/2859464795 >> 172.21.15.53:0/2859464795 conn(0x7fc7a406dbc0 msgr2=0x7fc7a4070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.728+0000 7fc7abf80700 1 -- 172.21.15.53:0/2859464795 shutdown_connections 2023-11-06T01:39:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.728+0000 7fc7abf80700 1 -- 172.21.15.53:0/2859464795 wait complete. 2023-11-06T01:39:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.728+0000 7fc7abf80700 1 Processor -- start 2023-11-06T01:39:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.728+0000 7fc7abf80700 1 -- start start 2023-11-06T01:39:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7abf80700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a4108ea0 0x7fc7a41a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7abf80700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a410b7d0 0x7fc7a41a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7abf80700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a410e190 0x7fc7a41a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7abf80700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a41a7420 con 0x7fc7a410b7d0 2023-11-06T01:39:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7abf80700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a41a7590 con 0x7fc7a4108ea0 2023-11-06T01:39:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7abf80700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a41a7750 con 0x7fc7a410e190 2023-11-06T01:39:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7a951b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a410b7d0 0x7fc7a41a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7aa51d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a410e190 0x7fc7a41a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7aa51d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a410e190 0x7fc7a41a6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55720/0 (socket says 172.21.15.53:55720) 2023-11-06T01:39:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7aa51d700 1 -- 172.21.15.53:0/431611196 learned_addr learned my addr 172.21.15.53:0/431611196 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7a9d1c700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a4108ea0 0x7fc7a41a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7aa51d700 1 -- 172.21.15.53:0/431611196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a4108ea0 msgr2=0x7fc7a41a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7aa51d700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a4108ea0 0x7fc7a41a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7aa51d700 1 -- 172.21.15.53:0/431611196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a410b7d0 msgr2=0x7fc7a41a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7aa51d700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a410b7d0 0x7fc7a41a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7aa51d700 1 -- 172.21.15.53:0/431611196 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc794009520 con 0x7fc7a410e190 2023-11-06T01:39:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.729+0000 7fc7a951b700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a410b7d0 0x7fc7a41a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.730+0000 7fc7aa51d700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a410e190 0x7fc7a41a6dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00ef30 tx=0x7fc79c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.730+0000 7fc79affd700 1 -- 172.21.15.53:0/431611196 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c018070 con 0x7fc7a410e190 2023-11-06T01:39:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.730+0000 7fc79affd700 1 -- 172.21.15.53:0/431611196 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc79c00f040 con 0x7fc7a410e190 2023-11-06T01:39:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.730+0000 7fc7abf80700 1 -- 172.21.15.53:0/431611196 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a41a79d0 con 0x7fc7a410e190 2023-11-06T01:39:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.730+0000 7fc79affd700 1 -- 172.21.15.53:0/431611196 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00ccf0 con 0x7fc7a410e190 2023-11-06T01:39:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.730+0000 7fc7abf80700 1 -- 172.21.15.53:0/431611196 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a41ae1f0 con 0x7fc7a410e190 2023-11-06T01:39:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.733+0000 7fc79affd700 1 -- 172.21.15.53:0/431611196 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fc79c003a00 con 0x7fc7a410e190 2023-11-06T01:39:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.734+0000 7fc79affd700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7900a0520 0x7fc7900a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.734+0000 7fc79affd700 1 -- 172.21.15.53:0/431611196 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc79c0cb4e0 con 0x7fc7a410e190 2023-11-06T01:39:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.734+0000 7fc7a9d1c700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7900a0520 0x7fc7900a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7b8110b80 1 Processor -- start 2023-11-06T01:39:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7b8110b80 1 -- start start 2023-11-06T01:39:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7a9d1c700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7900a0520 0x7fc7900a29e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4072160 tx=0x7fc7a000f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2680 0x55b3041c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 0x55b3041c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c3ed0 0x55b3041c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30411a9b0 con 0x55b3041c2fe0 2023-11-06T01:39:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b304118370 con 0x55b3041c2680 2023-11-06T01:39:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b304113c60 con 0x55b3041c3ed0 2023-11-06T01:39:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7a9d1c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2680 0x55b3041c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7a9d1c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2680 0x55b3041c2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52922/0 (socket says 172.21.15.53:52922) 2023-11-06T01:39:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7a951b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 0x55b3041c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.735+0000 7fc7a9d1c700 1 -- 172.21.15.53:0/2297439887 learned_addr learned my addr 172.21.15.53:0/2297439887 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.736+0000 7fc7aa51d700 1 --2- 172.21.15.53:0/2297439887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c3ed0 0x55b3041c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.736+0000 7fc7a9d1c700 1 -- 172.21.15.53:0/2297439887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c3ed0 msgr2=0x55b3041c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.736+0000 7fc7a9d1c700 1 --2- 172.21.15.53:0/2297439887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c3ed0 0x55b3041c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.736+0000 7fc7a9d1c700 1 -- 172.21.15.53:0/2297439887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 msgr2=0x55b3041c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.736+0000 7fc7a9d1c700 1 --2- 172.21.15.53:0/2297439887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 0x55b3041c38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.736+0000 7fc7a9d1c700 1 -- 172.21.15.53:0/2297439887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b303f41d80 con 0x55b3041c2680 2023-11-06T01:39:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.737+0000 7fc7a9d1c700 1 --2- 172.21.15.53:0/2297439887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2680 0x55b3041c2aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000dd40 tx=0x7fc7a0015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28e507ec8542fd1c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.738+0000 7fc78ffff700 1 -- 172.21.15.53:0/2297439887 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0009e60 con 0x55b3041c2680 2023-11-06T01:39:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.738+0000 7fc78ffff700 1 -- 172.21.15.53:0/2297439887 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a0013040 con 0x55b3041c2680 2023-11-06T01:39:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.738+0000 7fc78ffff700 1 -- 172.21.15.53:0/2297439887 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00053c0 con 0x55b3041c2680 2023-11-06T01:39:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.739+0000 7fc7b8110b80 1 -- 172.21.15.53:0/2297439887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2680 msgr2=0x55b3041c2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.739+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/2297439887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2680 0x55b3041c2aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000dd40 tx=0x7fc7a0015000 comp rx=0 tx=0).stop 2023-11-06T01:39:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.739+0000 7fc7b8110b80 1 -- 172.21.15.53:0/2297439887 shutdown_connections 2023-11-06T01:39:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.739+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/2297439887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2680 0x55b3041c2aa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.739+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/2297439887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c3ed0 0x55b3041c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.739+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/2297439887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 0x55b3041c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.739+0000 7fc7b8110b80 1 -- 172.21.15.53:0/2297439887 >> 172.21.15.53:0/2297439887 conn(0x55b304122070 msgr2=0x55b3041bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.739+0000 7fc7b8110b80 1 -- 172.21.15.53:0/2297439887 shutdown_connections 2023-11-06T01:39:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.739+0000 7fc7b8110b80 1 -- 172.21.15.53:0/2297439887 wait complete. 2023-11-06T01:39:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7b8110b80 1 -- start start 2023-11-06T01:39:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7b8110b80 1 Processor -- start 2023-11-06T01:39:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c2680 0x55b3041cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 0x55b3041cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c3ed0 0x55b3041d3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3040b04b0 con 0x55b3041c3ed0 2023-11-06T01:39:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3040b0150 con 0x55b3041c2fe0 2023-11-06T01:39:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b303f978c0 con 0x55b3041c2680 2023-11-06T01:39:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7b8110b80 10 client.0 _ll_register_callbacks cb 0x55b303ffda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7a951b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 0x55b3041cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7a9d1c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c2680 0x55b3041cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7a951b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 0x55b3041cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52930/0 (socket says 172.21.15.53:52930) 2023-11-06T01:39:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7a9d1c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c2680 0x55b3041cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55730/0 (socket says 172.21.15.53:55730) 2023-11-06T01:39:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7a951b700 1 -- 172.21.15.53:0/3664199592 learned_addr learned my addr 172.21.15.53:0/3664199592 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.740+0000 7fc7aa51d700 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c3ed0 0x55b3041d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc7a951b700 1 -- 172.21.15.53:0/3664199592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c2680 msgr2=0x55b3041cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc7a951b700 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c2680 0x55b3041cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc7a951b700 1 -- 172.21.15.53:0/3664199592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c3ed0 msgr2=0x55b3041d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc7a951b700 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c3ed0 0x55b3041d3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc7a951b700 1 -- 172.21.15.53:0/3664199592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a000dab0 con 0x55b3041c2fe0 2023-11-06T01:39:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc7aa51d700 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c3ed0 0x55b3041d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc7a951b700 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 0x55b3041cc940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4072730 tx=0x7fc79400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df834888ca36620 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc78d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:39:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc78d7fa700 1 -- 172.21.15.53:0/3664199592 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79401c040 con 0x55b3041c2fe0 2023-11-06T01:39:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc78d7fa700 1 -- 172.21.15.53:0/3664199592 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc794005040 con 0x55b3041c2fe0 2023-11-06T01:39:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc7b8110b80 1 -- 172.21.15.53:0/3664199592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b30413ecb0 con 0x55b3041c2fe0 2023-11-06T01:39:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc78d7fa700 1 -- 172.21.15.53:0/3664199592 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc794005200 con 0x55b3041c2fe0 2023-11-06T01:39:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc7b8110b80 10 client.25667 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.741+0000 7fc7b8110b80 1 -- 172.21.15.53:0/3664199592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b303f2ac00 con 0x55b3041c2fe0 2023-11-06T01:39:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.742+0000 7fc78d7fa700 1 -- 172.21.15.53:0/3664199592 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b30413ecb0 con 0x55b3041c2fe0 2023-11-06T01:39:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.742+0000 7fc78d7fa700 1 -- 172.21.15.53:0/3664199592 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fc794015020 con 0x55b3041c2fe0 2023-11-06T01:39:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.742+0000 7fc7b8110b80 10 client.25667 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.742+0000 7fc7b8110b80 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.742+0000 7fc7b8110b80 20 client.25667 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.742+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3041d82a0 0x55b3041da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.742+0000 7fc7b8110b80 4 client.25667 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:39:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.742+0000 7fc7b8110b80 1 -- 172.21.15.53:0/3664199592 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b304191db0 con 0x55b3041d82a0 2023-11-06T01:39:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.742+0000 7fc7a9d1c700 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3041d82a0 0x55b3041da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.743+0000 7fc78d7fa700 10 client.25667 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.743+0000 7fc7a9d1c700 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3041d82a0 0x55b3041da6c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee33bae2d399f517 server_cookie=6cea2bedd02d34c3 in_seq=0 out_seq=0 2023-11-06T01:39:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.743+0000 7fc78d7fa700 1 -- 172.21.15.53:0/3664199592 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b304191db0 con 0x55b3041d82a0 2023-11-06T01:39:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.743+0000 7fc78d7fa700 10 client.25667 handle_command_reply: tid=0 2023-11-06T01:39:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.743+0000 7fc7b8110b80 1 client.25667 shutdown 2023-11-06T01:39:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.743+0000 7fc7b8110b80 1 -- 172.21.15.53:0/3664199592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 msgr2=0x55b3041cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.743+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 0x55b3041cc940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4072730 tx=0x7fc79400b790 comp rx=0 tx=0).stop 2023-11-06T01:39:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.744+0000 7fc7b8110b80 1 -- 172.21.15.53:0/3664199592 shutdown_connections 2023-11-06T01:39:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.744+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3041d82a0 0x55b3041da6c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.744+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c3ed0 0x55b3041d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.744+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c2680 0x55b3041cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.744+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/3664199592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 0x55b3041cc940 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.744+0000 7fc7b8110b80 1 -- 172.21.15.53:0/3664199592 >> 172.21.15.53:0/3664199592 conn(0x55b304122070 msgr2=0x55b3041bd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.744+0000 7fc78d7fa700 0 client.25667 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.744+0000 7fc7b8110b80 1 -- 172.21.15.53:0/3664199592 shutdown_connections 2023-11-06T01:39:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.744+0000 7fc7b8110b80 1 -- 172.21.15.53:0/3664199592 wait complete. 2023-11-06T01:39:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.744+0000 7fc7b8110b80 20 client.25667 trim_cache size 0 max 16384 2023-11-06T01:39:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.765+0000 7fc7b8110b80 1 Processor -- start 2023-11-06T01:39:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7b8110b80 1 -- start start 2023-11-06T01:39:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c2680 0x55b3041d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 0x55b3041c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c3ed0 0x55b3041bec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3041b8500 con 0x55b3041c2fe0 2023-11-06T01:39:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3041cbf70 con 0x55b3041c3ed0 2023-11-06T01:39:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3041bf6f0 con 0x55b3041c2680 2023-11-06T01:39:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7a951b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 0x55b3041c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7a9d1c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c2680 0x55b3041d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7a951b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 0x55b3041c42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41500/0 (socket says 172.21.15.53:41500) 2023-11-06T01:39:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7a951b700 1 -- 172.21.15.53:0/2170745282 learned_addr learned my addr 172.21.15.53:0/2170745282 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7aa51d700 1 --2- 172.21.15.53:0/2170745282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c3ed0 0x55b3041bec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7a951b700 1 -- 172.21.15.53:0/2170745282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c2680 msgr2=0x55b3041d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7a951b700 1 --2- 172.21.15.53:0/2170745282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c2680 0x55b3041d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7a951b700 1 -- 172.21.15.53:0/2170745282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c3ed0 msgr2=0x55b3041bec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7a951b700 1 --2- 172.21.15.53:0/2170745282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c3ed0 0x55b3041bec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7a951b700 1 -- 172.21.15.53:0/2170745282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b303f681f0 con 0x55b3041c2fe0 2023-11-06T01:39:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.766+0000 7fc7aa51d700 1 --2- 172.21.15.53:0/2170745282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c3ed0 0x55b3041bec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.767+0000 7fc7a951b700 1 --2- 172.21.15.53:0/2170745282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 0x55b3041c42b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc794005a40 tx=0x7fc794009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71062b2653dee8cf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.767+0000 7fc798ff9700 1 -- 172.21.15.53:0/2170745282 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79401c040 con 0x55b3041c2fe0 2023-11-06T01:39:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.767+0000 7fc798ff9700 1 -- 172.21.15.53:0/2170745282 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc79401d040 con 0x55b3041c2fe0 2023-11-06T01:39:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.767+0000 7fc798ff9700 1 -- 172.21.15.53:0/2170745282 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794005330 con 0x55b3041c2fe0 2023-11-06T01:39:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.768+0000 7fc7b8110b80 1 -- 172.21.15.53:0/2170745282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 msgr2=0x55b3041c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.768+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/2170745282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 0x55b3041c42b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc794005a40 tx=0x7fc794009c00 comp rx=0 tx=0).stop 2023-11-06T01:39:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.768+0000 7fc7b8110b80 1 -- 172.21.15.53:0/2170745282 shutdown_connections 2023-11-06T01:39:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.768+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/2170745282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c3ed0 0x55b3041bec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.768+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/2170745282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c2680 0x55b3041d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.768+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/2170745282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2fe0 0x55b3041c42b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.768+0000 7fc7b8110b80 1 -- 172.21.15.53:0/2170745282 >> 172.21.15.53:0/2170745282 conn(0x55b304122070 msgr2=0x55b3041cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.768+0000 7fc7b8110b80 1 -- 172.21.15.53:0/2170745282 shutdown_connections 2023-11-06T01:39:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.768+0000 7fc7b8110b80 1 -- 172.21.15.53:0/2170745282 wait complete. 2023-11-06T01:39:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.768+0000 7fc7b8110b80 1 -- start start 2023-11-06T01:39:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.768+0000 7fc7b8110b80 1 Processor -- start 2023-11-06T01:39:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2680 0x55b3041d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 0x55b3041d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7b8110b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c3ed0 0x55b304227ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b304228410 con 0x55b3041c2680 2023-11-06T01:39:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b304228550 con 0x55b3041c2fe0 2023-11-06T01:39:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7b8110b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3042286e0 con 0x55b3041c3ed0 2023-11-06T01:39:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7b8110b80 10 client.0 _ll_register_callbacks cb 0x7fc79400f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7a9d1c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2680 0x55b3041d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7a9d1c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2680 0x55b3041d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41504/0 (socket says 172.21.15.53:41504) 2023-11-06T01:39:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7a9d1c700 1 -- 172.21.15.53:0/1450252262 learned_addr learned my addr 172.21.15.53:0/1450252262 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7aa51d700 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c3ed0 0x55b304227ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7a951b700 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 0x55b3041d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7a9d1c700 1 -- 172.21.15.53:0/1450252262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c3ed0 msgr2=0x55b304227ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7a9d1c700 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c3ed0 0x55b304227ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7a9d1c700 1 -- 172.21.15.53:0/1450252262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 msgr2=0x55b3041d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7a9d1c700 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 0x55b3041d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7a9d1c700 1 -- 172.21.15.53:0/1450252262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a41ae1f0 con 0x55b3041c2680 2023-11-06T01:39:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.769+0000 7fc7a951b700 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 0x55b3041d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:39:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.772+0000 7fc7a9d1c700 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2680 0x55b3041d0b20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0026040 tx=0x7fc7a0009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9185e071928f561f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.772+0000 7fc78d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:39:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.772+0000 7fc78d7fa700 1 -- 172.21.15.53:0/1450252262 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0034070 con 0x55b3041c2680 2023-11-06T01:39:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.772+0000 7fc78d7fa700 1 -- 172.21.15.53:0/1450252262 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc7a00052c0 con 0x55b3041c2680 2023-11-06T01:39:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.772+0000 7fc7b8110b80 1 -- 172.21.15.53:0/1450252262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7a0013040 con 0x55b3041c2680 2023-11-06T01:39:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.772+0000 7fc78d7fa700 1 -- 172.21.15.53:0/1450252262 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7a0005480 con 0x55b3041c2680 2023-11-06T01:39:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.773+0000 7fc7b8110b80 10 client.16104 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.773+0000 7fc7b8110b80 1 -- 172.21.15.53:0/1450252262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b303f2ac00 con 0x55b3041c2680 2023-11-06T01:39:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.773+0000 7fc78d7fa700 1 -- 172.21.15.53:0/1450252262 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a0048080 con 0x55b3041c2680 2023-11-06T01:39:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.773+0000 7fc78d7fa700 1 -- 172.21.15.53:0/1450252262 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fc7a0042030 con 0x55b3041c2680 2023-11-06T01:39:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.773+0000 7fc7b8110b80 10 client.16104 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.773+0000 7fc7b8110b80 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.773+0000 7fc7b8110b80 20 client.16104 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.773+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b30422a000 0x55b30422c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.773+0000 7fc7b8110b80 4 client.16104 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:39:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.773+0000 7fc7b8110b80 1 -- 172.21.15.53:0/1450252262 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b30410cce0 con 0x55b30422a000 2023-11-06T01:39:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.773+0000 7fc7a951b700 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b30422a000 0x55b30422c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.774+0000 7fc78d7fa700 10 client.16104 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.774+0000 7fc7a951b700 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b30422a000 0x55b30422c420 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=710c6723065e7552 server_cookie=d33c3fa8e4014b04 in_seq=0 out_seq=0 2023-11-06T01:39:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.788+0000 7fc78d7fa700 1 -- 172.21.15.53:0/1450252262 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b30410cce0 con 0x55b30422a000 2023-11-06T01:39:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.788+0000 7fc78d7fa700 10 client.16104 handle_command_reply: tid=0 2023-11-06T01:39:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.788+0000 7fc7b8110b80 1 client.16104 shutdown 2023-11-06T01:39:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.790+0000 7fc7b8110b80 1 -- 172.21.15.53:0/1450252262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2680 msgr2=0x55b3041d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.790+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2680 0x55b3041d0b20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0026040 tx=0x7fc7a0009a70 comp rx=0 tx=0).stop 2023-11-06T01:39:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.791+0000 7fc7b8110b80 1 -- 172.21.15.53:0/1450252262 shutdown_connections 2023-11-06T01:39:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.791+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b30422a000 0x55b30422c420 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.791+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3041c3ed0 0x55b304227ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.791+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3041c2680 0x55b3041d0b20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.791+0000 7fc78d7fa700 0 client.16104 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.791+0000 7fc7b8110b80 1 --2- 172.21.15.53:0/1450252262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3041c2fe0 0x55b3041d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.791+0000 7fc7b8110b80 1 -- 172.21.15.53:0/1450252262 >> 172.21.15.53:0/1450252262 conn(0x55b304122070 msgr2=0x55b3041c70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.791+0000 7fc7b8110b80 1 -- 172.21.15.53:0/1450252262 shutdown_connections 2023-11-06T01:39:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.791+0000 7fc7b8110b80 1 -- 172.21.15.53:0/1450252262 wait complete. 2023-11-06T01:39:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:39:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:39:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:39:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:39:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.791+0000 7fc7b8110b80 20 client.16104 trim_cache size 0 max 16384 2023-11-06T01:39:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 -- 172.21.15.53:0/431611196 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7900a0520 msgr2=0x7fc7900a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7900a0520 0x7fc7900a29e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4072160 tx=0x7fc7a000f040 comp rx=0 tx=0).stop 2023-11-06T01:39:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 -- 172.21.15.53:0/431611196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a410e190 msgr2=0x7fc7a41a6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a410e190 0x7fc7a41a6dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00ef30 tx=0x7fc79c00c3a0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 -- 172.21.15.53:0/431611196 shutdown_connections 2023-11-06T01:39:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc7900a0520 0x7fc7900a29e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a4108ea0 0x7fc7a41a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7a410e190 0x7fc7a41a6dc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 --2- 172.21.15.53:0/431611196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a410b7d0 0x7fc7a41a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 -- 172.21.15.53:0/431611196 >> 172.21.15.53:0/431611196 conn(0x7fc7a406dbc0 msgr2=0x7fc7a410a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 -- 172.21.15.53:0/431611196 shutdown_connections 2023-11-06T01:39:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:30.794+0000 7fc7abf80700 1 -- 172.21.15.53:0/431611196 wait complete. 2023-11-06T01:39:31.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:31 smithi053 ceph-mon[99472]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 2.00k op/s 2023-11-06T01:39:31.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:31 smithi062 ceph-mon[103924]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 2.00k op/s 2023-11-06T01:39:31.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:31 smithi163 ceph-mon[104416]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 2.00k op/s 2023-11-06T01:39:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 315.0 s, 2230.4 tps, lat 45.428 ms stddev 53.678 2023-11-06T01:39:33.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:33 smithi062 ceph-mon[103924]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2023-11-06T01:39:33.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:33 smithi163 ceph-mon[104416]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2023-11-06T01:39:33.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:33 smithi053 ceph-mon[99472]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2023-11-06T01:39:35.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:35 smithi062 ceph-mon[103924]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 1.02k op/s 2023-11-06T01:39:35.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:35 smithi163 ceph-mon[104416]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 1.02k op/s 2023-11-06T01:39:35.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:35 smithi053 ceph-mon[99472]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 1.02k op/s 2023-11-06T01:39:36.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:39:36.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.794+0000 7feff78e2700 1 -- 172.21.15.53:0/1712244589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feff0100890 msgr2=0x7feff0108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:36.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.794+0000 7feff78e2700 1 --2- 172.21.15.53:0/1712244589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feff0100890 0x7feff0108130 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fefe800b0a0 tx=0x7fefe800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.795+0000 7feff78e2700 1 -- 172.21.15.53:0/1712244589 shutdown_connections 2023-11-06T01:39:36.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.795+0000 7feff78e2700 1 --2- 172.21.15.53:0/1712244589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feff0100890 0x7feff0108130 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.795+0000 7feff78e2700 1 --2- 172.21.15.53:0/1712244589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff0101280 0x7feff01016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.795+0000 7feff78e2700 1 --2- 172.21.15.53:0/1712244589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff00ffed0 0x7feff0100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.795+0000 7feff78e2700 1 -- 172.21.15.53:0/1712244589 >> 172.21.15.53:0/1712244589 conn(0x7feff00fa960 msgr2=0x7feff00fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:36.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.795+0000 7feff78e2700 1 -- 172.21.15.53:0/1712244589 shutdown_connections 2023-11-06T01:39:36.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.795+0000 7feff78e2700 1 -- 172.21.15.53:0/1712244589 wait complete. 2023-11-06T01:39:36.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff78e2700 1 Processor -- start 2023-11-06T01:39:36.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff78e2700 1 -- start start 2023-11-06T01:39:36.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff78e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff00ffed0 0x7feff0196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff78e2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feff0100890 0x7feff0197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff78e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff0101280 0x7feff019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff78e2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff019e6d0 con 0x7feff0100890 2023-11-06T01:39:36.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff78e2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff019e810 con 0x7feff0101280 2023-11-06T01:39:36.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff78e2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff019e9a0 con 0x7feff00ffed0 2023-11-06T01:39:36.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff4e7d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feff0100890 0x7feff0197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff567e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff00ffed0 0x7feff0196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff5e7f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff0101280 0x7feff019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff4e7d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feff0100890 0x7feff0197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41520/0 (socket says 172.21.15.53:41520) 2023-11-06T01:39:36.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff567e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff00ffed0 0x7feff0196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55758/0 (socket says 172.21.15.53:55758) 2023-11-06T01:39:36.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff4e7d700 1 -- 172.21.15.53:0/3944576568 learned_addr learned my addr 172.21.15.53:0/3944576568 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:36.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.796+0000 7feff5e7f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff0101280 0x7feff019e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52960/0 (socket says 172.21.15.53:52960) 2023-11-06T01:39:36.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7feff5e7f700 1 -- 172.21.15.53:0/3944576568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff00ffed0 msgr2=0x7feff0196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:36.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7feff5e7f700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff00ffed0 0x7feff0196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7feff5e7f700 1 -- 172.21.15.53:0/3944576568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feff0100890 msgr2=0x7feff0197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:36.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7feff5e7f700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feff0100890 0x7feff0197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7feff5e7f700 1 -- 172.21.15.53:0/3944576568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe8009d00 con 0x7feff0101280 2023-11-06T01:39:36.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7feff4e7d700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feff0100890 0x7feff0197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:36.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7feff5e7f700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff0101280 0x7feff019e100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fefe8007a60 tx=0x7fefe80078e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:36.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7fefe67fc700 1 -- 172.21.15.53:0/3944576568 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe800e040 con 0x7feff0101280 2023-11-06T01:39:36.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7feff78e2700 1 -- 172.21.15.53:0/3944576568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff019ec20 con 0x7feff0101280 2023-11-06T01:39:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7fefe67fc700 1 -- 172.21.15.53:0/3944576568 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefe800be40 con 0x7feff0101280 2023-11-06T01:39:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7fefe67fc700 1 -- 172.21.15.53:0/3944576568 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe80052a0 con 0x7feff0101280 2023-11-06T01:39:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.797+0000 7feff78e2700 1 -- 172.21.15.53:0/3944576568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff019f0f0 con 0x7feff0101280 2023-11-06T01:39:36.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.801+0000 7fefe67fc700 1 -- 172.21.15.53:0/3944576568 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fefe801d470 con 0x7feff0101280 2023-11-06T01:39:36.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7fefe67fc700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fefdc0a05a0 0x7fefdc0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7ff003a72b80 1 Processor -- start 2023-11-06T01:39:36.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7ff003a72b80 1 -- start start 2023-11-06T01:39:36.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7feff567e700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fefdc0a05a0 0x7fefdc0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7fefe67fc700 1 -- 172.21.15.53:0/3944576568 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fefe80ccd70 con 0x7feff0101280 2023-11-06T01:39:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63520f0 0x555aa6352530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 0x555aa6350e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 0x555aa6351870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7ff003a72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa62a8a10 con 0x555aa6352a70 2023-11-06T01:39:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7ff003a72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa62a63d0 con 0x555aa63520f0 2023-11-06T01:39:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7ff003a72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa62a1cc0 con 0x555aa63513f0 2023-11-06T01:39:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7feff4e7d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 0x555aa6350e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7feff567e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63520f0 0x555aa6352530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7feff4e7d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 0x555aa6350e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41528/0 (socket says 172.21.15.53:41528) 2023-11-06T01:39:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7feff567e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63520f0 0x555aa6352530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52964/0 (socket says 172.21.15.53:52964) 2023-11-06T01:39:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7feff4e7d700 1 -- 172.21.15.53:0/3414186702 learned_addr learned my addr 172.21.15.53:0/3414186702 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.802+0000 7feff5e7f700 1 --2- 172.21.15.53:0/3414186702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 0x555aa6351870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7feff567e700 1 -- 172.21.15.53:0/3414186702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 msgr2=0x555aa6351870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7feff567e700 1 --2- 172.21.15.53:0/3414186702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 0x555aa6351870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7feff567e700 1 -- 172.21.15.53:0/3414186702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 msgr2=0x555aa6350e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7feff567e700 1 --2- 172.21.15.53:0/3414186702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 0x555aa6350e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7feff567e700 1 -- 172.21.15.53:0/3414186702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aa60cfde0 con 0x555aa63520f0 2023-11-06T01:39:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7feff4e7d700 1 --2- 172.21.15.53:0/3414186702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 0x555aa6350e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7feff567e700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fefdc0a05a0 0x7fefdc0a2a60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feff0067cd0 tx=0x7fefec0118e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7feff5e7f700 1 --2- 172.21.15.53:0/3414186702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 0x555aa6351870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7feff567e700 1 --2- 172.21.15.53:0/3414186702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63520f0 0x555aa6352530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fefec011ce0 tx=0x7fefec020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f649e120ff92c54e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:36.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7fefdb7fe700 1 -- 172.21.15.53:0/3414186702 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec020e80 con 0x555aa63520f0 2023-11-06T01:39:36.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7fefdb7fe700 1 -- 172.21.15.53:0/3414186702 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefec028890 con 0x555aa63520f0 2023-11-06T01:39:36.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.803+0000 7fefdb7fe700 1 -- 172.21.15.53:0/3414186702 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec00f6f0 con 0x555aa63520f0 2023-11-06T01:39:36.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.804+0000 7ff003a72b80 1 -- 172.21.15.53:0/3414186702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63520f0 msgr2=0x555aa6352530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:36.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.804+0000 7ff003a72b80 1 --2- 172.21.15.53:0/3414186702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63520f0 0x555aa6352530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fefec011ce0 tx=0x7fefec020000 comp rx=0 tx=0).stop 2023-11-06T01:39:36.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.804+0000 7ff003a72b80 1 -- 172.21.15.53:0/3414186702 shutdown_connections 2023-11-06T01:39:36.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.804+0000 7ff003a72b80 1 --2- 172.21.15.53:0/3414186702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63520f0 0x555aa6352530 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.804+0000 7ff003a72b80 1 --2- 172.21.15.53:0/3414186702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 0x555aa6351870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.804+0000 7ff003a72b80 1 --2- 172.21.15.53:0/3414186702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 0x555aa6350e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.804+0000 7ff003a72b80 1 -- 172.21.15.53:0/3414186702 >> 172.21.15.53:0/3414186702 conn(0x555aa62b00d0 msgr2=0x555aa634b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.804+0000 7ff003a72b80 1 -- 172.21.15.53:0/3414186702 shutdown_connections 2023-11-06T01:39:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.804+0000 7ff003a72b80 1 -- 172.21.15.53:0/3414186702 wait complete. 2023-11-06T01:39:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.804+0000 7ff003a72b80 1 -- start start 2023-11-06T01:39:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.805+0000 7ff003a72b80 1 Processor -- start 2023-11-06T01:39:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.805+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63513f0 0x555aa63631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.805+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63520f0 0x555aa6363700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.805+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 0x555aa636a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.805+0000 7ff003a72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa623e510 con 0x555aa6352a70 2023-11-06T01:39:36.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.805+0000 7ff003a72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa623e1b0 con 0x555aa63513f0 2023-11-06T01:39:36.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.805+0000 7ff003a72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6125920 con 0x555aa63520f0 2023-11-06T01:39:36.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.805+0000 7ff003a72b80 10 client.0 _ll_register_callbacks cb 0x555aa618ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:36.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.806+0000 7feff567e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63513f0 0x555aa63631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.806+0000 7feff567e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63513f0 0x555aa63631c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52970/0 (socket says 172.21.15.53:52970) 2023-11-06T01:39:36.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.806+0000 7feff567e700 1 -- 172.21.15.53:0/1571725953 learned_addr learned my addr 172.21.15.53:0/1571725953 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:36.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.806+0000 7feff5e7f700 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 0x555aa636a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.806+0000 7feff567e700 1 -- 172.21.15.53:0/1571725953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63520f0 msgr2=0x555aa6363700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:39:36.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.806+0000 7feff567e700 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63520f0 0x555aa6363700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.806+0000 7feff567e700 1 -- 172.21.15.53:0/1571725953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 msgr2=0x555aa636a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:36.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.806+0000 7feff567e700 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 0x555aa636a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.806+0000 7feff567e700 1 -- 172.21.15.53:0/1571725953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff019f0f0 con 0x555aa63513f0 2023-11-06T01:39:36.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.806+0000 7feff567e700 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63513f0 0x555aa63631c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefec009ef0 tx=0x7fefec028bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3da955b579fc4a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:36.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.806+0000 7fefd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:39:36.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.807+0000 7fefd8ff9700 1 -- 172.21.15.53:0/1571725953 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec02f070 con 0x555aa63513f0 2023-11-06T01:39:36.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.807+0000 7fefd8ff9700 1 -- 172.21.15.53:0/1571725953 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fefec028490 con 0x555aa63513f0 2023-11-06T01:39:36.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.807+0000 7ff003a72b80 1 -- 172.21.15.53:0/1571725953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555aa62ccd10 con 0x555aa63513f0 2023-11-06T01:39:36.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.807+0000 7fefd8ff9700 1 -- 172.21.15.53:0/1571725953 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefec028650 con 0x555aa63513f0 2023-11-06T01:39:36.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.807+0000 7ff003a72b80 10 client.25685 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:36.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.807+0000 7ff003a72b80 1 -- 172.21.15.53:0/1571725953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fefe00137d0 con 0x555aa63513f0 2023-11-06T01:39:36.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.807+0000 7fefd8ff9700 1 -- 172.21.15.53:0/1571725953 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555aa62ccd10 con 0x555aa63513f0 2023-11-06T01:39:36.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.807+0000 7fefd8ff9700 1 -- 172.21.15.53:0/1571725953 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fefec03c070 con 0x555aa63513f0 2023-11-06T01:39:36.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.807+0000 7ff003a72b80 10 client.25685 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:36.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.807+0000 7ff003a72b80 10 client.25685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:36.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.808+0000 7ff003a72b80 20 client.25685 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:36.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.808+0000 7ff003a72b80 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555aa636eec0 0x555aa63712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.808+0000 7ff003a72b80 4 client.25685 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:39:36.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.808+0000 7ff003a72b80 1 -- 172.21.15.53:0/1571725953 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555aa631fe10 con 0x555aa636eec0 2023-11-06T01:39:36.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.808+0000 7feff4e7d700 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555aa636eec0 0x555aa63712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.808+0000 7feff4e7d700 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555aa636eec0 0x555aa63712e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d666f95cab7595a server_cookie=6ef5c614d769bb22 in_seq=0 out_seq=0 2023-11-06T01:39:36.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.808+0000 7fefd8ff9700 10 client.25685 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:36.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7fefd8ff9700 1 -- 172.21.15.53:0/1571725953 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555aa631fe10 con 0x555aa636eec0 2023-11-06T01:39:36.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7fefd8ff9700 10 client.25685 handle_command_reply: tid=0 2023-11-06T01:39:36.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7ff003a72b80 1 client.25685 shutdown 2023-11-06T01:39:36.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7ff003a72b80 1 -- 172.21.15.53:0/1571725953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63513f0 msgr2=0x555aa63631c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:36.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7ff003a72b80 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63513f0 0x555aa63631c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefec009ef0 tx=0x7fefec028bf0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7ff003a72b80 1 -- 172.21.15.53:0/1571725953 shutdown_connections 2023-11-06T01:39:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7ff003a72b80 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa63513f0 0x555aa63631c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7ff003a72b80 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555aa636eec0 0x555aa63712e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7ff003a72b80 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa6352a70 0x555aa636a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7ff003a72b80 1 --2- 172.21.15.53:0/1571725953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63520f0 0x555aa6363700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7fefd8ff9700 0 client.25685 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7ff003a72b80 1 -- 172.21.15.53:0/1571725953 >> 172.21.15.53:0/1571725953 conn(0x555aa62b00d0 msgr2=0x555aa634b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7ff003a72b80 1 -- 172.21.15.53:0/1571725953 shutdown_connections 2023-11-06T01:39:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.809+0000 7ff003a72b80 1 -- 172.21.15.53:0/1571725953 wait complete. 2023-11-06T01:39:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.810+0000 7ff003a72b80 20 client.25685 trim_cache size 0 max 16384 2023-11-06T01:39:36.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7ff003a72b80 1 Processor -- start 2023-11-06T01:39:36.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7ff003a72b80 1 -- start start 2023-11-06T01:39:36.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63513f0 0x555aa6350e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63520f0 0x555aa63686f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 0x555aa6362f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7ff003a72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa62ccd10 con 0x555aa63513f0 2023-11-06T01:39:36.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7ff003a72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa63594a0 con 0x555aa6352a70 2023-11-06T01:39:36.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7ff003a72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6359e60 con 0x555aa63520f0 2023-11-06T01:39:36.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7feff567e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63513f0 0x555aa6350e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7feff567e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63513f0 0x555aa6350e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41538/0 (socket says 172.21.15.53:41538) 2023-11-06T01:39:36.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7feff4e7d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63520f0 0x555aa63686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7feff567e700 1 -- 172.21.15.53:0/1256769111 learned_addr learned my addr 172.21.15.53:0/1256769111 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:36.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.832+0000 7feff5e7f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 0x555aa6362f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.833+0000 7feff567e700 1 -- 172.21.15.53:0/1256769111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63520f0 msgr2=0x555aa63686f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:36.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.833+0000 7feff567e700 1 --2- 172.21.15.53:0/1256769111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63520f0 0x555aa63686f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.833+0000 7feff567e700 1 -- 172.21.15.53:0/1256769111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 msgr2=0x555aa6362f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:36.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.833+0000 7feff567e700 1 --2- 172.21.15.53:0/1256769111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 0x555aa6362f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.833+0000 7feff567e700 1 -- 172.21.15.53:0/1256769111 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec020450 con 0x555aa63513f0 2023-11-06T01:39:36.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.833+0000 7feff4e7d700 1 --2- 172.21.15.53:0/1256769111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63520f0 0x555aa63686f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:36.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.833+0000 7feff5e7f700 1 --2- 172.21.15.53:0/1256769111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 0x555aa6362f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:36.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.833+0000 7feff567e700 1 --2- 172.21.15.53:0/1256769111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63513f0 0x555aa6350e60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fefec004d70 tx=0x7fefec03d480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9677e9e0df3ff2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:36.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.833+0000 7fefdbfff700 1 -- 172.21.15.53:0/1256769111 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec02f030 con 0x555aa63513f0 2023-11-06T01:39:36.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.834+0000 7fefdbfff700 1 -- 172.21.15.53:0/1256769111 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefec028d10 con 0x555aa63513f0 2023-11-06T01:39:36.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.834+0000 7fefdbfff700 1 -- 172.21.15.53:0/1256769111 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec028700 con 0x555aa63513f0 2023-11-06T01:39:36.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.834+0000 7ff003a72b80 1 -- 172.21.15.53:0/1256769111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63513f0 msgr2=0x555aa6350e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:36.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.834+0000 7ff003a72b80 1 --2- 172.21.15.53:0/1256769111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63513f0 0x555aa6350e60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fefec004d70 tx=0x7fefec03d480 comp rx=0 tx=0).stop 2023-11-06T01:39:36.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 -- 172.21.15.53:0/1256769111 shutdown_connections 2023-11-06T01:39:36.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 --2- 172.21.15.53:0/1256769111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 0x555aa6362f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 --2- 172.21.15.53:0/1256769111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63513f0 0x555aa6350e60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 --2- 172.21.15.53:0/1256769111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63520f0 0x555aa63686f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 -- 172.21.15.53:0/1256769111 >> 172.21.15.53:0/1256769111 conn(0x555aa62b00d0 msgr2=0x555aa636b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:36.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 -- 172.21.15.53:0/1256769111 shutdown_connections 2023-11-06T01:39:36.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 -- 172.21.15.53:0/1256769111 wait complete. 2023-11-06T01:39:36.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 -- start start 2023-11-06T01:39:36.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 Processor -- start 2023-11-06T01:39:36.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 0x555aa63684f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63520f0 0x555aa6366b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 0x555aa6367040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.835+0000 7ff003a72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6367610 con 0x555aa63520f0 2023-11-06T01:39:36.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7ff003a72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6367780 con 0x555aa6352a70 2023-11-06T01:39:36.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7ff003a72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6367940 con 0x555aa63513f0 2023-11-06T01:39:36.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7ff003a72b80 10 client.0 _ll_register_callbacks cb 0x7fefe0009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:36.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff4e7d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63520f0 0x555aa6366b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff4e7d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63520f0 0x555aa6366b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41546/0 (socket says 172.21.15.53:41546) 2023-11-06T01:39:36.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff567e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 0x555aa63684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff4e7d700 1 -- 172.21.15.53:0/3190875637 learned_addr learned my addr 172.21.15.53:0/3190875637 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:36.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff5e7f700 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 0x555aa6367040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff4e7d700 1 -- 172.21.15.53:0/3190875637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 msgr2=0x555aa63684f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:36.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff4e7d700 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 0x555aa63684f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff4e7d700 1 -- 172.21.15.53:0/3190875637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 msgr2=0x555aa6367040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:36.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff4e7d700 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 0x555aa6367040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff4e7d700 1 -- 172.21.15.53:0/3190875637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe0013960 con 0x555aa63520f0 2023-11-06T01:39:36.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff567e700 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 0x555aa63684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:36.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff5e7f700 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 0x555aa6367040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:36.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7feff4e7d700 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63520f0 0x555aa6366b00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x555aa6352e50 tx=0x7fefe0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d362d666afb216 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:36.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7fefd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:39:36.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7fefd8ff9700 1 -- 172.21.15.53:0/3190875637 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0004030 con 0x555aa63520f0 2023-11-06T01:39:36.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7fefd8ff9700 1 -- 172.21.15.53:0/3190875637 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fefe0004190 con 0x555aa63520f0 2023-11-06T01:39:36.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.836+0000 7ff003a72b80 1 -- 172.21.15.53:0/3190875637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefec028890 con 0x555aa63520f0 2023-11-06T01:39:36.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7fefd8ff9700 1 -- 172.21.15.53:0/3190875637 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefe0004350 con 0x555aa63520f0 2023-11-06T01:39:36.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7ff003a72b80 10 client.16122 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:36.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7ff003a72b80 1 -- 172.21.15.53:0/3190875637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7feff019ec20 con 0x555aa63520f0 2023-11-06T01:39:36.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7fefd8ff9700 1 -- 172.21.15.53:0/3190875637 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefec028890 con 0x555aa63520f0 2023-11-06T01:39:36.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7fefd8ff9700 1 -- 172.21.15.53:0/3190875637 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fefe0005340 con 0x555aa63520f0 2023-11-06T01:39:36.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7ff003a72b80 10 client.16122 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:36.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7ff003a72b80 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:36.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7ff003a72b80 20 client.16122 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:36.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7ff003a72b80 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555aa63c2270 0x555aa63c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:36.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7ff003a72b80 4 client.16122 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:39:36.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7ff003a72b80 1 -- 172.21.15.53:0/3190875637 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555aa629ad40 con 0x555aa63c2270 2023-11-06T01:39:36.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.837+0000 7feff567e700 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555aa63c2270 0x555aa63c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:36.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7fefd8ff9700 10 client.16122 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:36.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7feff567e700 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555aa63c2270 0x555aa63c4690 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d425ad949adaa2a server_cookie=2e10c1b3e3f7cc95 in_seq=0 out_seq=0 2023-11-06T01:39:36.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7fefd8ff9700 1 -- 172.21.15.53:0/3190875637 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x555aa629ad40 con 0x555aa63c2270 2023-11-06T01:39:36.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7fefd8ff9700 10 client.16122 handle_command_reply: tid=0 2023-11-06T01:39:36.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 1 client.16122 shutdown 2023-11-06T01:39:36.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 1 -- 172.21.15.53:0/3190875637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63520f0 msgr2=0x555aa6366b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:36.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63520f0 0x555aa6366b00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x555aa6352e50 tx=0x7fefe0009260 comp rx=0 tx=0).stop 2023-11-06T01:39:36.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 1 -- 172.21.15.53:0/3190875637 shutdown_connections 2023-11-06T01:39:36.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555aa6352a70 0x555aa6367040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555aa63c2270 0x555aa63c4690 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555aa63513f0 0x555aa63684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7fefd8ff9700 0 client.16122 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:36.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 1 --2- 172.21.15.53:0/3190875637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555aa63520f0 0x555aa6366b00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 1 -- 172.21.15.53:0/3190875637 >> 172.21.15.53:0/3190875637 conn(0x555aa62b00d0 msgr2=0x555aa6362f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:36.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 1 -- 172.21.15.53:0/3190875637 shutdown_connections 2023-11-06T01:39:36.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 1 -- 172.21.15.53:0/3190875637 wait complete. 2023-11-06T01:39:36.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.838+0000 7ff003a72b80 20 client.16122 trim_cache size 0 max 16384 2023-11-06T01:39:36.856 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:39:36.856 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:39:36.856 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:39:36.857 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:39:36.857 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:39:36.857 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:39:36.857 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:39:36.857 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:39:36.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:39:36.858 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:39:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.839+0000 7feff78e2700 1 -- 172.21.15.53:0/3944576568 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fefdc0a05a0 msgr2=0x7fefdc0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:36.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.839+0000 7feff78e2700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fefdc0a05a0 0x7fefdc0a2a60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feff0067cd0 tx=0x7fefec0118e0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.839+0000 7feff78e2700 1 -- 172.21.15.53:0/3944576568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff0101280 msgr2=0x7feff019e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.839+0000 7feff78e2700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff0101280 0x7feff019e100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fefe8007a60 tx=0x7fefe80078e0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.839+0000 7feff78e2700 1 -- 172.21.15.53:0/3944576568 shutdown_connections 2023-11-06T01:39:36.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.839+0000 7feff78e2700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fefdc0a05a0 0x7fefdc0a2a60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.839+0000 7feff78e2700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feff0101280 0x7feff019e100 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.839+0000 7feff78e2700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feff00ffed0 0x7feff0196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.839+0000 7feff78e2700 1 --2- 172.21.15.53:0/3944576568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feff0100890 0x7feff0197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.839+0000 7feff78e2700 1 -- 172.21.15.53:0/3944576568 >> 172.21.15.53:0/3944576568 conn(0x7feff00fa960 msgr2=0x7feff00fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.841+0000 7feff78e2700 1 -- 172.21.15.53:0/3944576568 shutdown_connections 2023-11-06T01:39:36.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:39:36.841+0000 7feff78e2700 1 -- 172.21.15.53:0/3944576568 wait complete. 2023-11-06T01:39:36.862 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:39:37.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 320.0 s, 2347.4 tps, lat 42.475 ms stddev 51.028 2023-11-06T01:39:37.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:37 smithi163 ceph-mon[104416]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 968 op/s 2023-11-06T01:39:37.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:37 smithi053 ceph-mon[99472]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 968 op/s 2023-11-06T01:39:37.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:37 smithi062 ceph-mon[103924]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 968 op/s 2023-11-06T01:39:39.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:39 smithi163 ceph-mon[104416]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 688 op/s 2023-11-06T01:39:39.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:39 smithi053 ceph-mon[99472]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 688 op/s 2023-11-06T01:39:39.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:39 smithi062 ceph-mon[103924]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 688 op/s 2023-11-06T01:39:41.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:41 smithi163 ceph-mon[104416]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 717 op/s 2023-11-06T01:39:41.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:41 smithi053 ceph-mon[99472]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 717 op/s 2023-11-06T01:39:41.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:41 smithi062 ceph-mon[103924]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 717 op/s 2023-11-06T01:39:42.190 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 325.0 s, 2319.3 tps, lat 43.094 ms stddev 50.452 2023-11-06T01:39:43.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:43 smithi163 ceph-mon[104416]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 788 op/s 2023-11-06T01:39:43.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:43 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:39:43.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:43 smithi053 ceph-mon[99472]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 788 op/s 2023-11-06T01:39:43.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:43 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:39:43.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:43 smithi062 ceph-mon[103924]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 788 op/s 2023-11-06T01:39:43.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:43 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:39:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.944+0000 7f3e6bff5700 1 -- 172.21.15.53:0/912692563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e64101820 msgr2=0x7f3e641080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.944+0000 7f3e6bff5700 1 --2- 172.21.15.53:0/912692563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e64101820 0x7f3e641080d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00b0d0 tx=0x7f3e5c00b3e0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.944+0000 7f3e6bff5700 1 -- 172.21.15.53:0/912692563 shutdown_connections 2023-11-06T01:39:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.944+0000 7f3e6bff5700 1 --2- 172.21.15.53:0/912692563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e64101820 0x7f3e641080d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.944+0000 7f3e6bff5700 1 --2- 172.21.15.53:0/912692563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e640ffc90 0x7f3e641000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.944+0000 7f3e6bff5700 1 --2- 172.21.15.53:0/912692563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e64100e80 0x7f3e641012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.944+0000 7f3e6bff5700 1 -- 172.21.15.53:0/912692563 >> 172.21.15.53:0/912692563 conn(0x7f3e640fb1e0 msgr2=0x7f3e640fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.945+0000 7f3e6bff5700 1 -- 172.21.15.53:0/912692563 shutdown_connections 2023-11-06T01:39:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.945+0000 7f3e6bff5700 1 -- 172.21.15.53:0/912692563 wait complete. 2023-11-06T01:39:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.945+0000 7f3e6bff5700 1 Processor -- start 2023-11-06T01:39:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.945+0000 7f3e6bff5700 1 -- start start 2023-11-06T01:39:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.945+0000 7f3e6bff5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e640ffc90 0x7f3e64198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.945+0000 7f3e6bff5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e64100e80 0x7f3e64199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.945+0000 7f3e6bff5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e64101820 0x7f3e6419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.945+0000 7f3e6bff5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6419a850 con 0x7f3e640ffc90 2023-11-06T01:39:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.945+0000 7f3e6bff5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6419a9c0 con 0x7f3e64100e80 2023-11-06T01:39:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.945+0000 7f3e6bff5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6419ab80 con 0x7f3e64101820 2023-11-06T01:39:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69d91700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e640ffc90 0x7f3e64198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69590700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e64100e80 0x7f3e64199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69d91700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e640ffc90 0x7f3e64198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41556/0 (socket says 172.21.15.53:41556) 2023-11-06T01:39:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69590700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e64100e80 0x7f3e64199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:52996/0 (socket says 172.21.15.53:52996) 2023-11-06T01:39:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e6a592700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e64101820 0x7f3e6419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69d91700 1 -- 172.21.15.53:0/1572615218 learned_addr learned my addr 172.21.15.53:0/1572615218 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69590700 1 -- 172.21.15.53:0/1572615218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e64101820 msgr2=0x7f3e6419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69590700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e64101820 0x7f3e6419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69590700 1 -- 172.21.15.53:0/1572615218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e640ffc90 msgr2=0x7f3e64198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69590700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e640ffc90 0x7f3e64198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69590700 1 -- 172.21.15.53:0/1572615218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e54009520 con 0x7f3e64100e80 2023-11-06T01:39:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69d91700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e640ffc90 0x7f3e64198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e6a592700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e64101820 0x7f3e6419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e69590700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e64100e80 0x7f3e64199410 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400e9f0 tx=0x7f3e5400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e5affd700 1 -- 172.21.15.53:0/1572615218 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400cb70 con 0x7f3e64100e80 2023-11-06T01:39:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.946+0000 7f3e6bff5700 1 -- 172.21.15.53:0/1572615218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c009d00 con 0x7f3e64100e80 2023-11-06T01:39:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.947+0000 7f3e6bff5700 1 -- 172.21.15.53:0/1572615218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e641a1580 con 0x7f3e64100e80 2023-11-06T01:39:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.947+0000 7f3e5affd700 1 -- 172.21.15.53:0/1572615218 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5400ee90 con 0x7f3e64100e80 2023-11-06T01:39:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.947+0000 7f3e5affd700 1 -- 172.21.15.53:0/1572615218 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54003c90 con 0x7f3e64100e80 2023-11-06T01:39:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.949+0000 7f3e5affd700 1 -- 172.21.15.53:0/1572615218 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f3e5400ccd0 con 0x7f3e64100e80 2023-11-06T01:39:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.950+0000 7f3e5affd700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3e500a0580 0x7f3e500a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.950+0000 7f3e5affd700 1 -- 172.21.15.53:0/1572615218 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3e54014070 con 0x7f3e64100e80 2023-11-06T01:39:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.950+0000 7f3e69d91700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3e500a0580 0x7f3e500a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.950+0000 7f3e69d91700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3e500a0580 0x7f3e500a2a40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3e60005950 tx=0x7f3e60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e78185b80 1 Processor -- start 2023-11-06T01:39:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e78185b80 1 -- start start 2023-11-06T01:39:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 0x555b6e6de8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6deed0 0x555b6e6df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e78185b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e6359b0 con 0x555b6e6ddfe0 2023-11-06T01:39:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e78185b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e633370 con 0x555b6e6dd680 2023-11-06T01:39:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e78185b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e62ec60 con 0x555b6e6deed0 2023-11-06T01:39:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e69d91700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e69590700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 0x555b6e6de8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e6a592700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6deed0 0x555b6e6df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e69d91700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6ddaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53002/0 (socket says 172.21.15.53:53002) 2023-11-06T01:39:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.951+0000 7f3e69d91700 1 -- 172.21.15.53:0/3312553856 learned_addr learned my addr 172.21.15.53:0/3312553856 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.952+0000 7f3e6a592700 1 -- 172.21.15.53:0/3312553856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 msgr2=0x555b6e6ddaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.952+0000 7f3e6a592700 1 --2- 172.21.15.53:0/3312553856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6ddaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.952+0000 7f3e6a592700 1 -- 172.21.15.53:0/3312553856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 msgr2=0x555b6e6de8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.952+0000 7f3e6a592700 1 --2- 172.21.15.53:0/3312553856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 0x555b6e6de8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.952+0000 7f3e6a592700 1 -- 172.21.15.53:0/3312553856 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b6e45cd80 con 0x555b6e6deed0 2023-11-06T01:39:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.952+0000 7f3e69d91700 1 --2- 172.21.15.53:0/3312553856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.952+0000 7f3e69590700 1 --2- 172.21.15.53:0/3312553856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 0x555b6e6de8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.952+0000 7f3e6a592700 1 --2- 172.21.15.53:0/3312553856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6deed0 0x555b6e6df350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fd0 tx=0x7f3e5c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c3f7be946c5d0e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.952+0000 7f3e4ffff700 1 -- 172.21.15.53:0/3312553856 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00e040 con 0x555b6e6deed0 2023-11-06T01:39:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.952+0000 7f3e4ffff700 1 -- 172.21.15.53:0/3312553856 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5c003a00 con 0x555b6e6deed0 2023-11-06T01:39:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.952+0000 7f3e4ffff700 1 -- 172.21.15.53:0/3312553856 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0054e0 con 0x555b6e6deed0 2023-11-06T01:39:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.953+0000 7f3e78185b80 1 -- 172.21.15.53:0/3312553856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6deed0 msgr2=0x555b6e6df350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.953+0000 7f3e78185b80 1 --2- 172.21.15.53:0/3312553856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6deed0 0x555b6e6df350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fd0 tx=0x7f3e5c009b80 comp rx=0 tx=0).stop 2023-11-06T01:39:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.953+0000 7f3e78185b80 1 -- 172.21.15.53:0/3312553856 shutdown_connections 2023-11-06T01:39:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.953+0000 7f3e78185b80 1 --2- 172.21.15.53:0/3312553856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.953+0000 7f3e78185b80 1 --2- 172.21.15.53:0/3312553856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6deed0 0x555b6e6df350 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.953+0000 7f3e78185b80 1 --2- 172.21.15.53:0/3312553856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 0x555b6e6de8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.953+0000 7f3e78185b80 1 -- 172.21.15.53:0/3312553856 >> 172.21.15.53:0/3312553856 conn(0x555b6e63d070 msgr2=0x555b6e6da040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.953+0000 7f3e78185b80 1 -- 172.21.15.53:0/3312553856 shutdown_connections 2023-11-06T01:39:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.953+0000 7f3e78185b80 1 -- 172.21.15.53:0/3312553856 wait complete. 2023-11-06T01:39:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.953+0000 7f3e78185b80 1 -- start start 2023-11-06T01:39:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.953+0000 7f3e78185b80 1 Processor -- start 2023-11-06T01:39:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6e7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6ddfe0 0x555b6e6e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 0x555b6e6ee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e78185b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e5cb4b0 con 0x555b6e6deed0 2023-11-06T01:39:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e78185b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e5cb150 con 0x555b6e6dd680 2023-11-06T01:39:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e78185b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e4b28c0 con 0x555b6e6ddfe0 2023-11-06T01:39:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e78185b80 10 client.0 _ll_register_callbacks cb 0x555b6e518a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e69d91700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e6a592700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 0x555b6e6ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e69d91700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6e7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53008/0 (socket says 172.21.15.53:53008) 2023-11-06T01:39:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e69d91700 1 -- 172.21.15.53:0/2448279662 learned_addr learned my addr 172.21.15.53:0/2448279662 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e69590700 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6ddfe0 0x555b6e6e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e6a592700 1 -- 172.21.15.53:0/2448279662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6ddfe0 msgr2=0x555b6e6e78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e6a592700 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6ddfe0 0x555b6e6e78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e6a592700 1 -- 172.21.15.53:0/2448279662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 msgr2=0x555b6e6e7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e6a592700 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6e7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e6a592700 1 -- 172.21.15.53:0/2448279662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e60017070 con 0x555b6e6deed0 2023-11-06T01:39:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e69d91700 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.954+0000 7f3e69590700 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6ddfe0 0x555b6e6e78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e6a592700 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 0x555b6e6ee900 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00b770 tx=0x7f3e5c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd27374bbea5b5c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:39:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2448279662 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00e040 con 0x555b6e6deed0 2023-11-06T01:39:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2448279662 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3e5c009410 con 0x555b6e6deed0 2023-11-06T01:39:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e78185b80 1 -- 172.21.15.53:0/2448279662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b6e659cb0 con 0x555b6e6deed0 2023-11-06T01:39:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2448279662 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e5c0055f0 con 0x555b6e6deed0 2023-11-06T01:39:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e78185b80 10 client.16134 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e78185b80 1 -- 172.21.15.53:0/2448279662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f3e641a1580 con 0x555b6e6deed0 2023-11-06T01:39:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2448279662 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b6e659cb0 con 0x555b6e6deed0 2023-11-06T01:39:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2448279662 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f3e5c02c030 con 0x555b6e6deed0 2023-11-06T01:39:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e78185b80 10 client.16134 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e78185b80 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e78185b80 20 client.16134 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555b6e6f2ca0 0x555b6e6f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e78185b80 4 client.16134 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:39:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.955+0000 7f3e78185b80 1 -- 172.21.15.53:0/2448279662 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b6e6acdb0 con 0x555b6e6f2ca0 2023-11-06T01:39:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.956+0000 7f3e69d91700 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555b6e6f2ca0 0x555b6e6f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.957+0000 7f3e69d91700 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555b6e6f2ca0 0x555b6e6f5090 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98cb940a57c9b4f server_cookie=c98b79d821ae1fa9 in_seq=0 out_seq=0 2023-11-06T01:39:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e4d7fa700 10 client.16134 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2448279662 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555b6e6acdb0 con 0x555b6e6f2ca0 2023-11-06T01:39:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e4d7fa700 10 client.16134 handle_command_reply: tid=0 2023-11-06T01:39:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e78185b80 1 client.16134 shutdown 2023-11-06T01:39:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e78185b80 1 -- 172.21.15.53:0/2448279662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 msgr2=0x555b6e6ee900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 0x555b6e6ee900 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00b770 tx=0x7f3e5c00bd50 comp rx=0 tx=0).stop 2023-11-06T01:39:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e78185b80 1 -- 172.21.15.53:0/2448279662 shutdown_connections 2023-11-06T01:39:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555b6e6f2ca0 0x555b6e6f5090 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 0x555b6e6ee900 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2448279662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6ddfe0 0x555b6e6e78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e78185b80 1 -- 172.21.15.53:0/2448279662 >> 172.21.15.53:0/2448279662 conn(0x555b6e63d070 msgr2=0x555b6e6d83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e4d7fa700 0 client.16134 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.959+0000 7f3e78185b80 1 -- 172.21.15.53:0/2448279662 shutdown_connections 2023-11-06T01:39:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.960+0000 7f3e78185b80 1 -- 172.21.15.53:0/2448279662 wait complete. 2023-11-06T01:39:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.960+0000 7f3e78185b80 20 client.16134 trim_cache size 0 max 16384 2023-11-06T01:39:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.978+0000 7f3e78185b80 1 Processor -- start 2023-11-06T01:39:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.978+0000 7f3e78185b80 1 -- start start 2023-11-06T01:39:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.978+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6e5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.978+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6ddfe0 0x555b6e6ee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.978+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 0x555b6e6e56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.978+0000 7f3e78185b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e6d3530 con 0x555b6e6deed0 2023-11-06T01:39:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.978+0000 7f3e78185b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e6e6f20 con 0x555b6e6dd680 2023-11-06T01:39:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.978+0000 7f3e78185b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e6d0a90 con 0x555b6e6ddfe0 2023-11-06T01:39:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e6a592700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 0x555b6e6e56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e69d91700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e6a592700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 0x555b6e6e56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41582/0 (socket says 172.21.15.53:41582) 2023-11-06T01:39:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e6a592700 1 -- 172.21.15.53:0/3306737829 learned_addr learned my addr 172.21.15.53:0/3306737829 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e69590700 1 --2- 172.21.15.53:0/3306737829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6ddfe0 0x555b6e6ee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e6a592700 1 -- 172.21.15.53:0/3306737829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6ddfe0 msgr2=0x555b6e6ee8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e6a592700 1 --2- 172.21.15.53:0/3306737829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6ddfe0 0x555b6e6ee8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e6a592700 1 -- 172.21.15.53:0/3306737829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 msgr2=0x555b6e6e5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e6a592700 1 --2- 172.21.15.53:0/3306737829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6e5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e6a592700 1 -- 172.21.15.53:0/3306737829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e64108a20 con 0x555b6e6deed0 2023-11-06T01:39:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e6a592700 1 --2- 172.21.15.53:0/3306737829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 0x555b6e6e56d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c000f80 tx=0x7f3e5c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9915c14a6235679b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e58ff9700 1 -- 172.21.15.53:0/3306737829 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00e040 con 0x555b6e6deed0 2023-11-06T01:39:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e58ff9700 1 -- 172.21.15.53:0/3306737829 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5c028070 con 0x555b6e6deed0 2023-11-06T01:39:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.979+0000 7f3e58ff9700 1 -- 172.21.15.53:0/3306737829 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0056e0 con 0x555b6e6deed0 2023-11-06T01:39:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.980+0000 7f3e78185b80 1 -- 172.21.15.53:0/3306737829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 msgr2=0x555b6e6e56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.980+0000 7f3e78185b80 1 --2- 172.21.15.53:0/3306737829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 0x555b6e6e56d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c000f80 tx=0x7f3e5c009b80 comp rx=0 tx=0).stop 2023-11-06T01:39:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.983+0000 7f3e78185b80 1 -- 172.21.15.53:0/3306737829 shutdown_connections 2023-11-06T01:39:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.983+0000 7f3e78185b80 1 --2- 172.21.15.53:0/3306737829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6dd680 0x555b6e6e5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.983+0000 7f3e78185b80 1 --2- 172.21.15.53:0/3306737829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6deed0 0x555b6e6e56d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.983+0000 7f3e78185b80 1 --2- 172.21.15.53:0/3306737829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6ddfe0 0x555b6e6ee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.983+0000 7f3e78185b80 1 -- 172.21.15.53:0/3306737829 >> 172.21.15.53:0/3306737829 conn(0x555b6e63d070 msgr2=0x555b6e6d7660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.983+0000 7f3e78185b80 1 -- 172.21.15.53:0/3306737829 shutdown_connections 2023-11-06T01:39:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.983+0000 7f3e78185b80 1 -- 172.21.15.53:0/3306737829 wait complete. 2023-11-06T01:39:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.983+0000 7f3e78185b80 1 -- start start 2023-11-06T01:39:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.983+0000 7f3e78185b80 1 Processor -- start 2023-11-06T01:39:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6dd680 0x555b6e6ebef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 0x555b6e6ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e78185b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6deed0 0x555b6e6ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e78185b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e6ed7f0 con 0x555b6e6ddfe0 2023-11-06T01:39:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e78185b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e6ed960 con 0x555b6e6deed0 2023-11-06T01:39:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e78185b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6e6edb20 con 0x555b6e6dd680 2023-11-06T01:39:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e78185b80 10 client.0 _ll_register_callbacks cb 0x7f3e540950a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e69590700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 0x555b6e6ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e69d91700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6dd680 0x555b6e6ebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e6a592700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6deed0 0x555b6e6ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e69590700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 0x555b6e6ec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41586/0 (socket says 172.21.15.53:41586) 2023-11-06T01:39:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e69590700 1 -- 172.21.15.53:0/2496273135 learned_addr learned my addr 172.21.15.53:0/2496273135 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e69590700 1 -- 172.21.15.53:0/2496273135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6dd680 msgr2=0x555b6e6ebef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e69590700 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6dd680 0x555b6e6ebef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e69590700 1 -- 172.21.15.53:0/2496273135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6deed0 msgr2=0x555b6e6ed2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e69590700 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6deed0 0x555b6e6ed2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e69590700 1 -- 172.21.15.53:0/2496273135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e641a1580 con 0x555b6e6ddfe0 2023-11-06T01:39:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e6a592700 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6deed0 0x555b6e6ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e69590700 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 0x555b6e6ec430 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3e54099120 tx=0x7f3e54095760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26c52d701b6bb5b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:39:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.984+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2496273135 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5409d250 con 0x555b6e6ddfe0 2023-11-06T01:39:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2496273135 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3e540a1490 con 0x555b6e6ddfe0 2023-11-06T01:39:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e78185b80 1 -- 172.21.15.53:0/2496273135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e5c003a00 con 0x555b6e6ddfe0 2023-11-06T01:39:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2496273135 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e540c97d0 con 0x555b6e6ddfe0 2023-11-06T01:39:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e78185b80 10 client.16140 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e78185b80 1 -- 172.21.15.53:0/2496273135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555b6e445c00 con 0x555b6e6ddfe0 2023-11-06T01:39:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2496273135 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5c003a00 con 0x555b6e6ddfe0 2023-11-06T01:39:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2496273135 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f3e540c9b80 con 0x555b6e6ddfe0 2023-11-06T01:39:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e78185b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e78185b80 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e78185b80 20 client.16140 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555b6e744ea0 0x555b6e7472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e78185b80 4 client.16140 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:39:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e78185b80 1 -- 172.21.15.53:0/2496273135 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555b6e627ce0 con 0x555b6e744ea0 2023-11-06T01:39:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.985+0000 7f3e69d91700 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555b6e744ea0 0x555b6e7472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.986+0000 7f3e4d7fa700 10 client.16140 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:43.986+0000 7f3e69d91700 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555b6e744ea0 0x555b6e7472c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a59941774e4961 server_cookie=a25f99485b14da8e in_seq=0 out_seq=0 2023-11-06T01:39:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e4d7fa700 1 -- 172.21.15.53:0/2496273135 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555b6e627ce0 con 0x555b6e744ea0 2023-11-06T01:39:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e4d7fa700 10 client.16140 handle_command_reply: tid=0 2023-11-06T01:39:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 1 client.16140 shutdown 2023-11-06T01:39:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 1 -- 172.21.15.53:0/2496273135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 msgr2=0x555b6e6ec430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 0x555b6e6ec430 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3e54099120 tx=0x7f3e54095760 comp rx=0 tx=0).stop 2023-11-06T01:39:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 1 -- 172.21.15.53:0/2496273135 shutdown_connections 2023-11-06T01:39:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b6e6deed0 0x555b6e6ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x555b6e744ea0 0x555b6e7472c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b6e6dd680 0x555b6e6ebef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 1 --2- 172.21.15.53:0/2496273135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b6e6ddfe0 0x555b6e6ec430 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e4d7fa700 0 client.16140 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 1 -- 172.21.15.53:0/2496273135 >> 172.21.15.53:0/2496273135 conn(0x555b6e63d070 msgr2=0x555b6e6e5190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 1 -- 172.21.15.53:0/2496273135 shutdown_connections 2023-11-06T01:39:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 1 -- 172.21.15.53:0/2496273135 wait complete. 2023-11-06T01:39:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.068+0000 7f3e78185b80 20 client.16140 trim_cache size 0 max 16384 2023-11-06T01:39:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:39:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:39:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:39:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:39:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 -- 172.21.15.53:0/1572615218 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3e500a0580 msgr2=0x7f3e500a2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3e500a0580 0x7f3e500a2a40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3e60005950 tx=0x7f3e60009260 comp rx=0 tx=0).stop 2023-11-06T01:39:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 -- 172.21.15.53:0/1572615218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e64100e80 msgr2=0x7f3e64199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e64100e80 0x7f3e64199410 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400e9f0 tx=0x7f3e5400ed00 comp rx=0 tx=0).stop 2023-11-06T01:39:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 -- 172.21.15.53:0/1572615218 shutdown_connections 2023-11-06T01:39:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3e500a0580 0x7f3e500a2a40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e64101820 0x7f3e6419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e640ffc90 0x7f3e64198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 --2- 172.21.15.53:0/1572615218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e64100e80 0x7f3e64199410 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 -- 172.21.15.53:0/1572615218 >> 172.21.15.53:0/1572615218 conn(0x7f3e640fb1e0 msgr2=0x7f3e640fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 -- 172.21.15.53:0/1572615218 shutdown_connections 2023-11-06T01:39:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:44.069+0000 7f3e6bff5700 1 -- 172.21.15.53:0/1572615218 wait complete. 2023-11-06T01:39:44.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:39:44.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:39:44.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:39:44.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:39:44.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:39:44.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:39:44.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:39:44.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:39:44.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:39:45.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:45 smithi163 ceph-mon[104416]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 659 op/s 2023-11-06T01:39:45.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:45 smithi053 ceph-mon[99472]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 659 op/s 2023-11-06T01:39:45.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:45 smithi062 ceph-mon[103924]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 659 op/s 2023-11-06T01:39:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 330.0 s, 2137.4 tps, lat 46.876 ms stddev 50.241 2023-11-06T01:39:47.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:47 smithi163 ceph-mon[104416]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 762 op/s 2023-11-06T01:39:47.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:47 smithi053 ceph-mon[99472]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 762 op/s 2023-11-06T01:39:47.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:47 smithi062 ceph-mon[103924]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 762 op/s 2023-11-06T01:39:49.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:49 smithi053 ceph-mon[99472]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 616 op/s 2023-11-06T01:39:49.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:49 smithi062 ceph-mon[103924]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 616 op/s 2023-11-06T01:39:49.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:49 smithi163 ceph-mon[104416]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 616 op/s 2023-11-06T01:39:51.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:51 smithi053 ceph-mon[99472]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 MiB/s wr, 660 op/s 2023-11-06T01:39:51.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:51 smithi062 ceph-mon[103924]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 MiB/s wr, 660 op/s 2023-11-06T01:39:51.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:51 smithi163 ceph-mon[104416]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 MiB/s wr, 660 op/s 2023-11-06T01:39:52.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 335.0 s, 1539.4 tps, lat 64.645 ms stddev 74.945 2023-11-06T01:39:53.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:53 smithi053 ceph-mon[99472]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 792 op/s 2023-11-06T01:39:53.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:53 smithi062 ceph-mon[103924]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 792 op/s 2023-11-06T01:39:53.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:53 smithi163 ceph-mon[104416]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 792 op/s 2023-11-06T01:39:55.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:55 smithi053 ceph-mon[99472]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 767 op/s 2023-11-06T01:39:55.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:55 smithi062 ceph-mon[103924]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 767 op/s 2023-11-06T01:39:55.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:55 smithi163 ceph-mon[104416]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 767 op/s 2023-11-06T01:39:57.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 340.0 s, 1971.7 tps, lat 51.014 ms stddev 56.309 2023-11-06T01:39:57.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.238+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2535454073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba0104b80 msgr2=0x7f0ba010b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.238+0000 7f0ba81d1700 1 --2- 172.21.15.53:0/2535454073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba0104b80 0x7f0ba010b430 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800b0d0 tx=0x7f0b9800b3e0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.241+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2535454073 shutdown_connections 2023-11-06T01:39:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.241+0000 7f0ba81d1700 1 --2- 172.21.15.53:0/2535454073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba0104b80 0x7f0ba010b430 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.241+0000 7f0ba81d1700 1 --2- 172.21.15.53:0/2535454073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ba0102ff0 0x7f0ba0103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.241+0000 7f0ba81d1700 1 --2- 172.21.15.53:0/2535454073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba01041e0 0x7f0ba0104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.241+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2535454073 >> 172.21.15.53:0/2535454073 conn(0x7f0ba00fe540 msgr2=0x7f0ba01009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.249+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2535454073 shutdown_connections 2023-11-06T01:39:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.258+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2535454073 wait complete. 2023-11-06T01:39:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.259+0000 7f0ba81d1700 1 Processor -- start 2023-11-06T01:39:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.259+0000 7f0ba81d1700 1 -- start start 2023-11-06T01:39:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.259+0000 7f0ba81d1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba0102ff0 0x7f0ba019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.259+0000 7f0ba81d1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ba01041e0 0x7f0ba019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.259+0000 7f0ba81d1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba0104b80 0x7f0ba019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.259+0000 7f0ba81d1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba019ebd0 con 0x7f0ba0104b80 2023-11-06T01:39:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.259+0000 7f0ba81d1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba019ed40 con 0x7f0ba0102ff0 2023-11-06T01:39:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.259+0000 7f0ba81d1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba019ef00 con 0x7f0ba01041e0 2023-11-06T01:39:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.259+0000 7f0ba676e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba0104b80 0x7f0ba019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.259+0000 7f0ba676e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba0104b80 0x7f0ba019e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41598/0 (socket says 172.21.15.53:41598) 2023-11-06T01:39:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.259+0000 7f0ba676e700 1 -- 172.21.15.53:0/2590385763 learned_addr learned my addr 172.21.15.53:0/2590385763 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.260+0000 7f0ba576c700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ba01041e0 0x7f0ba019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.260+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba0102ff0 0x7f0ba019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.260+0000 7f0ba676e700 1 -- 172.21.15.53:0/2590385763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ba01041e0 msgr2=0x7f0ba019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.260+0000 7f0ba676e700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ba01041e0 0x7f0ba019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.260+0000 7f0ba676e700 1 -- 172.21.15.53:0/2590385763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba0102ff0 msgr2=0x7f0ba019d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.260+0000 7f0ba676e700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba0102ff0 0x7f0ba019d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.260+0000 7f0ba676e700 1 -- 172.21.15.53:0/2590385763 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b98009d00 con 0x7f0ba0104b80 2023-11-06T01:39:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.260+0000 7f0ba676e700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba0104b80 0x7f0ba019e570 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0b980097f0 tx=0x7f0b9800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.260+0000 7f0b96ffd700 1 -- 172.21.15.53:0/2590385763 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9800e040 con 0x7f0ba0104b80 2023-11-06T01:39:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.260+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2590385763 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba019f180 con 0x7f0ba0104b80 2023-11-06T01:39:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.261+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2590385763 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba01a59a0 con 0x7f0ba0104b80 2023-11-06T01:39:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.261+0000 7f0b96ffd700 1 -- 172.21.15.53:0/2590385763 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9801dd50 con 0x7f0ba0104b80 2023-11-06T01:39:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.261+0000 7f0b96ffd700 1 -- 172.21.15.53:0/2590385763 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9801bdf0 con 0x7f0ba0104b80 2023-11-06T01:39:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.266+0000 7f0b96ffd700 1 -- 172.21.15.53:0/2590385763 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f0b9801b450 con 0x7f0ba0104b80 2023-11-06T01:39:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.267+0000 7f0bb4361b80 1 Processor -- start 2023-11-06T01:39:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.267+0000 7f0bb4361b80 1 -- start start 2023-11-06T01:39:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.268+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebe680 0x5569e5ebeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.268+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ebf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.268+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5ec0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.268+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5e169b0 con 0x5569e5ebfed0 2023-11-06T01:39:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.268+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5e14370 con 0x5569e5ebefe0 2023-11-06T01:39:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.268+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5e0fc60 con 0x5569e5ebe680 2023-11-06T01:39:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.268+0000 7f0b96ffd700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b8c0a0620 0x7f0b8c0a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.268+0000 7f0b96ffd700 1 -- 172.21.15.53:0/2590385763 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0b980cd610 con 0x7f0ba0104b80 2023-11-06T01:39:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.268+0000 7f0ba5f6d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ebf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.268+0000 7f0ba5f6d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ebf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53040/0 (socket says 172.21.15.53:53040) 2023-11-06T01:39:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.268+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/2302995298 learned_addr learned my addr 172.21.15.53:0/2302995298 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0ba676e700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b8c0a0620 0x7f0b8c0a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0ba576c700 1 --2- 172.21.15.53:0/2302995298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebe680 0x5569e5ebeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/2302995298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebe680 msgr2=0x5569e5ebeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/2302995298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebe680 0x5569e5ebeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/2302995298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 msgr2=0x5569e5ec0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/2302995298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5ec0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/2302995298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569e5c3dd80 con 0x5569e5ebefe0 2023-11-06T01:39:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0ba576c700 1 --2- 172.21.15.53:0/2302995298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebe680 0x5569e5ebeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:39:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/2302995298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ebf8f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009aa0 tx=0x7f0b90009db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c75e5d2ceaf85300 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0ba676e700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b8c0a0620 0x7f0b8c0a2ae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800b820 tx=0x7f0b9800b3e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0b8bfff700 1 -- 172.21.15.53:0/2302995298 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000e990 con 0x5569e5ebefe0 2023-11-06T01:39:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0b8bfff700 1 -- 172.21.15.53:0/2302995298 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b90013470 con 0x5569e5ebefe0 2023-11-06T01:39:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.269+0000 7f0b8bfff700 1 -- 172.21.15.53:0/2302995298 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b900125d0 con 0x5569e5ebefe0 2023-11-06T01:39:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.270+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2302995298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 msgr2=0x5569e5ebf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.270+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/2302995298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ebf8f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009aa0 tx=0x7f0b90009db0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.270+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2302995298 shutdown_connections 2023-11-06T01:39:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.270+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/2302995298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5ec0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.270+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/2302995298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebe680 0x5569e5ebeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.270+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/2302995298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ebf8f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.270+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2302995298 >> 172.21.15.53:0/2302995298 conn(0x5569e5e1e070 msgr2=0x5569e5ebb040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.272+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2302995298 shutdown_connections 2023-11-06T01:39:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.272+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2302995298 wait complete. 2023-11-06T01:39:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.272+0000 7f0bb4361b80 1 -- start start 2023-11-06T01:39:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.272+0000 7f0bb4361b80 1 Processor -- start 2023-11-06T01:39:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebe680 0x5569e5ed9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebefe0 0x5569e5ed9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5ee09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5dac4b0 con 0x5569e5ebfed0 2023-11-06T01:39:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5dac150 con 0x5569e5ebe680 2023-11-06T01:39:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5c938c0 con 0x5569e5ebefe0 2023-11-06T01:39:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0bb4361b80 10 client.0 _ll_register_callbacks cb 0x5569e5cf9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0ba5f6d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebefe0 0x5569e5ed9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0ba576c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebe680 0x5569e5ed9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0ba5f6d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebefe0 0x5569e5ed9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55850/0 (socket says 172.21.15.53:55850) 2023-11-06T01:39:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/3490315113 learned_addr learned my addr 172.21.15.53:0/3490315113 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0ba576c700 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5ee09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/3490315113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebe680 msgr2=0x5569e5ed9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebe680 0x5569e5ed9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/3490315113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 msgr2=0x5569e5ee09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5ee09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/3490315113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b900096d0 con 0x5569e5ebefe0 2023-11-06T01:39:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.273+0000 7f0ba576c700 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5ee09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:39:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.274+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebefe0 0x5569e5ed9990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000ec40 tx=0x7f0b9000ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51f95e9acd80047d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.274+0000 7f0b897fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:39:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.274+0000 7f0b897fa700 1 -- 172.21.15.53:0/3490315113 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000ee90 con 0x5569e5ebefe0 2023-11-06T01:39:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.274+0000 7f0b897fa700 1 -- 172.21.15.53:0/3490315113 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0b90006dc0 con 0x5569e5ebefe0 2023-11-06T01:39:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.274+0000 7f0bb4361b80 1 -- 172.21.15.53:0/3490315113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569e5e3acb0 con 0x5569e5ebefe0 2023-11-06T01:39:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.274+0000 7f0b897fa700 1 -- 172.21.15.53:0/3490315113 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b9001c660 con 0x5569e5ebefe0 2023-11-06T01:39:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.274+0000 7f0bb4361b80 10 client.25780 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.274+0000 7f0bb4361b80 1 -- 172.21.15.53:0/3490315113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5569e5c26c00 con 0x5569e5ebefe0 2023-11-06T01:39:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.274+0000 7f0b897fa700 1 -- 172.21.15.53:0/3490315113 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b90033080 con 0x5569e5ebefe0 2023-11-06T01:39:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.274+0000 7f0b897fa700 1 -- 172.21.15.53:0/3490315113 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f0b9002d030 con 0x5569e5ebefe0 2023-11-06T01:39:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.275+0000 7f0bb4361b80 10 client.25780 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.275+0000 7f0bb4361b80 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.275+0000 7f0bb4361b80 20 client.25780 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.275+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569e5ee5180 0x5569e5ee75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.275+0000 7f0bb4361b80 4 client.25780 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:39:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.275+0000 7f0bb4361b80 1 -- 172.21.15.53:0/3490315113 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569e5e8ddb0 con 0x5569e5ee5180 2023-11-06T01:39:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.275+0000 7f0ba576c700 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569e5ee5180 0x5569e5ee75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.275+0000 7f0b897fa700 10 client.25780 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.275+0000 7f0ba576c700 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569e5ee5180 0x5569e5ee75a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf96a2a096c68951 server_cookie=292f0769f864102f in_seq=0 out_seq=0 2023-11-06T01:39:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0b897fa700 1 -- 172.21.15.53:0/3490315113 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5569e5e8ddb0 con 0x5569e5ee5180 2023-11-06T01:39:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0b897fa700 10 client.25780 handle_command_reply: tid=0 2023-11-06T01:39:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0bb4361b80 1 client.25780 shutdown 2023-11-06T01:39:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0bb4361b80 1 -- 172.21.15.53:0/3490315113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebefe0 msgr2=0x5569e5ed9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebefe0 0x5569e5ed9990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000ec40 tx=0x7f0b9000ec70 comp rx=0 tx=0).stop 2023-11-06T01:39:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0bb4361b80 1 -- 172.21.15.53:0/3490315113 shutdown_connections 2023-11-06T01:39:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebe680 0x5569e5ed9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569e5ee5180 0x5569e5ee75a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5ee09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/3490315113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebefe0 0x5569e5ed9990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0b897fa700 0 client.25780 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0bb4361b80 1 -- 172.21.15.53:0/3490315113 >> 172.21.15.53:0/3490315113 conn(0x5569e5e1e070 msgr2=0x5569e5eb93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0bb4361b80 1 -- 172.21.15.53:0/3490315113 shutdown_connections 2023-11-06T01:39:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.276+0000 7f0bb4361b80 1 -- 172.21.15.53:0/3490315113 wait complete. 2023-11-06T01:39:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.279+0000 7f0bb4361b80 20 client.25780 trim_cache size 0 max 16384 2023-11-06T01:39:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.317+0000 7f0bb4361b80 1 Processor -- start 2023-11-06T01:39:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.317+0000 7f0bb4361b80 1 -- start start 2023-11-06T01:39:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.317+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebe680 0x5569e5ec61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.317+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ee0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.317+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5ebac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.317+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5eb4500 con 0x5569e5ebfed0 2023-11-06T01:39:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.317+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5ed8fc0 con 0x5569e5ebefe0 2023-11-06T01:39:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.317+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5ebf3c0 con 0x5569e5ebe680 2023-11-06T01:39:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.320+0000 7f0ba5f6d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ee0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.320+0000 7f0ba5f6d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ee0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53056/0 (socket says 172.21.15.53:53056) 2023-11-06T01:39:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.320+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/544908609 learned_addr learned my addr 172.21.15.53:0/544908609 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.321+0000 7f0ba576c700 1 --2- 172.21.15.53:0/544908609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebe680 0x5569e5ec61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.322+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/544908609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebe680 msgr2=0x5569e5ec61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.322+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/544908609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebe680 0x5569e5ec61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.322+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/544908609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 msgr2=0x5569e5ebac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.322+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/544908609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5ebac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.322+0000 7f0ba5f6d700 1 -- 172.21.15.53:0/544908609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569e5c641f0 con 0x5569e5ebefe0 2023-11-06T01:39:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.323+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/544908609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ee0990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0b90013bd0 tx=0x7f0b90009db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b048d22a38c1fac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.323+0000 7f0b94ff9700 1 -- 172.21.15.53:0/544908609 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90024ea0 con 0x5569e5ebefe0 2023-11-06T01:39:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.324+0000 7f0b94ff9700 1 -- 172.21.15.53:0/544908609 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b90013e90 con 0x5569e5ebefe0 2023-11-06T01:39:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.324+0000 7f0b94ff9700 1 -- 172.21.15.53:0/544908609 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9002fe90 con 0x5569e5ebefe0 2023-11-06T01:39:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.324+0000 7f0bb4361b80 1 -- 172.21.15.53:0/544908609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 msgr2=0x5569e5ee0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.324+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/544908609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ee0990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0b90013bd0 tx=0x7f0b90009db0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.336+0000 7f0bb4361b80 1 -- 172.21.15.53:0/544908609 shutdown_connections 2023-11-06T01:39:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.336+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/544908609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5ebac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.336+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/544908609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebe680 0x5569e5ec61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.336+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/544908609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebefe0 0x5569e5ee0990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.336+0000 7f0bb4361b80 1 -- 172.21.15.53:0/544908609 >> 172.21.15.53:0/544908609 conn(0x5569e5e1e070 msgr2=0x5569e5eb8660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.343+0000 7f0bb4361b80 1 -- 172.21.15.53:0/544908609 shutdown_connections 2023-11-06T01:39:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.344+0000 7f0bb4361b80 1 -- 172.21.15.53:0/544908609 wait complete. 2023-11-06T01:39:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.344+0000 7f0bb4361b80 1 -- start start 2023-11-06T01:39:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.344+0000 7f0bb4361b80 1 Processor -- start 2023-11-06T01:39:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.345+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebe680 0x5569e5ede910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.345+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebefe0 0x5569e5edee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.345+0000 7f0bb4361b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5edfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.345+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5ee03a0 con 0x5569e5ebfed0 2023-11-06T01:39:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.345+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5ee0510 con 0x5569e5ebe680 2023-11-06T01:39:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.345+0000 7f0bb4361b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e5ee06d0 con 0x5569e5ebefe0 2023-11-06T01:39:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.345+0000 7f0bb4361b80 10 client.0 _ll_register_callbacks cb 0x7f0b900098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:39:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.345+0000 7f0ba576c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebe680 0x5569e5ede910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.345+0000 7f0ba576c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebe680 0x5569e5ede910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53062/0 (socket says 172.21.15.53:53062) 2023-11-06T01:39:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.345+0000 7f0ba576c700 1 -- 172.21.15.53:0/2000092228 learned_addr learned my addr 172.21.15.53:0/2000092228 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:39:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.346+0000 7f0ba576c700 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5edfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.346+0000 7f0ba576c700 1 -- 172.21.15.53:0/2000092228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebefe0 msgr2=0x5569e5edee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.346+0000 7f0ba576c700 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebefe0 0x5569e5edee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.346+0000 7f0ba576c700 1 -- 172.21.15.53:0/2000092228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebe680 msgr2=0x5569e5ede910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:39:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.346+0000 7f0ba576c700 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebe680 0x5569e5ede910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.346+0000 7f0ba576c700 1 -- 172.21.15.53:0/2000092228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569e5c26c00 con 0x5569e5ebfed0 2023-11-06T01:39:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.347+0000 7f0ba576c700 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5edfe60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c007560 tx=0x7f0b9c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d678cf79d1217ea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:39:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.347+0000 7f0b897fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:39:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.347+0000 7f0b897fa700 1 -- 172.21.15.53:0/2000092228 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c0220d0 con 0x5569e5ebfed0 2023-11-06T01:39:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.347+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2000092228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b90013470 con 0x5569e5ebfed0 2023-11-06T01:39:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.347+0000 7f0b897fa700 1 -- 172.21.15.53:0/2000092228 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0b9c022230 con 0x5569e5ebfed0 2023-11-06T01:39:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.347+0000 7f0b897fa700 1 -- 172.21.15.53:0/2000092228 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b9c0223f0 con 0x5569e5ebfed0 2023-11-06T01:39:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.347+0000 7f0bb4361b80 10 client.16152 fetch_fsmap learned FSMap version 40 2023-11-06T01:39:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.347+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2000092228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5569e5c3c250 con 0x5569e5ebfed0 2023-11-06T01:39:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.347+0000 7f0b897fa700 1 -- 172.21.15.53:0/2000092228 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b90013470 con 0x5569e5ebfed0 2023-11-06T01:39:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.347+0000 7f0b897fa700 1 -- 172.21.15.53:0/2000092228 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f0b9c020b80 con 0x5569e5ebfed0 2023-11-06T01:39:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.348+0000 7f0bb4361b80 10 client.16152 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:39:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.348+0000 7f0bb4361b80 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:39:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.348+0000 7f0bb4361b80 20 client.16152 populate_metadata read hostname 'smithi053' 2023-11-06T01:39:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.348+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569e5ec7680 0x5569e5ec9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:39:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.348+0000 7f0bb4361b80 4 client.16152 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:39:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.348+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2000092228 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5569e5e08ce0 con 0x5569e5ec7680 2023-11-06T01:39:57.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.354+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569e5ec7680 0x5569e5ec9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:39:57.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.355+0000 7f0b897fa700 10 client.16152 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:57.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.355+0000 7f0ba5f6d700 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569e5ec7680 0x5569e5ec9aa0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c158df50ddfa1595 server_cookie=4e748c1a5163663c in_seq=0 out_seq=0 2023-11-06T01:39:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.441+0000 7f0b897fa700 1 -- 172.21.15.53:0/2000092228 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5569e5e08ce0 con 0x5569e5ec7680 2023-11-06T01:39:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.441+0000 7f0b897fa700 10 client.16152 handle_command_reply: tid=0 2023-11-06T01:39:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.441+0000 7f0bb4361b80 1 client.16152 shutdown 2023-11-06T01:39:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.441+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2000092228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 msgr2=0x5569e5edfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.441+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5edfe60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c007560 tx=0x7f0b9c0047c0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.442+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2000092228 shutdown_connections 2023-11-06T01:39:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.442+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569e5ebe680 0x5569e5ede910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.442+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5569e5ec7680 0x5569e5ec9aa0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.442+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569e5ebfed0 0x5569e5edfe60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.442+0000 7f0b897fa700 0 client.16152 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:39:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.442+0000 7f0bb4361b80 1 --2- 172.21.15.53:0/2000092228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569e5ebefe0 0x5569e5edee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.442+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2000092228 >> 172.21.15.53:0/2000092228 conn(0x5569e5e1e070 msgr2=0x5569e5ee1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.442+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2000092228 shutdown_connections 2023-11-06T01:39:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.442+0000 7f0bb4361b80 1 -- 172.21.15.53:0/2000092228 wait complete. 2023-11-06T01:39:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.442+0000 7f0bb4361b80 20 client.16152 trim_cache size 0 max 16384 2023-11-06T01:39:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:39:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:39:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:39:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:39:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2590385763 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b8c0a0620 msgr2=0x7f0b8c0a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b8c0a0620 0x7f0b8c0a2ae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800b820 tx=0x7f0b9800b3e0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2590385763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba0104b80 msgr2=0x7f0ba019e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:39:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba0104b80 0x7f0ba019e570 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0b980097f0 tx=0x7f0b9800cee0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2590385763 shutdown_connections 2023-11-06T01:39:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0b8c0a0620 0x7f0b8c0a2ae0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba0102ff0 0x7f0ba019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba0104b80 0x7f0ba019e570 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 --2- 172.21.15.53:0/2590385763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ba01041e0 0x7f0ba019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:39:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2590385763 >> 172.21.15.53:0/2590385763 conn(0x7f0ba00fe540 msgr2=0x7f0ba0078880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:39:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2590385763 shutdown_connections 2023-11-06T01:39:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:39:57.443+0000 7f0ba81d1700 1 -- 172.21.15.53:0/2590385763 wait complete. 2023-11-06T01:39:57.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:57 smithi053 ceph-mon[99472]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 918 op/s 2023-11-06T01:39:57.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:57 smithi062 ceph-mon[103924]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 918 op/s 2023-11-06T01:39:57.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:57 smithi163 ceph-mon[104416]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 918 op/s 2023-11-06T01:39:59.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:39:59 smithi053 ceph-mon[99472]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 790 op/s 2023-11-06T01:39:59.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:39:59 smithi062 ceph-mon[103924]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 790 op/s 2023-11-06T01:39:59.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:39:59 smithi163 ceph-mon[104416]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 790 op/s 2023-11-06T01:40:00.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:00 smithi053 ceph-mon[99472]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 846 op/s 2023-11-06T01:40:00.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:00 smithi053 ceph-mon[99472]: overall HEALTH_OK 2023-11-06T01:40:00.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:00 smithi062 ceph-mon[103924]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 846 op/s 2023-11-06T01:40:00.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:00 smithi062 ceph-mon[103924]: overall HEALTH_OK 2023-11-06T01:40:00.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:00 smithi163 ceph-mon[104416]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 846 op/s 2023-11-06T01:40:00.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:00 smithi163 ceph-mon[104416]: overall HEALTH_OK 2023-11-06T01:40:02.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 345.0 s, 2141.9 tps, lat 45.861 ms stddev 49.818 2023-11-06T01:40:03.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:03 smithi163 ceph-mon[104416]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 990 op/s 2023-11-06T01:40:03.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:03 smithi053 ceph-mon[99472]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 990 op/s 2023-11-06T01:40:03.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:03 smithi062 ceph-mon[103924]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 990 op/s 2023-11-06T01:40:05.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:05 smithi163 ceph-mon[104416]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 904 op/s 2023-11-06T01:40:05.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:05 smithi053 ceph-mon[99472]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 904 op/s 2023-11-06T01:40:05.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:05 smithi062 ceph-mon[103924]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 904 op/s 2023-11-06T01:40:06.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:40:07.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.087+0000 7f7b6a933700 1 -- 172.21.15.53:0/1015244161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b640ff680 msgr2=0x7f7b64101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:07.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.087+0000 7f7b6a933700 1 --2- 172.21.15.53:0/1015244161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b640ff680 0x7f7b64101ab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00b0a0 tx=0x7f7b5c00b3b0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.087+0000 7f7b6a933700 1 -- 172.21.15.53:0/1015244161 shutdown_connections 2023-11-06T01:40:07.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.088+0000 7f7b6a933700 1 --2- 172.21.15.53:0/1015244161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b641049c0 0x7f7b64104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.088+0000 7f7b6a933700 1 --2- 172.21.15.53:0/1015244161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b640ff680 0x7f7b64101ab0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.088+0000 7f7b6a933700 1 --2- 172.21.15.53:0/1015244161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64101ff0 0x7f7b64104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.088+0000 7f7b6a933700 1 -- 172.21.15.53:0/1015244161 >> 172.21.15.53:0/1015244161 conn(0x7f7b640fb220 msgr2=0x7f7b640fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:07.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.088+0000 7f7b6a933700 1 -- 172.21.15.53:0/1015244161 shutdown_connections 2023-11-06T01:40:07.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.088+0000 7f7b6a933700 1 -- 172.21.15.53:0/1015244161 wait complete. 2023-11-06T01:40:07.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.088+0000 7f7b6a933700 1 Processor -- start 2023-11-06T01:40:07.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.089+0000 7f7b6a933700 1 -- start start 2023-11-06T01:40:07.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.089+0000 7f7b6a933700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b640ff680 0x7f7b6419d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.089+0000 7f7b6a933700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64101ff0 0x7f7b6419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.089+0000 7f7b6a933700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b641049c0 0x7f7b641a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.089+0000 7f7b6a933700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b641a4db0 con 0x7f7b64101ff0 2023-11-06T01:40:07.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.089+0000 7f7b6a933700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b641a4ef0 con 0x7f7b640ff680 2023-11-06T01:40:07.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.089+0000 7f7b6a933700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b641a5080 con 0x7f7b641049c0 2023-11-06T01:40:07.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.089+0000 7f7b63fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b640ff680 0x7f7b6419d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.089+0000 7f7b68ed0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b641049c0 0x7f7b641a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.089+0000 7f7b63fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b640ff680 0x7f7b6419d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53076/0 (socket says 172.21.15.53:53076) 2023-11-06T01:40:07.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.089+0000 7f7b63fff700 1 -- 172.21.15.53:0/1194348810 learned_addr learned my addr 172.21.15.53:0/1194348810 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:07.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.090+0000 7f7b637fe700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64101ff0 0x7f7b6419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.090+0000 7f7b63fff700 1 -- 172.21.15.53:0/1194348810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b641049c0 msgr2=0x7f7b641a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:07.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.090+0000 7f7b63fff700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b641049c0 0x7f7b641a47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.090+0000 7f7b63fff700 1 -- 172.21.15.53:0/1194348810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64101ff0 msgr2=0x7f7b6419d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:07.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.090+0000 7f7b63fff700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64101ff0 0x7f7b6419d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.090+0000 7f7b63fff700 1 -- 172.21.15.53:0/1194348810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c009d00 con 0x7f7b640ff680 2023-11-06T01:40:07.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.090+0000 7f7b63fff700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b640ff680 0x7f7b6419d230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00b070 tx=0x7f7b5c004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:07.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.091+0000 7f7b617fa700 1 -- 172.21.15.53:0/1194348810 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c00e040 con 0x7f7b640ff680 2023-11-06T01:40:07.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.091+0000 7f7b617fa700 1 -- 172.21.15.53:0/1194348810 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b5c009440 con 0x7f7b640ff680 2023-11-06T01:40:07.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.091+0000 7f7b617fa700 1 -- 172.21.15.53:0/1194348810 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c004020 con 0x7f7b640ff680 2023-11-06T01:40:07.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.091+0000 7f7b6a933700 1 -- 172.21.15.53:0/1194348810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b641a5300 con 0x7f7b640ff680 2023-11-06T01:40:07.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.091+0000 7f7b6a933700 1 -- 172.21.15.53:0/1194348810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b641a57b0 con 0x7f7b640ff680 2023-11-06T01:40:07.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.094+0000 7f7b617fa700 1 -- 172.21.15.53:0/1194348810 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f7b5c003a00 con 0x7f7b640ff680 2023-11-06T01:40:07.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.095+0000 7f7b617fa700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b500a05a0 0x7f7b500a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.095+0000 7f7b617fa700 1 -- 172.21.15.53:0/1194348810 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7b5c0cd110 con 0x7f7b640ff680 2023-11-06T01:40:07.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.096+0000 7f7b76ac3b80 1 Processor -- start 2023-11-06T01:40:07.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.096+0000 7f7b76ac3b80 1 -- start start 2023-11-06T01:40:07.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.096+0000 7f7b637fe700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b500a05a0 0x7f7b500a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.096+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc4a0f0 0x558d2bc4a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.096+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4aa70 0x558d2bc48e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.096+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc49870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.096+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bba0a10 con 0x558d2bc4a0f0 2023-11-06T01:40:07.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.097+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bb9e3d0 con 0x558d2bc4aa70 2023-11-06T01:40:07.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.097+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bb99cc0 con 0x558d2bc493f0 2023-11-06T01:40:07.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.098+0000 7f7b63fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc4a0f0 0x558d2bc4a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.098+0000 7f7b63fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc4a0f0 0x558d2bc4a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41648/0 (socket says 172.21.15.53:41648) 2023-11-06T01:40:07.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.098+0000 7f7b63fff700 1 -- 172.21.15.53:0/3865481133 learned_addr learned my addr 172.21.15.53:0/3865481133 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:07.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.098+0000 7f7b637fe700 1 --2- 172.21.15.53:0/3865481133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4aa70 0x558d2bc48e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.098+0000 7f7b68ed0700 1 --2- 172.21.15.53:0/3865481133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc49870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.098+0000 7f7b637fe700 1 -- 172.21.15.53:0/3865481133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 msgr2=0x558d2bc49870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:07.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.098+0000 7f7b637fe700 1 --2- 172.21.15.53:0/3865481133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc49870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.098+0000 7f7b637fe700 1 -- 172.21.15.53:0/3865481133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc4a0f0 msgr2=0x558d2bc4a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:07.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.098+0000 7f7b637fe700 1 --2- 172.21.15.53:0/3865481133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc4a0f0 0x558d2bc4a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.098+0000 7f7b637fe700 1 -- 172.21.15.53:0/3865481133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d2b9c7de0 con 0x558d2bc4aa70 2023-11-06T01:40:07.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.099+0000 7f7b637fe700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b500a05a0 0x7f7b500a2a60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7b54005950 tx=0x7f7b5400b840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:07.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.099+0000 7f7b637fe700 1 --2- 172.21.15.53:0/3865481133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4aa70 0x558d2bc48e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400b570 tx=0x7f7b5401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a057f7315cf6f0e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:07.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.099+0000 7f7b5a7fc700 1 -- 172.21.15.53:0/3865481133 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54011dc0 con 0x558d2bc4aa70 2023-11-06T01:40:07.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.099+0000 7f7b5a7fc700 1 -- 172.21.15.53:0/3865481133 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b54019070 con 0x558d2bc4aa70 2023-11-06T01:40:07.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.099+0000 7f7b5a7fc700 1 -- 172.21.15.53:0/3865481133 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54003dc0 con 0x558d2bc4aa70 2023-11-06T01:40:07.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.100+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3865481133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4aa70 msgr2=0x558d2bc48e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:07.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.100+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/3865481133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4aa70 0x558d2bc48e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400b570 tx=0x7f7b5401d000 comp rx=0 tx=0).stop 2023-11-06T01:40:07.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.100+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3865481133 shutdown_connections 2023-11-06T01:40:07.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.100+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/3865481133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc49870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.100+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/3865481133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc4a0f0 0x558d2bc4a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.100+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/3865481133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4aa70 0x558d2bc48e60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.100+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3865481133 >> 172.21.15.53:0/3865481133 conn(0x558d2bba80d0 msgr2=0x558d2bc43940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:07.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.100+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3865481133 shutdown_connections 2023-11-06T01:40:07.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.100+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3865481133 wait complete. 2023-11-06T01:40:07.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.101+0000 7f7b76ac3b80 1 -- start start 2023-11-06T01:40:07.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.101+0000 7f7b76ac3b80 1 Processor -- start 2023-11-06T01:40:07.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.102+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.102+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc4a0f0 0x558d2bc52ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.102+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4aa70 0x558d2bc59f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.102+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bb36510 con 0x558d2bc4a0f0 2023-11-06T01:40:07.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.102+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bb361b0 con 0x558d2bc4aa70 2023-11-06T01:40:07.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.102+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2ba1d920 con 0x558d2bc493f0 2023-11-06T01:40:07.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.102+0000 7f7b76ac3b80 10 client.0 _ll_register_callbacks cb 0x558d2ba83a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:07.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b63fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b63fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55894/0 (socket says 172.21.15.53:55894) 2023-11-06T01:40:07.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b63fff700 1 -- 172.21.15.53:0/3495305684 learned_addr learned my addr 172.21.15.53:0/3495305684 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:07.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b637fe700 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc4a0f0 0x558d2bc52ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b68ed0700 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4aa70 0x558d2bc59f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b63fff700 1 -- 172.21.15.53:0/3495305684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4aa70 msgr2=0x558d2bc59f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:07.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b63fff700 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4aa70 0x558d2bc59f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b63fff700 1 -- 172.21.15.53:0/3495305684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc4a0f0 msgr2=0x558d2bc52ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:07.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b63fff700 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc4a0f0 0x558d2bc52ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b63fff700 1 -- 172.21.15.53:0/3495305684 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b54011380 con 0x558d2bc493f0 2023-11-06T01:40:07.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b63fff700 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc529a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c01f000 tx=0x7f7b5c01d990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6816aa78afcaa358 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:07.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.103+0000 7f7b437fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:40:07.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.104+0000 7f7b437fe700 1 -- 172.21.15.53:0/3495305684 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c09ae40 con 0x558d2bc493f0 2023-11-06T01:40:07.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.104+0000 7f7b437fe700 1 -- 172.21.15.53:0/3495305684 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7b5c0b5020 con 0x558d2bc493f0 2023-11-06T01:40:07.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.104+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3495305684 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d2bbc4d10 con 0x558d2bc493f0 2023-11-06T01:40:07.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.104+0000 7f7b437fe700 1 -- 172.21.15.53:0/3495305684 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b5c09a340 con 0x558d2bc493f0 2023-11-06T01:40:07.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.104+0000 7f7b76ac3b80 10 client.25795 fetch_fsmap learned FSMap version 40 2023-11-06T01:40:07.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.104+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3495305684 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558d2b9b0c60 con 0x558d2bc493f0 2023-11-06T01:40:07.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.105+0000 7f7b437fe700 1 -- 172.21.15.53:0/3495305684 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d2bbc4d10 con 0x558d2bc493f0 2023-11-06T01:40:07.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.105+0000 7f7b437fe700 1 -- 172.21.15.53:0/3495305684 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7b5c099ed0 con 0x558d2bc493f0 2023-11-06T01:40:07.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.105+0000 7f7b76ac3b80 10 client.25795 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:40:07.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.105+0000 7f7b76ac3b80 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:07.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.105+0000 7f7b76ac3b80 20 client.25795 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:07.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.105+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d2bc6f4b0 0x558d2bc718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.105+0000 7f7b76ac3b80 4 client.25795 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:40:07.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.105+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3495305684 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d2bc17e10 con 0x558d2bc6f4b0 2023-11-06T01:40:07.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.105+0000 7f7b637fe700 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d2bc6f4b0 0x558d2bc718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.106+0000 7f7b637fe700 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d2bc6f4b0 0x558d2bc718d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e80e5b19ecc863af server_cookie=34d65b80f6cd43a in_seq=0 out_seq=0 2023-11-06T01:40:07.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.106+0000 7f7b437fe700 10 client.25795 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:07.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.107+0000 7f7b437fe700 1 -- 172.21.15.53:0/3495305684 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558d2bc17e10 con 0x558d2bc6f4b0 2023-11-06T01:40:07.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.107+0000 7f7b437fe700 10 client.25795 handle_command_reply: tid=0 2023-11-06T01:40:07.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.107+0000 7f7b76ac3b80 1 client.25795 shutdown 2023-11-06T01:40:07.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.108+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3495305684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 msgr2=0x558d2bc529a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:07.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.108+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc529a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c01f000 tx=0x7f7b5c01d990 comp rx=0 tx=0).stop 2023-11-06T01:40:07.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.108+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3495305684 shutdown_connections 2023-11-06T01:40:07.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.109+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4aa70 0x558d2bc59f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.109+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d2bc6f4b0 0x558d2bc718d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.109+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc529a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.109+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/3495305684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc4a0f0 0x558d2bc52ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.109+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3495305684 >> 172.21.15.53:0/3495305684 conn(0x558d2bba80d0 msgr2=0x558d2bc430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:07.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.109+0000 7f7b437fe700 0 client.25795 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:07.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.109+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3495305684 shutdown_connections 2023-11-06T01:40:07.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.109+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/3495305684 wait complete. 2023-11-06T01:40:07.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.109+0000 7f7b76ac3b80 20 client.25795 trim_cache size 0 max 16384 2023-11-06T01:40:07.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.130+0000 7f7b76ac3b80 1 Processor -- start 2023-11-06T01:40:07.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.130+0000 7f7b76ac3b80 1 -- start start 2023-11-06T01:40:07.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bba80d0 0x558d2bc50d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc59ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4a0f0 0x558d2bc48e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bc52570 con 0x558d2bba80d0 2023-11-06T01:40:07.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bc17e10 con 0x558d2bc4a0f0 2023-11-06T01:40:07.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bbc4d10 con 0x558d2bc493f0 2023-11-06T01:40:07.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b63fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bba80d0 0x558d2bc50d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b63fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bba80d0 0x558d2bc50d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41660/0 (socket says 172.21.15.53:41660) 2023-11-06T01:40:07.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b63fff700 1 -- 172.21.15.53:0/514781342 learned_addr learned my addr 172.21.15.53:0/514781342 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:07.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b68ed0700 1 --2- 172.21.15.53:0/514781342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4a0f0 0x558d2bc48e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b637fe700 1 --2- 172.21.15.53:0/514781342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc59ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b63fff700 1 -- 172.21.15.53:0/514781342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 msgr2=0x558d2bc59ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:07.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b63fff700 1 --2- 172.21.15.53:0/514781342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc59ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b63fff700 1 -- 172.21.15.53:0/514781342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4a0f0 msgr2=0x558d2bc48e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:07.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b63fff700 1 --2- 172.21.15.53:0/514781342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4a0f0 0x558d2bc48e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b63fff700 1 -- 172.21.15.53:0/514781342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d2bc3fff0 con 0x558d2bba80d0 2023-11-06T01:40:07.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b68ed0700 1 --2- 172.21.15.53:0/514781342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4a0f0 0x558d2bc48e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:07.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b63fff700 1 --2- 172.21.15.53:0/514781342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bba80d0 0x558d2bc50d10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c098d30 tx=0x7f7b5c01d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688e76591fbf7438 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:07.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.131+0000 7f7b5affd700 1 -- 172.21.15.53:0/514781342 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0a2a40 con 0x558d2bba80d0 2023-11-06T01:40:07.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.132+0000 7f7b5affd700 1 -- 172.21.15.53:0/514781342 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b5c09a340 con 0x558d2bba80d0 2023-11-06T01:40:07.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.132+0000 7f7b5affd700 1 -- 172.21.15.53:0/514781342 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c09ac40 con 0x558d2bba80d0 2023-11-06T01:40:07.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.132+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/514781342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bba80d0 msgr2=0x558d2bc50d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:07.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.132+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/514781342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bba80d0 0x558d2bc50d10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c098d30 tx=0x7f7b5c01d8b0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.132+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/514781342 shutdown_connections 2023-11-06T01:40:07.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.133+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/514781342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bc4a0f0 0x558d2bc48e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.133+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/514781342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bba80d0 0x558d2bc50d10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.133+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/514781342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc493f0 0x558d2bc59ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.133+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/514781342 >> 172.21.15.53:0/514781342 conn(0x558d2bc43c70 msgr2=0x558d2bc50600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:07.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.133+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/514781342 shutdown_connections 2023-11-06T01:40:07.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.133+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/514781342 wait complete. 2023-11-06T01:40:07.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.133+0000 7f7b76ac3b80 1 -- start start 2023-11-06T01:40:07.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.133+0000 7f7b76ac3b80 1 Processor -- start 2023-11-06T01:40:07.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.134+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bba80d0 0x558d2bc59a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.134+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc493f0 0x558d2bc58020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.134+0000 7f7b76ac3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc4a0f0 0x558d2bc58560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.134+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bc5a230 con 0x558d2bc493f0 2023-11-06T01:40:07.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.134+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bc58aa0 con 0x558d2bba80d0 2023-11-06T01:40:07.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.134+0000 7f7b76ac3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bc58c60 con 0x558d2bc4a0f0 2023-11-06T01:40:07.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.134+0000 7f7b76ac3b80 10 client.0 _ll_register_callbacks cb 0x7f7b5c009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:07.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.134+0000 7f7b637fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc493f0 0x558d2bc58020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.134+0000 7f7b637fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc493f0 0x558d2bc58020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41668/0 (socket says 172.21.15.53:41668) 2023-11-06T01:40:07.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.134+0000 7f7b637fe700 1 -- 172.21.15.53:0/8321469 learned_addr learned my addr 172.21.15.53:0/8321469 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:07.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.134+0000 7f7b63fff700 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bba80d0 0x558d2bc59a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.135+0000 7f7b63fff700 1 -- 172.21.15.53:0/8321469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc4a0f0 msgr2=0x558d2bc58560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:40:07.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.135+0000 7f7b63fff700 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc4a0f0 0x558d2bc58560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.135+0000 7f7b63fff700 1 -- 172.21.15.53:0/8321469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc493f0 msgr2=0x558d2bc58020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:07.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.135+0000 7f7b63fff700 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc493f0 0x558d2bc58020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.135+0000 7f7b63fff700 1 -- 172.21.15.53:0/8321469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c0097c0 con 0x558d2bba80d0 2023-11-06T01:40:07.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.135+0000 7f7b63fff700 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bba80d0 0x558d2bc59a10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x558d2bc444e0 tx=0x7f7b5c01d990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f2025bef29e14fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:07.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.135+0000 7f7b437fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:40:07.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.135+0000 7f7b437fe700 1 -- 172.21.15.53:0/8321469 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c01c3e0 con 0x558d2bba80d0 2023-11-06T01:40:07.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b437fe700 1 -- 172.21.15.53:0/8321469 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7b5c0b5070 con 0x558d2bba80d0 2023-11-06T01:40:07.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/8321469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b54019070 con 0x558d2bba80d0 2023-11-06T01:40:07.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b437fe700 1 -- 172.21.15.53:0/8321469 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b5c09b0e0 con 0x558d2bba80d0 2023-11-06T01:40:07.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b76ac3b80 10 client.25742 fetch_fsmap learned FSMap version 40 2023-11-06T01:40:07.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/8321469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558d2b9b0c60 con 0x558d2bba80d0 2023-11-06T01:40:07.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b437fe700 1 -- 172.21.15.53:0/8321469 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b54019070 con 0x558d2bba80d0 2023-11-06T01:40:07.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b437fe700 1 -- 172.21.15.53:0/8321469 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7b5c079b10 con 0x558d2bba80d0 2023-11-06T01:40:07.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b76ac3b80 10 client.25742 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:40:07.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b76ac3b80 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:07.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b76ac3b80 20 client.25742 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:07.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d2bc4aa70 0x558d2bcc3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:07.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.136+0000 7f7b76ac3b80 4 client.25742 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:40:07.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.137+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/8321469 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558d2bb92d40 con 0x558d2bc4aa70 2023-11-06T01:40:07.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.137+0000 7f7b637fe700 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d2bc4aa70 0x558d2bcc3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:07.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.137+0000 7f7b437fe700 10 client.25742 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:07.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.137+0000 7f7b637fe700 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d2bc4aa70 0x558d2bcc3f30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90882bbaf5bfb4da server_cookie=bd17719227e79b00 in_seq=0 out_seq=0 2023-11-06T01:40:07.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.137+0000 7f7b437fe700 1 -- 172.21.15.53:0/8321469 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558d2bb92d40 con 0x558d2bc4aa70 2023-11-06T01:40:07.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.137+0000 7f7b437fe700 10 client.25742 handle_command_reply: tid=0 2023-11-06T01:40:07.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.137+0000 7f7b76ac3b80 1 client.25742 shutdown 2023-11-06T01:40:07.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/8321469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bba80d0 msgr2=0x558d2bc59a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:07.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bba80d0 0x558d2bc59a10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x558d2bc444e0 tx=0x7f7b5c01d990 comp rx=0 tx=0).stop 2023-11-06T01:40:07.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/8321469 shutdown_connections 2023-11-06T01:40:07.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d2bba80d0 0x558d2bc59a10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558d2bc4aa70 0x558d2bcc3f30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d2bc4a0f0 0x558d2bc58560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b76ac3b80 1 --2- 172.21.15.53:0/8321469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d2bc493f0 0x558d2bc58020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b437fe700 0 client.25742 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:07.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/8321469 >> 172.21.15.53:0/8321469 conn(0x558d2bc43c70 msgr2=0x558d2bc52750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:07.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/8321469 shutdown_connections 2023-11-06T01:40:07.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b76ac3b80 1 -- 172.21.15.53:0/8321469 wait complete. 2023-11-06T01:40:07.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b76ac3b80 20 client.25742 trim_cache size 0 max 16384 2023-11-06T01:40:07.153 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:40:07.153 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:40:07.153 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:40:07.153 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:40:07.154 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:40:07.154 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:40:07.154 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:40:07.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:40:07.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:40:07.155 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:40:07.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b6a933700 1 -- 172.21.15.53:0/1194348810 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b500a05a0 msgr2=0x7f7b500a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:07.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b6a933700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b500a05a0 0x7f7b500a2a60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7b54005950 tx=0x7f7b5400b840 comp rx=0 tx=0).stop 2023-11-06T01:40:07.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b6a933700 1 -- 172.21.15.53:0/1194348810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b640ff680 msgr2=0x7f7b6419d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:07.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.138+0000 7f7b6a933700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b640ff680 0x7f7b6419d230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00b070 tx=0x7f7b5c004db0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.139+0000 7f7b6a933700 1 -- 172.21.15.53:0/1194348810 shutdown_connections 2023-11-06T01:40:07.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.139+0000 7f7b6a933700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b500a05a0 0x7f7b500a2a60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.139+0000 7f7b6a933700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b640ff680 0x7f7b6419d230 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.139+0000 7f7b6a933700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b641049c0 0x7f7b641a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.139+0000 7f7b6a933700 1 --2- 172.21.15.53:0/1194348810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64101ff0 0x7f7b6419d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:07.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.139+0000 7f7b6a933700 1 -- 172.21.15.53:0/1194348810 >> 172.21.15.53:0/1194348810 conn(0x7f7b640fb220 msgr2=0x7f7b64102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:07.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.139+0000 7f7b6a933700 1 -- 172.21.15.53:0/1194348810 shutdown_connections 2023-11-06T01:40:07.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:07.139+0000 7f7b6a933700 1 -- 172.21.15.53:0/1194348810 wait complete. 2023-11-06T01:40:07.173 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:40:07.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 350.0 s, 1687.0 tps, lat 59.920 ms stddev 121.619 2023-11-06T01:40:07.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:07 smithi163 ceph-mon[104416]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 888 op/s 2023-11-06T01:40:07.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:07 smithi053 ceph-mon[99472]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 888 op/s 2023-11-06T01:40:07.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:07 smithi062 ceph-mon[103924]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 888 op/s 2023-11-06T01:40:09.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:09 smithi163 ceph-mon[104416]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.48k op/s 2023-11-06T01:40:09.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:09 smithi053 ceph-mon[99472]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.48k op/s 2023-11-06T01:40:09.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:09 smithi062 ceph-mon[103924]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.48k op/s 2023-11-06T01:40:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.619+0000 7f8efa7bb700 1 -- 172.21.15.53:0/1192540373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef4108ea0 msgr2=0x7f8ef410b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.619+0000 7f8efa7bb700 1 --2- 172.21.15.53:0/1192540373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef4108ea0 0x7f8ef410b290 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8edc013ef0 tx=0x7f8edc0097f0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.621+0000 7f8efa7bb700 1 -- 172.21.15.53:0/1192540373 shutdown_connections 2023-11-06T01:40:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.621+0000 7f8efa7bb700 1 --2- 172.21.15.53:0/1192540373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef4107de0 0x7f8ef4108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.621+0000 7f8efa7bb700 1 --2- 172.21.15.53:0/1192540373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef4108ea0 0x7f8ef410b290 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.621+0000 7f8efa7bb700 1 --2- 172.21.15.53:0/1192540373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef410b7d0 0x7f8ef410dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.621+0000 7f8efa7bb700 1 -- 172.21.15.53:0/1192540373 >> 172.21.15.53:0/1192540373 conn(0x7f8ef406dbc0 msgr2=0x7f8ef4070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.636+0000 7f8efa7bb700 1 -- 172.21.15.53:0/1192540373 shutdown_connections 2023-11-06T01:40:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.638+0000 7f8efa7bb700 1 -- 172.21.15.53:0/1192540373 wait complete. 2023-11-06T01:40:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.639+0000 7f8efa7bb700 1 Processor -- start 2023-11-06T01:40:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8efa7bb700 1 -- start start 2023-11-06T01:40:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8efa7bb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef4107de0 0x7f8ef41ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8efa7bb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef4108ea0 0x7f8ef41ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8efa7bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef410b7d0 0x7f8ef41af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8efa7bb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef41afcc0 con 0x7f8ef4108ea0 2023-11-06T01:40:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8efa7bb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef41afe30 con 0x7f8ef410b7d0 2023-11-06T01:40:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8efa7bb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef41afff0 con 0x7f8ef4107de0 2023-11-06T01:40:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8ef3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef4107de0 0x7f8ef41ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8ef3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef4107de0 0x7f8ef41ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55918/0 (socket says 172.21.15.53:55918) 2023-11-06T01:40:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8ef3fff700 1 -- 172.21.15.53:0/3086299661 learned_addr learned my addr 172.21.15.53:0/3086299661 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8eeb7fe700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef4108ea0 0x7f8ef41ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.640+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef410b7d0 0x7f8ef41af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.641+0000 7f8ef3fff700 1 -- 172.21.15.53:0/3086299661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef410b7d0 msgr2=0x7f8ef41af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.641+0000 7f8ef3fff700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef410b7d0 0x7f8ef41af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.641+0000 7f8ef3fff700 1 -- 172.21.15.53:0/3086299661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef4108ea0 msgr2=0x7f8ef41ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.641+0000 7f8ef3fff700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef4108ea0 0x7f8ef41ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.641+0000 7f8ef3fff700 1 -- 172.21.15.53:0/3086299661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc013b50 con 0x7f8ef4107de0 2023-11-06T01:40:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.641+0000 7f8ef3fff700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef4107de0 0x7f8ef41ae2a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009c70 tx=0x7f8edc004030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.641+0000 7f8ef1ffb700 1 -- 172.21.15.53:0/3086299661 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0134d0 con 0x7f8ef4107de0 2023-11-06T01:40:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.641+0000 7f8ef1ffb700 1 -- 172.21.15.53:0/3086299661 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8edc00f450 con 0x7f8ef4107de0 2023-11-06T01:40:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.641+0000 7f8ef1ffb700 1 -- 172.21.15.53:0/3086299661 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc01b660 con 0x7f8ef4107de0 2023-11-06T01:40:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.641+0000 7f8efa7bb700 1 -- 172.21.15.53:0/3086299661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef41b0270 con 0x7f8ef4107de0 2023-11-06T01:40:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.641+0000 7f8efa7bb700 1 -- 172.21.15.53:0/3086299661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef41b6a90 con 0x7f8ef4107de0 2023-11-06T01:40:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8f0694bb80 1 Processor -- start 2023-11-06T01:40:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8f0694bb80 1 -- start start 2023-11-06T01:40:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8ef1ffb700 1 -- 172.21.15.53:0/3086299661 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f8edc00f5c0 con 0x7f8ef4107de0 2023-11-06T01:40:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b6262aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262fe0 0x55d3b62638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 0x55d3b6264350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b61ba9b0 con 0x55d3b6262680 2023-11-06T01:40:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b61b8370 con 0x55d3b6263ed0 2023-11-06T01:40:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b61b3c60 con 0x55d3b6262fe0 2023-11-06T01:40:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8ef8d58700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b6262aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8ef8d58700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b6262aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41684/0 (socket says 172.21.15.53:41684) 2023-11-06T01:40:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8ef8d58700 1 -- 172.21.15.53:0/548497038 learned_addr learned my addr 172.21.15.53:0/548497038 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.647+0000 7f8ef3fff700 1 --2- 172.21.15.53:0/548497038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262fe0 0x55d3b62638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.648+0000 7f8ef8d58700 1 -- 172.21.15.53:0/548497038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262fe0 msgr2=0x55d3b62638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.648+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/548497038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262fe0 0x55d3b62638f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.648+0000 7f8ef8d58700 1 -- 172.21.15.53:0/548497038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 msgr2=0x55d3b6264350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.648+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/548497038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 0x55d3b6264350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.648+0000 7f8ef8d58700 1 -- 172.21.15.53:0/548497038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3b5fe1d80 con 0x55d3b6262680 2023-11-06T01:40:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.648+0000 7f8ef1ffb700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8ee00a05f0 0x7f8ee00a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.648+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/548497038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b6262aa0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00a4d0 tx=0x7f8eec009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b067d45622d87f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.648+0000 7f8ef3fff700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8ee00a05f0 0x7f8ee00a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.648+0000 7f8ef1ffb700 1 -- 172.21.15.53:0/3086299661 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8edc0ccf90 con 0x7f8ef4107de0 2023-11-06T01:40:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.648+0000 7f8eea7fc700 1 -- 172.21.15.53:0/548497038 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec010040 con 0x55d3b6262680 2023-11-06T01:40:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.649+0000 7f8ef3fff700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8ee00a05f0 0x7f8ee00a2ab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8edc003af0 tx=0x7f8edc0d0910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.649+0000 7f8eea7fc700 1 -- 172.21.15.53:0/548497038 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eec0148e0 con 0x55d3b6262680 2023-11-06T01:40:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.649+0000 7f8eea7fc700 1 -- 172.21.15.53:0/548497038 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec013930 con 0x55d3b6262680 2023-11-06T01:40:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.650+0000 7f8f0694bb80 1 -- 172.21.15.53:0/548497038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 msgr2=0x55d3b6262aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.650+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/548497038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b6262aa0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00a4d0 tx=0x7f8eec009650 comp rx=0 tx=0).stop 2023-11-06T01:40:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.650+0000 7f8f0694bb80 1 -- 172.21.15.53:0/548497038 shutdown_connections 2023-11-06T01:40:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.650+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/548497038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 0x55d3b6264350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.650+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/548497038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b6262aa0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.650+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/548497038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262fe0 0x55d3b62638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.650+0000 7f8f0694bb80 1 -- 172.21.15.53:0/548497038 >> 172.21.15.53:0/548497038 conn(0x55d3b61c2070 msgr2=0x55d3b625f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.650+0000 7f8f0694bb80 1 -- 172.21.15.53:0/548497038 shutdown_connections 2023-11-06T01:40:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.650+0000 7f8f0694bb80 1 -- 172.21.15.53:0/548497038 wait complete. 2023-11-06T01:40:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.650+0000 7f8f0694bb80 1 -- start start 2023-11-06T01:40:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8f0694bb80 1 Processor -- start 2023-11-06T01:40:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6262680 0x55d3b626c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b626ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6263ed0 0x55d3b6273a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b61504b0 con 0x55d3b6262fe0 2023-11-06T01:40:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6150150 con 0x55d3b6262680 2023-11-06T01:40:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b60378c0 con 0x55d3b6263ed0 2023-11-06T01:40:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8f0694bb80 10 client.0 _ll_register_callbacks cb 0x55d3b609da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8eeb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b626ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8eeb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b626ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41694/0 (socket says 172.21.15.53:41694) 2023-11-06T01:40:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8ef8d58700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6262680 0x55d3b626c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8eeb7fe700 1 -- 172.21.15.53:0/2505758827 learned_addr learned my addr 172.21.15.53:0/2505758827 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6263ed0 0x55d3b6273a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8eeb7fe700 1 -- 172.21.15.53:0/2505758827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6263ed0 msgr2=0x55d3b6273a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8eeb7fe700 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6263ed0 0x55d3b6273a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8eeb7fe700 1 -- 172.21.15.53:0/2505758827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6262680 msgr2=0x55d3b626c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8eeb7fe700 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6262680 0x55d3b626c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.651+0000 7f8eeb7fe700 1 -- 172.21.15.53:0/2505758827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eec009d00 con 0x55d3b6262fe0 2023-11-06T01:40:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8eeb7fe700 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b626ca20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400e480 tx=0x7f8ee400e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee8fc7e088fd48e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8ed37fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:40:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8ed37fe700 1 -- 172.21.15.53:0/2505758827 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4013470 con 0x55d3b6262fe0 2023-11-06T01:40:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8ed37fe700 1 -- 172.21.15.53:0/2505758827 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8ee40135d0 con 0x55d3b6262fe0 2023-11-06T01:40:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8f0694bb80 1 -- 172.21.15.53:0/2505758827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3b61decb0 con 0x55d3b6262fe0 2023-11-06T01:40:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8ed37fe700 1 -- 172.21.15.53:0/2505758827 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ee4013790 con 0x55d3b6262fe0 2023-11-06T01:40:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8f0694bb80 10 client.16182 fetch_fsmap learned FSMap version 40 2023-11-06T01:40:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8f0694bb80 1 -- 172.21.15.53:0/2505758827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d3b5fcac00 con 0x55d3b6262fe0 2023-11-06T01:40:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8ed37fe700 1 -- 172.21.15.53:0/2505758827 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3b61decb0 con 0x55d3b6262fe0 2023-11-06T01:40:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8ed37fe700 1 -- 172.21.15.53:0/2505758827 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8ee4011070 con 0x55d3b6262fe0 2023-11-06T01:40:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8f0694bb80 10 client.16182 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:40:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8f0694bb80 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8f0694bb80 20 client.16182 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d3b6278350 0x55d3b627a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8f0694bb80 4 client.16182 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:40:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.652+0000 7f8f0694bb80 1 -- 172.21.15.53:0/2505758827 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3b6231db0 con 0x55d3b6278350 2023-11-06T01:40:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.653+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d3b6278350 0x55d3b627a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.653+0000 7f8ed37fe700 10 client.16182 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.653+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d3b6278350 0x55d3b627a770 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82553b194a17402 server_cookie=3aa6d29f52014011 in_seq=0 out_seq=0 2023-11-06T01:40:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8ed37fe700 1 -- 172.21.15.53:0/2505758827 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x7f8eec02c5b0 con 0x55d3b6278350 2023-11-06T01:40:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8ed37fe700 10 client.16182 handle_command_reply: tid=0 2023-11-06T01:40:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 1 client.16182 shutdown 2023-11-06T01:40:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 1 -- 172.21.15.53:0/2505758827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 msgr2=0x55d3b626ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b626ca20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400e480 tx=0x7f8ee400e790 comp rx=0 tx=0).stop 2023-11-06T01:40:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 1 -- 172.21.15.53:0/2505758827 shutdown_connections 2023-11-06T01:40:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6262680 0x55d3b626c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d3b6278350 0x55d3b627a770 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6263ed0 0x55d3b6273a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8ed37fe700 0 client.16182 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/2505758827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b626ca20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 1 -- 172.21.15.53:0/2505758827 >> 172.21.15.53:0/2505758827 conn(0x55d3b61c2070 msgr2=0x55d3b625d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 1 -- 172.21.15.53:0/2505758827 shutdown_connections 2023-11-06T01:40:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 1 -- 172.21.15.53:0/2505758827 wait complete. 2023-11-06T01:40:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.654+0000 7f8f0694bb80 20 client.16182 trim_cache size 0 max 16384 2023-11-06T01:40:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.675+0000 7f8f0694bb80 1 Processor -- start 2023-11-06T01:40:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.675+0000 7f8f0694bb80 1 -- start start 2023-11-06T01:40:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.675+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262680 0x55d3b6271a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.675+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b6264310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.675+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 0x55d3b62670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.675+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b626b990 con 0x55d3b6262fe0 2023-11-06T01:40:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.675+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6258500 con 0x55d3b6263ed0 2023-11-06T01:40:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.675+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b626c050 con 0x55d3b6262680 2023-11-06T01:40:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.675+0000 7f8eeb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b6264310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.676+0000 7f8ef8d58700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262680 0x55d3b6271a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.676+0000 7f8eeb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b6264310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41702/0 (socket says 172.21.15.53:41702) 2023-11-06T01:40:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.676+0000 7f8eeb7fe700 1 -- 172.21.15.53:0/1136332965 learned_addr learned my addr 172.21.15.53:0/1136332965 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.676+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/1136332965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 0x55d3b62670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.676+0000 7f8eeb7fe700 1 -- 172.21.15.53:0/1136332965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262680 msgr2=0x55d3b6271a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.676+0000 7f8eeb7fe700 1 --2- 172.21.15.53:0/1136332965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262680 0x55d3b6271a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.676+0000 7f8eeb7fe700 1 -- 172.21.15.53:0/1136332965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 msgr2=0x55d3b62670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.676+0000 7f8eeb7fe700 1 --2- 172.21.15.53:0/1136332965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 0x55d3b62670a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.676+0000 7f8eeb7fe700 1 -- 172.21.15.53:0/1136332965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3b60081f0 con 0x55d3b6262fe0 2023-11-06T01:40:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.676+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/1136332965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262680 0x55d3b6271a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.677+0000 7f8eeb7fe700 1 --2- 172.21.15.53:0/1136332965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b6264310 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40129f0 tx=0x7f8ee400ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815f2a94184e5b07 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.677+0000 7f8eeaffd700 1 -- 172.21.15.53:0/1136332965 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40125d0 con 0x55d3b6262fe0 2023-11-06T01:40:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.678+0000 7f8eeaffd700 1 -- 172.21.15.53:0/1136332965 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ee4012ea0 con 0x55d3b6262fe0 2023-11-06T01:40:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.678+0000 7f8eeaffd700 1 -- 172.21.15.53:0/1136332965 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4004570 con 0x55d3b6262fe0 2023-11-06T01:40:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.678+0000 7f8f0694bb80 1 -- 172.21.15.53:0/1136332965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 msgr2=0x55d3b6264310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.678+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/1136332965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b6264310 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40129f0 tx=0x7f8ee400ffd0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.678+0000 7f8f0694bb80 1 -- 172.21.15.53:0/1136332965 shutdown_connections 2023-11-06T01:40:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.678+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/1136332965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 0x55d3b62670a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.678+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/1136332965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262680 0x55d3b6271a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.678+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/1136332965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262fe0 0x55d3b6264310 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.679+0000 7f8f0694bb80 1 -- 172.21.15.53:0/1136332965 >> 172.21.15.53:0/1136332965 conn(0x55d3b61c2070 msgr2=0x55d3b626b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.679+0000 7f8f0694bb80 1 -- 172.21.15.53:0/1136332965 shutdown_connections 2023-11-06T01:40:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.679+0000 7f8f0694bb80 1 -- 172.21.15.53:0/1136332965 wait complete. 2023-11-06T01:40:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.679+0000 7f8f0694bb80 1 -- start start 2023-11-06T01:40:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.679+0000 7f8f0694bb80 1 Processor -- start 2023-11-06T01:40:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b62c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262fe0 0x55d3b62c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8f0694bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 0x55d3b62c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b62c8fe0 con 0x55d3b6262680 2023-11-06T01:40:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b62c9150 con 0x55d3b6263ed0 2023-11-06T01:40:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8f0694bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b62c9310 con 0x55d3b6262fe0 2023-11-06T01:40:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8f0694bb80 10 client.0 _ll_register_callbacks cb 0x7f8ee4009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8ef8d58700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b62c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8ef8d58700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b62c3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41706/0 (socket says 172.21.15.53:41706) 2023-11-06T01:40:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8ef8d58700 1 -- 172.21.15.53:0/776606982 learned_addr learned my addr 172.21.15.53:0/776606982 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 0x55d3b62c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8eeb7fe700 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262fe0 0x55d3b62c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8ef8d58700 1 -- 172.21.15.53:0/776606982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262fe0 msgr2=0x55d3b62c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262fe0 0x55d3b62c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8ef8d58700 1 -- 172.21.15.53:0/776606982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 msgr2=0x55d3b62c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 0x55d3b62c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8ef8d58700 1 -- 172.21.15.53:0/776606982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3b5fe1d80 con 0x55d3b6262680 2023-11-06T01:40:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8ef8d58700 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b62c3e30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00aac0 tx=0x7f8ee4009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec40605fbbf07e93 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.680+0000 7f8ed37fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:40:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.681+0000 7f8ed37fe700 1 -- 172.21.15.53:0/776606982 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec010030 con 0x55d3b6262680 2023-11-06T01:40:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.681+0000 7f8ed37fe700 1 -- 172.21.15.53:0/776606982 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8eec014a50 con 0x55d3b6262680 2023-11-06T01:40:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.681+0000 7f8f0694bb80 1 -- 172.21.15.53:0/776606982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8eec0148e0 con 0x55d3b6262680 2023-11-06T01:40:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.682+0000 7f8ed37fe700 1 -- 172.21.15.53:0/776606982 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eec014c10 con 0x55d3b6262680 2023-11-06T01:40:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.682+0000 7f8f0694bb80 10 client.16188 fetch_fsmap learned FSMap version 40 2023-11-06T01:40:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.682+0000 7f8f0694bb80 1 -- 172.21.15.53:0/776606982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d3b5fcac00 con 0x55d3b6262680 2023-11-06T01:40:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.683+0000 7f8ed37fe700 1 -- 172.21.15.53:0/776606982 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eec0148e0 con 0x55d3b6262680 2023-11-06T01:40:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.683+0000 7f8ed37fe700 1 -- 172.21.15.53:0/776606982 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8eec03c020 con 0x55d3b6262680 2023-11-06T01:40:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.683+0000 7f8f0694bb80 10 client.16188 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:40:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.683+0000 7f8f0694bb80 10 client.16188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.683+0000 7f8f0694bb80 20 client.16188 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.683+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d3b62cabb0 0x55d3b62ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.683+0000 7f8f0694bb80 4 client.16188 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:40:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.683+0000 7f8f0694bb80 1 -- 172.21.15.53:0/776606982 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d3b61acce0 con 0x55d3b62cabb0 2023-11-06T01:40:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.683+0000 7f8eeb7fe700 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d3b62cabb0 0x55d3b62ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.684+0000 7f8ed37fe700 10 client.16188 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.684+0000 7f8eeb7fe700 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d3b62cabb0 0x55d3b62ccfa0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b4484c6d1c24f46 server_cookie=b9f927b44159a3a1 in_seq=0 out_seq=0 2023-11-06T01:40:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.694+0000 7f8ed37fe700 1 -- 172.21.15.53:0/776606982 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d3b61acce0 con 0x55d3b62cabb0 2023-11-06T01:40:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.694+0000 7f8ed37fe700 10 client.16188 handle_command_reply: tid=0 2023-11-06T01:40:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.694+0000 7f8f0694bb80 1 client.16188 shutdown 2023-11-06T01:40:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.697+0000 7f8f0694bb80 1 -- 172.21.15.53:0/776606982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 msgr2=0x55d3b62c3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.697+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b62c3e30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00aac0 tx=0x7f8ee4009b10 comp rx=0 tx=0).stop 2023-11-06T01:40:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.697+0000 7f8f0694bb80 1 -- 172.21.15.53:0/776606982 shutdown_connections 2023-11-06T01:40:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.697+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3b6263ed0 0x55d3b62c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.697+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d3b62cabb0 0x55d3b62ccfa0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.697+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d3b6262680 0x55d3b62c3e30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.697+0000 7f8f0694bb80 1 --2- 172.21.15.53:0/776606982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6262fe0 0x55d3b62c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.697+0000 7f8f0694bb80 1 -- 172.21.15.53:0/776606982 >> 172.21.15.53:0/776606982 conn(0x55d3b61c2070 msgr2=0x55d3b62670a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.697+0000 7f8ed37fe700 0 client.16188 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.698+0000 7f8f0694bb80 1 -- 172.21.15.53:0/776606982 shutdown_connections 2023-11-06T01:40:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.698+0000 7f8f0694bb80 1 -- 172.21.15.53:0/776606982 wait complete. 2023-11-06T01:40:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.698+0000 7f8f0694bb80 20 client.16188 trim_cache size 0 max 16384 2023-11-06T01:40:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:40:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:40:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:40:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:40:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.699+0000 7f8efa7bb700 1 -- 172.21.15.53:0/3086299661 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8ee00a05f0 msgr2=0x7f8ee00a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.699+0000 7f8efa7bb700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8ee00a05f0 0x7f8ee00a2ab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8edc003af0 tx=0x7f8edc0d0910 comp rx=0 tx=0).stop 2023-11-06T01:40:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.699+0000 7f8efa7bb700 1 -- 172.21.15.53:0/3086299661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef4107de0 msgr2=0x7f8ef41ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.699+0000 7f8efa7bb700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef4107de0 0x7f8ef41ae2a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009c70 tx=0x7f8edc004030 comp rx=0 tx=0).stop 2023-11-06T01:40:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.699+0000 7f8efa7bb700 1 -- 172.21.15.53:0/3086299661 shutdown_connections 2023-11-06T01:40:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.699+0000 7f8efa7bb700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8ee00a05f0 0x7f8ee00a2ab0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.699+0000 7f8efa7bb700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef410b7d0 0x7f8ef41af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.699+0000 7f8efa7bb700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef4107de0 0x7f8ef41ae2a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.699+0000 7f8efa7bb700 1 --2- 172.21.15.53:0/3086299661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef4108ea0 0x7f8ef41ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.699+0000 7f8efa7bb700 1 -- 172.21.15.53:0/3086299661 >> 172.21.15.53:0/3086299661 conn(0x7f8ef406dbc0 msgr2=0x7f8ef410a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.701+0000 7f8efa7bb700 1 -- 172.21.15.53:0/3086299661 shutdown_connections 2023-11-06T01:40:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:10.701+0000 7f8efa7bb700 1 -- 172.21.15.53:0/3086299661 wait complete. 2023-11-06T01:40:11.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:11 smithi053 ceph-mon[99472]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 1.78k op/s 2023-11-06T01:40:11.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:11 smithi062 ceph-mon[103924]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 1.78k op/s 2023-11-06T01:40:11.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:11 smithi163 ceph-mon[104416]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 1.78k op/s 2023-11-06T01:40:12.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 355.0 s, 2324.6 tps, lat 43.269 ms stddev 54.471 2023-11-06T01:40:13.599 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:13 smithi053 ceph-mon[99472]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.04k op/s 2023-11-06T01:40:13.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:13 smithi062 ceph-mon[103924]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.04k op/s 2023-11-06T01:40:13.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:13 smithi163 ceph-mon[104416]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.04k op/s 2023-11-06T01:40:15.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:15 smithi053 ceph-mon[99472]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.89k op/s 2023-11-06T01:40:15.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:15 smithi062 ceph-mon[103924]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.89k op/s 2023-11-06T01:40:15.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:15 smithi163 ceph-mon[104416]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.89k op/s 2023-11-06T01:40:17.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 360.0 s, 2131.4 tps, lat 46.934 ms stddev 56.509 2023-11-06T01:40:17.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:17 smithi053 ceph-mon[99472]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 1.84k op/s 2023-11-06T01:40:17.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:17 smithi062 ceph-mon[103924]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 1.84k op/s 2023-11-06T01:40:17.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:17 smithi163 ceph-mon[104416]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 1.84k op/s 2023-11-06T01:40:19.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:19 smithi053 ceph-mon[99472]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 MiB/s wr, 1.87k op/s 2023-11-06T01:40:19.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:19 smithi062 ceph-mon[103924]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 MiB/s wr, 1.87k op/s 2023-11-06T01:40:19.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:19 smithi163 ceph-mon[104416]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 MiB/s wr, 1.87k op/s 2023-11-06T01:40:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:21 smithi053 ceph-mon[99472]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 MiB/s wr, 1.12k op/s 2023-11-06T01:40:21.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:21 smithi062 ceph-mon[103924]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 MiB/s wr, 1.12k op/s 2023-11-06T01:40:21.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:21 smithi163 ceph-mon[104416]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 MiB/s wr, 1.12k op/s 2023-11-06T01:40:22.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 365.0 s, 2428.2 tps, lat 40.910 ms stddev 46.461 2023-11-06T01:40:23.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:23 smithi053 ceph-mon[99472]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 908 op/s 2023-11-06T01:40:23.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:23 smithi062 ceph-mon[103924]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 908 op/s 2023-11-06T01:40:23.744 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:23 smithi163 ceph-mon[104416]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 908 op/s 2023-11-06T01:40:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.851+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/2625512484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf4101f70 msgr2=0x7f7bf4104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.851+0000 7f7bfbe3e700 1 --2- 172.21.15.53:0/2625512484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf4101f70 0x7f7bf4104400 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009990 tx=0x7f7be4009ca0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.851+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/2625512484 shutdown_connections 2023-11-06T01:40:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.851+0000 7f7bfbe3e700 1 --2- 172.21.15.53:0/2625512484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf40ff600 0x7f7bf4101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.851+0000 7f7bfbe3e700 1 --2- 172.21.15.53:0/2625512484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf4104940 0x7f7bf4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.851+0000 7f7bfbe3e700 1 --2- 172.21.15.53:0/2625512484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf4101f70 0x7f7bf4104400 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.851+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/2625512484 >> 172.21.15.53:0/2625512484 conn(0x7f7bf40fb1e0 msgr2=0x7f7bf40fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.852+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/2625512484 shutdown_connections 2023-11-06T01:40:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.852+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/2625512484 wait complete. 2023-11-06T01:40:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bfbe3e700 1 Processor -- start 2023-11-06T01:40:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bfbe3e700 1 -- start start 2023-11-06T01:40:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bfbe3e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf40ff600 0x7f7bf419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bfbe3e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf4101f70 0x7f7bf419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bfbe3e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf4104940 0x7f7bf419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bfbe3e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf419ebf0 con 0x7f7bf4104940 2023-11-06T01:40:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bfbe3e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf419ed60 con 0x7f7bf4101f70 2023-11-06T01:40:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bfbe3e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf419ef20 con 0x7f7bf40ff600 2023-11-06T01:40:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bf93d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf4101f70 0x7f7bf419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bf9bda700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf40ff600 0x7f7bf419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bfa3db700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf4104940 0x7f7bf419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bf93d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf4101f70 0x7f7bf419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53160/0 (socket says 172.21.15.53:53160) 2023-11-06T01:40:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bf93d9700 1 -- 172.21.15.53:0/1812151499 learned_addr learned my addr 172.21.15.53:0/1812151499 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.853+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf40ff600 0x7f7bf419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55960/0 (socket says 172.21.15.53:55960) 2023-11-06T01:40:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7bf93d9700 1 -- 172.21.15.53:0/1812151499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf40ff600 msgr2=0x7f7bf419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf40ff600 0x7f7bf419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7bf93d9700 1 -- 172.21.15.53:0/1812151499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf4104940 msgr2=0x7f7bf419e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf4104940 0x7f7bf419e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7bf93d9700 1 -- 172.21.15.53:0/1812151499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40095f0 con 0x7f7bf4101f70 2023-11-06T01:40:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf40ff600 0x7f7bf419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf4101f70 0x7f7bf419d710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7be40059d0 tx=0x7f7be40047a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7beaffd700 1 -- 172.21.15.53:0/1812151499 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be401d070 con 0x7f7bf4101f70 2023-11-06T01:40:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/1812151499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf419f1a0 con 0x7f7bf4101f70 2023-11-06T01:40:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7beaffd700 1 -- 172.21.15.53:0/1812151499 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7be400fb90 con 0x7f7bf4101f70 2023-11-06T01:40:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7beaffd700 1 -- 172.21.15.53:0/1812151499 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4017650 con 0x7f7bf4101f70 2023-11-06T01:40:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.854+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/1812151499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf41a59c0 con 0x7f7bf4101f70 2023-11-06T01:40:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.857+0000 7f7beaffd700 1 -- 172.21.15.53:0/1812151499 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f7be400fd00 con 0x7f7bf4101f70 2023-11-06T01:40:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.857+0000 7f7beaffd700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7be00a0520 0x7f7be00a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.858+0000 7f7beaffd700 1 -- 172.21.15.53:0/1812151499 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7be40cd000 con 0x7f7bf4101f70 2023-11-06T01:40:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.858+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7be00a0520 0x7f7be00a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.858+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7be00a0520 0x7f7be00a29e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009480 tx=0x7f7bec00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.859+0000 7f7c07fceb80 1 Processor -- start 2023-11-06T01:40:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.859+0000 7f7c07fceb80 1 -- start start 2023-11-06T01:40:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.859+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417680 0x562378417aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.859+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 0x5623784188f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.859+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378418ed0 0x562378419350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.859+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56237836f9b0 con 0x562378417680 2023-11-06T01:40:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.859+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56237836d370 con 0x562378417fe0 2023-11-06T01:40:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.859+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562378368c60 con 0x562378418ed0 2023-11-06T01:40:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bf9bda700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417680 0x562378417aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bf9bda700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417680 0x562378417aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41728/0 (socket says 172.21.15.53:41728) 2023-11-06T01:40:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bf9bda700 1 -- 172.21.15.53:0/1918639681 learned_addr learned my addr 172.21.15.53:0/1918639681 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bfa3db700 1 --2- 172.21.15.53:0/1918639681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378418ed0 0x562378419350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/1918639681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 0x5623784188f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bf9bda700 1 -- 172.21.15.53:0/1918639681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378418ed0 msgr2=0x562378419350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/1918639681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378418ed0 0x562378419350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bf9bda700 1 -- 172.21.15.53:0/1918639681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 msgr2=0x5623784188f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/1918639681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 0x5623784188f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bf9bda700 1 -- 172.21.15.53:0/1918639681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562378196d80 con 0x562378417680 2023-11-06T01:40:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bfa3db700 1 --2- 172.21.15.53:0/1918639681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378418ed0 0x562378419350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/1918639681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417680 0x562378417aa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00e960 tx=0x7f7bec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a4c1494488e610 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bdffff700 1 -- 172.21.15.53:0/1918639681 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec025070 con 0x562378417680 2023-11-06T01:40:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bdffff700 1 -- 172.21.15.53:0/1918639681 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bec00f040 con 0x562378417680 2023-11-06T01:40:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.860+0000 7f7bdffff700 1 -- 172.21.15.53:0/1918639681 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec021a80 con 0x562378417680 2023-11-06T01:40:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.861+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1918639681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417680 msgr2=0x562378417aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.861+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/1918639681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417680 0x562378417aa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00e960 tx=0x7f7bec014000 comp rx=0 tx=0).stop 2023-11-06T01:40:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.861+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1918639681 shutdown_connections 2023-11-06T01:40:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.861+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/1918639681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378418ed0 0x562378419350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.861+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/1918639681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417680 0x562378417aa0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.861+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/1918639681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 0x5623784188f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.861+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1918639681 >> 172.21.15.53:0/1918639681 conn(0x562378377070 msgr2=0x562378414040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.862+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1918639681 shutdown_connections 2023-11-06T01:40:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.862+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1918639681 wait complete. 2023-11-06T01:40:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.862+0000 7f7c07fceb80 1 -- start start 2023-11-06T01:40:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.862+0000 7f7c07fceb80 1 Processor -- start 2023-11-06T01:40:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.863+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 0x562378421460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.863+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 0x5623784219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.863+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 0x5623784289e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.863+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623783054b0 con 0x562378418ed0 2023-11-06T01:40:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.863+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562378305150 con 0x562378417fe0 2023-11-06T01:40:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.863+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623781ec8c0 con 0x562378417680 2023-11-06T01:40:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.863+0000 7f7c07fceb80 10 client.0 _ll_register_callbacks cb 0x562378252a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bf9bda700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 0x562378421460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bfa3db700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 0x5623784289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bf93d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 0x5623784219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bf93d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 0x5623784219a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53176/0 (socket says 172.21.15.53:53176) 2023-11-06T01:40:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bfa3db700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 0x5623784289e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41736/0 (socket says 172.21.15.53:41736) 2023-11-06T01:40:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bf93d9700 1 -- 172.21.15.53:0/3827089774 learned_addr learned my addr 172.21.15.53:0/3827089774 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bf93d9700 1 -- 172.21.15.53:0/3827089774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 msgr2=0x562378421460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 0x562378421460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bf93d9700 1 -- 172.21.15.53:0/3827089774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 msgr2=0x5623784289e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 0x5623784289e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bf93d9700 1 -- 172.21.15.53:0/3827089774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf00095d0 con 0x562378417fe0 2023-11-06T01:40:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bfa3db700 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 0x5623784289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 0x562378421460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 0x5623784219a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7be400ffb0 tx=0x7f7be4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72f0b08063e684ca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.864+0000 7f7bdd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:40:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.865+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/3827089774 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be40a26c0 con 0x562378417fe0 2023-11-06T01:40:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.865+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/3827089774 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7be409a970 con 0x562378417fe0 2023-11-06T01:40:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.865+0000 7f7c07fceb80 1 -- 172.21.15.53:0/3827089774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562378393cb0 con 0x562378417fe0 2023-11-06T01:40:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.865+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/3827089774 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7be409e7a0 con 0x562378417fe0 2023-11-06T01:40:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.865+0000 7f7c07fceb80 10 client.25766 fetch_fsmap learned FSMap version 40 2023-11-06T01:40:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.865+0000 7f7c07fceb80 1 -- 172.21.15.53:0/3827089774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f7bec00e640 con 0x562378417fe0 2023-11-06T01:40:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.865+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/3827089774 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562378393cb0 con 0x562378417fe0 2023-11-06T01:40:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.866+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/3827089774 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7be40998d0 con 0x562378417fe0 2023-11-06T01:40:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.866+0000 7f7c07fceb80 10 client.25766 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:40:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.866+0000 7f7c07fceb80 10 client.25766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.866+0000 7f7c07fceb80 20 client.25766 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.866+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56237842cef0 0x56237842f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.866+0000 7f7c07fceb80 4 client.25766 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:40:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.866+0000 7f7c07fceb80 1 -- 172.21.15.53:0/3827089774 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623783e6db0 con 0x56237842cef0 2023-11-06T01:40:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.866+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56237842cef0 0x56237842f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.866+0000 7f7bdd7fa700 10 client.25766 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.866+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56237842cef0 0x56237842f310 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adba2f8e7e2a6a15 server_cookie=ac4bdc839fc7c66f in_seq=0 out_seq=0 2023-11-06T01:40:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/3827089774 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5623783e6db0 con 0x56237842cef0 2023-11-06T01:40:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7bdd7fa700 10 client.25766 handle_command_reply: tid=0 2023-11-06T01:40:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7c07fceb80 1 client.25766 shutdown 2023-11-06T01:40:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7c07fceb80 1 -- 172.21.15.53:0/3827089774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 msgr2=0x5623784219a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 0x5623784219a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7be400ffb0 tx=0x7f7be4009ca0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7c07fceb80 1 -- 172.21.15.53:0/3827089774 shutdown_connections 2023-11-06T01:40:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56237842cef0 0x56237842f310 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 0x5623784289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7bdd7fa700 0 client.25766 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 0x562378421460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/3827089774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417fe0 0x5623784219a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7c07fceb80 1 -- 172.21.15.53:0/3827089774 >> 172.21.15.53:0/3827089774 conn(0x562378377070 msgr2=0x5623784123c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7c07fceb80 1 -- 172.21.15.53:0/3827089774 shutdown_connections 2023-11-06T01:40:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.867+0000 7f7c07fceb80 1 -- 172.21.15.53:0/3827089774 wait complete. 2023-11-06T01:40:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.868+0000 7f7c07fceb80 20 client.25766 trim_cache size 0 max 16384 2023-11-06T01:40:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.886+0000 7f7c07fceb80 1 Processor -- start 2023-11-06T01:40:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.886+0000 7f7c07fceb80 1 -- start start 2023-11-06T01:40:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.886+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 0x562378421460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.886+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417fe0 0x5623784289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.886+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378418ed0 0x5623784219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562378417280 con 0x562378417fe0 2023-11-06T01:40:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56237840f710 con 0x562378418ed0 2023-11-06T01:40:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56237840a7a0 con 0x562378417680 2023-11-06T01:40:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bf93d9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417fe0 0x5623784289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bf93d9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417fe0 0x5623784289a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41744/0 (socket says 172.21.15.53:41744) 2023-11-06T01:40:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bf9bda700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 0x562378421460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bf93d9700 1 -- 172.21.15.53:0/470202206 learned_addr learned my addr 172.21.15.53:0/470202206 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bfa3db700 1 --2- 172.21.15.53:0/470202206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378418ed0 0x5623784219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bf93d9700 1 -- 172.21.15.53:0/470202206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 msgr2=0x562378421460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/470202206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 0x562378421460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bf93d9700 1 -- 172.21.15.53:0/470202206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378418ed0 msgr2=0x5623784219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/470202206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378418ed0 0x5623784219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bf93d9700 1 -- 172.21.15.53:0/470202206 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56237817fc00 con 0x562378417fe0 2023-11-06T01:40:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/470202206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 0x562378421460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/470202206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417fe0 0x5623784289a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7be4096e70 tx=0x7f7be409af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363a1f25eabe2de8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.887+0000 7f7be8ff9700 1 -- 172.21.15.53:0/470202206 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be409a740 con 0x562378417fe0 2023-11-06T01:40:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.888+0000 7f7be8ff9700 1 -- 172.21.15.53:0/470202206 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7be409a970 con 0x562378417fe0 2023-11-06T01:40:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.888+0000 7f7be8ff9700 1 -- 172.21.15.53:0/470202206 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be40b8d20 con 0x562378417fe0 2023-11-06T01:40:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.888+0000 7f7c07fceb80 1 -- 172.21.15.53:0/470202206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417fe0 msgr2=0x5623784289a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.888+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/470202206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417fe0 0x5623784289a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7be4096e70 tx=0x7f7be409af90 comp rx=0 tx=0).stop 2023-11-06T01:40:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.891+0000 7f7c07fceb80 1 -- 172.21.15.53:0/470202206 shutdown_connections 2023-11-06T01:40:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.891+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/470202206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378418ed0 0x5623784219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.891+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/470202206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417680 0x562378421460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.891+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/470202206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378417fe0 0x5623784289a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.891+0000 7f7c07fceb80 1 -- 172.21.15.53:0/470202206 >> 172.21.15.53:0/470202206 conn(0x562378377070 msgr2=0x56237841f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.891+0000 7f7c07fceb80 1 -- 172.21.15.53:0/470202206 shutdown_connections 2023-11-06T01:40:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.891+0000 7f7c07fceb80 1 -- 172.21.15.53:0/470202206 wait complete. 2023-11-06T01:40:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.891+0000 7f7c07fceb80 1 -- start start 2023-11-06T01:40:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7c07fceb80 1 Processor -- start 2023-11-06T01:40:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417680 0x562378426820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417fe0 0x562378426d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7c07fceb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 0x562378427d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623784282b0 con 0x562378418ed0 2023-11-06T01:40:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562378428420 con 0x562378417680 2023-11-06T01:40:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7c07fceb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623784285e0 con 0x562378417fe0 2023-11-06T01:40:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7c07fceb80 10 client.0 _ll_register_callbacks cb 0x7f7be409a5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7bf93d9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417fe0 0x562378426d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7bf93d9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417fe0 0x562378426d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:55990/0 (socket says 172.21.15.53:55990) 2023-11-06T01:40:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7bf93d9700 1 -- 172.21.15.53:0/1301031630 learned_addr learned my addr 172.21.15.53:0/1301031630 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7bfa3db700 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 0x562378427d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.892+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417680 0x562378426820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bf93d9700 1 -- 172.21.15.53:0/1301031630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417680 msgr2=0x562378426820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417680 0x562378426820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bf93d9700 1 -- 172.21.15.53:0/1301031630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 msgr2=0x562378427d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 0x562378427d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bf93d9700 1 -- 172.21.15.53:0/1301031630 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562378196d80 con 0x562378417fe0 2023-11-06T01:40:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417680 0x562378426820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bf93d9700 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417fe0 0x562378426d60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7be400b680 tx=0x7f7be4096de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35765cd685b953a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bdd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:40:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/1301031630 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be409e7a0 con 0x562378417fe0 2023-11-06T01:40:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/1301031630 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7be40a2950 con 0x562378417fe0 2023-11-06T01:40:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1301031630 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bec00f040 con 0x562378417fe0 2023-11-06T01:40:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/1301031630 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7be40a2ba0 con 0x562378417fe0 2023-11-06T01:40:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7c07fceb80 10 client.25831 fetch_fsmap learned FSMap version 40 2023-11-06T01:40:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1301031630 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f7bec00e640 con 0x562378417fe0 2023-11-06T01:40:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.893+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/1301031630 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bec00f040 con 0x562378417fe0 2023-11-06T01:40:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.894+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/1301031630 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7be407f030 con 0x562378417fe0 2023-11-06T01:40:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.894+0000 7f7c07fceb80 10 client.25831 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:40:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.894+0000 7f7c07fceb80 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.894+0000 7f7c07fceb80 20 client.25831 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.894+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56237847f430 0x562378481850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.894+0000 7f7c07fceb80 4 client.25831 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:40:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.894+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1301031630 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562378361ce0 con 0x56237847f430 2023-11-06T01:40:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.894+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56237847f430 0x562378481850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.895+0000 7f7bf9bda700 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56237847f430 0x562378481850 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96822c499e4ccfac server_cookie=3b859b5f9321ca1b in_seq=0 out_seq=0 2023-11-06T01:40:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.896+0000 7f7bdd7fa700 10 client.25831 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7bdd7fa700 1 -- 172.21.15.53:0/1301031630 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562378361ce0 con 0x56237847f430 2023-11-06T01:40:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7bdd7fa700 10 client.25831 handle_command_reply: tid=0 2023-11-06T01:40:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7c07fceb80 1 client.25831 shutdown 2023-11-06T01:40:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1301031630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417fe0 msgr2=0x562378426d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417fe0 0x562378426d60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7be400b680 tx=0x7f7be4096de0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1301031630 shutdown_connections 2023-11-06T01:40:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562378417680 0x562378426820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56237847f430 0x562378481850 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562378418ed0 0x562378427d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7c07fceb80 1 --2- 172.21.15.53:0/1301031630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562378417fe0 0x562378426d60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7bdd7fa700 0 client.25831 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1301031630 >> 172.21.15.53:0/1301031630 conn(0x562378377070 msgr2=0x562378421400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1301031630 shutdown_connections 2023-11-06T01:40:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.907+0000 7f7c07fceb80 1 -- 172.21.15.53:0/1301031630 wait complete. 2023-11-06T01:40:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7c07fceb80 20 client.25831 trim_cache size 0 max 16384 2023-11-06T01:40:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:40:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:40:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:40:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:40:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/1812151499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7be00a0520 msgr2=0x7f7be00a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7be00a0520 0x7f7be00a29e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009480 tx=0x7f7bec00b040 comp rx=0 tx=0).stop 2023-11-06T01:40:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/1812151499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf4101f70 msgr2=0x7f7bf419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf4101f70 0x7f7bf419d710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7be40059d0 tx=0x7f7be40047a0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/1812151499 shutdown_connections 2023-11-06T01:40:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7be00a0520 0x7f7be00a29e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf4104940 0x7f7bf419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bf40ff600 0x7f7bf419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 --2- 172.21.15.53:0/1812151499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf4101f70 0x7f7bf419d710 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/1812151499 >> 172.21.15.53:0/1812151499 conn(0x7f7bf40fb1e0 msgr2=0x7f7bf4100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/1812151499 shutdown_connections 2023-11-06T01:40:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:23.908+0000 7f7bfbe3e700 1 -- 172.21.15.53:0/1812151499 wait complete. 2023-11-06T01:40:25.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:25 smithi053 ceph-mon[99472]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 672 op/s 2023-11-06T01:40:25.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:25 smithi062 ceph-mon[103924]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 672 op/s 2023-11-06T01:40:25.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:25 smithi163 ceph-mon[104416]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 672 op/s 2023-11-06T01:40:27.185 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 370.0 s, 2238.8 tps, lat 44.737 ms stddev 52.739 2023-11-06T01:40:27.603 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:27 smithi053 ceph-mon[99472]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 677 op/s 2023-11-06T01:40:27.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:27 smithi062 ceph-mon[103924]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 677 op/s 2023-11-06T01:40:27.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:27 smithi163 ceph-mon[104416]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 677 op/s 2023-11-06T01:40:29.598 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:29 smithi053 ceph-mon[99472]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 757 op/s 2023-11-06T01:40:29.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:29 smithi062 ceph-mon[103924]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 757 op/s 2023-11-06T01:40:29.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:29 smithi163 ceph-mon[104416]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 757 op/s 2023-11-06T01:40:31.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:31 smithi053 ceph-mon[99472]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 707 op/s 2023-11-06T01:40:31.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:31 smithi062 ceph-mon[103924]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 707 op/s 2023-11-06T01:40:31.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:31 smithi163 ceph-mon[104416]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 707 op/s 2023-11-06T01:40:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 375.0 s, 2183.8 tps, lat 46.082 ms stddev 53.868 2023-11-06T01:40:33.602 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:33 smithi053 ceph-mon[99472]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 820 op/s 2023-11-06T01:40:33.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:33 smithi062 ceph-mon[103924]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 820 op/s 2023-11-06T01:40:33.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:33 smithi163 ceph-mon[104416]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 820 op/s 2023-11-06T01:40:35.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:35 smithi062 ceph-mon[103924]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 769 op/s 2023-11-06T01:40:35.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:35 smithi163 ceph-mon[104416]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 769 op/s 2023-11-06T01:40:35.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:35 smithi053 ceph-mon[99472]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 769 op/s 2023-11-06T01:40:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.059+0000 7f206258c700 1 -- 172.21.15.53:0/2363107456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f205c1041e0 msgr2=0x7f205c104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.059+0000 7f206258c700 1 --2- 172.21.15.53:0/2363107456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f205c1041e0 0x7f205c104640 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f20500099c0 tx=0x7f2050009cd0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.059+0000 7f206258c700 1 -- 172.21.15.53:0/2363107456 shutdown_connections 2023-11-06T01:40:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.059+0000 7f206258c700 1 --2- 172.21.15.53:0/2363107456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c104b80 0x7f205c10b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.059+0000 7f206258c700 1 --2- 172.21.15.53:0/2363107456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f205c102ff0 0x7f205c103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.059+0000 7f206258c700 1 --2- 172.21.15.53:0/2363107456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f205c1041e0 0x7f205c104640 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.059+0000 7f206258c700 1 -- 172.21.15.53:0/2363107456 >> 172.21.15.53:0/2363107456 conn(0x7f205c0fe540 msgr2=0x7f205c1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.059+0000 7f206258c700 1 -- 172.21.15.53:0/2363107456 shutdown_connections 2023-11-06T01:40:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.059+0000 7f206258c700 1 -- 172.21.15.53:0/2363107456 wait complete. 2023-11-06T01:40:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f206258c700 1 Processor -- start 2023-11-06T01:40:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f206258c700 1 -- start start 2023-11-06T01:40:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f206258c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f205c102ff0 0x7f205c198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f206258c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f205c1041e0 0x7f205c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f206258c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c104b80 0x7f205c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f206258c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c19a7e0 con 0x7f205c102ff0 2023-11-06T01:40:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f206258c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c19a950 con 0x7f205c104b80 2023-11-06T01:40:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f206258c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c19ab10 con 0x7f205c1041e0 2023-11-06T01:40:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f205bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f205c102ff0 0x7f205c198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f205bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f205c102ff0 0x7f205c198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41762/0 (socket says 172.21.15.53:41762) 2023-11-06T01:40:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f205bfff700 1 -- 172.21.15.53:0/2873946181 learned_addr learned my addr 172.21.15.53:0/2873946181 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f2060b29700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c104b80 0x7f205c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f205b7fe700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f205c1041e0 0x7f205c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f205bfff700 1 -- 172.21.15.53:0/2873946181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f205c1041e0 msgr2=0x7f205c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f205bfff700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f205c1041e0 0x7f205c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f205bfff700 1 -- 172.21.15.53:0/2873946181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c104b80 msgr2=0x7f205c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f205bfff700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c104b80 0x7f205c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f205bfff700 1 -- 172.21.15.53:0/2873946181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20500095f0 con 0x7f205c102ff0 2023-11-06T01:40:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.060+0000 7f2060b29700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c104b80 0x7f205c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.061+0000 7f205bfff700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f205c102ff0 0x7f205c198df0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f204c00e970 tx=0x7f204c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.061+0000 7f20597fa700 1 -- 172.21.15.53:0/2873946181 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c009790 con 0x7f205c102ff0 2023-11-06T01:40:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.061+0000 7f206258c700 1 -- 172.21.15.53:0/2873946181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c19adf0 con 0x7f205c102ff0 2023-11-06T01:40:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.061+0000 7f20597fa700 1 -- 172.21.15.53:0/2873946181 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f204c00c7e0 con 0x7f205c102ff0 2023-11-06T01:40:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.061+0000 7f20597fa700 1 -- 172.21.15.53:0/2873946181 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c003c70 con 0x7f205c102ff0 2023-11-06T01:40:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.061+0000 7f206258c700 1 -- 172.21.15.53:0/2873946181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c1a1580 con 0x7f205c102ff0 2023-11-06T01:40:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.065+0000 7f20597fa700 1 -- 172.21.15.53:0/2873946181 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f204c00cc50 con 0x7f205c102ff0 2023-11-06T01:40:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.065+0000 7f20597fa700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f20440a0520 0x7f20440a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.066+0000 7f20597fa700 1 -- 172.21.15.53:0/2873946181 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f204c014070 con 0x7f205c102ff0 2023-11-06T01:40:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.066+0000 7f205b7fe700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f20440a0520 0x7f20440a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.066+0000 7f206e71cb80 1 Processor -- start 2023-11-06T01:40:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.066+0000 7f205b7fe700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f20440a0520 0x7f20440a29e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f205000b580 tx=0x7f2050005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.066+0000 7f206e71cb80 1 -- start start 2023-11-06T01:40:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b4b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b4b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b4b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f206e71cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b40d9b0 con 0x55996b4b5fe0 2023-11-06T01:40:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f206e71cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b40b370 con 0x55996b4b6ed0 2023-11-06T01:40:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f206e71cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b406c60 con 0x55996b4b5680 2023-11-06T01:40:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f205bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b4b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f205bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b4b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56010/0 (socket says 172.21.15.53:56010) 2023-11-06T01:40:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f205bfff700 1 -- 172.21.15.53:0/165311446 learned_addr learned my addr 172.21.15.53:0/165311446 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f205b7fe700 1 --2- 172.21.15.53:0/165311446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b4b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f2060b29700 1 --2- 172.21.15.53:0/165311446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b4b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f2060b29700 1 -- 172.21.15.53:0/165311446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 msgr2=0x55996b4b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.067+0000 7f2060b29700 1 --2- 172.21.15.53:0/165311446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b4b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.068+0000 7f2060b29700 1 -- 172.21.15.53:0/165311446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 msgr2=0x55996b4b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.068+0000 7f2060b29700 1 --2- 172.21.15.53:0/165311446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b4b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.068+0000 7f2060b29700 1 -- 172.21.15.53:0/165311446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55996b234d80 con 0x55996b4b6ed0 2023-11-06T01:40:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.068+0000 7f2060b29700 1 --2- 172.21.15.53:0/165311446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b4b7350 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2054003db0 tx=0x7f2054009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75c1c7e5334ff9e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.069+0000 7f20427fc700 1 -- 172.21.15.53:0/165311446 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205401c070 con 0x55996b4b6ed0 2023-11-06T01:40:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.069+0000 7f20427fc700 1 -- 172.21.15.53:0/165311446 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f205400f040 con 0x55996b4b6ed0 2023-11-06T01:40:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.069+0000 7f20427fc700 1 -- 172.21.15.53:0/165311446 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054007500 con 0x55996b4b6ed0 2023-11-06T01:40:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.070+0000 7f206e71cb80 1 -- 172.21.15.53:0/165311446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 msgr2=0x55996b4b7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.070+0000 7f206e71cb80 1 --2- 172.21.15.53:0/165311446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b4b7350 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2054003db0 tx=0x7f2054009920 comp rx=0 tx=0).stop 2023-11-06T01:40:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.070+0000 7f206e71cb80 1 -- 172.21.15.53:0/165311446 shutdown_connections 2023-11-06T01:40:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.070+0000 7f206e71cb80 1 --2- 172.21.15.53:0/165311446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b4b7350 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.070+0000 7f206e71cb80 1 --2- 172.21.15.53:0/165311446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b4b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.070+0000 7f206e71cb80 1 --2- 172.21.15.53:0/165311446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b4b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.070+0000 7f206e71cb80 1 -- 172.21.15.53:0/165311446 >> 172.21.15.53:0/165311446 conn(0x55996b415070 msgr2=0x55996b4b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.070+0000 7f206e71cb80 1 -- 172.21.15.53:0/165311446 shutdown_connections 2023-11-06T01:40:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.070+0000 7f206e71cb80 1 -- 172.21.15.53:0/165311446 wait complete. 2023-11-06T01:40:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.070+0000 7f206e71cb80 1 -- start start 2023-11-06T01:40:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f206e71cb80 1 Processor -- start 2023-11-06T01:40:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b4bf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b4bfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b4c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f206e71cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b3a34b0 con 0x55996b4b5fe0 2023-11-06T01:40:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f206e71cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b3a3150 con 0x55996b4b6ed0 2023-11-06T01:40:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f206e71cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b28a8c0 con 0x55996b4b5680 2023-11-06T01:40:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f206e71cb80 10 client.0 _ll_register_callbacks cb 0x55996b2f0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f2060b29700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b4c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f2060b29700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b4c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53216/0 (socket says 172.21.15.53:53216) 2023-11-06T01:40:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f205bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b4bf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f2060b29700 1 -- 172.21.15.53:0/4093417793 learned_addr learned my addr 172.21.15.53:0/4093417793 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f205b7fe700 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b4bfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f2060b29700 1 -- 172.21.15.53:0/4093417793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 msgr2=0x55996b4bf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f2060b29700 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b4bf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f2060b29700 1 -- 172.21.15.53:0/4093417793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 msgr2=0x55996b4bfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f2060b29700 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b4bfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f2060b29700 1 -- 172.21.15.53:0/4093417793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2054003a60 con 0x55996b4b6ed0 2023-11-06T01:40:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f205b7fe700 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b4bfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:40:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.071+0000 7f205bfff700 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b4bf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.072+0000 7f2060b29700 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b4c6aa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2054003d80 tx=0x7f2054004a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c919ccd9554ba07e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.072+0000 7f203b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:40:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.072+0000 7f203b7fe700 1 -- 172.21.15.53:0/4093417793 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205401c040 con 0x55996b4b6ed0 2023-11-06T01:40:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.072+0000 7f203b7fe700 1 -- 172.21.15.53:0/4093417793 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2054007700 con 0x55996b4b6ed0 2023-11-06T01:40:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.072+0000 7f206e71cb80 1 -- 172.21.15.53:0/4093417793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55996b431cb0 con 0x55996b4b6ed0 2023-11-06T01:40:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.072+0000 7f203b7fe700 1 -- 172.21.15.53:0/4093417793 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20540078c0 con 0x55996b4b6ed0 2023-11-06T01:40:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.072+0000 7f206e71cb80 10 client.25784 fetch_fsmap learned FSMap version 40 2023-11-06T01:40:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.072+0000 7f206e71cb80 1 -- 172.21.15.53:0/4093417793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55996b21dc00 con 0x55996b4b6ed0 2023-11-06T01:40:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.072+0000 7f203b7fe700 1 -- 172.21.15.53:0/4093417793 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55996b431cb0 con 0x55996b4b6ed0 2023-11-06T01:40:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.073+0000 7f203b7fe700 1 -- 172.21.15.53:0/4093417793 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f2054031020 con 0x55996b4b6ed0 2023-11-06T01:40:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.073+0000 7f206e71cb80 10 client.25784 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:40:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.073+0000 7f206e71cb80 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.073+0000 7f206e71cb80 20 client.25784 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.073+0000 7f206e71cb80 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55996b4cb150 0x55996b4cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.073+0000 7f206e71cb80 4 client.25784 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:40:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.073+0000 7f206e71cb80 1 -- 172.21.15.53:0/4093417793 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55996b484db0 con 0x55996b4cb150 2023-11-06T01:40:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.073+0000 7f205bfff700 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55996b4cb150 0x55996b4cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.073+0000 7f203b7fe700 10 client.25784 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.073+0000 7f205bfff700 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55996b4cb150 0x55996b4cd570 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f28570e7e35d25f7 server_cookie=17f67f2610518d37 in_seq=0 out_seq=0 2023-11-06T01:40:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.074+0000 7f203b7fe700 1 -- 172.21.15.53:0/4093417793 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55996b484db0 con 0x55996b4cb150 2023-11-06T01:40:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.074+0000 7f203b7fe700 10 client.25784 handle_command_reply: tid=0 2023-11-06T01:40:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.074+0000 7f206e71cb80 1 client.25784 shutdown 2023-11-06T01:40:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.074+0000 7f206e71cb80 1 -- 172.21.15.53:0/4093417793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 msgr2=0x55996b4c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.074+0000 7f206e71cb80 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b4c6aa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2054003d80 tx=0x7f2054004a20 comp rx=0 tx=0).stop 2023-11-06T01:40:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.074+0000 7f206e71cb80 1 -- 172.21.15.53:0/4093417793 shutdown_connections 2023-11-06T01:40:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.074+0000 7f206e71cb80 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b4c6aa0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.074+0000 7f206e71cb80 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55996b4cb150 0x55996b4cd570 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.074+0000 7f206e71cb80 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b4bf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.074+0000 7f203b7fe700 0 client.25784 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.074+0000 7f206e71cb80 1 --2- 172.21.15.53:0/4093417793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b4bfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.075+0000 7f206e71cb80 1 -- 172.21.15.53:0/4093417793 >> 172.21.15.53:0/4093417793 conn(0x55996b415070 msgr2=0x55996b4b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.075+0000 7f206e71cb80 1 -- 172.21.15.53:0/4093417793 shutdown_connections 2023-11-06T01:40:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.075+0000 7f206e71cb80 1 -- 172.21.15.53:0/4093417793 wait complete. 2023-11-06T01:40:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.075+0000 7f206e71cb80 20 client.25784 trim_cache size 0 max 16384 2023-11-06T01:40:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f206e71cb80 1 Processor -- start 2023-11-06T01:40:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f206e71cb80 1 -- start start 2023-11-06T01:40:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b5680 0x55996b4c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5fe0 0x55996b4b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b6ed0 0x55996b4bd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f206e71cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b4be9d0 con 0x55996b4b6ed0 2023-11-06T01:40:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f206e71cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b4ab500 con 0x55996b4b5680 2023-11-06T01:40:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f206e71cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b4bf090 con 0x55996b4b5fe0 2023-11-06T01:40:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f205bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b5680 0x55996b4c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f205b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5fe0 0x55996b4b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f205bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b5680 0x55996b4c4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53222/0 (socket says 172.21.15.53:53222) 2023-11-06T01:40:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f205bfff700 1 -- 172.21.15.53:0/2725074448 learned_addr learned my addr 172.21.15.53:0/2725074448 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.094+0000 7f2060b29700 1 --2- 172.21.15.53:0/2725074448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b6ed0 0x55996b4bd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.095+0000 7f205bfff700 1 -- 172.21.15.53:0/2725074448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5fe0 msgr2=0x55996b4b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.095+0000 7f205bfff700 1 --2- 172.21.15.53:0/2725074448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5fe0 0x55996b4b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.095+0000 7f205bfff700 1 -- 172.21.15.53:0/2725074448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b6ed0 msgr2=0x55996b4bd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.095+0000 7f205bfff700 1 --2- 172.21.15.53:0/2725074448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b6ed0 0x55996b4bd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.095+0000 7f205bfff700 1 -- 172.21.15.53:0/2725074448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55996b25b1f0 con 0x55996b4b5680 2023-11-06T01:40:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.095+0000 7f205b7fe700 1 --2- 172.21.15.53:0/2725074448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5fe0 0x55996b4b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.095+0000 7f2060b29700 1 --2- 172.21.15.53:0/2725074448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b6ed0 0x55996b4bd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.095+0000 7f205bfff700 1 --2- 172.21.15.53:0/2725074448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b5680 0x55996b4c4a50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f204c0c99b0 tx=0x7f204c0c9510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f15a606b03804f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.095+0000 7f2042ffd700 1 -- 172.21.15.53:0/2725074448 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c098d00 con 0x55996b4b5680 2023-11-06T01:40:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.095+0000 7f2042ffd700 1 -- 172.21.15.53:0/2725074448 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f204c099aa0 con 0x55996b4b5680 2023-11-06T01:40:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.095+0000 7f2042ffd700 1 -- 172.21.15.53:0/2725074448 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c097740 con 0x55996b4b5680 2023-11-06T01:40:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.096+0000 7f206e71cb80 1 -- 172.21.15.53:0/2725074448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b5680 msgr2=0x55996b4c4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.096+0000 7f206e71cb80 1 --2- 172.21.15.53:0/2725074448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b5680 0x55996b4c4a50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f204c0c99b0 tx=0x7f204c0c9510 comp rx=0 tx=0).stop 2023-11-06T01:40:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.097+0000 7f206e71cb80 1 -- 172.21.15.53:0/2725074448 shutdown_connections 2023-11-06T01:40:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.097+0000 7f206e71cb80 1 --2- 172.21.15.53:0/2725074448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b5680 0x55996b4c4a50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.097+0000 7f206e71cb80 1 --2- 172.21.15.53:0/2725074448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b6ed0 0x55996b4bd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.097+0000 7f206e71cb80 1 --2- 172.21.15.53:0/2725074448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5fe0 0x55996b4b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.097+0000 7f206e71cb80 1 -- 172.21.15.53:0/2725074448 >> 172.21.15.53:0/2725074448 conn(0x55996b415070 msgr2=0x55996b4be140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.098+0000 7f206e71cb80 1 -- 172.21.15.53:0/2725074448 shutdown_connections 2023-11-06T01:40:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.099+0000 7f206e71cb80 1 -- 172.21.15.53:0/2725074448 wait complete. 2023-11-06T01:40:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.099+0000 7f206e71cb80 1 -- start start 2023-11-06T01:40:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.099+0000 7f206e71cb80 1 Processor -- start 2023-11-06T01:40:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b516c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b519180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f206e71cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b51b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f206e71cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b51bf70 con 0x55996b4b5fe0 2023-11-06T01:40:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f206e71cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b51c0e0 con 0x55996b4b6ed0 2023-11-06T01:40:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f206e71cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996b52d2b0 con 0x55996b4b5680 2023-11-06T01:40:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f206e71cb80 10 client.0 _ll_register_callbacks cb 0x7f205000b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f205bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b516c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f205bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b516c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56030/0 (socket says 172.21.15.53:56030) 2023-11-06T01:40:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f205bfff700 1 -- 172.21.15.53:0/3764239020 learned_addr learned my addr 172.21.15.53:0/3764239020 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f2060b29700 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b51b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.100+0000 7f205b7fe700 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b519180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f2060b29700 1 -- 172.21.15.53:0/3764239020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 msgr2=0x55996b516c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f2060b29700 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b516c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f2060b29700 1 -- 172.21.15.53:0/3764239020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 msgr2=0x55996b519180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f2060b29700 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b519180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f2060b29700 1 -- 172.21.15.53:0/3764239020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c10bd80 con 0x55996b4b6ed0 2023-11-06T01:40:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f2060b29700 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b51b830 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2054003d50 tx=0x7f20540216a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71b2c6f859806a34 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f203b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:40:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f203b7fe700 1 -- 172.21.15.53:0/3764239020 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205401c040 con 0x55996b4b6ed0 2023-11-06T01:40:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f203b7fe700 1 -- 172.21.15.53:0/3764239020 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f205400e6f0 con 0x55996b4b6ed0 2023-11-06T01:40:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f206e71cb80 1 -- 172.21.15.53:0/3764239020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f205400f040 con 0x55996b4b6ed0 2023-11-06T01:40:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f203b7fe700 1 -- 172.21.15.53:0/3764239020 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2054038070 con 0x55996b4b6ed0 2023-11-06T01:40:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f206e71cb80 10 client.25790 fetch_fsmap learned FSMap version 40 2023-11-06T01:40:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f206e71cb80 1 -- 172.21.15.53:0/3764239020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55996b21dc00 con 0x55996b4b6ed0 2023-11-06T01:40:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.101+0000 7f203b7fe700 1 -- 172.21.15.53:0/3764239020 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f205400f040 con 0x55996b4b6ed0 2023-11-06T01:40:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.102+0000 7f203b7fe700 1 -- 172.21.15.53:0/3764239020 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f2054017110 con 0x55996b4b6ed0 2023-11-06T01:40:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.102+0000 7f206e71cb80 10 client.25790 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:40:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.102+0000 7f206e71cb80 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.102+0000 7f206e71cb80 20 client.25790 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.102+0000 7f206e71cb80 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55996b52e8d0 0x55996b530cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.102+0000 7f206e71cb80 4 client.25790 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:40:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.102+0000 7f206e71cb80 1 -- 172.21.15.53:0/3764239020 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55996b3ffce0 con 0x55996b52e8d0 2023-11-06T01:40:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.102+0000 7f205bfff700 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55996b52e8d0 0x55996b530cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.102+0000 7f203b7fe700 10 client.25790 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.102+0000 7f205bfff700 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55996b52e8d0 0x55996b530cc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c85aaa70503254 server_cookie=940067c07d37e280 in_seq=0 out_seq=0 2023-11-06T01:40:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.116+0000 7f203b7fe700 1 -- 172.21.15.53:0/3764239020 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55996b3ffce0 con 0x55996b52e8d0 2023-11-06T01:40:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.116+0000 7f203b7fe700 10 client.25790 handle_command_reply: tid=0 2023-11-06T01:40:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.116+0000 7f206e71cb80 1 client.25790 shutdown 2023-11-06T01:40:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.116+0000 7f206e71cb80 1 -- 172.21.15.53:0/3764239020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 msgr2=0x55996b51b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.116+0000 7f206e71cb80 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b51b830 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2054003d50 tx=0x7f20540216a0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.116+0000 7f206e71cb80 1 -- 172.21.15.53:0/3764239020 shutdown_connections 2023-11-06T01:40:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.116+0000 7f206e71cb80 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55996b4b6ed0 0x55996b51b830 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.116+0000 7f206e71cb80 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55996b52e8d0 0x55996b530cc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.116+0000 7f206e71cb80 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55996b4b5680 0x55996b516c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.116+0000 7f203b7fe700 0 client.25790 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.116+0000 7f206e71cb80 1 --2- 172.21.15.53:0/3764239020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55996b4b5fe0 0x55996b519180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206e71cb80 1 -- 172.21.15.53:0/3764239020 >> 172.21.15.53:0/3764239020 conn(0x55996b415070 msgr2=0x55996b4ba0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206e71cb80 1 -- 172.21.15.53:0/3764239020 shutdown_connections 2023-11-06T01:40:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206e71cb80 1 -- 172.21.15.53:0/3764239020 wait complete. 2023-11-06T01:40:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206e71cb80 20 client.25790 trim_cache size 0 max 16384 2023-11-06T01:40:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:40:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:40:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:40:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:40:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206258c700 1 -- 172.21.15.53:0/2873946181 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f20440a0520 msgr2=0x7f20440a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206258c700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f20440a0520 0x7f20440a29e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f205000b580 tx=0x7f2050005fb0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206258c700 1 -- 172.21.15.53:0/2873946181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f205c102ff0 msgr2=0x7f205c198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206258c700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f205c102ff0 0x7f205c198df0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f204c00e970 tx=0x7f204c00ec80 comp rx=0 tx=0).stop 2023-11-06T01:40:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206258c700 1 -- 172.21.15.53:0/2873946181 shutdown_connections 2023-11-06T01:40:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206258c700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f20440a0520 0x7f20440a29e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206258c700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c104b80 0x7f205c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206258c700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f205c102ff0 0x7f205c198df0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206258c700 1 --2- 172.21.15.53:0/2873946181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f205c1041e0 0x7f205c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.117+0000 7f206258c700 1 -- 172.21.15.53:0/2873946181 >> 172.21.15.53:0/2873946181 conn(0x7f205c0fe540 msgr2=0x7f205c107670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.118+0000 7f206258c700 1 -- 172.21.15.53:0/2873946181 shutdown_connections 2023-11-06T01:40:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:37.118+0000 7f206258c700 1 -- 172.21.15.53:0/2873946181 wait complete. 2023-11-06T01:40:37.175 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:40:37.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 380.0 s, 1944.0 tps, lat 51.205 ms stddev 56.185 2023-11-06T01:40:37.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.361+0000 7fc141675700 1 -- 172.21.15.53:0/1426723407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc13c106a80 msgr2=0x7fc13c108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.361+0000 7fc141675700 1 --2- 172.21.15.53:0/1426723407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc13c106a80 0x7fc13c108ef0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc13400b0a0 tx=0x7fc13400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.362+0000 7fc141675700 1 -- 172.21.15.53:0/1426723407 shutdown_connections 2023-11-06T01:40:37.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.362+0000 7fc141675700 1 --2- 172.21.15.53:0/1426723407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c101820 0x7fc13c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.362+0000 7fc141675700 1 --2- 172.21.15.53:0/1426723407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc13c106a80 0x7fc13c108ef0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.362+0000 7fc141675700 1 --2- 172.21.15.53:0/1426723407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc13c104150 0x7fc13c106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.362+0000 7fc141675700 1 -- 172.21.15.53:0/1426723407 >> 172.21.15.53:0/1426723407 conn(0x7fc13c0fb250 msgr2=0x7fc13c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.365+0000 7fc141675700 1 -- 172.21.15.53:0/1426723407 shutdown_connections 2023-11-06T01:40:37.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.365+0000 7fc141675700 1 -- 172.21.15.53:0/1426723407 wait complete. 2023-11-06T01:40:37.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.365+0000 7fc141675700 1 Processor -- start 2023-11-06T01:40:37.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc141675700 1 -- start start 2023-11-06T01:40:37.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc141675700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc13c101820 0x7fc13c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc141675700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc13c104150 0x7fc13c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc141675700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c106a80 0x7fc13c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc141675700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1a08f0 con 0x7fc13c101820 2023-11-06T01:40:37.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc141675700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1a0a60 con 0x7fc13c106a80 2023-11-06T01:40:37.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc141675700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1a0c20 con 0x7fc13c104150 2023-11-06T01:40:37.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc13c101820 0x7fc13c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c106a80 0x7fc13c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc13c101820 0x7fc13c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41804/0 (socket says 172.21.15.53:41804) 2023-11-06T01:40:37.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c106a80 0x7fc13c1a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53244/0 (socket says 172.21.15.53:53244) 2023-11-06T01:40:37.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13affd700 1 -- 172.21.15.53:0/2353209561 learned_addr learned my addr 172.21.15.53:0/2353209561 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:37.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13a7fc700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc13c104150 0x7fc13c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13b7fe700 1 -- 172.21.15.53:0/2353209561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc13c104150 msgr2=0x7fc13c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc13c104150 0x7fc13c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13b7fe700 1 -- 172.21.15.53:0/2353209561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc13c101820 msgr2=0x7fc13c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc13c101820 0x7fc13c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13b7fe700 1 -- 172.21.15.53:0/2353209561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc134009d00 con 0x7fc13c106a80 2023-11-06T01:40:37.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.366+0000 7fc13affd700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc13c101820 0x7fc13c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:37.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.367+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c106a80 0x7fc13c1a0320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc13400b070 tx=0x7fc13400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.367+0000 7fc123fff700 1 -- 172.21.15.53:0/2353209561 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13400e040 con 0x7fc13c106a80 2023-11-06T01:40:37.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.367+0000 7fc123fff700 1 -- 172.21.15.53:0/2353209561 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc134004c50 con 0x7fc13c106a80 2023-11-06T01:40:37.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.367+0000 7fc141675700 1 -- 172.21.15.53:0/2353209561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c1a0ea0 con 0x7fc13c106a80 2023-11-06T01:40:37.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.367+0000 7fc123fff700 1 -- 172.21.15.53:0/2353209561 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13401dea0 con 0x7fc13c106a80 2023-11-06T01:40:37.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.367+0000 7fc141675700 1 -- 172.21.15.53:0/2353209561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c1a1260 con 0x7fc13c106a80 2023-11-06T01:40:37.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.369+0000 7fc123fff700 1 -- 172.21.15.53:0/2353209561 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fc13401d470 con 0x7fc13c106a80 2023-11-06T01:40:37.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.370+0000 7fc123fff700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc1240a0520 0x7fc1240a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.370+0000 7fc123fff700 1 -- 172.21.15.53:0/2353209561 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc1340ccdc0 con 0x7fc13c106a80 2023-11-06T01:40:37.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.370+0000 7fc13affd700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc1240a0520 0x7fc1240a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc14d805b80 1 Processor -- start 2023-11-06T01:40:37.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc13affd700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc1240a0520 0x7fc1240a29e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc13c19a280 tx=0x7fc12c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc14d805b80 1 -- start start 2023-11-06T01:40:37.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3cc6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc6a70 0x5617e3cc4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc53f0 0x5617e3cc5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc14d805b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3c1ca10 con 0x5617e3cc6a70 2023-11-06T01:40:37.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc14d805b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3c1a3d0 con 0x5617e3cc53f0 2023-11-06T01:40:37.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc14d805b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3c15cc0 con 0x5617e3cc60f0 2023-11-06T01:40:37.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc13a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc6a70 0x5617e3cc4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc13a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc6a70 0x5617e3cc4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41814/0 (socket says 172.21.15.53:41814) 2023-11-06T01:40:37.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc13a7fc700 1 -- 172.21.15.53:0/974976218 learned_addr learned my addr 172.21.15.53:0/974976218 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:37.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc13affd700 1 --2- 172.21.15.53:0/974976218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3cc6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.371+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/974976218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc53f0 0x5617e3cc5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.372+0000 7fc13a7fc700 1 -- 172.21.15.53:0/974976218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 msgr2=0x5617e3cc6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.372+0000 7fc13a7fc700 1 --2- 172.21.15.53:0/974976218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3cc6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.372+0000 7fc13a7fc700 1 -- 172.21.15.53:0/974976218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc53f0 msgr2=0x5617e3cc5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.372+0000 7fc13a7fc700 1 --2- 172.21.15.53:0/974976218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc53f0 0x5617e3cc5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.372+0000 7fc13a7fc700 1 -- 172.21.15.53:0/974976218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617e3a43de0 con 0x5617e3cc6a70 2023-11-06T01:40:37.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.372+0000 7fc13affd700 1 --2- 172.21.15.53:0/974976218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3cc6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:37.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.372+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/974976218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc53f0 0x5617e3cc5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:37.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.372+0000 7fc13a7fc700 1 --2- 172.21.15.53:0/974976218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc6a70 0x5617e3cc4e60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc13000acb0 tx=0x7fc13000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d50189795be5aaa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.372+0000 7fc1217fa700 1 -- 172.21.15.53:0/974976218 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13000a7b0 con 0x5617e3cc6a70 2023-11-06T01:40:37.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.372+0000 7fc1217fa700 1 -- 172.21.15.53:0/974976218 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc13000fe50 con 0x5617e3cc6a70 2023-11-06T01:40:37.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.372+0000 7fc1217fa700 1 -- 172.21.15.53:0/974976218 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1300108a0 con 0x5617e3cc6a70 2023-11-06T01:40:37.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.373+0000 7fc14d805b80 1 -- 172.21.15.53:0/974976218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc6a70 msgr2=0x5617e3cc4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.373+0000 7fc14d805b80 1 --2- 172.21.15.53:0/974976218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc6a70 0x5617e3cc4e60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc13000acb0 tx=0x7fc13000f3c0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.373+0000 7fc14d805b80 1 -- 172.21.15.53:0/974976218 shutdown_connections 2023-11-06T01:40:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.373+0000 7fc14d805b80 1 --2- 172.21.15.53:0/974976218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc53f0 0x5617e3cc5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.373+0000 7fc14d805b80 1 --2- 172.21.15.53:0/974976218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3cc6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.373+0000 7fc14d805b80 1 --2- 172.21.15.53:0/974976218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc6a70 0x5617e3cc4e60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.373+0000 7fc14d805b80 1 -- 172.21.15.53:0/974976218 >> 172.21.15.53:0/974976218 conn(0x5617e3c240d0 msgr2=0x5617e3cbf940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.373+0000 7fc14d805b80 1 -- 172.21.15.53:0/974976218 shutdown_connections 2023-11-06T01:40:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.373+0000 7fc14d805b80 1 -- 172.21.15.53:0/974976218 wait complete. 2023-11-06T01:40:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.373+0000 7fc14d805b80 1 -- start start 2023-11-06T01:40:37.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.373+0000 7fc14d805b80 1 Processor -- start 2023-11-06T01:40:37.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3ccea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3ccef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc14d805b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3bb2510 con 0x5617e3cc53f0 2023-11-06T01:40:37.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc14d805b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3bb21b0 con 0x5617e3cc6a70 2023-11-06T01:40:37.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc14d805b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3a99920 con 0x5617e3cc60f0 2023-11-06T01:40:37.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc14d805b80 10 client.0 _ll_register_callbacks cb 0x5617e3affa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:37.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc13affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3ccea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc13b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc13affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3ccea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41818/0 (socket says 172.21.15.53:41818) 2023-11-06T01:40:37.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc13affd700 1 -- 172.21.15.53:0/21091368 learned_addr learned my addr 172.21.15.53:0/21091368 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:37.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc13b7fe700 1 -- 172.21.15.53:0/21091368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 msgr2=0x5617e3ccef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3ccef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc13b7fe700 1 -- 172.21.15.53:0/21091368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 msgr2=0x5617e3ccea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3ccea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc13b7fe700 1 -- 172.21.15.53:0/21091368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c1a1260 con 0x5617e3cc6a70 2023-11-06T01:40:37.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.377+0000 7fc13affd700 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3ccea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:37.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.378+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd5f80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1340ca8a0 tx=0x7fc134009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c0025fbab7c53d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.378+0000 7fc11eff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:40:37.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.378+0000 7fc11eff5700 1 -- 172.21.15.53:0/21091368 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13409e4c0 con 0x5617e3cc6a70 2023-11-06T01:40:37.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.378+0000 7fc11eff5700 1 -- 172.21.15.53:0/21091368 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc1340a2670 con 0x5617e3cc6a70 2023-11-06T01:40:37.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.378+0000 7fc14d805b80 1 -- 172.21.15.53:0/21091368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617e3c40d10 con 0x5617e3cc6a70 2023-11-06T01:40:37.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.378+0000 7fc11eff5700 1 -- 172.21.15.53:0/21091368 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc134099b80 con 0x5617e3cc6a70 2023-11-06T01:40:37.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.378+0000 7fc14d805b80 10 client.25802 fetch_fsmap learned FSMap version 40 2023-11-06T01:40:37.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.378+0000 7fc14d805b80 1 -- 172.21.15.53:0/21091368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fc12c017070 con 0x5617e3cc6a70 2023-11-06T01:40:37.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.378+0000 7fc11eff5700 1 -- 172.21.15.53:0/21091368 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc134064080 con 0x5617e3cc6a70 2023-11-06T01:40:37.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.379+0000 7fc11eff5700 1 -- 172.21.15.53:0/21091368 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fc13409e020 con 0x5617e3cc6a70 2023-11-06T01:40:37.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.379+0000 7fc14d805b80 10 client.25802 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:40:37.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.379+0000 7fc14d805b80 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:37.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.379+0000 7fc14d805b80 20 client.25802 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:37.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.379+0000 7fc14d805b80 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5617e3cda5a0 0x5617e3cdc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.379+0000 7fc14d805b80 4 client.25802 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:40:37.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.379+0000 7fc14d805b80 1 -- 172.21.15.53:0/21091368 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617e3c93e10 con 0x5617e3cda5a0 2023-11-06T01:40:37.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.380+0000 7fc13affd700 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5617e3cda5a0 0x5617e3cdc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.380+0000 7fc11eff5700 10 client.25802 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:37.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.380+0000 7fc13affd700 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5617e3cda5a0 0x5617e3cdc990 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ceeb032e153236 server_cookie=6dbf80fc385df5a9 in_seq=0 out_seq=0 2023-11-06T01:40:37.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc11eff5700 1 -- 172.21.15.53:0/21091368 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5617e3c93e10 con 0x5617e3cda5a0 2023-11-06T01:40:37.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc11eff5700 10 client.25802 handle_command_reply: tid=0 2023-11-06T01:40:37.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc14d805b80 1 client.25802 shutdown 2023-11-06T01:40:37.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc14d805b80 1 -- 172.21.15.53:0/21091368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 msgr2=0x5617e3cd5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc14d805b80 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd5f80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1340ca8a0 tx=0x7fc134009780 comp rx=0 tx=0).stop 2023-11-06T01:40:37.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc14d805b80 1 -- 172.21.15.53:0/21091368 shutdown_connections 2023-11-06T01:40:37.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc14d805b80 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd5f80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc14d805b80 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5617e3cda5a0 0x5617e3cdc990 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc14d805b80 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3ccea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc14d805b80 1 --2- 172.21.15.53:0/21091368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3ccef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc11eff5700 0 client.25802 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:37.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc14d805b80 1 -- 172.21.15.53:0/21091368 >> 172.21.15.53:0/21091368 conn(0x5617e3c240d0 msgr2=0x5617e3cbf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc14d805b80 1 -- 172.21.15.53:0/21091368 shutdown_connections 2023-11-06T01:40:37.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.381+0000 7fc14d805b80 1 -- 172.21.15.53:0/21091368 wait complete. 2023-11-06T01:40:37.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.382+0000 7fc14d805b80 20 client.25802 trim_cache size 0 max 16384 2023-11-06T01:40:37.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.404+0000 7fc14d805b80 1 Processor -- start 2023-11-06T01:40:37.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.404+0000 7fc14d805b80 1 -- start start 2023-11-06T01:40:37.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.404+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc53f0 0x5617e3cc4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.404+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc60f0 0x5617e3cd3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.404+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.404+0000 7fc14d805b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3ccdeb0 con 0x5617e3cc60f0 2023-11-06T01:40:37.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.404+0000 7fc14d805b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3cda3c0 con 0x5617e3cc6a70 2023-11-06T01:40:37.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.404+0000 7fc14d805b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3cce570 con 0x5617e3cc53f0 2023-11-06T01:40:37.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.404+0000 7fc13b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.404+0000 7fc13b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53268/0 (socket says 172.21.15.53:53268) 2023-11-06T01:40:37.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.404+0000 7fc13b7fe700 1 -- 172.21.15.53:0/242077496 learned_addr learned my addr 172.21.15.53:0/242077496 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:37.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.405+0000 7fc13affd700 1 --2- 172.21.15.53:0/242077496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc53f0 0x5617e3cc4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.405+0000 7fc13b7fe700 1 -- 172.21.15.53:0/242077496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc53f0 msgr2=0x5617e3cc4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.405+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/242077496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc53f0 0x5617e3cc4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.405+0000 7fc13b7fe700 1 -- 172.21.15.53:0/242077496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc60f0 msgr2=0x5617e3cd3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.405+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/242077496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc60f0 0x5617e3cd3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.405+0000 7fc13b7fe700 1 -- 172.21.15.53:0/242077496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13000a910 con 0x5617e3cc6a70 2023-11-06T01:40:37.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.405+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/242077496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd6870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc134026000 tx=0x7fc13409aed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90f00154b62585 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.406+0000 7fc121ffb700 1 -- 172.21.15.53:0/242077496 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1340a2670 con 0x5617e3cc6a70 2023-11-06T01:40:37.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.406+0000 7fc121ffb700 1 -- 172.21.15.53:0/242077496 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617e3c40d10 con 0x5617e3cc6a70 2023-11-06T01:40:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.406+0000 7fc121ffb700 1 -- 172.21.15.53:0/242077496 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1340a21e0 con 0x5617e3cc6a70 2023-11-06T01:40:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.406+0000 7fc14d805b80 1 -- 172.21.15.53:0/242077496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 msgr2=0x5617e3cd6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.406+0000 7fc14d805b80 1 --2- 172.21.15.53:0/242077496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd6870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc134026000 tx=0x7fc13409aed0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.406+0000 7fc14d805b80 1 -- 172.21.15.53:0/242077496 shutdown_connections 2023-11-06T01:40:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 --2- 172.21.15.53:0/242077496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd6870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 --2- 172.21.15.53:0/242077496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc53f0 0x5617e3cc4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 --2- 172.21.15.53:0/242077496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc60f0 0x5617e3cd3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 -- 172.21.15.53:0/242077496 >> 172.21.15.53:0/242077496 conn(0x5617e3c240d0 msgr2=0x5617e3ccd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 -- 172.21.15.53:0/242077496 shutdown_connections 2023-11-06T01:40:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 -- 172.21.15.53:0/242077496 wait complete. 2023-11-06T01:40:37.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 -- start start 2023-11-06T01:40:37.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 Processor -- start 2023-11-06T01:40:37.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3cd3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3cd2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3cd2fb0 con 0x5617e3cc53f0 2023-11-06T01:40:37.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3cd3120 con 0x5617e3cc6a70 2023-11-06T01:40:37.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617e3d34870 con 0x5617e3cc60f0 2023-11-06T01:40:37.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.407+0000 7fc14d805b80 10 client.0 _ll_register_callbacks cb 0x7fc13409a550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:37.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3cd3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3cd3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41832/0 (socket says 172.21.15.53:41832) 2023-11-06T01:40:37.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13affd700 1 -- 172.21.15.53:0/3876623836 learned_addr learned my addr 172.21.15.53:0/3876623836 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:37.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13b7fe700 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13a7fc700 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3cd2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13affd700 1 -- 172.21.15.53:0/3876623836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 msgr2=0x5617e3cd2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13affd700 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3cd2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13affd700 1 -- 172.21.15.53:0/3876623836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 msgr2=0x5617e3cd28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:37.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13affd700 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13affd700 1 -- 172.21.15.53:0/3876623836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c017070 con 0x5617e3cc53f0 2023-11-06T01:40:37.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13a7fc700 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3cd2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:40:37.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc13affd700 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3cd3d80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5617e3cdd220 tx=0x7fc12c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ebe11af860ea59 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:37.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc11eff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:40:37.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc11eff5700 1 -- 172.21.15.53:0/3876623836 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c024040 con 0x5617e3cc53f0 2023-11-06T01:40:37.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc11eff5700 1 -- 172.21.15.53:0/3876623836 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc12c029d60 con 0x5617e3cc53f0 2023-11-06T01:40:37.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.408+0000 7fc14d805b80 1 -- 172.21.15.53:0/3876623836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc13000fe50 con 0x5617e3cc53f0 2023-11-06T01:40:37.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc11eff5700 1 -- 172.21.15.53:0/3876623836 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc12c032420 con 0x5617e3cc53f0 2023-11-06T01:40:37.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc14d805b80 10 client.16236 fetch_fsmap learned FSMap version 40 2023-11-06T01:40:37.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc14d805b80 1 -- 172.21.15.53:0/3876623836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5617e3a2cc60 con 0x5617e3cc53f0 2023-11-06T01:40:37.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc11eff5700 1 -- 172.21.15.53:0/3876623836 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc13000fe50 con 0x5617e3cc53f0 2023-11-06T01:40:37.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc11eff5700 1 -- 172.21.15.53:0/3876623836 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fc12c01e540 con 0x5617e3cc53f0 2023-11-06T01:40:37.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc14d805b80 10 client.16236 fetch_fsmap finished waiting for FSMap version 40 2023-11-06T01:40:37.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc14d805b80 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:37.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc14d805b80 20 client.16236 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:37.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc14d805b80 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5617e3d36200 0x5617e3d38620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:37.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc14d805b80 4 client.16236 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:40:37.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc14d805b80 1 -- 172.21.15.53:0/3876623836 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5617e3c0ed40 con 0x5617e3d36200 2023-11-06T01:40:37.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.409+0000 7fc13a7fc700 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5617e3d36200 0x5617e3d38620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:37.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc11eff5700 10 client.16236 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:37.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc13a7fc700 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5617e3d36200 0x5617e3d38620 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=621b411b4e334fae server_cookie=25a44be9f457598d in_seq=0 out_seq=0 2023-11-06T01:40:37.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc11eff5700 1 -- 172.21.15.53:0/3876623836 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5617e3c0ed40 con 0x5617e3d36200 2023-11-06T01:40:37.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc11eff5700 10 client.16236 handle_command_reply: tid=0 2023-11-06T01:40:37.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc14d805b80 1 client.16236 shutdown 2023-11-06T01:40:37.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc14d805b80 1 -- 172.21.15.53:0/3876623836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 msgr2=0x5617e3cd3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc14d805b80 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3cd3d80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5617e3cdd220 tx=0x7fc12c0047c0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc14d805b80 1 -- 172.21.15.53:0/3876623836 shutdown_connections 2023-11-06T01:40:37.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc14d805b80 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617e3cc6a70 0x5617e3cd28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc14d805b80 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5617e3d36200 0x5617e3d38620 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc14d805b80 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617e3cc53f0 0x5617e3cd3d80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc14d805b80 1 --2- 172.21.15.53:0/3876623836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617e3cc60f0 0x5617e3cd2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.410+0000 7fc14d805b80 1 -- 172.21.15.53:0/3876623836 >> 172.21.15.53:0/3876623836 conn(0x5617e3c240d0 msgr2=0x5617e3cc9310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.411+0000 7fc11eff5700 0 client.16236 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:37.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.413+0000 7fc14d805b80 1 -- 172.21.15.53:0/3876623836 shutdown_connections 2023-11-06T01:40:37.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.413+0000 7fc14d805b80 1 -- 172.21.15.53:0/3876623836 wait complete. 2023-11-06T01:40:37.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc14d805b80 20 client.16236 trim_cache size 0 max 16384 2023-11-06T01:40:37.429 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:40:37.429 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-06T01:40:37.429 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:40:37.429 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:40:37.430 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:40:37.430 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:40:37.430 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:40:37.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:40:37.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:40:37.431 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:40:37.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 -- 172.21.15.53:0/2353209561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc1240a0520 msgr2=0x7fc1240a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc1240a0520 0x7fc1240a29e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc13c19a280 tx=0x7fc12c009260 comp rx=0 tx=0).stop 2023-11-06T01:40:37.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 -- 172.21.15.53:0/2353209561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c106a80 msgr2=0x7fc13c1a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:37.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c106a80 0x7fc13c1a0320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc13400b070 tx=0x7fc13400bab0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 -- 172.21.15.53:0/2353209561 shutdown_connections 2023-11-06T01:40:37.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc1240a0520 0x7fc1240a29e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c106a80 0x7fc13c1a0320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc13c101820 0x7fc13c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 --2- 172.21.15.53:0/2353209561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc13c104150 0x7fc13c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:37.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 -- 172.21.15.53:0/2353209561 >> 172.21.15.53:0/2353209561 conn(0x7fc13c0fb250 msgr2=0x7fc13c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:37.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 -- 172.21.15.53:0/2353209561 shutdown_connections 2023-11-06T01:40:37.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:40:37.414+0000 7fc141675700 1 -- 172.21.15.53:0/2353209561 wait complete. 2023-11-06T01:40:37.448 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:40:37.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:37 smithi053 ceph-mon[99472]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 742 op/s 2023-11-06T01:40:37.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:37 smithi062 ceph-mon[103924]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 742 op/s 2023-11-06T01:40:37.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:37 smithi163 ceph-mon[104416]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 742 op/s 2023-11-06T01:40:39.654 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:39 smithi062 ceph-mon[103924]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 812 op/s 2023-11-06T01:40:39.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:39 smithi163 ceph-mon[104416]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 812 op/s 2023-11-06T01:40:39.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:39 smithi053 ceph-mon[99472]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 812 op/s 2023-11-06T01:40:41.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:41 smithi062 ceph-mon[103924]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 739 op/s 2023-11-06T01:40:41.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:41 smithi163 ceph-mon[104416]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 739 op/s 2023-11-06T01:40:41.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:41 smithi053 ceph-mon[99472]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 739 op/s 2023-11-06T01:40:42.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 385.0 s, 1272.6 tps, lat 78.832 ms stddev 92.394 2023-11-06T01:40:43.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:43 smithi062 ceph-mon[103924]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 842 op/s 2023-11-06T01:40:43.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:43 smithi163 ceph-mon[104416]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 842 op/s 2023-11-06T01:40:43.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:43 smithi053 ceph-mon[99472]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 842 op/s 2023-11-06T01:40:44.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:44 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:40:44.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:44 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:40:44.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:44 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:40:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:45 smithi062 ceph-mon[103924]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 769 op/s 2023-11-06T01:40:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:45 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:40:45.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:45 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:40:45.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:45 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:40:45.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:45 smithi163 ceph-mon[104416]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 769 op/s 2023-11-06T01:40:45.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:45 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:40:45.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:45 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:40:45.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:45 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:40:45.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:45 smithi053 ceph-mon[99472]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 769 op/s 2023-11-06T01:40:45.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:40:45.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:40:45.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:40:46.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:46 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:40:46.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:46 smithi062 ceph-mon[103924]: scrub summary: active paths [/] 2023-11-06T01:40:46.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:46 smithi062 ceph-mon[103924]: scrub summary: idle+waiting paths [/] 2023-11-06T01:40:46.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:46 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:40:46.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:46 smithi163 ceph-mon[104416]: scrub summary: active paths [/] 2023-11-06T01:40:46.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:46 smithi163 ceph-mon[104416]: scrub summary: idle+waiting paths [/] 2023-11-06T01:40:46.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:46 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:40:46.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:46 smithi053 ceph-mon[99472]: scrub summary: active paths [/] 2023-11-06T01:40:46.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:46 smithi053 ceph-mon[99472]: scrub summary: idle+waiting paths [/] 2023-11-06T01:40:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 390.0 s, 1693.6 tps, lat 59.068 ms stddev 66.229 2023-11-06T01:40:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:47 smithi062 ceph-mon[103924]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 746 op/s 2023-11-06T01:40:47.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:47 smithi163 ceph-mon[104416]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 746 op/s 2023-11-06T01:40:47.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:47 smithi053 ceph-mon[99472]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 746 op/s 2023-11-06T01:40:49.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:49 smithi062 ceph-mon[103924]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 842 op/s 2023-11-06T01:40:49.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:49 smithi163 ceph-mon[104416]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 842 op/s 2023-11-06T01:40:49.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:49 smithi053 ceph-mon[99472]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 842 op/s 2023-11-06T01:40:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.274+0000 7f706f188700 1 -- 172.21.15.53:0/997234032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7068069b40 msgr2=0x7f706810fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.274+0000 7f706f188700 1 --2- 172.21.15.53:0/997234032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7068069b40 0x7f706810fd80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7058009990 tx=0x7f7058009ca0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.274+0000 7f706f188700 1 -- 172.21.15.53:0/997234032 shutdown_connections 2023-11-06T01:40:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.274+0000 7f706f188700 1 --2- 172.21.15.53:0/997234032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7068069b40 0x7f706810fd80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.274+0000 7f706f188700 1 --2- 172.21.15.53:0/997234032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7068108000 0x7f706810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.274+0000 7f706f188700 1 --2- 172.21.15.53:0/997234032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f706810a9c0 0x7f706810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.274+0000 7f706f188700 1 -- 172.21.15.53:0/997234032 >> 172.21.15.53:0/997234032 conn(0x7f706806df60 msgr2=0x7f70680703c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.274+0000 7f706f188700 1 -- 172.21.15.53:0/997234032 shutdown_connections 2023-11-06T01:40:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.274+0000 7f706f188700 1 -- 172.21.15.53:0/997234032 wait complete. 2023-11-06T01:40:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f706f188700 1 Processor -- start 2023-11-06T01:40:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f706f188700 1 -- start start 2023-11-06T01:40:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f706f188700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7068069b40 0x7f706819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f706f188700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7068108000 0x7f706819f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f706f188700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f706810a9c0 0x7f70681a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f706f188700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70681a0ed0 con 0x7f7068108000 2023-11-06T01:40:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f706f188700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70681a1040 con 0x7f706810a9c0 2023-11-06T01:40:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f706f188700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70681a1200 con 0x7f7068069b40 2023-11-06T01:40:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f7067fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7068108000 0x7f706819f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f706cf24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7068069b40 0x7f706819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f7067fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7068108000 0x7f706819f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41848/0 (socket says 172.21.15.53:41848) 2023-11-06T01:40:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f7067fff700 1 -- 172.21.15.53:0/3166974814 learned_addr learned my addr 172.21.15.53:0/3166974814 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f706cf24700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7068069b40 0x7f706819f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56086/0 (socket says 172.21.15.53:56086) 2023-11-06T01:40:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.275+0000 7f706d725700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f706810a9c0 0x7f70681a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f7067fff700 1 -- 172.21.15.53:0/3166974814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7068069b40 msgr2=0x7f706819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f7067fff700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7068069b40 0x7f706819f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f7067fff700 1 -- 172.21.15.53:0/3166974814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f706810a9c0 msgr2=0x7f70681a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f7067fff700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f706810a9c0 0x7f70681a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f7067fff700 1 -- 172.21.15.53:0/3166974814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70580095f0 con 0x7f7068108000 2023-11-06T01:40:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f706cf24700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7068069b40 0x7f706819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f7067fff700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7068108000 0x7f706819f9f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f705c00b500 tx=0x7f705c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f7065ffb700 1 -- 172.21.15.53:0/3166974814 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c011830 con 0x7f7068108000 2023-11-06T01:40:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f706f188700 1 -- 172.21.15.53:0/3166974814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70681a14e0 con 0x7f7068108000 2023-11-06T01:40:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f7065ffb700 1 -- 172.21.15.53:0/3166974814 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f705c011e50 con 0x7f7068108000 2023-11-06T01:40:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f7065ffb700 1 -- 172.21.15.53:0/3166974814 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c010de0 con 0x7f7068108000 2023-11-06T01:40:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.276+0000 7f706f188700 1 -- 172.21.15.53:0/3166974814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70681a7d20 con 0x7f7068108000 2023-11-06T01:40:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.280+0000 7f7065ffb700 1 -- 172.21.15.53:0/3166974814 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f705c010480 con 0x7f7068108000 2023-11-06T01:40:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.281+0000 7f7065ffb700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70500a0520 0x7f70500a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.281+0000 7f707b318b80 1 Processor -- start 2023-11-06T01:40:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.281+0000 7f7065ffb700 1 -- 172.21.15.53:0/3166974814 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f705c0cb440 con 0x7f7068108000 2023-11-06T01:40:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.281+0000 7f707b318b80 1 -- start start 2023-11-06T01:40:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.281+0000 7f706cf24700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70500a0520 0x7f70500a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.281+0000 7f707b318b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e0250eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.281+0000 7f707b318b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 0x55d6e0251d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.281+0000 7f707b318b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02522e0 0x55d6e0252760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.281+0000 7f707b318b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e01a49b0 con 0x55d6e02522e0 2023-11-06T01:40:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.281+0000 7f707b318b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e01a2370 con 0x55d6e0250a90 2023-11-06T01:40:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.281+0000 7f707b318b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e019dc60 con 0x55d6e02513f0 2023-11-06T01:40:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f706cf24700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e0250eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f7067fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 0x55d6e0251d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f706cf24700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e0250eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53292/0 (socket says 172.21.15.53:53292) 2023-11-06T01:40:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f706cf24700 1 -- 172.21.15.53:0/915938355 learned_addr learned my addr 172.21.15.53:0/915938355 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f706d725700 1 --2- 172.21.15.53:0/915938355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02522e0 0x55d6e0252760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f706cf24700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70500a0520 0x7f70500a29e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f706000a800 tx=0x7f70600096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f7067fff700 1 -- 172.21.15.53:0/915938355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 msgr2=0x55d6e0250eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f7067fff700 1 --2- 172.21.15.53:0/915938355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e0250eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f7067fff700 1 -- 172.21.15.53:0/915938355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02522e0 msgr2=0x55d6e0252760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f7067fff700 1 --2- 172.21.15.53:0/915938355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02522e0 0x55d6e0252760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f7067fff700 1 -- 172.21.15.53:0/915938355 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6dffcbd80 con 0x55d6e02513f0 2023-11-06T01:40:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f7067fff700 1 --2- 172.21.15.53:0/915938355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 0x55d6e0251d00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f705c000c00 tx=0x7f705c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf6d9f35ee2b7928 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f704effd700 1 -- 172.21.15.53:0/915938355 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c099380 con 0x55d6e02513f0 2023-11-06T01:40:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f704effd700 1 -- 172.21.15.53:0/915938355 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f705c098c80 con 0x55d6e02513f0 2023-11-06T01:40:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.282+0000 7f704effd700 1 -- 172.21.15.53:0/915938355 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c0a0860 con 0x55d6e02513f0 2023-11-06T01:40:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.283+0000 7f707b318b80 1 -- 172.21.15.53:0/915938355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 msgr2=0x55d6e0251d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.283+0000 7f707b318b80 1 --2- 172.21.15.53:0/915938355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 0x55d6e0251d00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f705c000c00 tx=0x7f705c00bf10 comp rx=0 tx=0).stop 2023-11-06T01:40:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.283+0000 7f707b318b80 1 -- 172.21.15.53:0/915938355 shutdown_connections 2023-11-06T01:40:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.283+0000 7f707b318b80 1 --2- 172.21.15.53:0/915938355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e0250eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.283+0000 7f707b318b80 1 --2- 172.21.15.53:0/915938355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02522e0 0x55d6e0252760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.283+0000 7f707b318b80 1 --2- 172.21.15.53:0/915938355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 0x55d6e0251d00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.283+0000 7f707b318b80 1 -- 172.21.15.53:0/915938355 >> 172.21.15.53:0/915938355 conn(0x55d6e01ac070 msgr2=0x55d6e0249040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.285+0000 7f707b318b80 1 -- 172.21.15.53:0/915938355 shutdown_connections 2023-11-06T01:40:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.285+0000 7f707b318b80 1 -- 172.21.15.53:0/915938355 wait complete. 2023-11-06T01:40:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.286+0000 7f707b318b80 1 -- start start 2023-11-06T01:40:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.286+0000 7f707b318b80 1 Processor -- start 2023-11-06T01:40:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f707b318b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e0250a90 0x55d6e025a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f707b318b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02513f0 0x55d6e025ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f707b318b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e02522e0 0x55d6e0261e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f707b318b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e013a4b0 con 0x55d6e02513f0 2023-11-06T01:40:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f707b318b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e013a150 con 0x55d6e02522e0 2023-11-06T01:40:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f707b318b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e00218c0 con 0x55d6e0250a90 2023-11-06T01:40:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f707b318b80 10 client.0 _ll_register_callbacks cb 0x55d6e0087a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f706cf24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e0250a90 0x55d6e025a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f706cf24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e0250a90 0x55d6e025a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56100/0 (socket says 172.21.15.53:56100) 2023-11-06T01:40:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f706cf24700 1 -- 172.21.15.53:0/610981650 learned_addr learned my addr 172.21.15.53:0/610981650 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f7067fff700 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02513f0 0x55d6e025ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f706d725700 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e02522e0 0x55d6e0261e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f706cf24700 1 -- 172.21.15.53:0/610981650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e02522e0 msgr2=0x55d6e0261e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f706cf24700 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e02522e0 0x55d6e0261e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f706cf24700 1 -- 172.21.15.53:0/610981650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02513f0 msgr2=0x55d6e025ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f706cf24700 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02513f0 0x55d6e025ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f706cf24700 1 -- 172.21.15.53:0/610981650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70681a7d20 con 0x55d6e0250a90 2023-11-06T01:40:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f706cf24700 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e0250a90 0x55d6e025a8d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7060017640 tx=0x7f7060013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd39e343d24713a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f7047fff700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:40:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.287+0000 7f7047fff700 1 -- 172.21.15.53:0/610981650 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060010070 con 0x55d6e0250a90 2023-11-06T01:40:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f707b318b80 1 -- 172.21.15.53:0/610981650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6e01c8cb0 con 0x55d6e0250a90 2023-11-06T01:40:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f7047fff700 1 -- 172.21.15.53:0/610981650 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f706001d040 con 0x55d6e0250a90 2023-11-06T01:40:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f7047fff700 1 -- 172.21.15.53:0/610981650 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7060018e40 con 0x55d6e0250a90 2023-11-06T01:40:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f707b318b80 10 client.25870 fetch_fsmap learned FSMap version 41 2023-11-06T01:40:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f707b318b80 1 -- 172.21.15.53:0/610981650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d6dffb4c00 con 0x55d6e0250a90 2023-11-06T01:40:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f7047fff700 1 -- 172.21.15.53:0/610981650 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7060030020 con 0x55d6e0250a90 2023-11-06T01:40:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f7047fff700 1 -- 172.21.15.53:0/610981650 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 41) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f706002a870 con 0x55d6e0250a90 2023-11-06T01:40:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f707b318b80 10 client.25870 fetch_fsmap finished waiting for FSMap version 41 2023-11-06T01:40:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f707b318b80 10 client.25870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f707b318b80 20 client.25870 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f707b318b80 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d6e02667b0 0x55d6e0268bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f707b318b80 4 client.25870 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:40:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f707b318b80 1 -- 172.21.15.53:0/610981650 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6e021bdb0 con 0x55d6e02667b0 2023-11-06T01:40:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.288+0000 7f7067fff700 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d6e02667b0 0x55d6e0268bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.289+0000 7f7047fff700 10 client.25870 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.289+0000 7f7067fff700 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d6e02667b0 0x55d6e0268bd0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a482047515dfdec1 server_cookie=4cb47ada070a664b in_seq=0 out_seq=0 2023-11-06T01:40:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.289+0000 7f7047fff700 1 -- 172.21.15.53:0/610981650 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d6e021bdb0 con 0x55d6e02667b0 2023-11-06T01:40:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.289+0000 7f7047fff700 10 client.25870 handle_command_reply: tid=0 2023-11-06T01:40:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.289+0000 7f707b318b80 1 client.25870 shutdown 2023-11-06T01:40:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f707b318b80 1 -- 172.21.15.53:0/610981650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e0250a90 msgr2=0x55d6e025a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f707b318b80 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e0250a90 0x55d6e025a8d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7060017640 tx=0x7f7060013000 comp rx=0 tx=0).stop 2023-11-06T01:40:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f707b318b80 1 -- 172.21.15.53:0/610981650 shutdown_connections 2023-11-06T01:40:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f707b318b80 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e02522e0 0x55d6e0261e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f707b318b80 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d6e02667b0 0x55d6e0268bd0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f707b318b80 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e0250a90 0x55d6e025a8d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f707b318b80 1 --2- 172.21.15.53:0/610981650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02513f0 0x55d6e025ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f7047fff700 0 client.25870 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f707b318b80 1 -- 172.21.15.53:0/610981650 >> 172.21.15.53:0/610981650 conn(0x55d6e01ac070 msgr2=0x55d6e02473c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f707b318b80 1 -- 172.21.15.53:0/610981650 shutdown_connections 2023-11-06T01:40:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f707b318b80 1 -- 172.21.15.53:0/610981650 wait complete. 2023-11-06T01:40:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.290+0000 7f707b318b80 20 client.25870 trim_cache size 0 max 16384 2023-11-06T01:40:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.323+0000 7f707b318b80 1 Processor -- start 2023-11-06T01:40:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.325+0000 7f707b318b80 1 -- start start 2023-11-06T01:40:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.325+0000 7f707b318b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e025fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.325+0000 7f707b318b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02513f0 0x55d6e0258710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.325+0000 7f707b318b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02522e0 0x55d6e02502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.325+0000 7f707b318b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e0259d80 con 0x55d6e02513f0 2023-11-06T01:40:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.325+0000 7f707b318b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e0242500 con 0x55d6e0250a90 2023-11-06T01:40:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.325+0000 7f707b318b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e0263b10 con 0x55d6e02522e0 2023-11-06T01:40:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.325+0000 7f706cf24700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e025fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.325+0000 7f706cf24700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e025fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53306/0 (socket says 172.21.15.53:53306) 2023-11-06T01:40:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.325+0000 7f706cf24700 1 -- 172.21.15.53:0/1606098092 learned_addr learned my addr 172.21.15.53:0/1606098092 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.325+0000 7f706d725700 1 --2- 172.21.15.53:0/1606098092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02522e0 0x55d6e02502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.326+0000 7f706d725700 1 -- 172.21.15.53:0/1606098092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 msgr2=0x55d6e025fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.326+0000 7f706d725700 1 --2- 172.21.15.53:0/1606098092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e025fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.326+0000 7f706d725700 1 -- 172.21.15.53:0/1606098092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02513f0 msgr2=0x55d6e0258710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:40:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.326+0000 7f706d725700 1 --2- 172.21.15.53:0/1606098092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02513f0 0x55d6e0258710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.326+0000 7f706d725700 1 -- 172.21.15.53:0/1606098092 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6dfff21f0 con 0x55d6e02522e0 2023-11-06T01:40:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.326+0000 7f706d725700 1 --2- 172.21.15.53:0/1606098092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02522e0 0x55d6e02502a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70580036f0 tx=0x7f705801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d78753928d0bc31 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.326+0000 7f704f7fe700 1 -- 172.21.15.53:0/1606098092 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705801d030 con 0x55d6e02522e0 2023-11-06T01:40:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.326+0000 7f704f7fe700 1 -- 172.21.15.53:0/1606098092 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f705801e040 con 0x55d6e02522e0 2023-11-06T01:40:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.327+0000 7f704f7fe700 1 -- 172.21.15.53:0/1606098092 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705800f460 con 0x55d6e02522e0 2023-11-06T01:40:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.327+0000 7f707b318b80 1 -- 172.21.15.53:0/1606098092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02522e0 msgr2=0x55d6e02502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.327+0000 7f707b318b80 1 --2- 172.21.15.53:0/1606098092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02522e0 0x55d6e02502a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70580036f0 tx=0x7f705801a040 comp rx=0 tx=0).stop 2023-11-06T01:40:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.328+0000 7f707b318b80 1 -- 172.21.15.53:0/1606098092 shutdown_connections 2023-11-06T01:40:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.328+0000 7f707b318b80 1 --2- 172.21.15.53:0/1606098092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e025fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.328+0000 7f707b318b80 1 --2- 172.21.15.53:0/1606098092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02522e0 0x55d6e02502a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.328+0000 7f707b318b80 1 --2- 172.21.15.53:0/1606098092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02513f0 0x55d6e0258710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.328+0000 7f707b318b80 1 -- 172.21.15.53:0/1606098092 >> 172.21.15.53:0/1606098092 conn(0x55d6e01ac070 msgr2=0x55d6e0259520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.341+0000 7f707b318b80 1 -- 172.21.15.53:0/1606098092 shutdown_connections 2023-11-06T01:40:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.341+0000 7f707b318b80 1 -- 172.21.15.53:0/1606098092 wait complete. 2023-11-06T01:40:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.341+0000 7f707b318b80 1 -- start start 2023-11-06T01:40:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.341+0000 7f707b318b80 1 Processor -- start 2023-11-06T01:40:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.342+0000 7f707b318b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e024c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.342+0000 7f707b318b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 0x55d6e024ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.342+0000 7f707b318b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02522e0 0x55d6e024f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.342+0000 7f707b318b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e024fde0 con 0x55d6e02522e0 2023-11-06T01:40:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.342+0000 7f707b318b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e024ff50 con 0x55d6e0250a90 2023-11-06T01:40:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.342+0000 7f707b318b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e0250110 con 0x55d6e02513f0 2023-11-06T01:40:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.342+0000 7f707b318b80 10 client.0 _ll_register_callbacks cb 0x7f7060018a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:40:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.342+0000 7f706cf24700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e024c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.342+0000 7f706cf24700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e024c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53310/0 (socket says 172.21.15.53:53310) 2023-11-06T01:40:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.342+0000 7f706cf24700 1 -- 172.21.15.53:0/1560007074 learned_addr learned my addr 172.21.15.53:0/1560007074 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:40:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.342+0000 7f7067fff700 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 0x55d6e024ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f7067fff700 1 -- 172.21.15.53:0/1560007074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 msgr2=0x55d6e024c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:40:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f7067fff700 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e024c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f7067fff700 1 -- 172.21.15.53:0/1560007074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02522e0 msgr2=0x55d6e024f8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:40:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f7067fff700 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02522e0 0x55d6e024f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f7067fff700 1 -- 172.21.15.53:0/1560007074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706810d920 con 0x55d6e02513f0 2023-11-06T01:40:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f706cf24700 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e024c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:40:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f7067fff700 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 0x55d6e024ca10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55d6e02590f0 tx=0x7f705c00bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca468df4130c4481 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:40:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f7047fff700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:40:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f7047fff700 1 -- 172.21.15.53:0/1560007074 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c0b1240 con 0x55d6e02513f0 2023-11-06T01:40:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f707b318b80 1 -- 172.21.15.53:0/1560007074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f705c098c80 con 0x55d6e02513f0 2023-11-06T01:40:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f7047fff700 1 -- 172.21.15.53:0/1560007074 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f705c09cab0 con 0x55d6e02513f0 2023-11-06T01:40:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.343+0000 7f7047fff700 1 -- 172.21.15.53:0/1560007074 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f705c0a0cf0 con 0x55d6e02513f0 2023-11-06T01:40:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.344+0000 7f707b318b80 10 client.25876 fetch_fsmap learned FSMap version 41 2023-11-06T01:40:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.344+0000 7f707b318b80 1 -- 172.21.15.53:0/1560007074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d6dffca250 con 0x55d6e02513f0 2023-11-06T01:40:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.344+0000 7f7047fff700 1 -- 172.21.15.53:0/1560007074 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f705c098c80 con 0x55d6e02513f0 2023-11-06T01:40:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.344+0000 7f7047fff700 1 -- 172.21.15.53:0/1560007074 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 41) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f705c0b8040 con 0x55d6e02513f0 2023-11-06T01:40:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.344+0000 7f707b318b80 10 client.25876 fetch_fsmap finished waiting for FSMap version 41 2023-11-06T01:40:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.344+0000 7f707b318b80 10 client.25876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:40:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.344+0000 7f707b318b80 20 client.25876 populate_metadata read hostname 'smithi053' 2023-11-06T01:40:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.344+0000 7f707b318b80 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d6e02bd910 0x55d6e02bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:40:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.344+0000 7f707b318b80 4 client.25876 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:40:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.344+0000 7f707b318b80 1 -- 172.21.15.53:0/1560007074 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d6e0196ce0 con 0x55d6e02bd910 2023-11-06T01:40:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.345+0000 7f706cf24700 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d6e02bd910 0x55d6e02bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:40:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.345+0000 7f706cf24700 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d6e02bd910 0x55d6e02bfd30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d770ff1619356bd6 server_cookie=54c1623af52bf397 in_seq=0 out_seq=0 2023-11-06T01:40:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.345+0000 7f7047fff700 10 client.25876 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f7047fff700 1 -- 172.21.15.53:0/1560007074 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d6e0196ce0 con 0x55d6e02bd910 2023-11-06T01:40:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f7047fff700 10 client.25876 handle_command_reply: tid=0 2023-11-06T01:40:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f707b318b80 1 client.25876 shutdown 2023-11-06T01:40:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f707b318b80 1 -- 172.21.15.53:0/1560007074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 msgr2=0x55d6e024ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f707b318b80 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 0x55d6e024ca10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55d6e02590f0 tx=0x7f705c00bd10 comp rx=0 tx=0).stop 2023-11-06T01:40:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f707b318b80 1 -- 172.21.15.53:0/1560007074 shutdown_connections 2023-11-06T01:40:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f707b318b80 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6e0250a90 0x55d6e024c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f707b318b80 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d6e02bd910 0x55d6e02bfd30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f707b318b80 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6e02522e0 0x55d6e024f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f7047fff700 0 client.25876 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:40:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f707b318b80 1 --2- 172.21.15.53:0/1560007074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6e02513f0 0x55d6e024ca10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.355+0000 7f707b318b80 1 -- 172.21.15.53:0/1560007074 >> 172.21.15.53:0/1560007074 conn(0x55d6e01ac070 msgr2=0x55d6e0258710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.356+0000 7f707b318b80 1 -- 172.21.15.53:0/1560007074 shutdown_connections 2023-11-06T01:40:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.356+0000 7f707b318b80 1 -- 172.21.15.53:0/1560007074 wait complete. 2023-11-06T01:40:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.356+0000 7f707b318b80 20 client.25876 trim_cache size 0 max 16384 2023-11-06T01:40:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:40:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:40:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:40:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:40:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 -- 172.21.15.53:0/3166974814 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70500a0520 msgr2=0x7f70500a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70500a0520 0x7f70500a29e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f706000a800 tx=0x7f70600096d0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 -- 172.21.15.53:0/3166974814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7068108000 msgr2=0x7f706819f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:40:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7068108000 0x7f706819f9f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f705c00b500 tx=0x7f705c00b810 comp rx=0 tx=0).stop 2023-11-06T01:40:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 -- 172.21.15.53:0/3166974814 shutdown_connections 2023-11-06T01:40:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f70500a0520 0x7f70500a29e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f706810a9c0 0x7f70681a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7068069b40 0x7f706819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 --2- 172.21.15.53:0/3166974814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7068108000 0x7f706819f9f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:40:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 -- 172.21.15.53:0/3166974814 >> 172.21.15.53:0/3166974814 conn(0x7f706806df60 msgr2=0x7f706810b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:40:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 -- 172.21.15.53:0/3166974814 shutdown_connections 2023-11-06T01:40:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:40:50.357+0000 7f706f188700 1 -- 172.21.15.53:0/3166974814 wait complete. 2023-11-06T01:40:51.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:51 smithi062 ceph-mon[103924]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 758 op/s 2023-11-06T01:40:51.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:51 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:40:51.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:51 smithi163 ceph-mon[104416]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 758 op/s 2023-11-06T01:40:51.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:51 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:40:51.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:51 smithi053 ceph-mon[99472]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 758 op/s 2023-11-06T01:40:51.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:51 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:40:52.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 395.0 s, 1907.8 tps, lat 51.904 ms stddev 55.838 2023-11-06T01:40:53.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:53 smithi163 ceph-mon[104416]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 19 MiB/s wr, 845 op/s 2023-11-06T01:40:53.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:53 smithi053 ceph-mon[99472]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 19 MiB/s wr, 845 op/s 2023-11-06T01:40:53.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:53 smithi062 ceph-mon[103924]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 19 MiB/s wr, 845 op/s 2023-11-06T01:40:55.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:55 smithi163 ceph-mon[104416]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 753 op/s 2023-11-06T01:40:55.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:55 smithi053 ceph-mon[99472]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 753 op/s 2023-11-06T01:40:55.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:55 smithi062 ceph-mon[103924]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 753 op/s 2023-11-06T01:40:57.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 400.0 s, 1335.4 tps, lat 75.485 ms stddev 118.623 2023-11-06T01:40:57.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:57 smithi163 ceph-mon[104416]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 769 op/s 2023-11-06T01:40:57.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:57 smithi053 ceph-mon[99472]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 769 op/s 2023-11-06T01:40:57.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:57 smithi062 ceph-mon[103924]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 769 op/s 2023-11-06T01:40:59.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:40:59 smithi053 ceph-mon[99472]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 1.51k op/s 2023-11-06T01:40:59.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:40:59 smithi062 ceph-mon[103924]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 1.51k op/s 2023-11-06T01:40:59.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:40:59 smithi163 ceph-mon[104416]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 1.51k op/s 2023-11-06T01:41:01.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:01 smithi053 ceph-mon[99472]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 1.70k op/s 2023-11-06T01:41:01.902 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:01 smithi062 ceph-mon[103924]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 1.70k op/s 2023-11-06T01:41:01.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:01 smithi163 ceph-mon[104416]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 1.70k op/s 2023-11-06T01:41:02.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 405.0 s, 1981.0 tps, lat 50.187 ms stddev 58.081 2023-11-06T01:41:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.529+0000 7f8dbee82700 1 -- 172.21.15.53:0/3014425380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8db8101860 msgr2=0x7f8db8108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.529+0000 7f8dbee82700 1 --2- 172.21.15.53:0/3014425380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8db8101860 0x7f8db8108110 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8db000b0a0 tx=0x7f8db000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.529+0000 7f8dbee82700 1 -- 172.21.15.53:0/3014425380 shutdown_connections 2023-11-06T01:41:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.529+0000 7f8dbee82700 1 --2- 172.21.15.53:0/3014425380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8db8101860 0x7f8db8108110 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.529+0000 7f8dbee82700 1 --2- 172.21.15.53:0/3014425380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8db80ffcd0 0x7f8db81000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.529+0000 7f8dbee82700 1 --2- 172.21.15.53:0/3014425380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db8100ec0 0x7f8db8101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.529+0000 7f8dbee82700 1 -- 172.21.15.53:0/3014425380 >> 172.21.15.53:0/3014425380 conn(0x7f8db80fb240 msgr2=0x7f8db80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.529+0000 7f8dbee82700 1 -- 172.21.15.53:0/3014425380 shutdown_connections 2023-11-06T01:41:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.529+0000 7f8dbee82700 1 -- 172.21.15.53:0/3014425380 wait complete. 2023-11-06T01:41:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.530+0000 7f8dbee82700 1 Processor -- start 2023-11-06T01:41:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.530+0000 7f8dbee82700 1 -- start start 2023-11-06T01:41:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.530+0000 7f8dbee82700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8db80ffcd0 0x7f8db819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.530+0000 7f8dbee82700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db8100ec0 0x7f8db819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.530+0000 7f8dbee82700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8db8101860 0x7f8db819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.530+0000 7f8dbee82700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db819ec50 con 0x7f8db8101860 2023-11-06T01:41:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.530+0000 7f8dbee82700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db819edc0 con 0x7f8db8100ec0 2023-11-06T01:41:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.530+0000 7f8dbee82700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db819ef80 con 0x7f8db80ffcd0 2023-11-06T01:41:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.531+0000 7f8dbd41f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8db8101860 0x7f8db819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.531+0000 7f8dbd41f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8db8101860 0x7f8db819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41882/0 (socket says 172.21.15.53:41882) 2023-11-06T01:41:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.531+0000 7f8dbd41f700 1 -- 172.21.15.53:0/2622012044 learned_addr learned my addr 172.21.15.53:0/2622012044 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.531+0000 7f8db7fff700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db8100ec0 0x7f8db819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.531+0000 7f8dbcc1e700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8db80ffcd0 0x7f8db819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.531+0000 7f8dbd41f700 1 -- 172.21.15.53:0/2622012044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8db80ffcd0 msgr2=0x7f8db819d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.531+0000 7f8dbd41f700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8db80ffcd0 0x7f8db819d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.531+0000 7f8dbd41f700 1 -- 172.21.15.53:0/2622012044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db8100ec0 msgr2=0x7f8db819de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.531+0000 7f8dbd41f700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db8100ec0 0x7f8db819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.531+0000 7f8dbd41f700 1 -- 172.21.15.53:0/2622012044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db0009d00 con 0x7f8db8101860 2023-11-06T01:41:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.531+0000 7f8dbcc1e700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8db80ffcd0 0x7f8db819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:41:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.532+0000 7f8dbd41f700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8db8101860 0x7f8db819e460 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009fd0 tx=0x7f8db00097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.532+0000 7f8db5ffb700 1 -- 172.21.15.53:0/2622012044 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db000e040 con 0x7f8db8101860 2023-11-06T01:41:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.532+0000 7f8dbee82700 1 -- 172.21.15.53:0/2622012044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db819f1a0 con 0x7f8db8101860 2023-11-06T01:41:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.532+0000 7f8dbee82700 1 -- 172.21.15.53:0/2622012044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db81a5950 con 0x7f8db8101860 2023-11-06T01:41:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.532+0000 7f8db5ffb700 1 -- 172.21.15.53:0/2622012044 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8db001d480 con 0x7f8db8101860 2023-11-06T01:41:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.532+0000 7f8db5ffb700 1 -- 172.21.15.53:0/2622012044 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db001b6b0 con 0x7f8db8101860 2023-11-06T01:41:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.535+0000 7f8db5ffb700 1 -- 172.21.15.53:0/2622012044 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f8db001b810 con 0x7f8db8101860 2023-11-06T01:41:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.537+0000 7f8dcb012b80 1 Processor -- start 2023-11-06T01:41:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.537+0000 7f8dcb012b80 1 -- start start 2023-11-06T01:41:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.538+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.538+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2fe0 0x55e07dca38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.538+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca3ed0 0x55e07dca4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.538+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07dbfa9b0 con 0x55e07dca2fe0 2023-11-06T01:41:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.538+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07dbf8370 con 0x55e07dca3ed0 2023-11-06T01:41:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.538+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07dbf3c60 con 0x55e07dca2680 2023-11-06T01:41:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.538+0000 7f8db7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.538+0000 7f8db7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56128/0 (socket says 172.21.15.53:56128) 2023-11-06T01:41:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.538+0000 7f8db7fff700 1 -- 172.21.15.53:0/762805632 learned_addr learned my addr 172.21.15.53:0/762805632 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.538+0000 7f8db7fff700 1 --2- 172.21.15.53:0/762805632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca3ed0 0x55e07dca4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.538+0000 7f8db5ffb700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8da80a0520 0x7f8da80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.539+0000 7f8db7fff700 1 -- 172.21.15.53:0/762805632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca3ed0 msgr2=0x55e07dca4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.539+0000 7f8db7fff700 1 --2- 172.21.15.53:0/762805632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca3ed0 0x55e07dca4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.539+0000 7f8db7fff700 1 -- 172.21.15.53:0/762805632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2fe0 msgr2=0x55e07dca38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.539+0000 7f8db7fff700 1 --2- 172.21.15.53:0/762805632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2fe0 0x55e07dca38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.539+0000 7f8db7fff700 1 -- 172.21.15.53:0/762805632 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e07da21d80 con 0x55e07dca2680 2023-11-06T01:41:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.539+0000 7f8db5ffb700 1 -- 172.21.15.53:0/2622012044 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8db00ce540 con 0x7f8db8101860 2023-11-06T01:41:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.539+0000 7f8db7fff700 1 --2- 172.21.15.53:0/762805632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca2aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8da4015590 tx=0x7f8da400c190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d04326df90510138 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.539+0000 7f8dbd41f700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8da80a0520 0x7f8da80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.539+0000 7f8da2ffd700 1 -- 172.21.15.53:0/762805632 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4015db0 con 0x55e07dca2680 2023-11-06T01:41:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.539+0000 7f8da2ffd700 1 -- 172.21.15.53:0/762805632 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8da40044e0 con 0x55e07dca2680 2023-11-06T01:41:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.539+0000 7f8da2ffd700 1 -- 172.21.15.53:0/762805632 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4017410 con 0x55e07dca2680 2023-11-06T01:41:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.540+0000 7f8dcb012b80 1 -- 172.21.15.53:0/762805632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 msgr2=0x55e07dca2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.540+0000 7f8dbd41f700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8da80a0520 0x7f8da80a29e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8db00062a0 tx=0x7f8db0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.540+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/762805632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca2aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8da4015590 tx=0x7f8da400c190 comp rx=0 tx=0).stop 2023-11-06T01:41:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.541+0000 7f8dcb012b80 1 -- 172.21.15.53:0/762805632 shutdown_connections 2023-11-06T01:41:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.541+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/762805632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca3ed0 0x55e07dca4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.541+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/762805632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca2aa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.541+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/762805632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2fe0 0x55e07dca38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.541+0000 7f8dcb012b80 1 -- 172.21.15.53:0/762805632 >> 172.21.15.53:0/762805632 conn(0x55e07dc02070 msgr2=0x55e07dc9f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.541+0000 7f8dcb012b80 1 -- 172.21.15.53:0/762805632 shutdown_connections 2023-11-06T01:41:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.541+0000 7f8dcb012b80 1 -- 172.21.15.53:0/762805632 wait complete. 2023-11-06T01:41:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dcb012b80 1 -- start start 2023-11-06T01:41:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dcb012b80 1 Processor -- start 2023-11-06T01:41:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2680 0x55e07dcac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2fe0 0x55e07dcac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca3ed0 0x55e07dcb3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07db904b0 con 0x55e07dca2680 2023-11-06T01:41:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07db90150 con 0x55e07dca3ed0 2023-11-06T01:41:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07da778c0 con 0x55e07dca2fe0 2023-11-06T01:41:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dcb012b80 10 client.0 _ll_register_callbacks cb 0x55e07dadda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dbcc1e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2fe0 0x55e07dcac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dbcc1e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2fe0 0x55e07dcac9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56136/0 (socket says 172.21.15.53:56136) 2023-11-06T01:41:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8dbcc1e700 1 -- 172.21.15.53:0/3926320182 learned_addr learned my addr 172.21.15.53:0/3926320182 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.542+0000 7f8db7fff700 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca3ed0 0x55e07dcb3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8dbcc1e700 1 -- 172.21.15.53:0/3926320182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca3ed0 msgr2=0x55e07dcb3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8dbcc1e700 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca3ed0 0x55e07dcb3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8dbcc1e700 1 -- 172.21.15.53:0/3926320182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2680 msgr2=0x55e07dcac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8dbcc1e700 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2680 0x55e07dcac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8dbcc1e700 1 -- 172.21.15.53:0/3926320182 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da4015300 con 0x55e07dca2fe0 2023-11-06T01:41:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8dbcc1e700 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2fe0 0x55e07dcac9e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00beb0 tx=0x7f8dac00f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda820105b058c98 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8d97fff700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:41:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8d97fff700 1 -- 172.21.15.53:0/3926320182 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac01a460 con 0x55e07dca2fe0 2023-11-06T01:41:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8d97fff700 1 -- 172.21.15.53:0/3926320182 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8dac011440 con 0x55e07dca2fe0 2023-11-06T01:41:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3926320182 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e07dc1ecb0 con 0x55e07dca2fe0 2023-11-06T01:41:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8d97fff700 1 -- 172.21.15.53:0/3926320182 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dac011600 con 0x55e07dca2fe0 2023-11-06T01:41:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8dcb012b80 10 client.25888 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.543+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3926320182 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55e07da0ac00 con 0x55e07dca2fe0 2023-11-06T01:41:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.544+0000 7f8d97fff700 1 -- 172.21.15.53:0/3926320182 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e07dc1ecb0 con 0x55e07dca2fe0 2023-11-06T01:41:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.544+0000 7f8d97fff700 1 -- 172.21.15.53:0/3926320182 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8dac01ab20 con 0x55e07dca2fe0 2023-11-06T01:41:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.544+0000 7f8dcb012b80 10 client.25888 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.544+0000 7f8dcb012b80 10 client.25888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.544+0000 7f8dcb012b80 20 client.25888 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.544+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e07dcb83e0 0x55e07dcba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.544+0000 7f8dcb012b80 4 client.25888 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:41:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.544+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3926320182 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e07dc71db0 con 0x55e07dcb83e0 2023-11-06T01:41:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.545+0000 7f8db7fff700 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e07dcb83e0 0x55e07dcba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.547+0000 7f8d97fff700 10 client.25888 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.547+0000 7f8db7fff700 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e07dcb83e0 0x55e07dcba800 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7889cf205cd544c8 server_cookie=5f92ea4e4e5f4ed3 in_seq=0 out_seq=0 2023-11-06T01:41:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.547+0000 7f8d97fff700 1 -- 172.21.15.53:0/3926320182 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e07dc71db0 con 0x55e07dcb83e0 2023-11-06T01:41:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.547+0000 7f8d97fff700 10 client.25888 handle_command_reply: tid=0 2023-11-06T01:41:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.547+0000 7f8dcb012b80 1 client.25888 shutdown 2023-11-06T01:41:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3926320182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2fe0 msgr2=0x55e07dcac9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2fe0 0x55e07dcac9e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00beb0 tx=0x7f8dac00f640 comp rx=0 tx=0).stop 2023-11-06T01:41:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3926320182 shutdown_connections 2023-11-06T01:41:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca3ed0 0x55e07dcb3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e07dcb83e0 0x55e07dcba800 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2680 0x55e07dcac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3926320182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2fe0 0x55e07dcac9e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8d97fff700 0 client.25888 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3926320182 >> 172.21.15.53:0/3926320182 conn(0x55e07dc02070 msgr2=0x55e07dc9d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3926320182 shutdown_connections 2023-11-06T01:41:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3926320182 wait complete. 2023-11-06T01:41:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.548+0000 7f8dcb012b80 20 client.25888 trim_cache size 0 max 16384 2023-11-06T01:41:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.572+0000 7f8dcb012b80 1 Processor -- start 2023-11-06T01:41:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.573+0000 7f8dcb012b80 1 -- start start 2023-11-06T01:41:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.573+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.573+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca2fe0 0x55e07dcb39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.573+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca3ed0 0x55e07dca70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.573+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07dc98500 con 0x55e07dca3ed0 2023-11-06T01:41:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.573+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07dcac010 con 0x55e07dca2fe0 2023-11-06T01:41:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.573+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07dcb5690 con 0x55e07dca2680 2023-11-06T01:41:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.573+0000 7f8db7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.573+0000 7f8db7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56144/0 (socket says 172.21.15.53:56144) 2023-11-06T01:41:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.573+0000 7f8db7fff700 1 -- 172.21.15.53:0/153086262 learned_addr learned my addr 172.21.15.53:0/153086262 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.574+0000 7f8db7fff700 1 -- 172.21.15.53:0/153086262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca2fe0 msgr2=0x55e07dcb39e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:41:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.574+0000 7f8db7fff700 1 --2- 172.21.15.53:0/153086262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca2fe0 0x55e07dcb39e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.574+0000 7f8db7fff700 1 -- 172.21.15.53:0/153086262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca3ed0 msgr2=0x55e07dca70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.574+0000 7f8db7fff700 1 --2- 172.21.15.53:0/153086262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca3ed0 0x55e07dca70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.574+0000 7f8db7fff700 1 -- 172.21.15.53:0/153086262 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e07dc95ed0 con 0x55e07dca2680 2023-11-06T01:41:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.574+0000 7f8db7fff700 1 --2- 172.21.15.53:0/153086262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca42b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8da4015560 tx=0x7f8da4003b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf443b1dd3fcd5c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.574+0000 7f8da37fe700 1 -- 172.21.15.53:0/153086262 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4015d80 con 0x55e07dca2680 2023-11-06T01:41:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.574+0000 7f8da37fe700 1 -- 172.21.15.53:0/153086262 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8da40040a0 con 0x55e07dca2680 2023-11-06T01:41:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.574+0000 7f8da37fe700 1 -- 172.21.15.53:0/153086262 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da40476a0 con 0x55e07dca2680 2023-11-06T01:41:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.575+0000 7f8dcb012b80 1 -- 172.21.15.53:0/153086262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 msgr2=0x55e07dca42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.575+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/153086262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca42b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8da4015560 tx=0x7f8da4003b70 comp rx=0 tx=0).stop 2023-11-06T01:41:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.575+0000 7f8dcb012b80 1 -- 172.21.15.53:0/153086262 shutdown_connections 2023-11-06T01:41:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.575+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/153086262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca3ed0 0x55e07dca70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.575+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/153086262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca2680 0x55e07dca42b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.575+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/153086262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca2fe0 0x55e07dcb39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.575+0000 7f8dcb012b80 1 -- 172.21.15.53:0/153086262 >> 172.21.15.53:0/153086262 conn(0x55e07dc02070 msgr2=0x55e07dcaa390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.577+0000 7f8dcb012b80 1 -- 172.21.15.53:0/153086262 shutdown_connections 2023-11-06T01:41:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.577+0000 7f8dcb012b80 1 -- 172.21.15.53:0/153086262 wait complete. 2023-11-06T01:41:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.577+0000 7f8dcb012b80 1 -- start start 2023-11-06T01:41:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.577+0000 7f8dcb012b80 1 Processor -- start 2023-11-06T01:41:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca2680 0x55e07dcb19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2fe0 0x55e07dcb1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8dcb012b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca3ed0 0x55e07dcb2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07dcb3450 con 0x55e07dca2fe0 2023-11-06T01:41:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07dcb35c0 con 0x55e07dca2680 2023-11-06T01:41:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8dcb012b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07dcb3780 con 0x55e07dca3ed0 2023-11-06T01:41:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8dcb012b80 10 client.0 _ll_register_callbacks cb 0x7f8dac00b910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8db7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca2680 0x55e07dcb19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8db7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca2680 0x55e07dcb19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53346/0 (socket says 172.21.15.53:53346) 2023-11-06T01:41:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8db7fff700 1 -- 172.21.15.53:0/3888948214 learned_addr learned my addr 172.21.15.53:0/3888948214 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8db7fff700 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca3ed0 0x55e07dcb2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8dbcc1e700 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2fe0 0x55e07dcb1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8db7fff700 1 -- 172.21.15.53:0/3888948214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca3ed0 msgr2=0x55e07dcb2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8db7fff700 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca3ed0 0x55e07dcb2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8db7fff700 1 -- 172.21.15.53:0/3888948214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2fe0 msgr2=0x55e07dcb1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8db7fff700 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2fe0 0x55e07dcb1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.578+0000 7f8db7fff700 1 -- 172.21.15.53:0/3888948214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e07da21d80 con 0x55e07dca2680 2023-11-06T01:41:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.579+0000 7f8db7fff700 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca2680 0x55e07dcb19c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8da40039e0 tx=0x7f8da400cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36bb86f3c84499f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.579+0000 7f8d97fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:41:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.579+0000 7f8d97fff700 1 -- 172.21.15.53:0/3888948214 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400c040 con 0x55e07dca2680 2023-11-06T01:41:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.579+0000 7f8d97fff700 1 -- 172.21.15.53:0/3888948214 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8da4003be0 con 0x55e07dca2680 2023-11-06T01:41:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.579+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3888948214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8da40044e0 con 0x55e07dca2680 2023-11-06T01:41:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.579+0000 7f8d97fff700 1 -- 172.21.15.53:0/3888948214 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8da4003da0 con 0x55e07dca2680 2023-11-06T01:41:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.579+0000 7f8dcb012b80 10 client.25832 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.579+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3888948214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55e07da481f0 con 0x55e07dca2680 2023-11-06T01:41:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.579+0000 7f8d97fff700 1 -- 172.21.15.53:0/3888948214 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8da40044e0 con 0x55e07dca2680 2023-11-06T01:41:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.579+0000 7f8d97fff700 1 -- 172.21.15.53:0/3888948214 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8da4051440 con 0x55e07dca2680 2023-11-06T01:41:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.580+0000 7f8dcb012b80 10 client.25832 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.580+0000 7f8dcb012b80 10 client.25832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.580+0000 7f8dcb012b80 20 client.25832 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.580+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e07dd0ad30 0x55e07dd0d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.580+0000 7f8dcb012b80 4 client.25832 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:41:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.580+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3888948214 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e07dbecce0 con 0x55e07dd0ad30 2023-11-06T01:41:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.580+0000 7f8dbcc1e700 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e07dd0ad30 0x55e07dd0d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.583+0000 7f8d97fff700 10 client.25832 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.583+0000 7f8dbcc1e700 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e07dd0ad30 0x55e07dd0d150 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=214154620b112fbd server_cookie=bc6e34195c86ea58 in_seq=0 out_seq=0 2023-11-06T01:41:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.593+0000 7f8d97fff700 1 -- 172.21.15.53:0/3888948214 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e07dbecce0 con 0x55e07dd0ad30 2023-11-06T01:41:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.593+0000 7f8d97fff700 10 client.25832 handle_command_reply: tid=0 2023-11-06T01:41:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.593+0000 7f8dcb012b80 1 client.25832 shutdown 2023-11-06T01:41:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.594+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3888948214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca2680 msgr2=0x55e07dcb19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.594+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca2680 0x55e07dcb19c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8da40039e0 tx=0x7f8da400cca0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.594+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3888948214 shutdown_connections 2023-11-06T01:41:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.594+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e07dca2680 0x55e07dcb19c0 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8da40039e0 tx=0x7f8da400cca0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.594+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55e07dd0ad30 0x55e07dd0d150 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.594+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e07dca3ed0 0x55e07dcb2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.594+0000 7f8dcb012b80 1 --2- 172.21.15.53:0/3888948214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e07dca2fe0 0x55e07dcb1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.594+0000 7f8d97fff700 0 client.25832 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.594+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3888948214 >> 172.21.15.53:0/3888948214 conn(0x55e07dc02070 msgr2=0x55e07dcb4250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.595+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3888948214 shutdown_connections 2023-11-06T01:41:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.595+0000 7f8dcb012b80 1 -- 172.21.15.53:0/3888948214 wait complete. 2023-11-06T01:41:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.595+0000 7f8dcb012b80 20 client.25832 trim_cache size 0 max 16384 2023-11-06T01:41:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:41:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:41:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:41:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:41:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.595+0000 7f8dbee82700 1 -- 172.21.15.53:0/2622012044 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8da80a0520 msgr2=0x7f8da80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.595+0000 7f8dbee82700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8da80a0520 0x7f8da80a29e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8db00062a0 tx=0x7f8db0009040 comp rx=0 tx=0).stop 2023-11-06T01:41:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.595+0000 7f8dbee82700 1 -- 172.21.15.53:0/2622012044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8db8101860 msgr2=0x7f8db819e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.595+0000 7f8dbee82700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8db8101860 0x7f8db819e460 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009fd0 tx=0x7f8db00097e0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.596+0000 7f8dbee82700 1 -- 172.21.15.53:0/2622012044 shutdown_connections 2023-11-06T01:41:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.596+0000 7f8dbee82700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8da80a0520 0x7f8da80a29e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.596+0000 7f8dbee82700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8db8101860 0x7f8db819e460 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.596+0000 7f8dbee82700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8db80ffcd0 0x7f8db819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.596+0000 7f8dbee82700 1 --2- 172.21.15.53:0/2622012044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db8100ec0 0x7f8db819de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.596+0000 7f8dbee82700 1 -- 172.21.15.53:0/2622012044 >> 172.21.15.53:0/2622012044 conn(0x7f8db80fb240 msgr2=0x7f8db810d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.596+0000 7f8dbee82700 1 -- 172.21.15.53:0/2622012044 shutdown_connections 2023-11-06T01:41:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:03.596+0000 7f8dbee82700 1 -- 172.21.15.53:0/2622012044 wait complete. 2023-11-06T01:41:03.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:03 smithi053 ceph-mon[99472]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 1.97k op/s 2023-11-06T01:41:03.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:03 smithi062 ceph-mon[103924]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 1.97k op/s 2023-11-06T01:41:03.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:03 smithi163 ceph-mon[104416]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 1.97k op/s 2023-11-06T01:41:05.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:05 smithi053 ceph-mon[99472]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 MiB/s wr, 1.90k op/s 2023-11-06T01:41:05.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:05 smithi062 ceph-mon[103924]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 MiB/s wr, 1.90k op/s 2023-11-06T01:41:05.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:05 smithi163 ceph-mon[104416]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 MiB/s wr, 1.90k op/s 2023-11-06T01:41:07.184 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 410.0 s, 2070.6 tps, lat 48.600 ms stddev 56.065 2023-11-06T01:41:07.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:41:07.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.630+0000 7f0d88e8c700 1 -- 172.21.15.53:0/3272819759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d840ffd00 msgr2=0x7f0d84100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:07.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.630+0000 7f0d88e8c700 1 --2- 172.21.15.53:0/3272819759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d840ffd00 0x7f0d84100120 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009990 tx=0x7f0d74009ca0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.630+0000 7f0d88e8c700 1 -- 172.21.15.53:0/3272819759 shutdown_connections 2023-11-06T01:41:07.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.630+0000 7f0d88e8c700 1 --2- 172.21.15.53:0/3272819759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84101890 0x7f0d84108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.630+0000 7f0d88e8c700 1 --2- 172.21.15.53:0/3272819759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d840ffd00 0x7f0d84100120 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.630+0000 7f0d88e8c700 1 --2- 172.21.15.53:0/3272819759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d84100ef0 0x7f0d84101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.630+0000 7f0d88e8c700 1 -- 172.21.15.53:0/3272819759 >> 172.21.15.53:0/3272819759 conn(0x7f0d840fb290 msgr2=0x7f0d840fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:07.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.630+0000 7f0d88e8c700 1 -- 172.21.15.53:0/3272819759 shutdown_connections 2023-11-06T01:41:07.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.630+0000 7f0d88e8c700 1 -- 172.21.15.53:0/3272819759 wait complete. 2023-11-06T01:41:07.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d88e8c700 1 Processor -- start 2023-11-06T01:41:07.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d88e8c700 1 -- start start 2023-11-06T01:41:07.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d88e8c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d840ffd00 0x7f0d84196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d88e8c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84100ef0 0x7f0d84196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d88e8c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d84101890 0x7f0d8419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d88e8c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8419e610 con 0x7f0d84100ef0 2023-11-06T01:41:07.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d88e8c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8419e750 con 0x7f0d840ffd00 2023-11-06T01:41:07.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d88e8c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8419e8e0 con 0x7f0d84101890 2023-11-06T01:41:07.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d81d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84100ef0 0x7f0d84196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d8259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d840ffd00 0x7f0d84196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d8259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d840ffd00 0x7f0d84196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53360/0 (socket says 172.21.15.53:53360) 2023-11-06T01:41:07.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d82d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d84101890 0x7f0d8419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d8259c700 1 -- 172.21.15.53:0/619019796 learned_addr learned my addr 172.21.15.53:0/619019796 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:07.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d81d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84100ef0 0x7f0d84196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41924/0 (socket says 172.21.15.53:41924) 2023-11-06T01:41:07.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d81d9b700 1 -- 172.21.15.53:0/619019796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d84101890 msgr2=0x7f0d8419e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:07.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d81d9b700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d84101890 0x7f0d8419e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d81d9b700 1 -- 172.21.15.53:0/619019796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d840ffd00 msgr2=0x7f0d84196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:07.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d81d9b700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d840ffd00 0x7f0d84196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d81d9b700 1 -- 172.21.15.53:0/619019796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d740095f0 con 0x7f0d84100ef0 2023-11-06T01:41:07.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.631+0000 7f0d8259c700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d840ffd00 0x7f0d84196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:07.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.632+0000 7f0d81d9b700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84100ef0 0x7f0d84196fd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800e9f0 tx=0x7f0d7800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:07.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.632+0000 7f0d737fe700 1 -- 172.21.15.53:0/619019796 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800c9c0 con 0x7f0d84100ef0 2023-11-06T01:41:07.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.632+0000 7f0d88e8c700 1 -- 172.21.15.53:0/619019796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8419ebc0 con 0x7f0d84100ef0 2023-11-06T01:41:07.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.632+0000 7f0d88e8c700 1 -- 172.21.15.53:0/619019796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8419f120 con 0x7f0d84100ef0 2023-11-06T01:41:07.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.633+0000 7f0d737fe700 1 -- 172.21.15.53:0/619019796 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d78010430 con 0x7f0d84100ef0 2023-11-06T01:41:07.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.633+0000 7f0d737fe700 1 -- 172.21.15.53:0/619019796 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78018680 con 0x7f0d84100ef0 2023-11-06T01:41:07.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.637+0000 7f0d9501cb80 1 Processor -- start 2023-11-06T01:41:07.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.637+0000 7f0d9501cb80 1 -- start start 2023-11-06T01:41:07.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.637+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 0x55658a60c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.637+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60ca70 0x55658a60ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.637+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60b3f0 0x55658a60b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.637+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a562a10 con 0x55658a60ca70 2023-11-06T01:41:07.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.637+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a5603d0 con 0x55658a60c0f0 2023-11-06T01:41:07.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.637+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a55bcc0 con 0x55658a60b3f0 2023-11-06T01:41:07.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.638+0000 7f0d82d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 0x55658a60c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.638+0000 7f0d82d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 0x55658a60c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53366/0 (socket says 172.21.15.53:53366) 2023-11-06T01:41:07.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.638+0000 7f0d82d9d700 1 -- 172.21.15.53:0/1614955458 learned_addr learned my addr 172.21.15.53:0/1614955458 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:07.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.638+0000 7f0d8259c700 1 --2- 172.21.15.53:0/1614955458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60ca70 0x55658a60ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.638+0000 7f0d737fe700 1 -- 172.21.15.53:0/619019796 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f0d7800cb20 con 0x7f0d84100ef0 2023-11-06T01:41:07.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.638+0000 7f0d82d9d700 1 -- 172.21.15.53:0/1614955458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60b3f0 msgr2=0x55658a60b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:07.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.638+0000 7f0d82d9d700 1 --2- 172.21.15.53:0/1614955458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60b3f0 0x55658a60b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.638+0000 7f0d82d9d700 1 -- 172.21.15.53:0/1614955458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60ca70 msgr2=0x55658a60ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:07.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.638+0000 7f0d82d9d700 1 --2- 172.21.15.53:0/1614955458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60ca70 0x55658a60ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.638+0000 7f0d82d9d700 1 -- 172.21.15.53:0/1614955458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55658a389de0 con 0x55658a60c0f0 2023-11-06T01:41:07.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.639+0000 7f0d82d9d700 1 --2- 172.21.15.53:0/1614955458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 0x55658a60c530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c011680 tx=0x7f0d7c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e07a3cd1707e25c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:07.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.639+0000 7f0d70ff9700 1 -- 172.21.15.53:0/1614955458 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00e040 con 0x55658a60c0f0 2023-11-06T01:41:07.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.639+0000 7f0d70ff9700 1 -- 172.21.15.53:0/1614955458 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d7c00f040 con 0x55658a60c0f0 2023-11-06T01:41:07.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.639+0000 7f0d70ff9700 1 -- 172.21.15.53:0/1614955458 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0048c0 con 0x55658a60c0f0 2023-11-06T01:41:07.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.639+0000 7f0d737fe700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0d6c0a0620 0x7f0d6c0a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.639+0000 7f0d8259c700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0d6c0a0620 0x7f0d6c0a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.639+0000 7f0d737fe700 1 -- 172.21.15.53:0/619019796 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d78014070 con 0x7f0d84100ef0 2023-11-06T01:41:07.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d9501cb80 1 -- 172.21.15.53:0/1614955458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 msgr2=0x55658a60c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:07.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/1614955458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 0x55658a60c530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c011680 tx=0x7f0d7c009920 comp rx=0 tx=0).stop 2023-11-06T01:41:07.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d9501cb80 1 -- 172.21.15.53:0/1614955458 shutdown_connections 2023-11-06T01:41:07.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/1614955458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 0x55658a60c530 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/1614955458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60b3f0 0x55658a60b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/1614955458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60ca70 0x55658a60ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d9501cb80 1 -- 172.21.15.53:0/1614955458 >> 172.21.15.53:0/1614955458 conn(0x55658a56a0d0 msgr2=0x55658a605940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:07.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d8259c700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0d6c0a0620 0x7f0d6c0a2ae0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55658a60ce50 tx=0x7f0d74005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:07.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d9501cb80 1 -- 172.21.15.53:0/1614955458 shutdown_connections 2023-11-06T01:41:07.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d9501cb80 1 -- 172.21.15.53:0/1614955458 wait complete. 2023-11-06T01:41:07.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d9501cb80 1 -- start start 2023-11-06T01:41:07.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.640+0000 7f0d9501cb80 1 Processor -- start 2023-11-06T01:41:07.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a6259f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60c0f0 0x55658a625f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60ca70 0x55658a62cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a4f8510 con 0x55658a60b3f0 2023-11-06T01:41:07.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a4f81b0 con 0x55658a60ca70 2023-11-06T01:41:07.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a3df920 con 0x55658a60c0f0 2023-11-06T01:41:07.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d9501cb80 10 client.0 _ll_register_callbacks cb 0x55658a445a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:07.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d8259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a6259f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d8259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a6259f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41936/0 (socket says 172.21.15.53:41936) 2023-11-06T01:41:07.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d8259c700 1 -- 172.21.15.53:0/3568058710 learned_addr learned my addr 172.21.15.53:0/3568058710 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:07.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d81d9b700 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60c0f0 0x55658a625f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d82d9d700 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60ca70 0x55658a62cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d8259c700 1 -- 172.21.15.53:0/3568058710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60c0f0 msgr2=0x55658a625f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:07.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d8259c700 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60c0f0 0x55658a625f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d8259c700 1 -- 172.21.15.53:0/3568058710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60ca70 msgr2=0x55658a62cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:07.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d8259c700 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60ca70 0x55658a62cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d8259c700 1 -- 172.21.15.53:0/3568058710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c011330 con 0x55658a60b3f0 2023-11-06T01:41:07.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d8259c700 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a6259f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0d74004880 tx=0x7f0d74009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c291bab8d43fe3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:07.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:41:07.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3568058710 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74026870 con 0x55658a60b3f0 2023-11-06T01:41:07.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3568058710 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d740269d0 con 0x55658a60b3f0 2023-11-06T01:41:07.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.641+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3568058710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55658a586d10 con 0x55658a60b3f0 2023-11-06T01:41:07.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3568058710 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d74026b90 con 0x55658a60b3f0 2023-11-06T01:41:07.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d9501cb80 10 client.16266 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:07.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3568058710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55658a372c60 con 0x55658a60b3f0 2023-11-06T01:41:07.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3568058710 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55658a586d10 con 0x55658a60b3f0 2023-11-06T01:41:07.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3568058710 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0d74031430 con 0x55658a60b3f0 2023-11-06T01:41:07.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d9501cb80 10 client.16266 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:07.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d9501cb80 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:07.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d9501cb80 20 client.16266 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:07.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55658a631740 0x55658a633b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d9501cb80 4 client.16266 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:41:07.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3568058710 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55658a5d9e10 con 0x55658a631740 2023-11-06T01:41:07.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.642+0000 7f0d81d9b700 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55658a631740 0x55658a633b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.643+0000 7f0d81d9b700 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55658a631740 0x55658a633b60 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353774e6c2dab9a9 server_cookie=a3e58a15d605a005 in_seq=0 out_seq=0 2023-11-06T01:41:07.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.643+0000 7f0d69ffb700 10 client.16266 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:07.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3568058710 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55658a5d9e10 con 0x55658a631740 2023-11-06T01:41:07.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d69ffb700 10 client.16266 handle_command_reply: tid=0 2023-11-06T01:41:07.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d9501cb80 1 client.16266 shutdown 2023-11-06T01:41:07.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3568058710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 msgr2=0x55658a6259f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:07.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a6259f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0d74004880 tx=0x7f0d74009310 comp rx=0 tx=0).stop 2023-11-06T01:41:07.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3568058710 shutdown_connections 2023-11-06T01:41:07.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60ca70 0x55658a62cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55658a631740 0x55658a633b60 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a6259f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3568058710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60c0f0 0x55658a625f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3568058710 >> 172.21.15.53:0/3568058710 conn(0x55658a56a0d0 msgr2=0x55658a6050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:07.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d69ffb700 0 client.16266 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:07.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3568058710 shutdown_connections 2023-11-06T01:41:07.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.644+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3568058710 wait complete. 2023-11-06T01:41:07.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.645+0000 7f0d9501cb80 20 client.16266 trim_cache size 0 max 16384 2023-11-06T01:41:07.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d9501cb80 1 Processor -- start 2023-11-06T01:41:07.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d9501cb80 1 -- start start 2023-11-06T01:41:07.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60b3f0 0x55658a60ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60c0f0 0x55658a62aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60ca70 0x55658a6257a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a624510 con 0x55658a60c0f0 2023-11-06T01:41:07.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a631510 con 0x55658a60ca70 2023-11-06T01:41:07.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a607170 con 0x55658a60b3f0 2023-11-06T01:41:07.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d81d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60c0f0 0x55658a62aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d81d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60c0f0 0x55658a62aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41946/0 (socket says 172.21.15.53:41946) 2023-11-06T01:41:07.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d81d9b700 1 -- 172.21.15.53:0/3439989889 learned_addr learned my addr 172.21.15.53:0/3439989889 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:07.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d82d9d700 1 --2- 172.21.15.53:0/3439989889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60ca70 0x55658a6257a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.664+0000 7f0d8259c700 1 --2- 172.21.15.53:0/3439989889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60b3f0 0x55658a60ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.665+0000 7f0d81d9b700 1 -- 172.21.15.53:0/3439989889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60b3f0 msgr2=0x55658a60ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:07.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.665+0000 7f0d81d9b700 1 --2- 172.21.15.53:0/3439989889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60b3f0 0x55658a60ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.665+0000 7f0d81d9b700 1 -- 172.21.15.53:0/3439989889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60ca70 msgr2=0x55658a6257a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:07.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.665+0000 7f0d81d9b700 1 --2- 172.21.15.53:0/3439989889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60ca70 0x55658a6257a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.665+0000 7f0d81d9b700 1 -- 172.21.15.53:0/3439989889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55658a3b0250 con 0x55658a60c0f0 2023-11-06T01:41:07.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.665+0000 7f0d81d9b700 1 --2- 172.21.15.53:0/3439989889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60c0f0 0x55658a62aef0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0d78095bd0 tx=0x7f0d780cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f7e0607178f80d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:07.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.665+0000 7f0d717fa700 1 -- 172.21.15.53:0/3439989889 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78099860 con 0x55658a60c0f0 2023-11-06T01:41:07.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.665+0000 7f0d717fa700 1 -- 172.21.15.53:0/3439989889 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d78099160 con 0x55658a60c0f0 2023-11-06T01:41:07.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.665+0000 7f0d717fa700 1 -- 172.21.15.53:0/3439989889 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780c9510 con 0x55658a60c0f0 2023-11-06T01:41:07.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3439989889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60c0f0 msgr2=0x55658a62aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:07.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3439989889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60c0f0 0x55658a62aef0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0d78095bd0 tx=0x7f0d780cb040 comp rx=0 tx=0).stop 2023-11-06T01:41:07.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3439989889 shutdown_connections 2023-11-06T01:41:07.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3439989889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60ca70 0x55658a6257a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3439989889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60b3f0 0x55658a60ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3439989889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60c0f0 0x55658a62aef0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3439989889 >> 172.21.15.53:0/3439989889 conn(0x55658a56a0d0 msgr2=0x55658a612760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:07.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3439989889 shutdown_connections 2023-11-06T01:41:07.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3439989889 wait complete. 2023-11-06T01:41:07.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 -- start start 2023-11-06T01:41:07.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 Processor -- start 2023-11-06T01:41:07.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a62aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 0x55658a6290c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60ca70 0x55658a629600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a62b2d0 con 0x55658a60b3f0 2023-11-06T01:41:07.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a629b70 con 0x55658a60c0f0 2023-11-06T01:41:07.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658a629d30 con 0x55658a60ca70 2023-11-06T01:41:07.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.666+0000 7f0d9501cb80 10 client.0 _ll_register_callbacks cb 0x7f0d74005730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:07.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d8259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a62aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d82d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60ca70 0x55658a629600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d8259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a62aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41950/0 (socket says 172.21.15.53:41950) 2023-11-06T01:41:07.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d8259c700 1 -- 172.21.15.53:0/3966460454 learned_addr learned my addr 172.21.15.53:0/3966460454 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:07.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d81d9b700 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 0x55658a6290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d82d9d700 1 -- 172.21.15.53:0/3966460454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 msgr2=0x55658a6290c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:07.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d82d9d700 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 0x55658a6290c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d82d9d700 1 -- 172.21.15.53:0/3966460454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 msgr2=0x55658a62aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:07.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d82d9d700 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a62aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d82d9d700 1 -- 172.21.15.53:0/3966460454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8419f120 con 0x55658a60ca70 2023-11-06T01:41:07.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d8259c700 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a62aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:07.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d81d9b700 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 0x55658a6290c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:07.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d82d9d700 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60ca70 0x55658a629600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c000f80 tx=0x7f0d7c007730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c64ec1641fbd6ecf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:41:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3966460454 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00e040 con 0x55658a60ca70 2023-11-06T01:41:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3966460454 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d7c009390 con 0x55658a60ca70 2023-11-06T01:41:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3966460454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d7c00f040 con 0x55658a60ca70 2023-11-06T01:41:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3966460454 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d7c009ac0 con 0x55658a60ca70 2023-11-06T01:41:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.667+0000 7f0d9501cb80 10 client.25909 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.668+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3966460454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55658a3882b0 con 0x55658a60ca70 2023-11-06T01:41:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.668+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3966460454 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d7c00f040 con 0x55658a60ca70 2023-11-06T01:41:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.668+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3966460454 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0d7c031050 con 0x55658a60ca70 2023-11-06T01:41:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.668+0000 7f0d9501cb80 10 client.25909 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:07.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.668+0000 7f0d9501cb80 10 client.25909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:07.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.668+0000 7f0d9501cb80 20 client.25909 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:07.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.668+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55658a6848d0 0x55658a686cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:07.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.668+0000 7f0d9501cb80 4 client.25909 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:41:07.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.668+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3966460454 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55658a554d40 con 0x55658a6848d0 2023-11-06T01:41:07.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.668+0000 7f0d8259c700 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55658a6848d0 0x55658a686cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:07.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.669+0000 7f0d8259c700 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55658a6848d0 0x55658a686cf0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a02cf092cc1b3356 server_cookie=ea6d08a80ef63b29 in_seq=0 out_seq=0 2023-11-06T01:41:07.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.669+0000 7f0d69ffb700 10 client.25909 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:07.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.670+0000 7f0d69ffb700 1 -- 172.21.15.53:0/3966460454 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55658a554d40 con 0x55658a6848d0 2023-11-06T01:41:07.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.670+0000 7f0d69ffb700 10 client.25909 handle_command_reply: tid=0 2023-11-06T01:41:07.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.670+0000 7f0d9501cb80 1 client.25909 shutdown 2023-11-06T01:41:07.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.670+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3966460454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60ca70 msgr2=0x55658a629600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:07.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.670+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60ca70 0x55658a629600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c000f80 tx=0x7f0d7c007730 comp rx=0 tx=0).stop 2023-11-06T01:41:07.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3966460454 shutdown_connections 2023-11-06T01:41:07.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55658a6848d0 0x55658a686cf0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55658a60ca70 0x55658a629600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55658a60b3f0 0x55658a62aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d9501cb80 1 --2- 172.21.15.53:0/3966460454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55658a60c0f0 0x55658a6290c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3966460454 >> 172.21.15.53:0/3966460454 conn(0x55658a56a0d0 msgr2=0x55658a6257a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d69ffb700 0 client.25909 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3966460454 shutdown_connections 2023-11-06T01:41:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d9501cb80 1 -- 172.21.15.53:0/3966460454 wait complete. 2023-11-06T01:41:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d9501cb80 20 client.25909 trim_cache size 0 max 16384 2023-11-06T01:41:07.690 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:41:07.691 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:41:07.691 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:41:07.691 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:41:07.691 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:41:07.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:41:07.692 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:41:07.692 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:41:07.692 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:41:07.692 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:41:07.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d88e8c700 1 -- 172.21.15.53:0/619019796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0d6c0a0620 msgr2=0x7f0d6c0a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:07.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d88e8c700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0d6c0a0620 0x7f0d6c0a2ae0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55658a60ce50 tx=0x7f0d74005fb0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d88e8c700 1 -- 172.21.15.53:0/619019796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84100ef0 msgr2=0x7f0d84196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:07.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d88e8c700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84100ef0 0x7f0d84196fd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800e9f0 tx=0x7f0d7800ed00 comp rx=0 tx=0).stop 2023-11-06T01:41:07.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d88e8c700 1 -- 172.21.15.53:0/619019796 shutdown_connections 2023-11-06T01:41:07.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d88e8c700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f0d6c0a0620 0x7f0d6c0a2ae0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d88e8c700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d840ffd00 0x7f0d84196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d88e8c700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d84101890 0x7f0d8419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d88e8c700 1 --2- 172.21.15.53:0/619019796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84100ef0 0x7f0d84196fd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:07.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.671+0000 7f0d88e8c700 1 -- 172.21.15.53:0/619019796 >> 172.21.15.53:0/619019796 conn(0x7f0d840fb290 msgr2=0x7f0d840fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:07.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.672+0000 7f0d88e8c700 1 -- 172.21.15.53:0/619019796 shutdown_connections 2023-11-06T01:41:07.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:07.672+0000 7f0d88e8c700 1 -- 172.21.15.53:0/619019796 wait complete. 2023-11-06T01:41:07.696 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:41:07.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:07 smithi053 ceph-mon[99472]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2023-11-06T01:41:07.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:07 smithi062 ceph-mon[103924]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2023-11-06T01:41:07.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:07 smithi163 ceph-mon[104416]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2023-11-06T01:41:09.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:09 smithi053 ceph-mon[99472]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 1.85k op/s 2023-11-06T01:41:09.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:09 smithi062 ceph-mon[103924]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 1.85k op/s 2023-11-06T01:41:09.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:09 smithi163 ceph-mon[104416]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 1.85k op/s 2023-11-06T01:41:11.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:11 smithi053 ceph-mon[99472]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 1.11k op/s 2023-11-06T01:41:11.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:11 smithi062 ceph-mon[103924]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 1.11k op/s 2023-11-06T01:41:11.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:11 smithi163 ceph-mon[104416]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 1.11k op/s 2023-11-06T01:41:12.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 415.0 s, 2029.0 tps, lat 49.114 ms stddev 57.628 2023-11-06T01:41:13.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:13 smithi053 ceph-mon[99472]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 903 op/s 2023-11-06T01:41:13.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:13 smithi062 ceph-mon[103924]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 903 op/s 2023-11-06T01:41:13.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:13 smithi163 ceph-mon[104416]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 903 op/s 2023-11-06T01:41:15.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:15 smithi062 ceph-mon[103924]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 656 op/s 2023-11-06T01:41:15.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:15 smithi163 ceph-mon[104416]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 656 op/s 2023-11-06T01:41:16.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:15 smithi053 ceph-mon[99472]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 656 op/s 2023-11-06T01:41:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.746+0000 7fde463c3700 1 -- 172.21.15.53:0/844232242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde40102fd0 msgr2=0x7fde401033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.746+0000 7fde463c3700 1 --2- 172.21.15.53:0/844232242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde40102fd0 0x7fde401033f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fde300099c0 tx=0x7fde30009cd0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.747+0000 7fde463c3700 1 -- 172.21.15.53:0/844232242 shutdown_connections 2023-11-06T01:41:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.747+0000 7fde463c3700 1 --2- 172.21.15.53:0/844232242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde40104b60 0x7fde4010b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.747+0000 7fde463c3700 1 --2- 172.21.15.53:0/844232242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde40102fd0 0x7fde401033f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.747+0000 7fde463c3700 1 --2- 172.21.15.53:0/844232242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde401041c0 0x7fde40104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.747+0000 7fde463c3700 1 -- 172.21.15.53:0/844232242 >> 172.21.15.53:0/844232242 conn(0x7fde400fe540 msgr2=0x7fde40100980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.747+0000 7fde463c3700 1 -- 172.21.15.53:0/844232242 shutdown_connections 2023-11-06T01:41:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.747+0000 7fde463c3700 1 -- 172.21.15.53:0/844232242 wait complete. 2023-11-06T01:41:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde463c3700 1 Processor -- start 2023-11-06T01:41:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde463c3700 1 -- start start 2023-11-06T01:41:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde463c3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde40102fd0 0x7fde4006f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde463c3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde401041c0 0x7fde4006fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde463c3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde40104b60 0x7fde40070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde463c3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40071150 con 0x7fde401041c0 2023-11-06T01:41:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde463c3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde400712c0 con 0x7fde40102fd0 2023-11-06T01:41:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde463c3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40071480 con 0x7fde40104b60 2023-11-06T01:41:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde3ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde40102fd0 0x7fde4006f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde3f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde401041c0 0x7fde4006fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde44960700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde40104b60 0x7fde40070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde3ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde40102fd0 0x7fde4006f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53402/0 (socket says 172.21.15.53:53402) 2023-11-06T01:41:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.748+0000 7fde3ffff700 1 -- 172.21.15.53:0/3581556142 learned_addr learned my addr 172.21.15.53:0/3581556142 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde44960700 1 -- 172.21.15.53:0/3581556142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde40102fd0 msgr2=0x7fde4006f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde44960700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde40102fd0 0x7fde4006f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde44960700 1 -- 172.21.15.53:0/3581556142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde401041c0 msgr2=0x7fde4006fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde44960700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde401041c0 0x7fde4006fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde44960700 1 -- 172.21.15.53:0/3581556142 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde300095f0 con 0x7fde40104b60 2023-11-06T01:41:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde401041c0 0x7fde4006fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde3ffff700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde40102fd0 0x7fde4006f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde44960700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde40104b60 0x7fde40070af0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fde3800efc0 tx=0x7fde3800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde3d7fa700 1 -- 172.21.15.53:0/3581556142 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde38018070 con 0x7fde40104b60 2023-11-06T01:41:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde463c3700 1 -- 172.21.15.53:0/3581556142 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde40071700 con 0x7fde40104b60 2023-11-06T01:41:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde3d7fa700 1 -- 172.21.15.53:0/3581556142 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde3800f040 con 0x7fde40104b60 2023-11-06T01:41:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde3d7fa700 1 -- 172.21.15.53:0/3581556142 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3800cd40 con 0x7fde40104b60 2023-11-06T01:41:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.749+0000 7fde463c3700 1 -- 172.21.15.53:0/3581556142 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde40071f10 con 0x7fde40104b60 2023-11-06T01:41:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.754+0000 7fde52553b80 1 Processor -- start 2023-11-06T01:41:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.755+0000 7fde3d7fa700 1 -- 172.21.15.53:0/3581556142 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fde38003a00 con 0x7fde40104b60 2023-11-06T01:41:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde52553b80 1 -- start start 2023-11-06T01:41:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde52553b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f5e90 0x56479f8f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde52553b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f8f7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde52553b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f76e0 0x56479f8f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde52553b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f8459b0 con 0x56479f8f67f0 2023-11-06T01:41:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde52553b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f843370 con 0x56479f8f5e90 2023-11-06T01:41:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde52553b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f83ec60 con 0x56479f8f76e0 2023-11-06T01:41:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde3ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f8f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde3ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f8f7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41968/0 (socket says 172.21.15.53:41968) 2023-11-06T01:41:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde3ffff700 1 -- 172.21.15.53:0/3375786667 learned_addr learned my addr 172.21.15.53:0/3375786667 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/3375786667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f5e90 0x56479f8f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.756+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/3375786667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f76e0 0x56479f8f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde3d7fa700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde280a0520 0x7fde280a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde3ffff700 1 -- 172.21.15.53:0/3375786667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f76e0 msgr2=0x56479f8f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde3ffff700 1 --2- 172.21.15.53:0/3375786667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f76e0 0x56479f8f7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde3ffff700 1 -- 172.21.15.53:0/3375786667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f5e90 msgr2=0x56479f8f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde3ffff700 1 --2- 172.21.15.53:0/3375786667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f5e90 0x56479f8f62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde3ffff700 1 -- 172.21.15.53:0/3375786667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56479f66cd80 con 0x56479f8f67f0 2023-11-06T01:41:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde3d7fa700 1 -- 172.21.15.53:0/3581556142 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fde380cb590 con 0x7fde40104b60 2023-11-06T01:41:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde44960700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde280a0520 0x7fde280a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde3ffff700 1 --2- 172.21.15.53:0/3375786667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f8f7100 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fde300058e0 tx=0x7fde30005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5d531ae0844748 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde44960700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde280a0520 0x7fde280a29e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fde3800ef90 tx=0x7fde380092e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde267fc700 1 -- 172.21.15.53:0/3375786667 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3000bd40 con 0x56479f8f67f0 2023-11-06T01:41:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde267fc700 1 -- 172.21.15.53:0/3375786667 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde3001e040 con 0x56479f8f67f0 2023-11-06T01:41:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.757+0000 7fde267fc700 1 -- 172.21.15.53:0/3375786667 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde300056a0 con 0x56479f8f67f0 2023-11-06T01:41:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.758+0000 7fde52553b80 1 -- 172.21.15.53:0/3375786667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 msgr2=0x56479f8f7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.758+0000 7fde52553b80 1 --2- 172.21.15.53:0/3375786667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f8f7100 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fde300058e0 tx=0x7fde30005fb0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.758+0000 7fde52553b80 1 -- 172.21.15.53:0/3375786667 shutdown_connections 2023-11-06T01:41:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.758+0000 7fde52553b80 1 --2- 172.21.15.53:0/3375786667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f5e90 0x56479f8f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.758+0000 7fde52553b80 1 --2- 172.21.15.53:0/3375786667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f76e0 0x56479f8f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.758+0000 7fde52553b80 1 --2- 172.21.15.53:0/3375786667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f8f7100 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.758+0000 7fde52553b80 1 -- 172.21.15.53:0/3375786667 >> 172.21.15.53:0/3375786667 conn(0x56479f84d070 msgr2=0x56479f8ea040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.758+0000 7fde52553b80 1 -- 172.21.15.53:0/3375786667 shutdown_connections 2023-11-06T01:41:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.758+0000 7fde52553b80 1 -- 172.21.15.53:0/3375786667 wait complete. 2023-11-06T01:41:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde52553b80 1 -- start start 2023-11-06T01:41:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde52553b80 1 Processor -- start 2023-11-06T01:41:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde52553b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f8ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde52553b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f900180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde52553b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 0x56479f9071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde52553b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f7db4b0 con 0x56479f8f67f0 2023-11-06T01:41:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde52553b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f7db150 con 0x56479f8f76e0 2023-11-06T01:41:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde52553b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f6c28c0 con 0x56479f8f5e90 2023-11-06T01:41:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde52553b80 10 client.0 _ll_register_callbacks cb 0x56479f728a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde3f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f8ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde3ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f900180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde3f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f8ffc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56216/0 (socket says 172.21.15.53:56216) 2023-11-06T01:41:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde3f7fe700 1 -- 172.21.15.53:0/1865248030 learned_addr learned my addr 172.21.15.53:0/1865248030 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.759+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 0x56479f9071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.760+0000 7fde3f7fe700 1 -- 172.21.15.53:0/1865248030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 msgr2=0x56479f9071c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.760+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 0x56479f9071c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.760+0000 7fde3f7fe700 1 -- 172.21.15.53:0/1865248030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 msgr2=0x56479f900180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.760+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f900180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.760+0000 7fde3f7fe700 1 -- 172.21.15.53:0/1865248030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4010bd60 con 0x56479f8f5e90 2023-11-06T01:41:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.760+0000 7fde3ffff700 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f900180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.760+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f8ffc40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fde34014fc0 tx=0x7fde34013460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb40c1202391459 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.760+0000 7fde1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:41:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde1f7fe700 1 -- 172.21.15.53:0/1865248030 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34010e10 con 0x56479f8f5e90 2023-11-06T01:41:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde1f7fe700 1 -- 172.21.15.53:0/1865248030 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fde34013110 con 0x56479f8f5e90 2023-11-06T01:41:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde52553b80 1 -- 172.21.15.53:0/1865248030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56479f869cb0 con 0x56479f8f5e90 2023-11-06T01:41:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde1f7fe700 1 -- 172.21.15.53:0/1865248030 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde340132d0 con 0x56479f8f5e90 2023-11-06T01:41:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde52553b80 10 client.25921 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde52553b80 1 -- 172.21.15.53:0/1865248030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56479f655c00 con 0x56479f8f5e90 2023-11-06T01:41:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde1f7fe700 1 -- 172.21.15.53:0/1865248030 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56479f869cb0 con 0x56479f8f5e90 2023-11-06T01:41:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde1f7fe700 1 -- 172.21.15.53:0/1865248030 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fde34005020 con 0x56479f8f5e90 2023-11-06T01:41:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde52553b80 10 client.25921 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde52553b80 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde52553b80 20 client.25921 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.761+0000 7fde52553b80 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56479f90bad0 0x56479f90def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.762+0000 7fde52553b80 4 client.25921 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:41:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.762+0000 7fde52553b80 1 -- 172.21.15.53:0/1865248030 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56479f8bcdb0 con 0x56479f90bad0 2023-11-06T01:41:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.762+0000 7fde3ffff700 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56479f90bad0 0x56479f90def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.762+0000 7fde1f7fe700 10 client.25921 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.762+0000 7fde3ffff700 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56479f90bad0 0x56479f90def0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e062099607423fa8 server_cookie=ad6998022951b5a5 in_seq=0 out_seq=0 2023-11-06T01:41:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde1f7fe700 1 -- 172.21.15.53:0/1865248030 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56479f8bcdb0 con 0x56479f90bad0 2023-11-06T01:41:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde1f7fe700 10 client.25921 handle_command_reply: tid=0 2023-11-06T01:41:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 1 client.25921 shutdown 2023-11-06T01:41:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 1 -- 172.21.15.53:0/1865248030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 msgr2=0x56479f8ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f8ffc40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fde34014fc0 tx=0x7fde34013460 comp rx=0 tx=0).stop 2023-11-06T01:41:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 1 -- 172.21.15.53:0/1865248030 shutdown_connections 2023-11-06T01:41:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 0x56479f9071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56479f90bad0 0x56479f90def0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f8ffc40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 1 --2- 172.21.15.53:0/1865248030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f900180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde1f7fe700 0 client.25921 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 1 -- 172.21.15.53:0/1865248030 >> 172.21.15.53:0/1865248030 conn(0x56479f84d070 msgr2=0x56479f8e83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 1 -- 172.21.15.53:0/1865248030 shutdown_connections 2023-11-06T01:41:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 1 -- 172.21.15.53:0/1865248030 wait complete. 2023-11-06T01:41:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.763+0000 7fde52553b80 20 client.25921 trim_cache size 0 max 16384 2023-11-06T01:41:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.787+0000 7fde52553b80 1 Processor -- start 2023-11-06T01:41:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.789+0000 7fde52553b80 1 -- start start 2023-11-06T01:41:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.789+0000 7fde52553b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f5e90 0x56479f8f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.789+0000 7fde52553b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f67f0 0x56479f907180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.789+0000 7fde52553b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 0x56479f8ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.789+0000 7fde52553b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f900e30 con 0x56479f8f5e90 2023-11-06T01:41:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.789+0000 7fde52553b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f900c90 con 0x56479f8f76e0 2023-11-06T01:41:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.789+0000 7fde52553b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f8e07a0 con 0x56479f8f67f0 2023-11-06T01:41:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.793+0000 7fde3f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f5e90 0x56479f8f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.793+0000 7fde3f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f5e90 0x56479f8f7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41986/0 (socket says 172.21.15.53:41986) 2023-11-06T01:41:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.793+0000 7fde3f7fe700 1 -- 172.21.15.53:0/930481534 learned_addr learned my addr 172.21.15.53:0/930481534 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.793+0000 7fde3f7fe700 1 -- 172.21.15.53:0/930481534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f67f0 msgr2=0x56479f907180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:41:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.793+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/930481534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f67f0 0x56479f907180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.793+0000 7fde3f7fe700 1 -- 172.21.15.53:0/930481534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 msgr2=0x56479f8ff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.793+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/930481534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 0x56479f8ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.793+0000 7fde3f7fe700 1 -- 172.21.15.53:0/930481534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56479f6931f0 con 0x56479f8f5e90 2023-11-06T01:41:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.793+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/930481534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f5e90 0x56479f8f7ac0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fde34000c00 tx=0x7fde34005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e64a67d8a4c6451 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.794+0000 7fde26ffd700 1 -- 172.21.15.53:0/930481534 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34010e10 con 0x56479f8f5e90 2023-11-06T01:41:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.794+0000 7fde26ffd700 1 -- 172.21.15.53:0/930481534 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde340050e0 con 0x56479f8f5e90 2023-11-06T01:41:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.794+0000 7fde26ffd700 1 -- 172.21.15.53:0/930481534 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34014120 con 0x56479f8f5e90 2023-11-06T01:41:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.795+0000 7fde52553b80 1 -- 172.21.15.53:0/930481534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f5e90 msgr2=0x56479f8f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.795+0000 7fde52553b80 1 --2- 172.21.15.53:0/930481534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f5e90 0x56479f8f7ac0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fde34000c00 tx=0x7fde34005dc0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.801+0000 7fde52553b80 1 -- 172.21.15.53:0/930481534 shutdown_connections 2023-11-06T01:41:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.801+0000 7fde52553b80 1 --2- 172.21.15.53:0/930481534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 0x56479f8ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.801+0000 7fde52553b80 1 --2- 172.21.15.53:0/930481534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f5e90 0x56479f8f7ac0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.801+0000 7fde52553b80 1 --2- 172.21.15.53:0/930481534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f67f0 0x56479f907180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.801+0000 7fde52553b80 1 -- 172.21.15.53:0/930481534 >> 172.21.15.53:0/930481534 conn(0x56479f84d070 msgr2=0x56479f8e9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.801+0000 7fde52553b80 1 -- 172.21.15.53:0/930481534 shutdown_connections 2023-11-06T01:41:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.801+0000 7fde52553b80 1 -- 172.21.15.53:0/930481534 wait complete. 2023-11-06T01:41:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.801+0000 7fde52553b80 1 -- start start 2023-11-06T01:41:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.802+0000 7fde52553b80 1 Processor -- start 2023-11-06T01:41:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.802+0000 7fde52553b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f904ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.802+0000 7fde52553b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f905010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.802+0000 7fde52553b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 0x56479f905ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.802+0000 7fde52553b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f906530 con 0x56479f8f67f0 2023-11-06T01:41:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.802+0000 7fde52553b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f9066a0 con 0x56479f8f76e0 2023-11-06T01:41:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.802+0000 7fde52553b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479f906860 con 0x56479f8f5e90 2023-11-06T01:41:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.802+0000 7fde52553b80 10 client.0 _ll_register_callbacks cb 0x7fde34014f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.804+0000 7fde3f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f904ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.804+0000 7fde3f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f904ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56230/0 (socket says 172.21.15.53:56230) 2023-11-06T01:41:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.804+0000 7fde3f7fe700 1 -- 172.21.15.53:0/3733498128 learned_addr learned my addr 172.21.15.53:0/3733498128 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.805+0000 7fde3f7fe700 1 -- 172.21.15.53:0/3733498128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 msgr2=0x56479f905ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.805+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/3733498128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 0x56479f905ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.805+0000 7fde3f7fe700 1 -- 172.21.15.53:0/3733498128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 msgr2=0x56479f905010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:41:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.805+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/3733498128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f905010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.805+0000 7fde3f7fe700 1 -- 172.21.15.53:0/3733498128 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56479f655c00 con 0x56479f8f5e90 2023-11-06T01:41:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.816+0000 7fde3f7fe700 1 --2- 172.21.15.53:0/3733498128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f904ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fde34014000 tx=0x7fde34009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=851b4a8c710bc631 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.816+0000 7fde1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:41:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.820+0000 7fde1f7fe700 1 -- 172.21.15.53:0/3733498128 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34013970 con 0x56479f8f5e90 2023-11-06T01:41:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.820+0000 7fde1f7fe700 1 -- 172.21.15.53:0/3733498128 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fde34013ad0 con 0x56479f8f5e90 2023-11-06T01:41:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.821+0000 7fde52553b80 1 -- 172.21.15.53:0/3733498128 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde3001e040 con 0x56479f8f5e90 2023-11-06T01:41:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.821+0000 7fde1f7fe700 1 -- 172.21.15.53:0/3733498128 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde34013c90 con 0x56479f8f5e90 2023-11-06T01:41:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.821+0000 7fde52553b80 10 client.25924 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.821+0000 7fde52553b80 1 -- 172.21.15.53:0/3733498128 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56479f66b250 con 0x56479f8f5e90 2023-11-06T01:41:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.821+0000 7fde1f7fe700 1 -- 172.21.15.53:0/3733498128 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde3001e040 con 0x56479f8f5e90 2023-11-06T01:41:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.821+0000 7fde1f7fe700 1 -- 172.21.15.53:0/3733498128 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fde340039c0 con 0x56479f8f5e90 2023-11-06T01:41:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.822+0000 7fde52553b80 10 client.25924 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.822+0000 7fde52553b80 10 client.25924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.822+0000 7fde52553b80 20 client.25924 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.822+0000 7fde52553b80 1 --2- 172.21.15.53:0/3733498128 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56479f8f3a80 0x56479f8f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.822+0000 7fde52553b80 4 client.25924 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:41:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.822+0000 7fde52553b80 1 -- 172.21.15.53:0/3733498128 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56479f837ce0 con 0x56479f8f3a80 2023-11-06T01:41:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.822+0000 7fde3ffff700 1 --2- 172.21.15.53:0/3733498128 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56479f8f3a80 0x56479f8f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.823+0000 7fde1f7fe700 10 client.25924 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.823+0000 7fde3ffff700 1 --2- 172.21.15.53:0/3733498128 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56479f8f3a80 0x56479f8f3e90 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6942a12c05bdceef server_cookie=58d4f0d624ccc339 in_seq=0 out_seq=0 2023-11-06T01:41:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde1f7fe700 1 -- 172.21.15.53:0/3733498128 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56479f837ce0 con 0x56479f8f3a80 2023-11-06T01:41:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde1f7fe700 10 client.25924 handle_command_reply: tid=0 2023-11-06T01:41:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde52553b80 1 client.25924 shutdown 2023-11-06T01:41:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde52553b80 1 -- 172.21.15.53:0/3733498128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 msgr2=0x56479f904ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde52553b80 1 --2- 172.21.15.53:0/3733498128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f904ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fde34014000 tx=0x7fde34009260 comp rx=0 tx=0).stop 2023-11-06T01:41:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde52553b80 1 -- 172.21.15.53:0/3733498128 shutdown_connections 2023-11-06T01:41:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde52553b80 1 --2- 172.21.15.53:0/3733498128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56479f8f76e0 0x56479f905ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde52553b80 1 --2- 172.21.15.53:0/3733498128 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56479f8f3a80 0x56479f8f3e90 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde52553b80 1 --2- 172.21.15.53:0/3733498128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56479f8f5e90 0x56479f904ad0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde1f7fe700 0 client.25924 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde52553b80 1 --2- 172.21.15.53:0/3733498128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56479f8f67f0 0x56479f905010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde52553b80 1 -- 172.21.15.53:0/3733498128 >> 172.21.15.53:0/3733498128 conn(0x56479f84d070 msgr2=0x56479f8fa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde52553b80 1 -- 172.21.15.53:0/3733498128 shutdown_connections 2023-11-06T01:41:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.832+0000 7fde52553b80 1 -- 172.21.15.53:0/3733498128 wait complete. 2023-11-06T01:41:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde52553b80 20 client.25924 trim_cache size 0 max 16384 2023-11-06T01:41:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:41:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:41:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:41:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:41:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 -- 172.21.15.53:0/3581556142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde280a0520 msgr2=0x7fde280a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde280a0520 0x7fde280a29e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fde3800ef90 tx=0x7fde380092e0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 -- 172.21.15.53:0/3581556142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde40104b60 msgr2=0x7fde40070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde40104b60 0x7fde40070af0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fde3800efc0 tx=0x7fde3800c3a0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 -- 172.21.15.53:0/3581556142 shutdown_connections 2023-11-06T01:41:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fde280a0520 0x7fde280a29e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde40102fd0 0x7fde4006f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde40104b60 0x7fde40070af0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 --2- 172.21.15.53:0/3581556142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde401041c0 0x7fde4006fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 -- 172.21.15.53:0/3581556142 >> 172.21.15.53:0/3581556142 conn(0x7fde400fe540 msgr2=0x7fde40107720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 -- 172.21.15.53:0/3581556142 shutdown_connections 2023-11-06T01:41:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:16.833+0000 7fde463c3700 1 -- 172.21.15.53:0/3581556142 wait complete. 2023-11-06T01:41:17.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 420.0 s, 1891.9 tps, lat 52.914 ms stddev 55.087 2023-11-06T01:41:17.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:17 smithi062 ceph-mon[103924]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 624 op/s 2023-11-06T01:41:17.998 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:17 smithi163 ceph-mon[104416]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 624 op/s 2023-11-06T01:41:18.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:17 smithi053 ceph-mon[99472]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 624 op/s 2023-11-06T01:41:19.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:19 smithi062 ceph-mon[103924]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 711 op/s 2023-11-06T01:41:19.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:19 smithi163 ceph-mon[104416]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 711 op/s 2023-11-06T01:41:20.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:19 smithi053 ceph-mon[99472]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 711 op/s 2023-11-06T01:41:20.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:20 smithi163 ceph-mon[104416]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 642 op/s 2023-11-06T01:41:21.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:20 smithi053 ceph-mon[99472]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 642 op/s 2023-11-06T01:41:21.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:20 smithi062 ceph-mon[103924]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 642 op/s 2023-11-06T01:41:22.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 425.0 s, 1794.0 tps, lat 55.678 ms stddev 58.693 2023-11-06T01:41:23.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:23 smithi163 ceph-mon[104416]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 741 op/s 2023-11-06T01:41:23.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:23 smithi053 ceph-mon[99472]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 741 op/s 2023-11-06T01:41:23.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:23 smithi062 ceph-mon[103924]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 741 op/s 2023-11-06T01:41:25.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:25 smithi163 ceph-mon[104416]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 639 op/s 2023-11-06T01:41:25.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:25 smithi053 ceph-mon[99472]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 639 op/s 2023-11-06T01:41:25.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:25 smithi062 ceph-mon[103924]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 639 op/s 2023-11-06T01:41:27.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 430.0 s, 1379.0 tps, lat 72.474 ms stddev 81.091 2023-11-06T01:41:27.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:27 smithi163 ceph-mon[104416]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 613 op/s 2023-11-06T01:41:27.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:27 smithi053 ceph-mon[99472]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 613 op/s 2023-11-06T01:41:27.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:27 smithi062 ceph-mon[103924]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 613 op/s 2023-11-06T01:41:29.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:29 smithi163 ceph-mon[104416]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 790 op/s 2023-11-06T01:41:29.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:29 smithi053 ceph-mon[99472]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 790 op/s 2023-11-06T01:41:29.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:29 smithi062 ceph-mon[103924]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 790 op/s 2023-11-06T01:41:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:29.999+0000 7f472efe7700 1 -- 172.21.15.53:0/3094213306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4728100e80 msgr2=0x7f47281012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:29.999+0000 7f472efe7700 1 --2- 172.21.15.53:0/3094213306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4728100e80 0x7f47281012e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f471c0099c0 tx=0x7f471c009cd0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:29.999+0000 7f472efe7700 1 -- 172.21.15.53:0/3094213306 shutdown_connections 2023-11-06T01:41:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:29.999+0000 7f472efe7700 1 --2- 172.21.15.53:0/3094213306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4728101820 0x7f47281080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:29.999+0000 7f472efe7700 1 --2- 172.21.15.53:0/3094213306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47280ffc90 0x7f47281000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:29.999+0000 7f472efe7700 1 --2- 172.21.15.53:0/3094213306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4728100e80 0x7f47281012e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:29.999+0000 7f472efe7700 1 -- 172.21.15.53:0/3094213306 >> 172.21.15.53:0/3094213306 conn(0x7f47280fb1e0 msgr2=0x7f47280fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:29.999+0000 7f472efe7700 1 -- 172.21.15.53:0/3094213306 shutdown_connections 2023-11-06T01:41:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.001+0000 7f472efe7700 1 -- 172.21.15.53:0/3094213306 wait complete. 2023-11-06T01:41:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.001+0000 7f472efe7700 1 Processor -- start 2023-11-06T01:41:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.001+0000 7f472efe7700 1 -- start start 2023-11-06T01:41:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.001+0000 7f472efe7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280ffc90 0x7f47281949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.001+0000 7f472efe7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4728100e80 0x7f4728194f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472efe7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4728101820 0x7f4728195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472efe7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4728196400 con 0x7f4728101820 2023-11-06T01:41:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472efe7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4728196570 con 0x7f47280ffc90 2023-11-06T01:41:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472efe7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4728196730 con 0x7f4728100e80 2023-11-06T01:41:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472cd83700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280ffc90 0x7f47281949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472d584700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4728101820 0x7f4728195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f4727fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4728100e80 0x7f4728194f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472cd83700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280ffc90 0x7f47281949e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53448/0 (socket says 172.21.15.53:53448) 2023-11-06T01:41:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f4727fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4728100e80 0x7f4728194f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56252/0 (socket says 172.21.15.53:56252) 2023-11-06T01:41:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472cd83700 1 -- 172.21.15.53:0/3240720961 learned_addr learned my addr 172.21.15.53:0/3240720961 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472d584700 1 -- 172.21.15.53:0/3240720961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4728100e80 msgr2=0x7f4728194f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472d584700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4728100e80 0x7f4728194f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472d584700 1 -- 172.21.15.53:0/3240720961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280ffc90 msgr2=0x7f47281949e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472d584700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280ffc90 0x7f47281949e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472d584700 1 -- 172.21.15.53:0/3240720961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c0095f0 con 0x7f4728101820 2023-11-06T01:41:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472cd83700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280ffc90 0x7f47281949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f472d584700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4728101820 0x7f4728195da0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4720011180 tx=0x7f4720011490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.002+0000 7f4725ffb700 1 -- 172.21.15.53:0/3240720961 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720004bb0 con 0x7f4728101820 2023-11-06T01:41:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.003+0000 7f472efe7700 1 -- 172.21.15.53:0/3240720961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4728196a10 con 0x7f4728101820 2023-11-06T01:41:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.003+0000 7f4725ffb700 1 -- 172.21.15.53:0/3240720961 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4720013040 con 0x7f4728101820 2023-11-06T01:41:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.003+0000 7f4725ffb700 1 -- 172.21.15.53:0/3240720961 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720019660 con 0x7f4728101820 2023-11-06T01:41:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.003+0000 7f472efe7700 1 -- 172.21.15.53:0/3240720961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472819d1d0 con 0x7f4728101820 2023-11-06T01:41:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.006+0000 7f4725ffb700 1 -- 172.21.15.53:0/3240720961 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4720005020 con 0x7f4728101820 2023-11-06T01:41:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.007+0000 7f4725ffb700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47100a0520 0x7f47100a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.007+0000 7f4725ffb700 1 -- 172.21.15.53:0/3240720961 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f47200cf5d0 con 0x7f4728101820 2023-11-06T01:41:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.007+0000 7f472cd83700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47100a0520 0x7f47100a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.008+0000 7f472cd83700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47100a0520 0x7f47100a29e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f47180105c0 tx=0x7f471800b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f473b177b80 1 Processor -- start 2023-11-06T01:41:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f473b177b80 1 -- start start 2023-11-06T01:41:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f473b177b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dc680 0x55a1596dcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f473b177b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dcfe0 0x55a1596dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f473b177b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596de350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f473b177b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1596349b0 con 0x55a1596dcfe0 2023-11-06T01:41:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f473b177b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a159632370 con 0x55a1596dded0 2023-11-06T01:41:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f473b177b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15962dc60 con 0x55a1596dc680 2023-11-06T01:41:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f472cd83700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dc680 0x55a1596dcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f472cd83700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dc680 0x55a1596dcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56258/0 (socket says 172.21.15.53:56258) 2023-11-06T01:41:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f472cd83700 1 -- 172.21.15.53:0/2494826966 learned_addr learned my addr 172.21.15.53:0/2494826966 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f4727fff700 1 --2- 172.21.15.53:0/2494826966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dcfe0 0x55a1596dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.010+0000 7f472d584700 1 --2- 172.21.15.53:0/2494826966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596de350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.011+0000 7f472cd83700 1 -- 172.21.15.53:0/2494826966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 msgr2=0x55a1596de350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.011+0000 7f472cd83700 1 --2- 172.21.15.53:0/2494826966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596de350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.011+0000 7f472cd83700 1 -- 172.21.15.53:0/2494826966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dcfe0 msgr2=0x55a1596dd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.011+0000 7f472cd83700 1 --2- 172.21.15.53:0/2494826966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dcfe0 0x55a1596dd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.011+0000 7f472cd83700 1 -- 172.21.15.53:0/2494826966 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a15945bd80 con 0x55a1596dc680 2023-11-06T01:41:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.011+0000 7f472cd83700 1 --2- 172.21.15.53:0/2494826966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dc680 0x55a1596dcaa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f471800bc60 tx=0x7f4718013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33046992814d337a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.011+0000 7f472d584700 1 --2- 172.21.15.53:0/2494826966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596de350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:41:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.011+0000 7f470effd700 1 -- 172.21.15.53:0/2494826966 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471801e620 con 0x55a1596dc680 2023-11-06T01:41:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.011+0000 7f470effd700 1 -- 172.21.15.53:0/2494826966 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4718015070 con 0x55a1596dc680 2023-11-06T01:41:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.011+0000 7f470effd700 1 -- 172.21.15.53:0/2494826966 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471801cae0 con 0x55a1596dc680 2023-11-06T01:41:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.012+0000 7f473b177b80 1 -- 172.21.15.53:0/2494826966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dc680 msgr2=0x55a1596dcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.012+0000 7f473b177b80 1 --2- 172.21.15.53:0/2494826966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dc680 0x55a1596dcaa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f471800bc60 tx=0x7f4718013000 comp rx=0 tx=0).stop 2023-11-06T01:41:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.012+0000 7f473b177b80 1 -- 172.21.15.53:0/2494826966 shutdown_connections 2023-11-06T01:41:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.012+0000 7f473b177b80 1 --2- 172.21.15.53:0/2494826966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596de350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.012+0000 7f473b177b80 1 --2- 172.21.15.53:0/2494826966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dc680 0x55a1596dcaa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.012+0000 7f473b177b80 1 --2- 172.21.15.53:0/2494826966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dcfe0 0x55a1596dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.012+0000 7f473b177b80 1 -- 172.21.15.53:0/2494826966 >> 172.21.15.53:0/2494826966 conn(0x55a15963c070 msgr2=0x55a1596d9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.012+0000 7f473b177b80 1 -- 172.21.15.53:0/2494826966 shutdown_connections 2023-11-06T01:41:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.012+0000 7f473b177b80 1 -- 172.21.15.53:0/2494826966 wait complete. 2023-11-06T01:41:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.012+0000 7f473b177b80 1 -- start start 2023-11-06T01:41:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.012+0000 7f473b177b80 1 Processor -- start 2023-11-06T01:41:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f473b177b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dc680 0x55a1596e6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f473b177b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 0x55a1596e6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f473b177b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596ed690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f473b177b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1595ca4b0 con 0x55a1596dc680 2023-11-06T01:41:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f473b177b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1595ca150 con 0x55a1596dded0 2023-11-06T01:41:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f473b177b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1594b18c0 con 0x55a1596dcfe0 2023-11-06T01:41:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f473b177b80 10 client.0 _ll_register_callbacks cb 0x55a159517a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f472cd83700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dc680 0x55a1596e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f4727fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 0x55a1596e6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f472cd83700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dc680 0x55a1596e6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42024/0 (socket says 172.21.15.53:42024) 2023-11-06T01:41:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f472cd83700 1 -- 172.21.15.53:0/4196347466 learned_addr learned my addr 172.21.15.53:0/4196347466 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f472d584700 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f4727fff700 1 -- 172.21.15.53:0/4196347466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 msgr2=0x55a1596ed690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f4727fff700 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596ed690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f4727fff700 1 -- 172.21.15.53:0/4196347466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dc680 msgr2=0x55a1596e6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f4727fff700 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dc680 0x55a1596e6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f4727fff700 1 -- 172.21.15.53:0/4196347466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47180114e0 con 0x55a1596dcfe0 2023-11-06T01:41:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.013+0000 7f472cd83700 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dc680 0x55a1596e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.014+0000 7f4727fff700 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 0x55a1596e6650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f471c0098e0 tx=0x7f471c00dc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69d1f4b506eb101c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.014+0000 7f4707fff700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:41:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.014+0000 7f4707fff700 1 -- 172.21.15.53:0/4196347466 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c01b040 con 0x55a1596dcfe0 2023-11-06T01:41:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.014+0000 7f4707fff700 1 -- 172.21.15.53:0/4196347466 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f471c00d220 con 0x55a1596dcfe0 2023-11-06T01:41:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.014+0000 7f473b177b80 1 -- 172.21.15.53:0/4196347466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a159658cb0 con 0x55a1596dcfe0 2023-11-06T01:41:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.014+0000 7f4707fff700 1 -- 172.21.15.53:0/4196347466 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f471c005310 con 0x55a1596dcfe0 2023-11-06T01:41:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.014+0000 7f473b177b80 10 client.25936 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.014+0000 7f473b177b80 1 -- 172.21.15.53:0/4196347466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55a159444c00 con 0x55a1596dcfe0 2023-11-06T01:41:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.014+0000 7f4707fff700 1 -- 172.21.15.53:0/4196347466 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a159658cb0 con 0x55a1596dcfe0 2023-11-06T01:41:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.015+0000 7f4707fff700 1 -- 172.21.15.53:0/4196347466 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f471c028020 con 0x55a1596dcfe0 2023-11-06T01:41:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.015+0000 7f473b177b80 10 client.25936 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.015+0000 7f473b177b80 10 client.25936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.015+0000 7f473b177b80 20 client.25936 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.015+0000 7f473b177b80 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a1596f1fd0 0x55a1596f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.015+0000 7f473b177b80 4 client.25936 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:41:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.015+0000 7f473b177b80 1 -- 172.21.15.53:0/4196347466 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1596abdb0 con 0x55a1596f1fd0 2023-11-06T01:41:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.015+0000 7f472cd83700 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a1596f1fd0 0x55a1596f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.016+0000 7f4707fff700 10 client.25936 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.016+0000 7f472cd83700 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a1596f1fd0 0x55a1596f43f0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43fac8b9fb368018 server_cookie=3c344a8aa9436e3c in_seq=0 out_seq=0 2023-11-06T01:41:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.016+0000 7f4707fff700 1 -- 172.21.15.53:0/4196347466 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a1596abdb0 con 0x55a1596f1fd0 2023-11-06T01:41:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.016+0000 7f4707fff700 10 client.25936 handle_command_reply: tid=0 2023-11-06T01:41:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.016+0000 7f473b177b80 1 client.25936 shutdown 2023-11-06T01:41:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f473b177b80 1 -- 172.21.15.53:0/4196347466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 msgr2=0x55a1596e6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f473b177b80 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 0x55a1596e6650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f471c0098e0 tx=0x7f471c00dc40 comp rx=0 tx=0).stop 2023-11-06T01:41:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f473b177b80 1 -- 172.21.15.53:0/4196347466 shutdown_connections 2023-11-06T01:41:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f473b177b80 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596ed690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f473b177b80 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a1596f1fd0 0x55a1596f43f0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f473b177b80 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dc680 0x55a1596e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f473b177b80 1 --2- 172.21.15.53:0/4196347466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 0x55a1596e6650 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f473b177b80 1 -- 172.21.15.53:0/4196347466 >> 172.21.15.53:0/4196347466 conn(0x55a15963c070 msgr2=0x55a1596d73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f4707fff700 0 client.25936 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f473b177b80 1 -- 172.21.15.53:0/4196347466 shutdown_connections 2023-11-06T01:41:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f473b177b80 1 -- 172.21.15.53:0/4196347466 wait complete. 2023-11-06T01:41:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.017+0000 7f473b177b80 20 client.25936 trim_cache size 0 max 16384 2023-11-06T01:41:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.042+0000 7f473b177b80 1 Processor -- start 2023-11-06T01:41:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.043+0000 7f473b177b80 1 -- start start 2023-11-06T01:41:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.044+0000 7f473b177b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dc680 0x55a1596fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.044+0000 7f473b177b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 0x55a1596de2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.044+0000 7f473b177b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.044+0000 7f473b177b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1596d8880 con 0x55a1596dc680 2023-11-06T01:41:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.044+0000 7f473b177b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1596d2500 con 0x55a1596dded0 2023-11-06T01:41:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.044+0000 7f473b177b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1596cf7a0 con 0x55a1596dcfe0 2023-11-06T01:41:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.044+0000 7f4727fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 0x55a1596de2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.044+0000 7f472d584700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.044+0000 7f4727fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 0x55a1596de2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56272/0 (socket says 172.21.15.53:56272) 2023-11-06T01:41:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.044+0000 7f472d584700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596e10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53472/0 (socket says 172.21.15.53:53472) 2023-11-06T01:41:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.044+0000 7f4727fff700 1 -- 172.21.15.53:0/143494767 learned_addr learned my addr 172.21.15.53:0/143494767 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.045+0000 7f4727fff700 1 -- 172.21.15.53:0/143494767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 msgr2=0x55a1596e10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.045+0000 7f4727fff700 1 --2- 172.21.15.53:0/143494767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596e10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.045+0000 7f4727fff700 1 -- 172.21.15.53:0/143494767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dc680 msgr2=0x55a1596fa0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:41:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.045+0000 7f4727fff700 1 --2- 172.21.15.53:0/143494767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dc680 0x55a1596fa0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.045+0000 7f4727fff700 1 -- 172.21.15.53:0/143494767 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1594821f0 con 0x55a1596dcfe0 2023-11-06T01:41:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.045+0000 7f4727fff700 1 --2- 172.21.15.53:0/143494767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 0x55a1596de2b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f471c00a010 tx=0x7f471c005a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9be40a95c1583ccf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.045+0000 7f470f7fe700 1 -- 172.21.15.53:0/143494767 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c01b040 con 0x55a1596dcfe0 2023-11-06T01:41:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.045+0000 7f470f7fe700 1 -- 172.21.15.53:0/143494767 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f471c01c040 con 0x55a1596dcfe0 2023-11-06T01:41:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.045+0000 7f470f7fe700 1 -- 172.21.15.53:0/143494767 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c004710 con 0x55a1596dcfe0 2023-11-06T01:41:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.046+0000 7f473b177b80 1 -- 172.21.15.53:0/143494767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 msgr2=0x55a1596de2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.046+0000 7f473b177b80 1 --2- 172.21.15.53:0/143494767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 0x55a1596de2b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f471c00a010 tx=0x7f471c005a80 comp rx=0 tx=0).stop 2023-11-06T01:41:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.046+0000 7f473b177b80 1 -- 172.21.15.53:0/143494767 shutdown_connections 2023-11-06T01:41:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.046+0000 7f473b177b80 1 --2- 172.21.15.53:0/143494767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dded0 0x55a1596e10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.046+0000 7f473b177b80 1 --2- 172.21.15.53:0/143494767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dc680 0x55a1596fa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.046+0000 7f473b177b80 1 --2- 172.21.15.53:0/143494767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596dcfe0 0x55a1596de2b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.046+0000 7f473b177b80 1 -- 172.21.15.53:0/143494767 >> 172.21.15.53:0/143494767 conn(0x55a15963c070 msgr2=0x55a1596e4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.049+0000 7f473b177b80 1 -- 172.21.15.53:0/143494767 shutdown_connections 2023-11-06T01:41:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.051+0000 7f473b177b80 1 -- 172.21.15.53:0/143494767 wait complete. 2023-11-06T01:41:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.051+0000 7f473b177b80 1 -- start start 2023-11-06T01:41:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.051+0000 7f473b177b80 1 Processor -- start 2023-11-06T01:41:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.052+0000 7f473b177b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dcfe0 0x55a1596f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.052+0000 7f473b177b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dded0 0x55a1596f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.052+0000 7f473b177b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596f95a0 0x55a15974b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.052+0000 7f473b177b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15974bbd0 con 0x55a1596dded0 2023-11-06T01:41:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.052+0000 7f473b177b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15974bd40 con 0x55a1596dcfe0 2023-11-06T01:41:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.052+0000 7f473b177b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15974bf00 con 0x55a1596f95a0 2023-11-06T01:41:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.052+0000 7f473b177b80 10 client.0 _ll_register_callbacks cb 0x55a159517a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.052+0000 7f4727fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dded0 0x55a1596f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.052+0000 7f4727fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dded0 0x55a1596f9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42038/0 (socket says 172.21.15.53:42038) 2023-11-06T01:41:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.052+0000 7f4727fff700 1 -- 172.21.15.53:0/1781266894 learned_addr learned my addr 172.21.15.53:0/1781266894 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.053+0000 7f4727fff700 1 -- 172.21.15.53:0/1781266894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596f95a0 msgr2=0x55a15974b690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:41:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.053+0000 7f4727fff700 1 --2- 172.21.15.53:0/1781266894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596f95a0 0x55a15974b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.053+0000 7f4727fff700 1 -- 172.21.15.53:0/1781266894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dcfe0 msgr2=0x55a1596f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.053+0000 7f4727fff700 1 --2- 172.21.15.53:0/1781266894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dcfe0 0x55a1596f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.053+0000 7f4727fff700 1 -- 172.21.15.53:0/1781266894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a159444c00 con 0x55a1596dded0 2023-11-06T01:41:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.053+0000 7f4727fff700 1 --2- 172.21.15.53:0/1781266894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dded0 0x55a1596f9060 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f471c00d810 tx=0x7f471c003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c5930e4f5f644d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.053+0000 7f4707fff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:41:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.053+0000 7f4707fff700 1 -- 172.21.15.53:0/1781266894 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c005510 con 0x55a1596dded0 2023-11-06T01:41:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.053+0000 7f4707fff700 1 -- 172.21.15.53:0/1781266894 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f471c005670 con 0x55a1596dded0 2023-11-06T01:41:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.053+0000 7f473b177b80 1 -- 172.21.15.53:0/1781266894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4718015070 con 0x55a1596dded0 2023-11-06T01:41:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.054+0000 7f4707fff700 1 -- 172.21.15.53:0/1781266894 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f471c004710 con 0x55a1596dded0 2023-11-06T01:41:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.054+0000 7f473b177b80 10 client.16302 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.054+0000 7f473b177b80 1 -- 172.21.15.53:0/1781266894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55a15945a250 con 0x55a1596dded0 2023-11-06T01:41:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.054+0000 7f4707fff700 1 -- 172.21.15.53:0/1781266894 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4718015070 con 0x55a1596dded0 2023-11-06T01:41:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.054+0000 7f4707fff700 1 -- 172.21.15.53:0/1781266894 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f471c028070 con 0x55a1596dded0 2023-11-06T01:41:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.055+0000 7f473b177b80 10 client.16302 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.055+0000 7f473b177b80 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.055+0000 7f473b177b80 20 client.16302 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.055+0000 7f473b177b80 1 --2- 172.21.15.53:0/1781266894 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a15974d4a0 0x55a15974f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.055+0000 7f473b177b80 4 client.16302 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:41:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.055+0000 7f473b177b80 1 -- 172.21.15.53:0/1781266894 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a159626ce0 con 0x55a15974d4a0 2023-11-06T01:41:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.058+0000 7f472cd83700 1 --2- 172.21.15.53:0/1781266894 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a15974d4a0 0x55a15974f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.063+0000 7f4707fff700 10 client.16302 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.063+0000 7f472cd83700 1 --2- 172.21.15.53:0/1781266894 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a15974d4a0 0x55a15974f8c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0341a39d87ea8d2 server_cookie=a1fadc9f316bd64d in_seq=0 out_seq=0 2023-11-06T01:41:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f4707fff700 1 -- 172.21.15.53:0/1781266894 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a159626ce0 con 0x55a15974d4a0 2023-11-06T01:41:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f4707fff700 10 client.16302 handle_command_reply: tid=0 2023-11-06T01:41:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 1 client.16302 shutdown 2023-11-06T01:41:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 1 -- 172.21.15.53:0/1781266894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dded0 msgr2=0x55a1596f9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 1 --2- 172.21.15.53:0/1781266894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dded0 0x55a1596f9060 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f471c00d810 tx=0x7f471c003ca0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 1 -- 172.21.15.53:0/1781266894 shutdown_connections 2023-11-06T01:41:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 1 --2- 172.21.15.53:0/1781266894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1596dcfe0 0x55a1596f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 1 --2- 172.21.15.53:0/1781266894 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55a15974d4a0 0x55a15974f8c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 1 --2- 172.21.15.53:0/1781266894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1596f95a0 0x55a15974b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f4707fff700 0 client.16302 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 1 --2- 172.21.15.53:0/1781266894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1596dded0 0x55a1596f9060 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 1 -- 172.21.15.53:0/1781266894 >> 172.21.15.53:0/1781266894 conn(0x55a15963c070 msgr2=0x55a1596d8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 1 -- 172.21.15.53:0/1781266894 shutdown_connections 2023-11-06T01:41:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 1 -- 172.21.15.53:0/1781266894 wait complete. 2023-11-06T01:41:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.111+0000 7f473b177b80 20 client.16302 trim_cache size 0 max 16384 2023-11-06T01:41:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:41:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:41:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:41:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:41:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.112+0000 7f472efe7700 1 -- 172.21.15.53:0/3240720961 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47100a0520 msgr2=0x7f47100a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.112+0000 7f472efe7700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47100a0520 0x7f47100a29e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f47180105c0 tx=0x7f471800b4a0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.112+0000 7f472efe7700 1 -- 172.21.15.53:0/3240720961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4728101820 msgr2=0x7f4728195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.112+0000 7f472efe7700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4728101820 0x7f4728195da0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4720011180 tx=0x7f4720011490 comp rx=0 tx=0).stop 2023-11-06T01:41:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.112+0000 7f472efe7700 1 -- 172.21.15.53:0/3240720961 shutdown_connections 2023-11-06T01:41:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.112+0000 7f472efe7700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47100a0520 0x7f47100a29e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.112+0000 7f472efe7700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47280ffc90 0x7f47281949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.112+0000 7f472efe7700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4728101820 0x7f4728195da0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.112+0000 7f472efe7700 1 --2- 172.21.15.53:0/3240720961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4728100e80 0x7f4728194f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.112+0000 7f472efe7700 1 -- 172.21.15.53:0/3240720961 >> 172.21.15.53:0/3240720961 conn(0x7f47280fb1e0 msgr2=0x7f47280fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.113+0000 7f472efe7700 1 -- 172.21.15.53:0/3240720961 shutdown_connections 2023-11-06T01:41:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:30.113+0000 7f472efe7700 1 -- 172.21.15.53:0/3240720961 wait complete. 2023-11-06T01:41:31.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:31 smithi163 ceph-mon[104416]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 767 op/s 2023-11-06T01:41:31.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:31 smithi053 ceph-mon[99472]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 767 op/s 2023-11-06T01:41:31.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:31 smithi062 ceph-mon[103924]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 767 op/s 2023-11-06T01:41:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 435.0 s, 1815.2 tps, lat 55.144 ms stddev 63.794 2023-11-06T01:41:33.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:33 smithi053 ceph-mon[99472]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 911 op/s 2023-11-06T01:41:33.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:33 smithi062 ceph-mon[103924]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 911 op/s 2023-11-06T01:41:33.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:33 smithi163 ceph-mon[104416]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 911 op/s 2023-11-06T01:41:35.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:35 smithi053 ceph-mon[99472]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 854 op/s 2023-11-06T01:41:35.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:35 smithi062 ceph-mon[103924]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 854 op/s 2023-11-06T01:41:35.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:35 smithi163 ceph-mon[104416]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 854 op/s 2023-11-06T01:41:37.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 440.0 s, 1299.4 tps, lat 77.082 ms stddev 88.544 2023-11-06T01:41:37.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:37 smithi053 ceph-mon[99472]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 887 op/s 2023-11-06T01:41:37.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:37 smithi062 ceph-mon[103924]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 887 op/s 2023-11-06T01:41:37.697 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:41:37.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:37 smithi163 ceph-mon[104416]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 887 op/s 2023-11-06T01:41:37.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.923+0000 7f78492f9700 1 -- 172.21.15.53:0/471741036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844100f80 msgr2=0x7f7844107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:37.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.923+0000 7f78492f9700 1 --2- 172.21.15.53:0/471741036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844100f80 0x7f7844107830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7834009920 tx=0x7f7834009c30 comp rx=0 tx=0).stop 2023-11-06T01:41:37.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.924+0000 7f78492f9700 1 -- 172.21.15.53:0/471741036 shutdown_connections 2023-11-06T01:41:37.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.924+0000 7f78492f9700 1 --2- 172.21.15.53:0/471741036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844100f80 0x7f7844107830 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.924+0000 7f78492f9700 1 --2- 172.21.15.53:0/471741036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78440ff3f0 0x7f78440ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.924+0000 7f78492f9700 1 --2- 172.21.15.53:0/471741036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78441005e0 0x7f7844100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.924+0000 7f78492f9700 1 -- 172.21.15.53:0/471741036 >> 172.21.15.53:0/471741036 conn(0x7f78440fa960 msgr2=0x7f78440fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:37.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.924+0000 7f78492f9700 1 -- 172.21.15.53:0/471741036 shutdown_connections 2023-11-06T01:41:37.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.924+0000 7f78492f9700 1 -- 172.21.15.53:0/471741036 wait complete. 2023-11-06T01:41:37.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.924+0000 7f78492f9700 1 Processor -- start 2023-11-06T01:41:37.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.924+0000 7f78492f9700 1 -- start start 2023-11-06T01:41:37.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f78492f9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78440ff3f0 0x7f784410c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f78492f9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78441005e0 0x7f784419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f78492f9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7844100f80 0x7f784419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f78492f9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784419e5d0 con 0x7f7844100f80 2023-11-06T01:41:37.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f78492f9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784419e740 con 0x7f78440ff3f0 2023-11-06T01:41:37.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f78492f9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784419e900 con 0x7f78441005e0 2023-11-06T01:41:37.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f7842ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78440ff3f0 0x7f784410c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f78437fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7844100f80 0x7f784419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f7842ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78440ff3f0 0x7f784410c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53488/0 (socket says 172.21.15.53:53488) 2023-11-06T01:41:37.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f7842ffd700 1 -- 172.21.15.53:0/983848581 learned_addr learned my addr 172.21.15.53:0/983848581 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:37.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f78427fc700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78441005e0 0x7f784419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f7842ffd700 1 -- 172.21.15.53:0/983848581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78441005e0 msgr2=0x7f784419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:37.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f7842ffd700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78441005e0 0x7f784419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f7842ffd700 1 -- 172.21.15.53:0/983848581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7844100f80 msgr2=0x7f784419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:37.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f7842ffd700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7844100f80 0x7f784419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f7842ffd700 1 -- 172.21.15.53:0/983848581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7834009520 con 0x7f78440ff3f0 2023-11-06T01:41:37.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f78437fe700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7844100f80 0x7f784419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:37.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f78427fc700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78441005e0 0x7f784419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:37.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.925+0000 7f7842ffd700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78440ff3f0 0x7f784410c0b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f783c00efb0 tx=0x7f783c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:37.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.926+0000 7f783bfff700 1 -- 172.21.15.53:0/983848581 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c018070 con 0x7f78440ff3f0 2023-11-06T01:41:37.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.926+0000 7f78492f9700 1 -- 172.21.15.53:0/983848581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784419eb80 con 0x7f78440ff3f0 2023-11-06T01:41:37.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.926+0000 7f783bfff700 1 -- 172.21.15.53:0/983848581 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f783c00f040 con 0x7f78440ff3f0 2023-11-06T01:41:37.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.926+0000 7f783bfff700 1 -- 172.21.15.53:0/983848581 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c004280 con 0x7f78440ff3f0 2023-11-06T01:41:37.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.926+0000 7f78492f9700 1 -- 172.21.15.53:0/983848581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784419f0e0 con 0x7f78440ff3f0 2023-11-06T01:41:37.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.929+0000 7f783bfff700 1 -- 172.21.15.53:0/983848581 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f783c00ca20 con 0x7f78440ff3f0 2023-11-06T01:41:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.930+0000 7f783bfff700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78300a0520 0x7f78300a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.930+0000 7f783bfff700 1 -- 172.21.15.53:0/983848581 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f783c0cb810 con 0x7f78440ff3f0 2023-11-06T01:41:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.930+0000 7f78427fc700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78300a0520 0x7f78300a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.930+0000 7f7855489b80 1 Processor -- start 2023-11-06T01:41:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.930+0000 7f7855489b80 1 -- start start 2023-11-06T01:41:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.930+0000 7f78427fc700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78300a0520 0x7f78300a29e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f782c011490 tx=0x7f782c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f7855489b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e9740f0 0x55923e974510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f7855489b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e972e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f7855489b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e973820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f7855489b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e8caa10 con 0x55923e974a50 2023-11-06T01:41:37.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f7855489b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e8c83d0 con 0x55923e9740f0 2023-11-06T01:41:37.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f7855489b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e8c3cc0 con 0x55923e9733a0 2023-11-06T01:41:37.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f78427fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e972e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f7842ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e9740f0 0x55923e974510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f78437fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e973820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f78427fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e972e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42060/0 (socket says 172.21.15.53:42060) 2023-11-06T01:41:37.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f78437fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e973820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56302/0 (socket says 172.21.15.53:56302) 2023-11-06T01:41:37.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f7842ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e9740f0 0x55923e974510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53496/0 (socket says 172.21.15.53:53496) 2023-11-06T01:41:37.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f78437fe700 1 -- 172.21.15.53:0/336909675 learned_addr learned my addr 172.21.15.53:0/336909675 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:37.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f78437fe700 1 -- 172.21.15.53:0/336909675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e9740f0 msgr2=0x55923e974510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:37.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f78437fe700 1 --2- 172.21.15.53:0/336909675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e9740f0 0x55923e974510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f78437fe700 1 -- 172.21.15.53:0/336909675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 msgr2=0x55923e972e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:37.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f78437fe700 1 --2- 172.21.15.53:0/336909675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e972e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f78437fe700 1 -- 172.21.15.53:0/336909675 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55923e6f1de0 con 0x55923e9733a0 2023-11-06T01:41:37.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.931+0000 7f78427fc700 1 --2- 172.21.15.53:0/336909675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e972e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:37.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.932+0000 7f78437fe700 1 --2- 172.21.15.53:0/336909675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e973820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7834009920 tx=0x7f78340058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c7a3946c6bb152 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:37.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.932+0000 7f78397fa700 1 -- 172.21.15.53:0/336909675 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783401c050 con 0x55923e9733a0 2023-11-06T01:41:37.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.932+0000 7f78397fa700 1 -- 172.21.15.53:0/336909675 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f783401d040 con 0x55923e9733a0 2023-11-06T01:41:37.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.932+0000 7f78397fa700 1 -- 172.21.15.53:0/336909675 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78340053c0 con 0x55923e9733a0 2023-11-06T01:41:37.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.933+0000 7f7855489b80 1 -- 172.21.15.53:0/336909675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 msgr2=0x55923e973820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:37.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.933+0000 7f7855489b80 1 --2- 172.21.15.53:0/336909675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e973820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7834009920 tx=0x7f78340058e0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.933+0000 7f7855489b80 1 -- 172.21.15.53:0/336909675 shutdown_connections 2023-11-06T01:41:37.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.933+0000 7f7855489b80 1 --2- 172.21.15.53:0/336909675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e9740f0 0x55923e974510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.933+0000 7f7855489b80 1 --2- 172.21.15.53:0/336909675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e973820 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.933+0000 7f7855489b80 1 --2- 172.21.15.53:0/336909675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e972e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.933+0000 7f7855489b80 1 -- 172.21.15.53:0/336909675 >> 172.21.15.53:0/336909675 conn(0x55923e8d20d0 msgr2=0x55923e96d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:37.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.933+0000 7f7855489b80 1 -- 172.21.15.53:0/336909675 shutdown_connections 2023-11-06T01:41:37.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.933+0000 7f7855489b80 1 -- 172.21.15.53:0/336909675 wait complete. 2023-11-06T01:41:37.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.933+0000 7f7855489b80 1 -- start start 2023-11-06T01:41:37.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f7855489b80 1 Processor -- start 2023-11-06T01:41:37.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f7855489b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e97c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f7855489b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e9740f0 0x55923e97ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f7855489b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e981ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f7855489b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e860510 con 0x55923e974a50 2023-11-06T01:41:37.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f7855489b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e8601b0 con 0x55923e9740f0 2023-11-06T01:41:37.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f7855489b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e747920 con 0x55923e9733a0 2023-11-06T01:41:37.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f7855489b80 10 client.0 _ll_register_callbacks cb 0x55923e7ada80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:37.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f78437fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e981ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f7842ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e97c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f78437fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e981ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42066/0 (socket says 172.21.15.53:42066) 2023-11-06T01:41:37.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f78437fe700 1 -- 172.21.15.53:0/4236868663 learned_addr learned my addr 172.21.15.53:0/4236868663 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:37.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.934+0000 7f78427fc700 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e9740f0 0x55923e97ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f78437fe700 1 -- 172.21.15.53:0/4236868663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 msgr2=0x55923e97c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:37.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f78437fe700 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e97c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f78437fe700 1 -- 172.21.15.53:0/4236868663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e9740f0 msgr2=0x55923e97ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:37.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f78437fe700 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e9740f0 0x55923e97ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f78437fe700 1 -- 172.21.15.53:0/4236868663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c017070 con 0x55923e974a50 2023-11-06T01:41:37.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f78437fe700 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e981ec0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f783400fa00 tx=0x7f78340057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e9736aa18c8c0a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:37.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f782a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:41:37.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f782a7fc700 1 -- 172.21.15.53:0/4236868663 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783401c030 con 0x55923e974a50 2023-11-06T01:41:37.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f782a7fc700 1 -- 172.21.15.53:0/4236868663 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f783400b970 con 0x55923e974a50 2023-11-06T01:41:37.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f7855489b80 1 -- 172.21.15.53:0/4236868663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55923e8eed10 con 0x55923e974a50 2023-11-06T01:41:37.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f782a7fc700 1 -- 172.21.15.53:0/4236868663 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f783400bb30 con 0x55923e974a50 2023-11-06T01:41:37.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f7855489b80 10 client.16314 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:37.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f7855489b80 1 -- 172.21.15.53:0/4236868663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f7844108180 con 0x55923e974a50 2023-11-06T01:41:37.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.935+0000 7f782a7fc700 1 -- 172.21.15.53:0/4236868663 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f783403d020 con 0x55923e974a50 2023-11-06T01:41:37.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.936+0000 7f782a7fc700 1 -- 172.21.15.53:0/4236868663 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7834009ca0 con 0x55923e974a50 2023-11-06T01:41:37.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.936+0000 7f7855489b80 10 client.16314 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:37.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.936+0000 7f7855489b80 10 client.16314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:37.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.936+0000 7f7855489b80 20 client.16314 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:37.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.936+0000 7f7855489b80 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55923e999380 0x55923e99b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.936+0000 7f7855489b80 4 client.16314 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:41:37.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.936+0000 7f7855489b80 1 -- 172.21.15.53:0/4236868663 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55923e941e10 con 0x55923e999380 2023-11-06T01:41:37.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.937+0000 7f7842ffd700 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55923e999380 0x55923e99b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.937+0000 7f7842ffd700 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55923e999380 0x55923e99b7a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d025ed6d1378ce81 server_cookie=2ccc1f07e6e2c190 in_seq=0 out_seq=0 2023-11-06T01:41:37.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.937+0000 7f782a7fc700 10 client.16314 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:37.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f782a7fc700 1 -- 172.21.15.53:0/4236868663 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55923e941e10 con 0x55923e999380 2023-11-06T01:41:37.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f782a7fc700 10 client.16314 handle_command_reply: tid=0 2023-11-06T01:41:37.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 1 client.16314 shutdown 2023-11-06T01:41:37.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 1 -- 172.21.15.53:0/4236868663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 msgr2=0x55923e981ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:37.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e981ec0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f783400fa00 tx=0x7f78340057a0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 1 -- 172.21.15.53:0/4236868663 shutdown_connections 2023-11-06T01:41:37.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55923e999380 0x55923e99b7a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e974a50 0x55923e981ec0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f782a7fc700 0 client.16314 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:37.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e97c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 1 --2- 172.21.15.53:0/4236868663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e9740f0 0x55923e97ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 1 -- 172.21.15.53:0/4236868663 >> 172.21.15.53:0/4236868663 conn(0x55923e8d20d0 msgr2=0x55923e96d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:37.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 1 -- 172.21.15.53:0/4236868663 shutdown_connections 2023-11-06T01:41:37.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 1 -- 172.21.15.53:0/4236868663 wait complete. 2023-11-06T01:41:37.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.938+0000 7f7855489b80 20 client.16314 trim_cache size 0 max 16384 2023-11-06T01:41:37.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7855489b80 1 Processor -- start 2023-11-06T01:41:37.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7855489b80 1 -- start start 2023-11-06T01:41:37.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7855489b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e981de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7855489b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9740f0 0x55923e97ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7855489b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e974a50 0x55923e971e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7855489b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e97d350 con 0x55923e9740f0 2023-11-06T01:41:37.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7855489b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e9683d0 con 0x55923e974a50 2023-11-06T01:41:37.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7855489b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e96e2b0 con 0x55923e9733a0 2023-11-06T01:41:37.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f78427fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9740f0 0x55923e97ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7842ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e981de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f78427fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9740f0 0x55923e97ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42072/0 (socket says 172.21.15.53:42072) 2023-11-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f78427fc700 1 -- 172.21.15.53:0/2878541181 learned_addr learned my addr 172.21.15.53:0/2878541181 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f78437fe700 1 --2- 172.21.15.53:0/2878541181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e974a50 0x55923e971e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7842ffd700 1 -- 172.21.15.53:0/2878541181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e974a50 msgr2=0x55923e971e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:37.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7842ffd700 1 --2- 172.21.15.53:0/2878541181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e974a50 0x55923e971e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7842ffd700 1 -- 172.21.15.53:0/2878541181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9740f0 msgr2=0x55923e97ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7842ffd700 1 --2- 172.21.15.53:0/2878541181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9740f0 0x55923e97ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f7842ffd700 1 -- 172.21.15.53:0/2878541181 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55923e6dac60 con 0x55923e9733a0 2023-11-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.961+0000 7f78427fc700 1 --2- 172.21.15.53:0/2878541181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9740f0 0x55923e97ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:37.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.962+0000 7f7842ffd700 1 --2- 172.21.15.53:0/2878541181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e981de0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f783c0c92f0 tx=0x7f783c016040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4288bf6d35e378 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.962+0000 7f7839ffb700 1 -- 172.21.15.53:0/2878541181 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c0997e0 con 0x55923e9733a0 2023-11-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.962+0000 7f7839ffb700 1 -- 172.21.15.53:0/2878541181 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f783c0990e0 con 0x55923e9733a0 2023-11-06T01:41:37.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.962+0000 7f7839ffb700 1 -- 172.21.15.53:0/2878541181 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c099340 con 0x55923e9733a0 2023-11-06T01:41:37.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.963+0000 7f7855489b80 1 -- 172.21.15.53:0/2878541181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 msgr2=0x55923e981de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:37.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.963+0000 7f7855489b80 1 --2- 172.21.15.53:0/2878541181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e981de0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f783c0c92f0 tx=0x7f783c016040 comp rx=0 tx=0).stop 2023-11-06T01:41:37.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.963+0000 7f7855489b80 1 -- 172.21.15.53:0/2878541181 shutdown_connections 2023-11-06T01:41:37.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.963+0000 7f7855489b80 1 --2- 172.21.15.53:0/2878541181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e974a50 0x55923e971e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.963+0000 7f7855489b80 1 --2- 172.21.15.53:0/2878541181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9733a0 0x55923e981de0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.963+0000 7f7855489b80 1 --2- 172.21.15.53:0/2878541181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9740f0 0x55923e97ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.963+0000 7f7855489b80 1 -- 172.21.15.53:0/2878541181 >> 172.21.15.53:0/2878541181 conn(0x55923e8d20d0 msgr2=0x55923e97c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:37.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.963+0000 7f7855489b80 1 -- 172.21.15.53:0/2878541181 shutdown_connections 2023-11-06T01:41:37.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.963+0000 7f7855489b80 1 -- 172.21.15.53:0/2878541181 wait complete. 2023-11-06T01:41:37.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.963+0000 7f7855489b80 1 -- start start 2023-11-06T01:41:37.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.963+0000 7f7855489b80 1 Processor -- start 2023-11-06T01:41:37.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7855489b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9733a0 0x55923e9e3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7855489b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9740f0 0x55923e9e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7855489b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e974a50 0x55923e9ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7855489b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e9ea640 con 0x55923e9733a0 2023-11-06T01:41:37.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7855489b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e9ea7b0 con 0x55923e974a50 2023-11-06T01:41:37.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7855489b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923e9ea970 con 0x55923e9740f0 2023-11-06T01:41:37.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7855489b80 10 client.0 _ll_register_callbacks cb 0x7f782c004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:37.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7842ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9733a0 0x55923e9e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7842ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9733a0 0x55923e9e3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42078/0 (socket says 172.21.15.53:42078) 2023-11-06T01:41:37.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7842ffd700 1 -- 172.21.15.53:0/2974965205 learned_addr learned my addr 172.21.15.53:0/2974965205 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:37.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f78427fc700 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9740f0 0x55923e9e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f78437fe700 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e974a50 0x55923e9ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7842ffd700 1 -- 172.21.15.53:0/2974965205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9740f0 msgr2=0x55923e9e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:37.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7842ffd700 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9740f0 0x55923e9e35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7842ffd700 1 -- 172.21.15.53:0/2974965205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e974a50 msgr2=0x55923e9ea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:37.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7842ffd700 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e974a50 0x55923e9ea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7842ffd700 1 -- 172.21.15.53:0/2974965205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c004220 con 0x55923e9733a0 2023-11-06T01:41:37.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f7842ffd700 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9733a0 0x55923e9e3060 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c6f0 tx=0x7f783c097370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee6599aee72f866 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:37.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f782a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:41:37.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.964+0000 7f782a7fc700 1 -- 172.21.15.53:0/2974965205 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c099540 con 0x55923e9733a0 2023-11-06T01:41:37.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f782a7fc700 1 -- 172.21.15.53:0/2974965205 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f783c0997e0 con 0x55923e9733a0 2023-11-06T01:41:37.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f7855489b80 1 -- 172.21.15.53:0/2974965205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f783401d040 con 0x55923e9733a0 2023-11-06T01:41:37.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f782a7fc700 1 -- 172.21.15.53:0/2974965205 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f783c09cf10 con 0x55923e9733a0 2023-11-06T01:41:37.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f7855489b80 10 client.16320 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:37.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f7855489b80 1 -- 172.21.15.53:0/2974965205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f783c09c040 con 0x55923e9733a0 2023-11-06T01:41:37.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f782a7fc700 1 -- 172.21.15.53:0/2974965205 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f783401d040 con 0x55923e9733a0 2023-11-06T01:41:37.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f782a7fc700 1 -- 172.21.15.53:0/2974965205 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f783c094ed0 con 0x55923e9733a0 2023-11-06T01:41:37.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f7855489b80 10 client.16320 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:37.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f7855489b80 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:37.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f7855489b80 20 client.16320 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:37.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f7855489b80 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55923e9ebef0 0x55923e9ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:37.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f7855489b80 4 client.16320 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:41:37.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f7855489b80 1 -- 172.21.15.53:0/2974965205 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55923e8bcd40 con 0x55923e9ebef0 2023-11-06T01:41:37.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.965+0000 7f78427fc700 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55923e9ebef0 0x55923e9ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:37.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f78427fc700 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55923e9ebef0 0x55923e9ee310 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692f3af992d87a0 server_cookie=cf28ba343911d403 in_seq=0 out_seq=0 2023-11-06T01:41:37.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f782a7fc700 10 client.16320 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:37.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f782a7fc700 1 -- 172.21.15.53:0/2974965205 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55923e8bcd40 con 0x55923e9ebef0 2023-11-06T01:41:37.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f782a7fc700 10 client.16320 handle_command_reply: tid=0 2023-11-06T01:41:37.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 1 client.16320 shutdown 2023-11-06T01:41:37.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 1 -- 172.21.15.53:0/2974965205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9733a0 msgr2=0x55923e9e3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:37.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9733a0 0x55923e9e3060 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c6f0 tx=0x7f783c097370 comp rx=0 tx=0).stop 2023-11-06T01:41:37.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 1 -- 172.21.15.53:0/2974965205 shutdown_connections 2023-11-06T01:41:37.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55923e974a50 0x55923e9ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55923e9ebef0 0x55923e9ee310 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55923e9733a0 0x55923e9e3060 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 1 --2- 172.21.15.53:0/2974965205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55923e9740f0 0x55923e9e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 1 -- 172.21.15.53:0/2974965205 >> 172.21.15.53:0/2974965205 conn(0x55923e8d20d0 msgr2=0x55923e97aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:37.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f782a7fc700 0 client.16320 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:37.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 1 -- 172.21.15.53:0/2974965205 shutdown_connections 2023-11-06T01:41:37.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 1 -- 172.21.15.53:0/2974965205 wait complete. 2023-11-06T01:41:37.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.966+0000 7f7855489b80 20 client.16320 trim_cache size 0 max 16384 2023-11-06T01:41:37.983 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:41:37.984 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:41:37.984 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:41:37.984 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:41:37.984 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:41:37.984 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:41:37.985 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:41:37.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:41:37.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:41:37.985 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:41:37.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 -- 172.21.15.53:0/983848581 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78300a0520 msgr2=0x7f78300a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:37.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78300a0520 0x7f78300a29e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f782c011490 tx=0x7f782c009260 comp rx=0 tx=0).stop 2023-11-06T01:41:37.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 -- 172.21.15.53:0/983848581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78440ff3f0 msgr2=0x7f784410c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:37.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78440ff3f0 0x7f784410c0b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f783c00efb0 tx=0x7f783c00c3a0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 -- 172.21.15.53:0/983848581 shutdown_connections 2023-11-06T01:41:37.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78300a0520 0x7f78300a29e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78440ff3f0 0x7f784410c0b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7844100f80 0x7f784419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 --2- 172.21.15.53:0/983848581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78441005e0 0x7f784419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:37.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 -- 172.21.15.53:0/983848581 >> 172.21.15.53:0/983848581 conn(0x7f78440fa960 msgr2=0x7f78440fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:37.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 -- 172.21.15.53:0/983848581 shutdown_connections 2023-11-06T01:41:37.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:41:37.967+0000 7f78492f9700 1 -- 172.21.15.53:0/983848581 wait complete. 2023-11-06T01:41:37.989 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:41:39.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:39 smithi053 ceph-mon[99472]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 996 op/s 2023-11-06T01:41:39.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:39 smithi062 ceph-mon[103924]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 996 op/s 2023-11-06T01:41:39.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:39 smithi163 ceph-mon[104416]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 996 op/s 2023-11-06T01:41:41.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:41 smithi053 ceph-mon[99472]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 19 MiB/s wr, 816 op/s 2023-11-06T01:41:41.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:41 smithi062 ceph-mon[103924]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 19 MiB/s wr, 816 op/s 2023-11-06T01:41:41.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:41 smithi163 ceph-mon[104416]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 19 MiB/s wr, 816 op/s 2023-11-06T01:41:42.187 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 445.0 s, 1206.3 tps, lat 82.495 ms stddev 149.393 2023-11-06T01:41:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.282+0000 7f883ef3a700 1 -- 172.21.15.53:0/1608270833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88381086f0 msgr2=0x7f883810f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.282+0000 7f883ef3a700 1 --2- 172.21.15.53:0/1608270833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88381086f0 0x7f883810f0d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f883000b0a0 tx=0x7f883000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.282+0000 7f883ef3a700 1 -- 172.21.15.53:0/1608270833 shutdown_connections 2023-11-06T01:41:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.282+0000 7f883ef3a700 1 --2- 172.21.15.53:0/1608270833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8838107d90 0x7f88381081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.282+0000 7f883ef3a700 1 --2- 172.21.15.53:0/1608270833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f883810f610 0x7f8838111a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.282+0000 7f883ef3a700 1 --2- 172.21.15.53:0/1608270833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88381086f0 0x7f883810f0d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.282+0000 7f883ef3a700 1 -- 172.21.15.53:0/1608270833 >> 172.21.15.53:0/1608270833 conn(0x7f883806dbc0 msgr2=0x7f8838070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.282+0000 7f883ef3a700 1 -- 172.21.15.53:0/1608270833 shutdown_connections 2023-11-06T01:41:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.282+0000 7f883ef3a700 1 -- 172.21.15.53:0/1608270833 wait complete. 2023-11-06T01:41:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.283+0000 7f883ef3a700 1 Processor -- start 2023-11-06T01:41:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.285+0000 7f883ef3a700 1 -- start start 2023-11-06T01:41:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.285+0000 7f883ef3a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8838107d90 0x7f883819d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.285+0000 7f883ef3a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88381086f0 0x7f883819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.285+0000 7f883ef3a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f883810f610 0x7f883819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.285+0000 7f883ef3a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883819ed30 con 0x7f883810f610 2023-11-06T01:41:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.285+0000 7f883ef3a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883819eea0 con 0x7f8838107d90 2023-11-06T01:41:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.285+0000 7f883ef3a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883819f060 con 0x7f88381086f0 2023-11-06T01:41:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f8837fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88381086f0 0x7f883819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f8837fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88381086f0 0x7f883819d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56332/0 (socket says 172.21.15.53:56332) 2023-11-06T01:41:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f8837fff700 1 -- 172.21.15.53:0/933358321 learned_addr learned my addr 172.21.15.53:0/933358321 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f883d4d7700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f883810f610 0x7f883819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f883ccd6700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8838107d90 0x7f883819d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f8837fff700 1 -- 172.21.15.53:0/933358321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8838107d90 msgr2=0x7f883819d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f8837fff700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8838107d90 0x7f883819d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f8837fff700 1 -- 172.21.15.53:0/933358321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f883810f610 msgr2=0x7f883819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f8837fff700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f883810f610 0x7f883819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f8837fff700 1 -- 172.21.15.53:0/933358321 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8830009d00 con 0x7f88381086f0 2023-11-06T01:41:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f8837fff700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88381086f0 0x7f883819d7e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8830007a60 tx=0x7f883000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.286+0000 7f8835ffb700 1 -- 172.21.15.53:0/933358321 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883000e040 con 0x7f88381086f0 2023-11-06T01:41:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.287+0000 7f8835ffb700 1 -- 172.21.15.53:0/933358321 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8830004c50 con 0x7f88381086f0 2023-11-06T01:41:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.287+0000 7f883ef3a700 1 -- 172.21.15.53:0/933358321 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883819f2e0 con 0x7f88381086f0 2023-11-06T01:41:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.287+0000 7f8835ffb700 1 -- 172.21.15.53:0/933358321 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88300052a0 con 0x7f88381086f0 2023-11-06T01:41:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.287+0000 7f883ef3a700 1 -- 172.21.15.53:0/933358321 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88381a5b20 con 0x7f88381086f0 2023-11-06T01:41:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.291+0000 7f884b0cab80 1 Processor -- start 2023-11-06T01:41:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.291+0000 7f884b0cab80 1 -- start start 2023-11-06T01:41:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f8835ffb700 1 -- 172.21.15.53:0/933358321 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f883001d470 con 0x7f88381086f0 2023-11-06T01:41:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 0x55d045a16aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a16fe0 0x55d045a178f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a17ed0 0x55d045a18350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f884b0cab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04596e9b0 con 0x55d045a16680 2023-11-06T01:41:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f884b0cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04596c370 con 0x55d045a17ed0 2023-11-06T01:41:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f884b0cab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d045967c60 con 0x55d045a16fe0 2023-11-06T01:41:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f883d4d7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 0x55d045a16aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f883d4d7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 0x55d045a16aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42098/0 (socket says 172.21.15.53:42098) 2023-11-06T01:41:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f883d4d7700 1 -- 172.21.15.53:0/2984706592 learned_addr learned my addr 172.21.15.53:0/2984706592 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f8837fff700 1 --2- 172.21.15.53:0/2984706592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a17ed0 0x55d045a18350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f883ccd6700 1 --2- 172.21.15.53:0/2984706592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a16fe0 0x55d045a178f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.292+0000 7f8835ffb700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f88240a05a0 0x7f88240a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.293+0000 7f883ccd6700 1 -- 172.21.15.53:0/2984706592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a17ed0 msgr2=0x55d045a18350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.293+0000 7f883ccd6700 1 --2- 172.21.15.53:0/2984706592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a17ed0 0x55d045a18350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.293+0000 7f883ccd6700 1 -- 172.21.15.53:0/2984706592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 msgr2=0x55d045a16aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.293+0000 7f883ccd6700 1 --2- 172.21.15.53:0/2984706592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 0x55d045a16aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.293+0000 7f8835ffb700 1 -- 172.21.15.53:0/933358321 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f88300cce60 con 0x7f88381086f0 2023-11-06T01:41:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.293+0000 7f883ccd6700 1 -- 172.21.15.53:0/2984706592 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d045795d80 con 0x55d045a16fe0 2023-11-06T01:41:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.293+0000 7f883ccd6700 1 --2- 172.21.15.53:0/2984706592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a16fe0 0x55d045a178f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8820009c80 tx=0x7f8820009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc9d8d0fc4bd175a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.293+0000 7f883d4d7700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f88240a05a0 0x7f88240a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.294+0000 7f882effd700 1 -- 172.21.15.53:0/2984706592 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820011cb0 con 0x55d045a16fe0 2023-11-06T01:41:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.294+0000 7f882effd700 1 -- 172.21.15.53:0/2984706592 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8820011e10 con 0x55d045a16fe0 2023-11-06T01:41:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.294+0000 7f882effd700 1 -- 172.21.15.53:0/2984706592 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820005280 con 0x55d045a16fe0 2023-11-06T01:41:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.295+0000 7f884b0cab80 1 -- 172.21.15.53:0/2984706592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a16fe0 msgr2=0x55d045a178f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.295+0000 7f883d4d7700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f88240a05a0 0x7f88240a2a60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8828009f50 tx=0x7f8828004750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.295+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2984706592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a16fe0 0x55d045a178f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8820009c80 tx=0x7f8820009f90 comp rx=0 tx=0).stop 2023-11-06T01:41:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.295+0000 7f884b0cab80 1 -- 172.21.15.53:0/2984706592 shutdown_connections 2023-11-06T01:41:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.295+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2984706592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a17ed0 0x55d045a18350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.295+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2984706592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 0x55d045a16aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.295+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2984706592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a16fe0 0x55d045a178f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.295+0000 7f884b0cab80 1 -- 172.21.15.53:0/2984706592 >> 172.21.15.53:0/2984706592 conn(0x55d045976070 msgr2=0x55d045a13040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.295+0000 7f884b0cab80 1 -- 172.21.15.53:0/2984706592 shutdown_connections 2023-11-06T01:41:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.296+0000 7f884b0cab80 1 -- 172.21.15.53:0/2984706592 wait complete. 2023-11-06T01:41:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.296+0000 7f884b0cab80 1 -- start start 2023-11-06T01:41:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.296+0000 7f884b0cab80 1 Processor -- start 2023-11-06T01:41:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 0x55d045a20520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16fe0 0x55d045a20a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 0x55d045a27aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f884b0cab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0459044b0 con 0x55d045a16680 2023-11-06T01:41:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f884b0cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d045904150 con 0x55d045a16fe0 2023-11-06T01:41:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f884b0cab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0457eb8c0 con 0x55d045a17ed0 2023-11-06T01:41:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f884b0cab80 10 client.0 _ll_register_callbacks cb 0x55d045851a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f883ccd6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 0x55d045a20520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f8837fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16fe0 0x55d045a20a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f883d4d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 0x55d045a27aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f883ccd6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 0x55d045a20520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42106/0 (socket says 172.21.15.53:42106) 2023-11-06T01:41:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f883ccd6700 1 -- 172.21.15.53:0/4096172130 learned_addr learned my addr 172.21.15.53:0/4096172130 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f883ccd6700 1 -- 172.21.15.53:0/4096172130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 msgr2=0x55d045a27aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f883ccd6700 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 0x55d045a27aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f883ccd6700 1 -- 172.21.15.53:0/4096172130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16fe0 msgr2=0x55d045a20a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f883ccd6700 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16fe0 0x55d045a20a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f883ccd6700 1 -- 172.21.15.53:0/4096172130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88200098e0 con 0x55d045a16680 2023-11-06T01:41:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f883d4d7700 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 0x55d045a27aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f8837fff700 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16fe0 0x55d045a20a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f883ccd6700 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 0x55d045a20520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8820005d90 tx=0x7f88200047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8218f7a465f6e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.297+0000 7f8817fff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:41:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.298+0000 7f8817fff700 1 -- 172.21.15.53:0/4096172130 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820005120 con 0x55d045a16680 2023-11-06T01:41:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.298+0000 7f8817fff700 1 -- 172.21.15.53:0/4096172130 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8820005280 con 0x55d045a16680 2023-11-06T01:41:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.298+0000 7f884b0cab80 1 -- 172.21.15.53:0/4096172130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d045992cb0 con 0x55d045a16680 2023-11-06T01:41:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.298+0000 7f8817fff700 1 -- 172.21.15.53:0/4096172130 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8820005440 con 0x55d045a16680 2023-11-06T01:41:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.298+0000 7f884b0cab80 10 client.16332 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.298+0000 7f884b0cab80 1 -- 172.21.15.53:0/4096172130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55d04577ec00 con 0x55d045a16680 2023-11-06T01:41:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.299+0000 7f8817fff700 1 -- 172.21.15.53:0/4096172130 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8820031020 con 0x55d045a16680 2023-11-06T01:41:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.299+0000 7f8817fff700 1 -- 172.21.15.53:0/4096172130 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8820022400 con 0x55d045a16680 2023-11-06T01:41:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.299+0000 7f884b0cab80 10 client.16332 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.299+0000 7f884b0cab80 10 client.16332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.299+0000 7f884b0cab80 20 client.16332 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.299+0000 7f884b0cab80 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d045a2c290 0x55d045a2e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.299+0000 7f884b0cab80 4 client.16332 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:41:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.299+0000 7f884b0cab80 1 -- 172.21.15.53:0/4096172130 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0459e5db0 con 0x55d045a2c290 2023-11-06T01:41:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.299+0000 7f8837fff700 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d045a2c290 0x55d045a2e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.300+0000 7f8817fff700 10 client.16332 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.300+0000 7f8837fff700 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d045a2c290 0x55d045a2e6b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607d1ec0eae9105c server_cookie=51eaf5f269c78a39 in_seq=0 out_seq=0 2023-11-06T01:41:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f8817fff700 1 -- 172.21.15.53:0/4096172130 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d0459e5db0 con 0x55d045a2c290 2023-11-06T01:41:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f8817fff700 10 client.16332 handle_command_reply: tid=0 2023-11-06T01:41:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 1 client.16332 shutdown 2023-11-06T01:41:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 1 -- 172.21.15.53:0/4096172130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 msgr2=0x55d045a20520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 0x55d045a20520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8820005d90 tx=0x7f88200047e0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 1 -- 172.21.15.53:0/4096172130 shutdown_connections 2023-11-06T01:41:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d045a2c290 0x55d045a2e6b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 0x55d045a27aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16680 0x55d045a20520 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 1 --2- 172.21.15.53:0/4096172130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16fe0 0x55d045a20a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 1 -- 172.21.15.53:0/4096172130 >> 172.21.15.53:0/4096172130 conn(0x55d045976070 msgr2=0x55d045a113c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f8817fff700 0 client.16332 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 1 -- 172.21.15.53:0/4096172130 shutdown_connections 2023-11-06T01:41:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 1 -- 172.21.15.53:0/4096172130 wait complete. 2023-11-06T01:41:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.301+0000 7f884b0cab80 20 client.16332 trim_cache size 0 max 16384 2023-11-06T01:41:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.325+0000 7f884b0cab80 1 Processor -- start 2023-11-06T01:41:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.325+0000 7f884b0cab80 1 -- start start 2023-11-06T01:41:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.325+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a16680 0x55d045a25a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.325+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16fe0 0x55d045a2e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.325+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a17ed0 0x55d045a1f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.325+0000 7f884b0cab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d045a0c500 con 0x55d045a17ed0 2023-11-06T01:41:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.325+0000 7f884b0cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d045a12840 con 0x55d045a16fe0 2023-11-06T01:41:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.325+0000 7f884b0cab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d045a173c0 con 0x55d045a16680 2023-11-06T01:41:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.326+0000 7f883d4d7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a17ed0 0x55d045a1f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.326+0000 7f883d4d7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a17ed0 0x55d045a1f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42116/0 (socket says 172.21.15.53:42116) 2023-11-06T01:41:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.326+0000 7f883d4d7700 1 -- 172.21.15.53:0/2590036143 learned_addr learned my addr 172.21.15.53:0/2590036143 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.326+0000 7f8837fff700 1 --2- 172.21.15.53:0/2590036143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16fe0 0x55d045a2e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.326+0000 7f883d4d7700 1 -- 172.21.15.53:0/2590036143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a16680 msgr2=0x55d045a25a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.326+0000 7f883d4d7700 1 --2- 172.21.15.53:0/2590036143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a16680 0x55d045a25a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.326+0000 7f883d4d7700 1 -- 172.21.15.53:0/2590036143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16fe0 msgr2=0x55d045a2e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.326+0000 7f883d4d7700 1 --2- 172.21.15.53:0/2590036143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16fe0 0x55d045a2e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.326+0000 7f883d4d7700 1 -- 172.21.15.53:0/2590036143 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0457bc1f0 con 0x55d045a17ed0 2023-11-06T01:41:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.327+0000 7f883d4d7700 1 --2- 172.21.15.53:0/2590036143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a17ed0 0x55d045a1f170 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f882801d450 tx=0x7f8828019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a3e998e9eb7f3d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.327+0000 7f882f7fe700 1 -- 172.21.15.53:0/2590036143 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280045b0 con 0x55d045a17ed0 2023-11-06T01:41:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.327+0000 7f882f7fe700 1 -- 172.21.15.53:0/2590036143 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f882801dca0 con 0x55d045a17ed0 2023-11-06T01:41:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.327+0000 7f882f7fe700 1 -- 172.21.15.53:0/2590036143 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828010710 con 0x55d045a17ed0 2023-11-06T01:41:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.328+0000 7f884b0cab80 1 -- 172.21.15.53:0/2590036143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a17ed0 msgr2=0x55d045a1f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.328+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2590036143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a17ed0 0x55d045a1f170 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f882801d450 tx=0x7f8828019000 comp rx=0 tx=0).stop 2023-11-06T01:41:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.328+0000 7f884b0cab80 1 -- 172.21.15.53:0/2590036143 shutdown_connections 2023-11-06T01:41:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.328+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2590036143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a17ed0 0x55d045a1f170 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.328+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2590036143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a16680 0x55d045a25a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.328+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2590036143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16fe0 0x55d045a2e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.328+0000 7f884b0cab80 1 -- 172.21.15.53:0/2590036143 >> 172.21.15.53:0/2590036143 conn(0x55d045976070 msgr2=0x55d045a12c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.329+0000 7f884b0cab80 1 -- 172.21.15.53:0/2590036143 shutdown_connections 2023-11-06T01:41:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.329+0000 7f884b0cab80 1 -- 172.21.15.53:0/2590036143 wait complete. 2023-11-06T01:41:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.329+0000 7f884b0cab80 1 -- start start 2023-11-06T01:41:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.329+0000 7f884b0cab80 1 Processor -- start 2023-11-06T01:41:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.330+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16680 0x55d045a77d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.330+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16fe0 0x55d045a7a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.330+0000 7f884b0cab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 0x55d045a7c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.330+0000 7f884b0cab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d045a7cd30 con 0x55d045a16fe0 2023-11-06T01:41:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.330+0000 7f884b0cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d045a7ce70 con 0x55d045a16680 2023-11-06T01:41:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.330+0000 7f884b0cab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d045a7cfb0 con 0x55d045a17ed0 2023-11-06T01:41:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.330+0000 7f884b0cab80 10 client.0 _ll_register_callbacks cb 0x7f8828009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.330+0000 7f883d4d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 0x55d045a7c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.330+0000 7f883d4d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 0x55d045a7c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56360/0 (socket says 172.21.15.53:56360) 2023-11-06T01:41:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.330+0000 7f883d4d7700 1 -- 172.21.15.53:0/2955168787 learned_addr learned my addr 172.21.15.53:0/2955168787 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.331+0000 7f883d4d7700 1 -- 172.21.15.53:0/2955168787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16680 msgr2=0x55d045a77d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:41:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.331+0000 7f883d4d7700 1 --2- 172.21.15.53:0/2955168787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16680 0x55d045a77d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.331+0000 7f883d4d7700 1 -- 172.21.15.53:0/2955168787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16fe0 msgr2=0x55d045a7a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.331+0000 7f883d4d7700 1 --2- 172.21.15.53:0/2955168787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16fe0 0x55d045a7a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.331+0000 7f883d4d7700 1 -- 172.21.15.53:0/2955168787 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8828003d80 con 0x55d045a17ed0 2023-11-06T01:41:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.331+0000 7f883d4d7700 1 --2- 172.21.15.53:0/2955168787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 0x55d045a7c7f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8828009ac0 tx=0x7f882801cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37b5ba4398d91e74 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.331+0000 7f8817fff700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:41:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.331+0000 7f8817fff700 1 -- 172.21.15.53:0/2955168787 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828010840 con 0x55d045a17ed0 2023-11-06T01:41:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.331+0000 7f884b0cab80 1 -- 172.21.15.53:0/2955168787 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8820011e10 con 0x55d045a17ed0 2023-11-06T01:41:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.331+0000 7f8817fff700 1 -- 172.21.15.53:0/2955168787 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f88280109a0 con 0x55d045a17ed0 2023-11-06T01:41:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.331+0000 7f8817fff700 1 -- 172.21.15.53:0/2955168787 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8828010b60 con 0x55d045a17ed0 2023-11-06T01:41:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.332+0000 7f884b0cab80 10 client.25972 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.332+0000 7f884b0cab80 1 -- 172.21.15.53:0/2955168787 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55d045794250 con 0x55d045a17ed0 2023-11-06T01:41:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.332+0000 7f8817fff700 1 -- 172.21.15.53:0/2955168787 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8828042020 con 0x55d045a17ed0 2023-11-06T01:41:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.332+0000 7f8817fff700 1 -- 172.21.15.53:0/2955168787 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8828028070 con 0x55d045a17ed0 2023-11-06T01:41:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.332+0000 7f884b0cab80 10 client.25972 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.332+0000 7f884b0cab80 10 client.25972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.332+0000 7f884b0cab80 20 client.25972 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.332+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2955168787 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d045a7e990 0x55d045a80df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.332+0000 7f884b0cab80 4 client.25972 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:41:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.332+0000 7f884b0cab80 1 -- 172.21.15.53:0/2955168787 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d045960ce0 con 0x55d045a7e990 2023-11-06T01:41:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.333+0000 7f883ccd6700 1 --2- 172.21.15.53:0/2955168787 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d045a7e990 0x55d045a80df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.333+0000 7f8817fff700 10 client.25972 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.333+0000 7f883ccd6700 1 --2- 172.21.15.53:0/2955168787 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d045a7e990 0x55d045a80df0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d296c9488899b0f5 server_cookie=d18074214a92e29 in_seq=0 out_seq=0 2023-11-06T01:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f8817fff700 1 -- 172.21.15.53:0/2955168787 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d045960ce0 con 0x55d045a7e990 2023-11-06T01:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f8817fff700 10 client.25972 handle_command_reply: tid=0 2023-11-06T01:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f884b0cab80 1 client.25972 shutdown 2023-11-06T01:41:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f884b0cab80 1 -- 172.21.15.53:0/2955168787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 msgr2=0x55d045a7c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2955168787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 0x55d045a7c7f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8828009ac0 tx=0x7f882801cc70 comp rx=0 tx=0).stop 2023-11-06T01:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f884b0cab80 1 -- 172.21.15.53:0/2955168787 shutdown_connections 2023-11-06T01:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2955168787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d045a16680 0x55d045a77d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2955168787 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d045a7e990 0x55d045a80df0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2955168787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d045a17ed0 0x55d045a7c7f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f884b0cab80 1 --2- 172.21.15.53:0/2955168787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d045a16fe0 0x55d045a7a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f8817fff700 0 client.25972 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f884b0cab80 1 -- 172.21.15.53:0/2955168787 >> 172.21.15.53:0/2955168787 conn(0x55d045976070 msgr2=0x55d045a2e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f884b0cab80 1 -- 172.21.15.53:0/2955168787 shutdown_connections 2023-11-06T01:41:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.341+0000 7f884b0cab80 1 -- 172.21.15.53:0/2955168787 wait complete. 2023-11-06T01:41:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.342+0000 7f884b0cab80 20 client.25972 trim_cache size 0 max 16384 2023-11-06T01:41:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:41:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:41:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:41:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:41:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 -- 172.21.15.53:0/933358321 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f88240a05a0 msgr2=0x7f88240a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f88240a05a0 0x7f88240a2a60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8828009f50 tx=0x7f8828004750 comp rx=0 tx=0).stop 2023-11-06T01:41:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 -- 172.21.15.53:0/933358321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88381086f0 msgr2=0x7f883819d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88381086f0 0x7f883819d7e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8830007a60 tx=0x7f883000bab0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 -- 172.21.15.53:0/933358321 shutdown_connections 2023-11-06T01:41:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f88240a05a0 0x7f88240a2a60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8838107d90 0x7f883819d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f883810f610 0x7f883819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 --2- 172.21.15.53:0/933358321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88381086f0 0x7f883819d7e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 -- 172.21.15.53:0/933358321 >> 172.21.15.53:0/933358321 conn(0x7f883806dbc0 msgr2=0x7f88380705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 -- 172.21.15.53:0/933358321 shutdown_connections 2023-11-06T01:41:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:43.343+0000 7f883ef3a700 1 -- 172.21.15.53:0/933358321 wait complete. 2023-11-06T01:41:43.600 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:43 smithi053 ceph-mon[99472]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 1.63k op/s 2023-11-06T01:41:43.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:43 smithi062 ceph-mon[103924]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 1.63k op/s 2023-11-06T01:41:43.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:43 smithi163 ceph-mon[104416]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 1.63k op/s 2023-11-06T01:41:45.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:45 smithi053 ceph-mon[99472]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 MiB/s wr, 1.89k op/s 2023-11-06T01:41:45.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:45 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:41:45.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:45 smithi062 ceph-mon[103924]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 MiB/s wr, 1.89k op/s 2023-11-06T01:41:45.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:45 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:41:45.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:45 smithi163 ceph-mon[104416]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 MiB/s wr, 1.89k op/s 2023-11-06T01:41:45.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:45 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:41:46.337 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.337 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.337 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.337 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.338 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.338 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.338 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.338 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.339 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:41:46.340 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:41:46.340 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:46 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:41:46.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.649 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.650 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:41:46.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:41:46.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:46 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:41:46.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.741 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.743 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.743 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.743 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.743 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.743 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-06T01:41:46.744 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:41:46.744 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:41:46.744 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:46 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:41:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 450.0 s, 1632.8 tps, lat 61.441 ms stddev 73.169 2023-11-06T01:41:47.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:47 smithi053 ceph-mon[99472]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 1.91k op/s 2023-11-06T01:41:47.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:47 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi163 to 196.2M 2023-11-06T01:41:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:47 smithi053 ceph-mon[99472]: Unable to set osd_memory_target on smithi163 to 205762457: error parsing value: Value '205762457' is below minimum 939524096 2023-11-06T01:41:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:47 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi062 to 196.2M 2023-11-06T01:41:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:47 smithi053 ceph-mon[99472]: Adjusting osd_memory_target on smithi053 to 197.2M 2023-11-06T01:41:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:47 smithi053 ceph-mon[99472]: Unable to set osd_memory_target on smithi062 to 205776076: error parsing value: Value '205776076' is below minimum 939524096 2023-11-06T01:41:47.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:47 smithi053 ceph-mon[99472]: Unable to set osd_memory_target on smithi053 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-06T01:41:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:47 smithi062 ceph-mon[103924]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 1.91k op/s 2023-11-06T01:41:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:47 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi163 to 196.2M 2023-11-06T01:41:47.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:47 smithi062 ceph-mon[103924]: Unable to set osd_memory_target on smithi163 to 205762457: error parsing value: Value '205762457' is below minimum 939524096 2023-11-06T01:41:47.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:47 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi062 to 196.2M 2023-11-06T01:41:47.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:47 smithi062 ceph-mon[103924]: Adjusting osd_memory_target on smithi053 to 197.2M 2023-11-06T01:41:47.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:47 smithi062 ceph-mon[103924]: Unable to set osd_memory_target on smithi062 to 205776076: error parsing value: Value '205776076' is below minimum 939524096 2023-11-06T01:41:47.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:47 smithi062 ceph-mon[103924]: Unable to set osd_memory_target on smithi053 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-06T01:41:47.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:47 smithi163 ceph-mon[104416]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 1.91k op/s 2023-11-06T01:41:47.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:47 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi163 to 196.2M 2023-11-06T01:41:47.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:47 smithi163 ceph-mon[104416]: Unable to set osd_memory_target on smithi163 to 205762457: error parsing value: Value '205762457' is below minimum 939524096 2023-11-06T01:41:47.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:47 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi062 to 196.2M 2023-11-06T01:41:47.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:47 smithi163 ceph-mon[104416]: Adjusting osd_memory_target on smithi053 to 197.2M 2023-11-06T01:41:47.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:47 smithi163 ceph-mon[104416]: Unable to set osd_memory_target on smithi062 to 205776076: error parsing value: Value '205776076' is below minimum 939524096 2023-11-06T01:41:47.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:47 smithi163 ceph-mon[104416]: Unable to set osd_memory_target on smithi053 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-06T01:41:49.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:49 smithi053 ceph-mon[99472]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-11-06T01:41:49.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:49 smithi062 ceph-mon[103924]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-11-06T01:41:49.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:49 smithi163 ceph-mon[104416]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-11-06T01:41:51.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:51 smithi053 ceph-mon[99472]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.82k op/s 2023-11-06T01:41:51.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:51 smithi062 ceph-mon[103924]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.82k op/s 2023-11-06T01:41:51.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:51 smithi163 ceph-mon[104416]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.82k op/s 2023-11-06T01:41:52.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 455.0 s, 2110.2 tps, lat 47.324 ms stddev 52.697 2023-11-06T01:41:53.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:53 smithi062 ceph-mon[103924]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-11-06T01:41:53.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:53 smithi163 ceph-mon[104416]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-11-06T01:41:53.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:53 smithi053 ceph-mon[99472]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-11-06T01:41:55.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:55 smithi062 ceph-mon[103924]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 1.11k op/s 2023-11-06T01:41:55.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:55 smithi163 ceph-mon[104416]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 1.11k op/s 2023-11-06T01:41:55.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:55 smithi053 ceph-mon[99472]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 1.11k op/s 2023-11-06T01:41:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.530+0000 7f51f5423700 1 -- 172.21.15.53:0/3618505265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0107d90 msgr2=0x7f51f010a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.530+0000 7f51f5423700 1 --2- 172.21.15.53:0/3618505265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0107d90 0x7f51f010a1c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f51e40098f0 tx=0x7f51e4009c00 comp rx=0 tx=0).stop 2023-11-06T01:41:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.531+0000 7f51f5423700 1 -- 172.21.15.53:0/3618505265 shutdown_connections 2023-11-06T01:41:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.531+0000 7f51f5423700 1 --2- 172.21.15.53:0/3618505265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f010d0d0 0x7f51f010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.531+0000 7f51f5423700 1 --2- 172.21.15.53:0/3618505265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0107d90 0x7f51f010a1c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.531+0000 7f51f5423700 1 --2- 172.21.15.53:0/3618505265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f010a700 0x7f51f010cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.531+0000 7f51f5423700 1 -- 172.21.15.53:0/3618505265 >> 172.21.15.53:0/3618505265 conn(0x7f51f006dbc0 msgr2=0x7f51f0070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.531+0000 7f51f5423700 1 -- 172.21.15.53:0/3618505265 shutdown_connections 2023-11-06T01:41:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.531+0000 7f51f5423700 1 -- 172.21.15.53:0/3618505265 wait complete. 2023-11-06T01:41:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.532+0000 7f51f5423700 1 Processor -- start 2023-11-06T01:41:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.532+0000 7f51f5423700 1 -- start start 2023-11-06T01:41:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.532+0000 7f51f5423700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0107d90 0x7f51f0114420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.532+0000 7f51f5423700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f010a700 0x7f51f0114960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.532+0000 7f51f5423700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f010d0d0 0x7f51f0115830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.532+0000 7f51f5423700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f0115e90 con 0x7f51f0107d90 2023-11-06T01:41:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.532+0000 7f51f5423700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f0116000 con 0x7f51f010a700 2023-11-06T01:41:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.532+0000 7f51f5423700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f01161c0 con 0x7f51f010d0d0 2023-11-06T01:41:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.532+0000 7f51eeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0107d90 0x7f51f0114420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.532+0000 7f51eeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0107d90 0x7f51f0114420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42132/0 (socket says 172.21.15.53:42132) 2023-11-06T01:41:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.532+0000 7f51eeffd700 1 -- 172.21.15.53:0/2867989826 learned_addr learned my addr 172.21.15.53:0/2867989826 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.533+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f010a700 0x7f51f0114960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.533+0000 7f51ef7fe700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f010d0d0 0x7f51f0115830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.533+0000 7f51eeffd700 1 -- 172.21.15.53:0/2867989826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f010d0d0 msgr2=0x7f51f0115830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.533+0000 7f51eeffd700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f010d0d0 0x7f51f0115830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.533+0000 7f51eeffd700 1 -- 172.21.15.53:0/2867989826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f010a700 msgr2=0x7f51f0114960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.533+0000 7f51eeffd700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f010a700 0x7f51f0114960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.533+0000 7f51eeffd700 1 -- 172.21.15.53:0/2867989826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e4009520 con 0x7f51f0107d90 2023-11-06T01:41:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.533+0000 7f51ef7fe700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f010d0d0 0x7f51f0115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.533+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f010a700 0x7f51f0114960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.533+0000 7f51eeffd700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0107d90 0x7f51f0114420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009230 tx=0x7f51e400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.533+0000 7f51d7fff700 1 -- 172.21.15.53:0/2867989826 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e401c070 con 0x7f51f0107d90 2023-11-06T01:41:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.534+0000 7f51f5423700 1 -- 172.21.15.53:0/2867989826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f0116440 con 0x7f51f0107d90 2023-11-06T01:41:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.534+0000 7f51d7fff700 1 -- 172.21.15.53:0/2867989826 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51e400bc40 con 0x7f51f0107d90 2023-11-06T01:41:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.534+0000 7f51f5423700 1 -- 172.21.15.53:0/2867989826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f0116c50 con 0x7f51f0107d90 2023-11-06T01:41:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.534+0000 7f51d7fff700 1 -- 172.21.15.53:0/2867989826 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4004e20 con 0x7f51f0107d90 2023-11-06T01:41:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.537+0000 7f51d7fff700 1 -- 172.21.15.53:0/2867989826 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f51e4005030 con 0x7f51f0107d90 2023-11-06T01:41:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.538+0000 7f51d7fff700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f51d80a0520 0x7f51d80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.538+0000 7f52015b3b80 1 Processor -- start 2023-11-06T01:41:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.538+0000 7f51d7fff700 1 -- 172.21.15.53:0/2867989826 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f51e40ccbb0 con 0x7f51f0107d90 2023-11-06T01:41:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.538+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f51d80a0520 0x7f51d80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.538+0000 7f52015b3b80 1 -- start start 2023-11-06T01:41:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.538+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70680 0x55f44de70aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.538+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de70fe0 0x55f44de718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.538+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de71ed0 0x55f44de72350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.538+0000 7f52015b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44ddc89b0 con 0x55f44de70fe0 2023-11-06T01:41:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.538+0000 7f52015b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44ddc6370 con 0x55f44de71ed0 2023-11-06T01:41:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.539+0000 7f52015b3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44ddc1c60 con 0x55f44de70680 2023-11-06T01:41:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.540+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f51d80a0520 0x7f51d80a29e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f51f0115470 tx=0x7f51e80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.540+0000 7f51ef7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de71ed0 0x55f44de72350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.540+0000 7f51eeffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70680 0x55f44de70aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.540+0000 7f51eeffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70680 0x55f44de70aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56380/0 (socket says 172.21.15.53:56380) 2023-11-06T01:41:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.540+0000 7f51eeffd700 1 -- 172.21.15.53:0/2015219629 learned_addr learned my addr 172.21.15.53:0/2015219629 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.540+0000 7f51ef7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de71ed0 0x55f44de72350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53580/0 (socket says 172.21.15.53:53580) 2023-11-06T01:41:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.541+0000 7f51eeffd700 1 -- 172.21.15.53:0/2015219629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de71ed0 msgr2=0x55f44de72350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.541+0000 7f51eeffd700 1 --2- 172.21.15.53:0/2015219629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de71ed0 0x55f44de72350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.541+0000 7f51eeffd700 1 -- 172.21.15.53:0/2015219629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de70fe0 msgr2=0x55f44de718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.541+0000 7f51eeffd700 1 --2- 172.21.15.53:0/2015219629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de70fe0 0x55f44de718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.541+0000 7f51eeffd700 1 -- 172.21.15.53:0/2015219629 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f44dbefd80 con 0x55f44de70680 2023-11-06T01:41:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.541+0000 7f51eeffd700 1 --2- 172.21.15.53:0/2015219629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70680 0x55f44de70aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f51e4096e90 tx=0x7f51e4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb15f416f0562b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.542+0000 7f51d57fa700 1 -- 172.21.15.53:0/2015219629 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e409e250 con 0x55f44de70680 2023-11-06T01:41:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.542+0000 7f51d57fa700 1 -- 172.21.15.53:0/2015219629 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51e40a2400 con 0x55f44de70680 2023-11-06T01:41:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.542+0000 7f51d57fa700 1 -- 172.21.15.53:0/2015219629 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e409a680 con 0x55f44de70680 2023-11-06T01:41:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.542+0000 7f52015b3b80 1 -- 172.21.15.53:0/2015219629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70680 msgr2=0x55f44de70aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.542+0000 7f52015b3b80 1 --2- 172.21.15.53:0/2015219629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70680 0x55f44de70aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f51e4096e90 tx=0x7f51e4009c00 comp rx=0 tx=0).stop 2023-11-06T01:41:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 -- 172.21.15.53:0/2015219629 shutdown_connections 2023-11-06T01:41:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 --2- 172.21.15.53:0/2015219629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de71ed0 0x55f44de72350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 --2- 172.21.15.53:0/2015219629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70680 0x55f44de70aa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 --2- 172.21.15.53:0/2015219629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de70fe0 0x55f44de718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 -- 172.21.15.53:0/2015219629 >> 172.21.15.53:0/2015219629 conn(0x55f44ddd0070 msgr2=0x55f44de6d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 -- 172.21.15.53:0/2015219629 shutdown_connections 2023-11-06T01:41:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 -- 172.21.15.53:0/2015219629 wait complete. 2023-11-06T01:41:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 -- start start 2023-11-06T01:41:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 Processor -- start 2023-11-06T01:41:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de70680 0x55f44de7a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70fe0 0x55f44de7a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de71ed0 0x55f44de81730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44dd5e4b0 con 0x55f44de70680 2023-11-06T01:41:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44dd5e150 con 0x55f44de70fe0 2023-11-06T01:41:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.543+0000 7f52015b3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44dc458c0 con 0x55f44de71ed0 2023-11-06T01:41:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f52015b3b80 10 client.0 _ll_register_callbacks cb 0x55f44dcaba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51eeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de70680 0x55f44de7a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51ee7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70fe0 0x55f44de7a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51eeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de70680 0x55f44de7a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42146/0 (socket says 172.21.15.53:42146) 2023-11-06T01:41:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51ee7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70fe0 0x55f44de7a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53584/0 (socket says 172.21.15.53:53584) 2023-11-06T01:41:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51eeffd700 1 -- 172.21.15.53:0/1664435275 learned_addr learned my addr 172.21.15.53:0/1664435275 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51ef7fe700 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de71ed0 0x55f44de81730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51eeffd700 1 -- 172.21.15.53:0/1664435275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de71ed0 msgr2=0x55f44de81730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51eeffd700 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de71ed0 0x55f44de81730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51eeffd700 1 -- 172.21.15.53:0/1664435275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70fe0 msgr2=0x55f44de7a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51eeffd700 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70fe0 0x55f44de7a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51eeffd700 1 -- 172.21.15.53:0/1664435275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f0116c50 con 0x55f44de70680 2023-11-06T01:41:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70fe0 0x55f44de7a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51eeffd700 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de70680 0x55f44de7a1b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f51e40a1ed0 tx=0x7f51e4096d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6411cff60a92ae00 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:41:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1664435275 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e409a880 con 0x55f44de70680 2023-11-06T01:41:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1664435275 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f51e40850c0 con 0x55f44de70680 2023-11-06T01:41:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f52015b3b80 1 -- 172.21.15.53:0/1664435275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f44ddeccb0 con 0x55f44de70680 2023-11-06T01:41:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1664435275 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51e409e250 con 0x55f44de70680 2023-11-06T01:41:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f52015b3b80 10 client.16344 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.544+0000 7f52015b3b80 1 -- 172.21.15.53:0/1664435275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55f44dbd8c00 con 0x55f44de70680 2023-11-06T01:41:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.545+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1664435275 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51e40b8020 con 0x55f44de70680 2023-11-06T01:41:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.545+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1664435275 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f51e40a0020 con 0x55f44de70680 2023-11-06T01:41:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.545+0000 7f52015b3b80 10 client.16344 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.546+0000 7f52015b3b80 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.546+0000 7f52015b3b80 20 client.16344 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.546+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f44de85b50 0x55f44de87f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.546+0000 7f52015b3b80 4 client.16344 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:41:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.546+0000 7f52015b3b80 1 -- 172.21.15.53:0/1664435275 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f44de3fdb0 con 0x55f44de85b50 2023-11-06T01:41:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.546+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f44de85b50 0x55f44de87f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.546+0000 7f51d2ff5700 10 client.16344 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.546+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f44de85b50 0x55f44de87f70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c482f8b7d3375ab server_cookie=28ad24d85be7db75 in_seq=0 out_seq=0 2023-11-06T01:41:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1664435275 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f44de3fdb0 con 0x55f44de85b50 2023-11-06T01:41:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f51d2ff5700 10 client.16344 handle_command_reply: tid=0 2023-11-06T01:41:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 1 client.16344 shutdown 2023-11-06T01:41:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 1 -- 172.21.15.53:0/1664435275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de70680 msgr2=0x55f44de7a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de70680 0x55f44de7a1b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f51e40a1ed0 tx=0x7f51e4096d90 comp rx=0 tx=0).stop 2023-11-06T01:41:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 1 -- 172.21.15.53:0/1664435275 shutdown_connections 2023-11-06T01:41:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f44de85b50 0x55f44de87f70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de71ed0 0x55f44de81730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f51d2ff5700 0 client.16344 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de70680 0x55f44de7a1b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1664435275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70fe0 0x55f44de7a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 1 -- 172.21.15.53:0/1664435275 >> 172.21.15.53:0/1664435275 conn(0x55f44ddd0070 msgr2=0x55f44de6b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 1 -- 172.21.15.53:0/1664435275 shutdown_connections 2023-11-06T01:41:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 1 -- 172.21.15.53:0/1664435275 wait complete. 2023-11-06T01:41:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.547+0000 7f52015b3b80 20 client.16344 trim_cache size 0 max 16384 2023-11-06T01:41:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.565+0000 7f52015b3b80 1 Processor -- start 2023-11-06T01:41:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.565+0000 7f52015b3b80 1 -- start start 2023-11-06T01:41:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.566+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70680 0x55f44de7f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.566+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44de79860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.566+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 0x55f44de81f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.566+0000 7f52015b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44de3fdb0 con 0x55f44de71ed0 2023-11-06T01:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.566+0000 7f52015b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44de68710 con 0x55f44de70680 2023-11-06T01:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.566+0000 7f52015b3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44de71d50 con 0x55f44de70fe0 2023-11-06T01:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.566+0000 7f51ee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44de79860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.566+0000 7f51ee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44de79860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56394/0 (socket says 172.21.15.53:56394) 2023-11-06T01:41:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.566+0000 7f51ee7fc700 1 -- 172.21.15.53:0/3662580179 learned_addr learned my addr 172.21.15.53:0/3662580179 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.566+0000 7f51ef7fe700 1 --2- 172.21.15.53:0/3662580179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 0x55f44de81f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.567+0000 7f51ee7fc700 1 -- 172.21.15.53:0/3662580179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70680 msgr2=0x55f44de7f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.567+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/3662580179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70680 0x55f44de7f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.567+0000 7f51ee7fc700 1 -- 172.21.15.53:0/3662580179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 msgr2=0x55f44de81f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.567+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/3662580179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 0x55f44de81f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.567+0000 7f51ee7fc700 1 -- 172.21.15.53:0/3662580179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f44dc161f0 con 0x55f44de70fe0 2023-11-06T01:41:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.567+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/3662580179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44de79860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55f44de88880 tx=0x7f51e800b050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8e885afefa38654 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.567+0000 7f51d5ffb700 1 -- 172.21.15.53:0/3662580179 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8028070 con 0x55f44de70fe0 2023-11-06T01:41:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.567+0000 7f51d5ffb700 1 -- 172.21.15.53:0/3662580179 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51e800f040 con 0x55f44de70fe0 2023-11-06T01:41:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.567+0000 7f51d5ffb700 1 -- 172.21.15.53:0/3662580179 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8015e00 con 0x55f44de70fe0 2023-11-06T01:41:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.568+0000 7f52015b3b80 1 -- 172.21.15.53:0/3662580179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 msgr2=0x55f44de79860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.568+0000 7f52015b3b80 1 --2- 172.21.15.53:0/3662580179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44de79860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55f44de88880 tx=0x7f51e800b050 comp rx=0 tx=0).stop 2023-11-06T01:41:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.568+0000 7f52015b3b80 1 -- 172.21.15.53:0/3662580179 shutdown_connections 2023-11-06T01:41:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.568+0000 7f52015b3b80 1 --2- 172.21.15.53:0/3662580179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70680 0x55f44de7f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.568+0000 7f52015b3b80 1 --2- 172.21.15.53:0/3662580179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 0x55f44de81f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.568+0000 7f52015b3b80 1 --2- 172.21.15.53:0/3662580179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44de79860 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.568+0000 7f52015b3b80 1 -- 172.21.15.53:0/3662580179 >> 172.21.15.53:0/3662580179 conn(0x55f44ddd0070 msgr2=0x55f44de782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f52015b3b80 1 -- 172.21.15.53:0/3662580179 shutdown_connections 2023-11-06T01:41:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f52015b3b80 1 -- 172.21.15.53:0/3662580179 wait complete. 2023-11-06T01:41:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f52015b3b80 1 -- start start 2023-11-06T01:41:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f52015b3b80 1 Processor -- start 2023-11-06T01:41:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70680 0x55f44decf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44decfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f52015b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 0x55f44ded6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f52015b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44ded6c70 con 0x55f44de71ed0 2023-11-06T01:41:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f52015b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44ded6de0 con 0x55f44de70680 2023-11-06T01:41:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f52015b3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44ded6fa0 con 0x55f44de70fe0 2023-11-06T01:41:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f52015b3b80 10 client.0 _ll_register_callbacks cb 0x7f51e000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:41:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f51ef7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 0x55f44ded6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.569+0000 7f51ee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44decfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51eeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70680 0x55f44decf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51ee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44decfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56402/0 (socket says 172.21.15.53:56402) 2023-11-06T01:41:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51ef7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 0x55f44ded6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42164/0 (socket says 172.21.15.53:42164) 2023-11-06T01:41:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51eeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70680 0x55f44decf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53598/0 (socket says 172.21.15.53:53598) 2023-11-06T01:41:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51ee7fc700 1 -- 172.21.15.53:0/1364359278 learned_addr learned my addr 172.21.15.53:0/1364359278 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:41:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51ee7fc700 1 -- 172.21.15.53:0/1364359278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70680 msgr2=0x55f44decf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70680 0x55f44decf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51ee7fc700 1 -- 172.21.15.53:0/1364359278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 msgr2=0x55f44ded6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:41:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 0x55f44ded6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51ee7fc700 1 -- 172.21.15.53:0/1364359278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e8016ac0 con 0x55f44de70fe0 2023-11-06T01:41:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51ef7fe700 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 0x55f44ded6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:41:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51ee7fc700 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44decfdb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51e8016830 tx=0x7f51e8017e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0bd28d2c2953d02 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:41:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:41:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1364359278 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8028040 con 0x55f44de70fe0 2023-11-06T01:41:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1364359278 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f51e8015440 con 0x55f44de70fe0 2023-11-06T01:41:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f52015b3b80 1 -- 172.21.15.53:0/1364359278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51e40a2400 con 0x55f44de70fe0 2023-11-06T01:41:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1364359278 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51e8015620 con 0x55f44de70fe0 2023-11-06T01:41:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f52015b3b80 10 client.25990 fetch_fsmap learned FSMap version 42 2023-11-06T01:41:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.570+0000 7f52015b3b80 1 -- 172.21.15.53:0/1364359278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55f44dbee250 con 0x55f44de70fe0 2023-11-06T01:41:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.571+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1364359278 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51e40a2400 con 0x55f44de70fe0 2023-11-06T01:41:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.571+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1364359278 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f51e8005160 con 0x55f44de70fe0 2023-11-06T01:41:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.571+0000 7f52015b3b80 10 client.25990 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:41:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.571+0000 7f52015b3b80 10 client.25990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:41:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.571+0000 7f52015b3b80 20 client.25990 populate_metadata read hostname 'smithi053' 2023-11-06T01:41:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.571+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f44ded8a60 0x55f44dedae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:41:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.571+0000 7f52015b3b80 4 client.25990 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:41:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.571+0000 7f52015b3b80 1 -- 172.21.15.53:0/1364359278 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f44ddbace0 con 0x55f44ded8a60 2023-11-06T01:41:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.571+0000 7f51eeffd700 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f44ded8a60 0x55f44dedae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:41:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.571+0000 7f51d2ff5700 10 client.25990 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.571+0000 7f51eeffd700 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f44ded8a60 0x55f44dedae80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc0587f7ce49b90 server_cookie=e561420512e961de in_seq=0 out_seq=0 2023-11-06T01:41:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f51d2ff5700 1 -- 172.21.15.53:0/1364359278 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f44ddbace0 con 0x55f44ded8a60 2023-11-06T01:41:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f51d2ff5700 10 client.25990 handle_command_reply: tid=0 2023-11-06T01:41:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f52015b3b80 1 client.25990 shutdown 2023-11-06T01:41:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f52015b3b80 1 -- 172.21.15.53:0/1364359278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 msgr2=0x55f44decfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44decfdb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51e8016830 tx=0x7f51e8017e60 comp rx=0 tx=0).stop 2023-11-06T01:41:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f52015b3b80 1 -- 172.21.15.53:0/1364359278 shutdown_connections 2023-11-06T01:41:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f44de70680 0x55f44decf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55f44ded8a60 0x55f44dedae80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f44de71ed0 0x55f44ded6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f52015b3b80 1 --2- 172.21.15.53:0/1364359278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f44de70fe0 0x55f44decfdb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f52015b3b80 1 -- 172.21.15.53:0/1364359278 >> 172.21.15.53:0/1364359278 conn(0x55f44ddd0070 msgr2=0x55f44de81f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.580+0000 7f51d2ff5700 0 client.25990 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:41:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f52015b3b80 1 -- 172.21.15.53:0/1364359278 shutdown_connections 2023-11-06T01:41:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f52015b3b80 1 -- 172.21.15.53:0/1364359278 wait complete. 2023-11-06T01:41:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f52015b3b80 20 client.25990 trim_cache size 0 max 16384 2023-11-06T01:41:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:41:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:41:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:41:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:41:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 -- 172.21.15.53:0/2867989826 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f51d80a0520 msgr2=0x7f51d80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f51d80a0520 0x7f51d80a29e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f51f0115470 tx=0x7f51e80091b0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 -- 172.21.15.53:0/2867989826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0107d90 msgr2=0x7f51f0114420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:41:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0107d90 0x7f51f0114420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009230 tx=0x7f51e400b840 comp rx=0 tx=0).stop 2023-11-06T01:41:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 -- 172.21.15.53:0/2867989826 shutdown_connections 2023-11-06T01:41:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f51d80a0520 0x7f51d80a29e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f010d0d0 0x7f51f0115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0107d90 0x7f51f0114420 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 --2- 172.21.15.53:0/2867989826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f010a700 0x7f51f0114960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:41:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 -- 172.21.15.53:0/2867989826 >> 172.21.15.53:0/2867989826 conn(0x7f51f006dbc0 msgr2=0x7f51f01096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:41:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 -- 172.21.15.53:0/2867989826 shutdown_connections 2023-11-06T01:41:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:41:56.582+0000 7f51f5423700 1 -- 172.21.15.53:0/2867989826 wait complete. 2023-11-06T01:41:57.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 460.0 s, 1553.6 tps, lat 63.526 ms stddev 70.386 2023-11-06T01:41:57.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:57 smithi062 ceph-mon[103924]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 674 op/s 2023-11-06T01:41:57.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:57 smithi163 ceph-mon[104416]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 674 op/s 2023-11-06T01:41:57.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:57 smithi053 ceph-mon[99472]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 674 op/s 2023-11-06T01:41:59.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:41:59 smithi062 ceph-mon[103924]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 654 op/s 2023-11-06T01:41:59.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:41:59 smithi163 ceph-mon[104416]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 654 op/s 2023-11-06T01:41:59.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:41:59 smithi053 ceph-mon[99472]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 654 op/s 2023-11-06T01:42:01.746 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:01 smithi163 ceph-mon[104416]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 662 op/s 2023-11-06T01:42:01.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:01 smithi053 ceph-mon[99472]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 662 op/s 2023-11-06T01:42:01.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:01 smithi062 ceph-mon[103924]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 662 op/s 2023-11-06T01:42:02.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 465.0 s, 1864.0 tps, lat 54.017 ms stddev 62.298 2023-11-06T01:42:03.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:03 smithi163 ceph-mon[104416]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 806 op/s 2023-11-06T01:42:03.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:03 smithi053 ceph-mon[99472]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 806 op/s 2023-11-06T01:42:03.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:03 smithi062 ceph-mon[103924]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 806 op/s 2023-11-06T01:42:05.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:05 smithi163 ceph-mon[104416]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 730 op/s 2023-11-06T01:42:05.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:05 smithi053 ceph-mon[99472]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 730 op/s 2023-11-06T01:42:05.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:05 smithi062 ceph-mon[103924]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 730 op/s 2023-11-06T01:42:07.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 470.0 s, 2139.8 tps, lat 47.177 ms stddev 60.314 2023-11-06T01:42:07.750 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:07 smithi163 ceph-mon[104416]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 712 op/s 2023-11-06T01:42:07.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:07 smithi053 ceph-mon[99472]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 712 op/s 2023-11-06T01:42:07.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:07 smithi062 ceph-mon[103924]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 712 op/s 2023-11-06T01:42:07.990 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:42:08.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.228+0000 7f445ccaf700 1 -- 172.21.15.53:0/262920641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4458107d90 msgr2=0x7f445810a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:08.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.228+0000 7f445ccaf700 1 --2- 172.21.15.53:0/262920641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4458107d90 0x7f445810a1c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f445000b0a0 tx=0x7f445000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.229+0000 7f445ccaf700 1 -- 172.21.15.53:0/262920641 shutdown_connections 2023-11-06T01:42:08.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.229+0000 7f445ccaf700 1 --2- 172.21.15.53:0/262920641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f445810d0d0 0x7f445810d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.229+0000 7f445ccaf700 1 --2- 172.21.15.53:0/262920641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4458107d90 0x7f445810a1c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.229+0000 7f445ccaf700 1 --2- 172.21.15.53:0/262920641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f445810a700 0x7f445810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.229+0000 7f445ccaf700 1 -- 172.21.15.53:0/262920641 >> 172.21.15.53:0/262920641 conn(0x7f445806db40 msgr2=0x7f445806ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:08.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.247+0000 7f445ccaf700 1 -- 172.21.15.53:0/262920641 shutdown_connections 2023-11-06T01:42:08.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.257+0000 7f445ccaf700 1 -- 172.21.15.53:0/262920641 wait complete. 2023-11-06T01:42:08.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.258+0000 7f445ccaf700 1 Processor -- start 2023-11-06T01:42:08.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.258+0000 7f445ccaf700 1 -- start start 2023-11-06T01:42:08.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.258+0000 7f445ccaf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4458107d90 0x7f44581142f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.258+0000 7f445ccaf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445810a700 0x7f4458114830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.258+0000 7f445ccaf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f445810d0d0 0x7f44581a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.258+0000 7f445ccaf700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4458117980 con 0x7f445810a700 2023-11-06T01:42:08.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.258+0000 7f445ccaf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4458117af0 con 0x7f445810d0d0 2023-11-06T01:42:08.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.258+0000 7f445ccaf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44581a5020 con 0x7f4458107d90 2023-11-06T01:42:08.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.259+0000 7f4455d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445810a700 0x7f4458114830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:08.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.259+0000 7f4455d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445810a700 0x7f4458114830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42176/0 (socket says 172.21.15.53:42176) 2023-11-06T01:42:08.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.259+0000 7f4455d9b700 1 -- 172.21.15.53:0/2407109548 learned_addr learned my addr 172.21.15.53:0/2407109548 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:08.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.259+0000 7f4455d9b700 1 -- 172.21.15.53:0/2407109548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4458107d90 msgr2=0x7f44581142f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-06T01:42:08.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.259+0000 7f4455d9b700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4458107d90 0x7f44581142f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.260+0000 7f4455d9b700 1 -- 172.21.15.53:0/2407109548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f445810d0d0 msgr2=0x7f44581a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:08.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.260+0000 7f4455d9b700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f445810d0d0 0x7f44581a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.260+0000 7f4455d9b700 1 -- 172.21.15.53:0/2407109548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4450009d00 con 0x7f445810a700 2023-11-06T01:42:08.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.261+0000 7f4455d9b700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445810a700 0x7f4458114830 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f444800c920 tx=0x7f444800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:08.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.264+0000 7f44477fe700 1 -- 172.21.15.53:0/2407109548 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448007b00 con 0x7f445810a700 2023-11-06T01:42:08.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.264+0000 7f44477fe700 1 -- 172.21.15.53:0/2407109548 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4448004cf0 con 0x7f445810a700 2023-11-06T01:42:08.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.264+0000 7f44477fe700 1 -- 172.21.15.53:0/2407109548 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448005650 con 0x7f445810a700 2023-11-06T01:42:08.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.264+0000 7f445ccaf700 1 -- 172.21.15.53:0/2407109548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44581a5300 con 0x7f445810a700 2023-11-06T01:42:08.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.264+0000 7f445ccaf700 1 -- 172.21.15.53:0/2407109548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44581a5840 con 0x7f445810a700 2023-11-06T01:42:08.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.270+0000 7f4468e3fb80 1 Processor -- start 2023-11-06T01:42:08.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.271+0000 7f4468e3fb80 1 -- start start 2023-11-06T01:42:08.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.271+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x557102485530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.271+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557102485a70 0x557102483e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.271+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102484870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.271+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571023dba10 con 0x5571024843f0 2023-11-06T01:42:08.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.271+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571023d93d0 con 0x557102485a70 2023-11-06T01:42:08.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.271+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571023d4cc0 con 0x5571024850f0 2023-11-06T01:42:08.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.271+0000 7f44477fe700 1 -- 172.21.15.53:0/2407109548 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4448007c60 con 0x7f445810a700 2023-11-06T01:42:08.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.271+0000 7f4456d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x557102485530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:08.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.271+0000 7f4456d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x557102485530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56418/0 (socket says 172.21.15.53:56418) 2023-11-06T01:42:08.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.271+0000 7f4456d9d700 1 -- 172.21.15.53:0/2789064295 learned_addr learned my addr 172.21.15.53:0/2789064295 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:08.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.272+0000 7f4456d9d700 1 -- 172.21.15.53:0/2789064295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557102485a70 msgr2=0x557102483e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:42:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.272+0000 7f4456d9d700 1 --2- 172.21.15.53:0/2789064295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557102485a70 0x557102483e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.272+0000 7f4456d9d700 1 -- 172.21.15.53:0/2789064295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 msgr2=0x557102484870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.272+0000 7f4456d9d700 1 --2- 172.21.15.53:0/2789064295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102484870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.272+0000 7f4456d9d700 1 -- 172.21.15.53:0/2789064295 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557102202de0 con 0x5571024850f0 2023-11-06T01:42:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.272+0000 7f44477fe700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44400a0620 0x7f44400a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.272+0000 7f44477fe700 1 -- 172.21.15.53:0/2407109548 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f44480cb850 con 0x7f445810a700 2023-11-06T01:42:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.274+0000 7f4456d9d700 1 --2- 172.21.15.53:0/2789064295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x557102485530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f444c0097f0 tx=0x7f444c009b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a5d3876b2e30de server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.276+0000 7f4444ff9700 1 -- 172.21.15.53:0/2789064295 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c011830 con 0x5571024850f0 2023-11-06T01:42:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.276+0000 7f4444ff9700 1 -- 172.21.15.53:0/2789064295 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f444c011e50 con 0x5571024850f0 2023-11-06T01:42:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.276+0000 7f4444ff9700 1 -- 172.21.15.53:0/2789064295 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c010380 con 0x5571024850f0 2023-11-06T01:42:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.277+0000 7f4468e3fb80 1 -- 172.21.15.53:0/2789064295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 msgr2=0x557102485530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.277+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/2789064295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x557102485530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f444c0097f0 tx=0x7f444c009b00 comp rx=0 tx=0).stop 2023-11-06T01:42:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.277+0000 7f445659c700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44400a0620 0x7f44400a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.278+0000 7f4468e3fb80 1 -- 172.21.15.53:0/2789064295 shutdown_connections 2023-11-06T01:42:08.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.278+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/2789064295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102484870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.278+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/2789064295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x557102485530 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.278+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/2789064295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557102485a70 0x557102483e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.278+0000 7f4468e3fb80 1 -- 172.21.15.53:0/2789064295 >> 172.21.15.53:0/2789064295 conn(0x5571023e30d0 msgr2=0x55710247e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:08.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.278+0000 7f445659c700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44400a0620 0x7f44400a2ae0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f445000b830 tx=0x7f4450006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:08.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.288+0000 7f4468e3fb80 1 -- 172.21.15.53:0/2789064295 shutdown_connections 2023-11-06T01:42:08.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.290+0000 7f4468e3fb80 1 -- 172.21.15.53:0/2789064295 wait complete. 2023-11-06T01:42:08.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.290+0000 7f4468e3fb80 1 -- start start 2023-11-06T01:42:08.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.291+0000 7f4468e3fb80 1 Processor -- start 2023-11-06T01:42:08.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.291+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x55710248da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.291+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x55710248dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.291+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557102485a70 0x557102494fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.291+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557102371510 con 0x5571024843f0 2023-11-06T01:42:08.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.291+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571023711b0 con 0x557102485a70 2023-11-06T01:42:08.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.291+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557102258920 con 0x5571024850f0 2023-11-06T01:42:08.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.291+0000 7f4468e3fb80 10 client.0 _ll_register_callbacks cb 0x5571022bea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:08.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.291+0000 7f445659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x55710248da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:08.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.291+0000 7f445659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x55710248da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42184/0 (socket says 172.21.15.53:42184) 2023-11-06T01:42:08.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.291+0000 7f445659c700 1 -- 172.21.15.53:0/1958630444 learned_addr learned my addr 172.21.15.53:0/1958630444 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.292+0000 7f445659c700 1 -- 172.21.15.53:0/1958630444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 msgr2=0x55710248dfa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:42:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.292+0000 7f445659c700 1 --2- 172.21.15.53:0/1958630444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x55710248dfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.292+0000 7f445659c700 1 -- 172.21.15.53:0/1958630444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557102485a70 msgr2=0x557102494fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:42:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.292+0000 7f445659c700 1 --2- 172.21.15.53:0/1958630444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557102485a70 0x557102494fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.292+0000 7f445659c700 1 -- 172.21.15.53:0/1958630444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c009420 con 0x5571024843f0 2023-11-06T01:42:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.292+0000 7f445659c700 1 --2- 172.21.15.53:0/1958630444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x55710248da60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4450015000 tx=0x7f445000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18573bf8ace81b99 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:08.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.292+0000 7f443dffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:42:08.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.292+0000 7f443dffb700 1 -- 172.21.15.53:0/1958630444 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445000e040 con 0x5571024843f0 2023-11-06T01:42:08.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.292+0000 7f443dffb700 1 -- 172.21.15.53:0/1958630444 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4450005480 con 0x5571024843f0 2023-11-06T01:42:08.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.292+0000 7f4468e3fb80 1 -- 172.21.15.53:0/1958630444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571023ffd10 con 0x5571024843f0 2023-11-06T01:42:08.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.293+0000 7f443dffb700 1 -- 172.21.15.53:0/1958630444 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4450006160 con 0x5571024843f0 2023-11-06T01:42:08.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.293+0000 7f4468e3fb80 10 client.16359 fetch_fsmap learned FSMap version 42 2023-11-06T01:42:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.293+0000 7f4468e3fb80 1 -- 172.21.15.53:0/1958630444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5571021ebc60 con 0x5571024843f0 2023-11-06T01:42:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.293+0000 7f443dffb700 1 -- 172.21.15.53:0/1958630444 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571023ffd10 con 0x5571024843f0 2023-11-06T01:42:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.293+0000 7f443dffb700 1 -- 172.21.15.53:0/1958630444 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f445001bcb0 con 0x5571024843f0 2023-11-06T01:42:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.294+0000 7f4468e3fb80 10 client.16359 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:42:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.294+0000 7f4468e3fb80 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.294+0000 7f4468e3fb80 20 client.16359 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.294+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/1958630444 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557102499990 0x55710249be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.294+0000 7f4468e3fb80 4 client.16359 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:42:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.294+0000 7f4468e3fb80 1 -- 172.21.15.53:0/1958630444 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557102452e10 con 0x557102499990 2023-11-06T01:42:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.297+0000 7f4455d9b700 1 --2- 172.21.15.53:0/1958630444 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557102499990 0x55710249be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.298+0000 7f4455d9b700 1 --2- 172.21.15.53:0/1958630444 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557102499990 0x55710249be10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d80a6069c2598abc server_cookie=e9d04df9ae20f897 in_seq=0 out_seq=0 2023-11-06T01:42:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.298+0000 7f443dffb700 10 client.16359 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:08.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.301+0000 7f443dffb700 1 -- 172.21.15.53:0/1958630444 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557102452e10 con 0x557102499990 2023-11-06T01:42:08.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.301+0000 7f443dffb700 10 client.16359 handle_command_reply: tid=0 2023-11-06T01:42:08.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.301+0000 7f4468e3fb80 1 client.16359 shutdown 2023-11-06T01:42:08.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.302+0000 7f4468e3fb80 1 -- 172.21.15.53:0/1958630444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 msgr2=0x55710248da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:08.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.302+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/1958630444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x55710248da60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4450015000 tx=0x7f445000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.303+0000 7f4468e3fb80 1 -- 172.21.15.53:0/1958630444 shutdown_connections 2023-11-06T01:42:08.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.303+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/1958630444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557102485a70 0x557102494fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.303+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/1958630444 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557102499990 0x55710249be10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.303+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/1958630444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x55710248da60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.303+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/1958630444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x55710248dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.303+0000 7f4468e3fb80 1 -- 172.21.15.53:0/1958630444 >> 172.21.15.53:0/1958630444 conn(0x5571023e30d0 msgr2=0x55710247e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:08.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.303+0000 7f443dffb700 0 client.16359 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:08.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.303+0000 7f4468e3fb80 1 -- 172.21.15.53:0/1958630444 shutdown_connections 2023-11-06T01:42:08.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.304+0000 7f4468e3fb80 1 -- 172.21.15.53:0/1958630444 wait complete. 2023-11-06T01:42:08.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.304+0000 7f4468e3fb80 20 client.16359 trim_cache size 0 max 16384 2023-11-06T01:42:08.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.330+0000 7f4468e3fb80 1 Processor -- start 2023-11-06T01:42:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.330+0000 7f4468e3fb80 1 -- start start 2023-11-06T01:42:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5571023e30d0 0x557102483e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102492f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x557102488310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55710248ec50 con 0x5571024843f0 2023-11-06T01:42:08.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557102479420 con 0x5571023e30d0 2023-11-06T01:42:08.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55710247f240 con 0x5571024850f0 2023-11-06T01:42:08.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4456d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x557102488310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:08.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f445659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5571023e30d0 0x557102483e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:08.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4455d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102492f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:08.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f445659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5571023e30d0 0x557102483e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53630/0 (socket says 172.21.15.53:53630) 2023-11-06T01:42:08.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4455d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102492f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42194/0 (socket says 172.21.15.53:42194) 2023-11-06T01:42:08.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f445659c700 1 -- 172.21.15.53:0/2932477298 learned_addr learned my addr 172.21.15.53:0/2932477298 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:08.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4456d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x557102488310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56436/0 (socket says 172.21.15.53:56436) 2023-11-06T01:42:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4455d9b700 1 -- 172.21.15.53:0/2932477298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 msgr2=0x557102488310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4455d9b700 1 --2- 172.21.15.53:0/2932477298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x557102488310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4455d9b700 1 -- 172.21.15.53:0/2932477298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5571023e30d0 msgr2=0x557102483e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4455d9b700 1 --2- 172.21.15.53:0/2932477298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5571023e30d0 0x557102483e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f4455d9b700 1 -- 172.21.15.53:0/2932477298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557102229250 con 0x5571024843f0 2023-11-06T01:42:08.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.331+0000 7f445659c700 1 --2- 172.21.15.53:0/2932477298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5571023e30d0 0x557102483e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:42:08.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.332+0000 7f4455d9b700 1 --2- 172.21.15.53:0/2932477298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102492f60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f44480c92a0 tx=0x7f44480cba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3421b38ed0a0cd1c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:08.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.332+0000 7f44457fa700 1 -- 172.21.15.53:0/2932477298 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444800de20 con 0x5571024843f0 2023-11-06T01:42:08.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.332+0000 7f44457fa700 1 -- 172.21.15.53:0/2932477298 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4448099730 con 0x5571024843f0 2023-11-06T01:42:08.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.332+0000 7f44457fa700 1 -- 172.21.15.53:0/2932477298 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448099290 con 0x5571024843f0 2023-11-06T01:42:08.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 -- 172.21.15.53:0/2932477298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 msgr2=0x557102492f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:08.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/2932477298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102492f60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f44480c92a0 tx=0x7f44480cba60 comp rx=0 tx=0).stop 2023-11-06T01:42:08.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 -- 172.21.15.53:0/2932477298 shutdown_connections 2023-11-06T01:42:08.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/2932477298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5571023e30d0 0x557102483e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/2932477298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571024850f0 0x557102488310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/2932477298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102492f60 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 -- 172.21.15.53:0/2932477298 >> 172.21.15.53:0/2932477298 conn(0x55710247ec70 msgr2=0x55710249b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 -- 172.21.15.53:0/2932477298 shutdown_connections 2023-11-06T01:42:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 -- 172.21.15.53:0/2932477298 wait complete. 2023-11-06T01:42:08.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 -- start start 2023-11-06T01:42:08.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 Processor -- start 2023-11-06T01:42:08.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571023e30d0 0x557102492d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102491390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5571024850f0 0x5571024918d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.333+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557102491f20 con 0x5571024843f0 2023-11-06T01:42:08.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557102492090 con 0x5571024850f0 2023-11-06T01:42:08.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f4468e3fb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557102492250 con 0x5571023e30d0 2023-11-06T01:42:08.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f4468e3fb80 10 client.0 _ll_register_callbacks cb 0x7f4450004d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:08.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f4455d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102491390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:08.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f445659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571023e30d0 0x557102492d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:08.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f445659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571023e30d0 0x557102492d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56438/0 (socket says 172.21.15.53:56438) 2023-11-06T01:42:08.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f4455d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102491390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42200/0 (socket says 172.21.15.53:42200) 2023-11-06T01:42:08.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f445659c700 1 -- 172.21.15.53:0/3506323439 learned_addr learned my addr 172.21.15.53:0/3506323439 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:08.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f4455d9b700 1 -- 172.21.15.53:0/3506323439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571023e30d0 msgr2=0x557102492d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:08.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f4455d9b700 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571023e30d0 0x557102492d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f4455d9b700 1 -- 172.21.15.53:0/3506323439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5571024850f0 msgr2=0x5571024918d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:42:08.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f4455d9b700 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5571024850f0 0x5571024918d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f4455d9b700 1 -- 172.21.15.53:0/3506323439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571021ebc60 con 0x5571024843f0 2023-11-06T01:42:08.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.334+0000 7f445659c700 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571023e30d0 0x557102492d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:42:08.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.335+0000 7f4455d9b700 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102491390 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55710248ed90 tx=0x7f4448097520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0712756c7e28fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:08.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.335+0000 7f443dffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:42:08.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.336+0000 7f443dffb700 1 -- 172.21.15.53:0/3506323439 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448098720 con 0x5571024843f0 2023-11-06T01:42:08.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.336+0000 7f443dffb700 1 -- 172.21.15.53:0/3506323439 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4448099030 con 0x5571024843f0 2023-11-06T01:42:08.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.336+0000 7f4468e3fb80 1 -- 172.21.15.53:0/3506323439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f444c011e50 con 0x5571024843f0 2023-11-06T01:42:08.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.336+0000 7f443dffb700 1 -- 172.21.15.53:0/3506323439 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4448099290 con 0x5571024843f0 2023-11-06T01:42:08.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.336+0000 7f4468e3fb80 10 client.16365 fetch_fsmap learned FSMap version 42 2023-11-06T01:42:08.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.336+0000 7f4468e3fb80 1 -- 172.21.15.53:0/3506323439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f44581a5840 con 0x5571024843f0 2023-11-06T01:42:08.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.337+0000 7f443dffb700 1 -- 172.21.15.53:0/3506323439 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f444c011e50 con 0x5571024843f0 2023-11-06T01:42:08.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.337+0000 7f443dffb700 1 -- 172.21.15.53:0/3506323439 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4448094f90 con 0x5571024843f0 2023-11-06T01:42:08.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.337+0000 7f4468e3fb80 10 client.16365 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:42:08.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.337+0000 7f4468e3fb80 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:08.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.337+0000 7f4468e3fb80 20 client.16365 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:08.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.337+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557102485a70 0x5571024eeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:08.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.337+0000 7f4468e3fb80 4 client.16365 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:42:08.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.337+0000 7f4468e3fb80 1 -- 172.21.15.53:0/3506323439 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5571023cdd40 con 0x557102485a70 2023-11-06T01:42:08.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.337+0000 7f445659c700 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557102485a70 0x5571024eeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:08.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f443dffb700 10 client.16365 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:08.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f445659c700 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557102485a70 0x5571024eeca0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466b4bbd42c76dd server_cookie=77ee2f300d00f723 in_seq=0 out_seq=0 2023-11-06T01:42:08.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f443dffb700 1 -- 172.21.15.53:0/3506323439 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5571023cdd40 con 0x557102485a70 2023-11-06T01:42:08.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f443dffb700 10 client.16365 handle_command_reply: tid=0 2023-11-06T01:42:08.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f4468e3fb80 1 client.16365 shutdown 2023-11-06T01:42:08.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f4468e3fb80 1 -- 172.21.15.53:0/3506323439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 msgr2=0x557102491390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:08.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102491390 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55710248ed90 tx=0x7f4448097520 comp rx=0 tx=0).stop 2023-11-06T01:42:08.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f4468e3fb80 1 -- 172.21.15.53:0/3506323439 shutdown_connections 2023-11-06T01:42:08.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5571024850f0 0x5571024918d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557102485a70 0x5571024eeca0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571023e30d0 0x557102492d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f4468e3fb80 1 --2- 172.21.15.53:0/3506323439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571024843f0 0x557102491390 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f443dffb700 0 client.16365 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:08.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f4468e3fb80 1 -- 172.21.15.53:0/3506323439 >> 172.21.15.53:0/3506323439 conn(0x55710247ec70 msgr2=0x557102488310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:08.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f4468e3fb80 1 -- 172.21.15.53:0/3506323439 shutdown_connections 2023-11-06T01:42:08.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.338+0000 7f4468e3fb80 1 -- 172.21.15.53:0/3506323439 wait complete. 2023-11-06T01:42:08.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f4468e3fb80 20 client.16365 trim_cache size 0 max 16384 2023-11-06T01:42:08.354 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:42:08.354 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:42:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:42:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:42:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:42:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:42:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:42:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:42:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:42:08.356 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:42:08.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 -- 172.21.15.53:0/2407109548 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44400a0620 msgr2=0x7f44400a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:08.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44400a0620 0x7f44400a2ae0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f445000b830 tx=0x7f4450006000 comp rx=0 tx=0).stop 2023-11-06T01:42:08.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 -- 172.21.15.53:0/2407109548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445810a700 msgr2=0x7f4458114830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:08.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445810a700 0x7f4458114830 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f444800c920 tx=0x7f444800cce0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 -- 172.21.15.53:0/2407109548 shutdown_connections 2023-11-06T01:42:08.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f44400a0620 0x7f44400a2ae0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f445810d0d0 0x7f44581a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4458107d90 0x7f44581142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 --2- 172.21.15.53:0/2407109548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445810a700 0x7f4458114830 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:08.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 -- 172.21.15.53:0/2407109548 >> 172.21.15.53:0/2407109548 conn(0x7f445806db40 msgr2=0x7f445810b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:08.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 -- 172.21.15.53:0/2407109548 shutdown_connections 2023-11-06T01:42:08.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:08.339+0000 7f445ccaf700 1 -- 172.21.15.53:0/2407109548 wait complete. 2023-11-06T01:42:08.360 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:42:09.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:09 smithi163 ceph-mon[104416]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 775 op/s 2023-11-06T01:42:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.756+0000 7fe119af6700 1 -- 172.21.15.53:0/3474384010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe114100ff0 msgr2=0x7fe1141078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.756+0000 7fe119af6700 1 --2- 172.21.15.53:0/3474384010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe114100ff0 0x7fe1141078a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe10c01c100 tx=0x7fe10c01c410 comp rx=0 tx=0).stop 2023-11-06T01:42:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.756+0000 7fe119af6700 1 -- 172.21.15.53:0/3474384010 shutdown_connections 2023-11-06T01:42:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.756+0000 7fe119af6700 1 --2- 172.21.15.53:0/3474384010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe114100ff0 0x7fe1141078a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.756+0000 7fe119af6700 1 --2- 172.21.15.53:0/3474384010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1140ff460 0x7fe1140ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.756+0000 7fe119af6700 1 --2- 172.21.15.53:0/3474384010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe114100650 0x7fe114100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.756+0000 7fe119af6700 1 -- 172.21.15.53:0/3474384010 >> 172.21.15.53:0/3474384010 conn(0x7fe1140fa9b0 msgr2=0x7fe1140fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.756+0000 7fe119af6700 1 -- 172.21.15.53:0/3474384010 shutdown_connections 2023-11-06T01:42:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.756+0000 7fe119af6700 1 -- 172.21.15.53:0/3474384010 wait complete. 2023-11-06T01:42:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.757+0000 7fe119af6700 1 Processor -- start 2023-11-06T01:42:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.757+0000 7fe119af6700 1 -- start start 2023-11-06T01:42:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.757+0000 7fe119af6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1140ff460 0x7fe11410a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.757+0000 7fe119af6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe114100650 0x7fe11410cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.757+0000 7fe119af6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe114100ff0 0x7fe11410f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.757+0000 7fe119af6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11410f900 con 0x7fe1140ff460 2023-11-06T01:42:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.757+0000 7fe119af6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11410fa70 con 0x7fe114100ff0 2023-11-06T01:42:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.757+0000 7fe119af6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11410fc30 con 0x7fe114100650 2023-11-06T01:42:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.758+0000 7fe112ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe114100650 0x7fe11410cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.758+0000 7fe112ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe114100650 0x7fe11410cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56452/0 (socket says 172.21.15.53:56452) 2023-11-06T01:42:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.758+0000 7fe113fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe114100ff0 0x7fe11410f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.758+0000 7fe112ffd700 1 -- 172.21.15.53:0/3704851599 learned_addr learned my addr 172.21.15.53:0/3704851599 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.758+0000 7fe112ffd700 1 -- 172.21.15.53:0/3704851599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe114100ff0 msgr2=0x7fe11410f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.758+0000 7fe112ffd700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe114100ff0 0x7fe11410f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.758+0000 7fe112ffd700 1 -- 172.21.15.53:0/3704851599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1140ff460 msgr2=0x7fe11410a670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:42:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.758+0000 7fe112ffd700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1140ff460 0x7fe11410a670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.758+0000 7fe112ffd700 1 -- 172.21.15.53:0/3704851599 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c01ad10 con 0x7fe114100650 2023-11-06T01:42:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.758+0000 7fe112ffd700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe114100650 0x7fe11410cbc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe10400e9a0 tx=0x7fe10400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.759+0000 7fe110ff9700 1 -- 172.21.15.53:0/3704851599 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104009790 con 0x7fe114100650 2023-11-06T01:42:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.759+0000 7fe110ff9700 1 -- 172.21.15.53:0/3704851599 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe10400cbd0 con 0x7fe114100650 2023-11-06T01:42:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.759+0000 7fe119af6700 1 -- 172.21.15.53:0/3704851599 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11410ff10 con 0x7fe114100650 2023-11-06T01:42:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.759+0000 7fe110ff9700 1 -- 172.21.15.53:0/3704851599 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104018680 con 0x7fe114100650 2023-11-06T01:42:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.759+0000 7fe119af6700 1 -- 172.21.15.53:0/3704851599 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe114110470 con 0x7fe114100650 2023-11-06T01:42:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.762+0000 7fe110ff9700 1 -- 172.21.15.53:0/3704851599 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe10400cd40 con 0x7fe114100650 2023-11-06T01:42:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.763+0000 7fe110ff9700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0fc0a0620 0x7fe0fc0a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.763+0000 7fe110ff9700 1 -- 172.21.15.53:0/3704851599 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe104014070 con 0x7fe114100650 2023-11-06T01:42:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.764+0000 7fe1137fe700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0fc0a0620 0x7fe0fc0a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.764+0000 7fe125c86b80 1 Processor -- start 2023-11-06T01:42:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.764+0000 7fe125c86b80 1 -- start start 2023-11-06T01:42:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.764+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5680 0x564834dc5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.764+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5fe0 0x564834dc68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 0x564834dc7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe125c86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834d1d9b0 con 0x564834dc5680 2023-11-06T01:42:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe125c86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834d1b370 con 0x564834dc5fe0 2023-11-06T01:42:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe125c86b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834d16c60 con 0x564834dc6ed0 2023-11-06T01:42:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe113fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 0x564834dc7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe113fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 0x564834dc7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56458/0 (socket says 172.21.15.53:56458) 2023-11-06T01:42:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe113fff700 1 -- 172.21.15.53:0/3374830256 learned_addr learned my addr 172.21.15.53:0/3374830256 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe112ffd700 1 --2- 172.21.15.53:0/3374830256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5fe0 0x564834dc68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe113fff700 1 -- 172.21.15.53:0/3374830256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5fe0 msgr2=0x564834dc68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe113fff700 1 --2- 172.21.15.53:0/3374830256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5fe0 0x564834dc68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe113fff700 1 -- 172.21.15.53:0/3374830256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5680 msgr2=0x564834dc5aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:42:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.765+0000 7fe113fff700 1 --2- 172.21.15.53:0/3374830256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5680 0x564834dc5aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.766+0000 7fe113fff700 1 -- 172.21.15.53:0/3374830256 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564834b44d80 con 0x564834dc6ed0 2023-11-06T01:42:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.766+0000 7fe112ffd700 1 --2- 172.21.15.53:0/3374830256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5fe0 0x564834dc68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:42:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.766+0000 7fe1137fe700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0fc0a0620 0x7fe0fc0a2ae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe108005950 tx=0x7fe1080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.766+0000 7fe113fff700 1 --2- 172.21.15.53:0/3374830256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 0x564834dc7350 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe10c01c100 tx=0x7fe10c0061f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd7c46bc9e229d50 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.766+0000 7fe0f9ffb700 1 -- 172.21.15.53:0/3374830256 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c01f040 con 0x564834dc6ed0 2023-11-06T01:42:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.767+0000 7fe0f9ffb700 1 -- 172.21.15.53:0/3374830256 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe10c01dd60 con 0x564834dc6ed0 2023-11-06T01:42:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.767+0000 7fe0f9ffb700 1 -- 172.21.15.53:0/3374830256 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c005820 con 0x564834dc6ed0 2023-11-06T01:42:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.768+0000 7fe125c86b80 1 -- 172.21.15.53:0/3374830256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 msgr2=0x564834dc7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.768+0000 7fe125c86b80 1 --2- 172.21.15.53:0/3374830256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 0x564834dc7350 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe10c01c100 tx=0x7fe10c0061f0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.768+0000 7fe125c86b80 1 -- 172.21.15.53:0/3374830256 shutdown_connections 2023-11-06T01:42:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.768+0000 7fe125c86b80 1 --2- 172.21.15.53:0/3374830256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 0x564834dc7350 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.768+0000 7fe125c86b80 1 --2- 172.21.15.53:0/3374830256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5680 0x564834dc5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.768+0000 7fe125c86b80 1 --2- 172.21.15.53:0/3374830256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5fe0 0x564834dc68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.768+0000 7fe125c86b80 1 -- 172.21.15.53:0/3374830256 >> 172.21.15.53:0/3374830256 conn(0x564834d25070 msgr2=0x564834dc2040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.771+0000 7fe125c86b80 1 -- 172.21.15.53:0/3374830256 shutdown_connections 2023-11-06T01:42:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.771+0000 7fe125c86b80 1 -- 172.21.15.53:0/3374830256 wait complete. 2023-11-06T01:42:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.771+0000 7fe125c86b80 1 -- start start 2023-11-06T01:42:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe125c86b80 1 Processor -- start 2023-11-06T01:42:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834dcf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5fe0 0x564834dcf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc6ed0 0x564834dd69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe125c86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834cb34b0 con 0x564834dc6ed0 2023-11-06T01:42:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe125c86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834cb3150 con 0x564834dc5fe0 2023-11-06T01:42:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe125c86b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834b9a8c0 con 0x564834dc5680 2023-11-06T01:42:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe125c86b80 10 client.0 _ll_register_callbacks cb 0x564834c00a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe1137fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834dcf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe1137fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834dcf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56464/0 (socket says 172.21.15.53:56464) 2023-11-06T01:42:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe1137fe700 1 -- 172.21.15.53:0/3970141753 learned_addr learned my addr 172.21.15.53:0/3970141753 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.772+0000 7fe112ffd700 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5fe0 0x564834dcf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe113fff700 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc6ed0 0x564834dd69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe1137fe700 1 -- 172.21.15.53:0/3970141753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5fe0 msgr2=0x564834dcf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe1137fe700 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5fe0 0x564834dcf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe1137fe700 1 -- 172.21.15.53:0/3970141753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc6ed0 msgr2=0x564834dd69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe1137fe700 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc6ed0 0x564834dd69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe1137fe700 1 -- 172.21.15.53:0/3970141753 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe114110470 con 0x564834dc5680 2023-11-06T01:42:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe1137fe700 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834dcf430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe10800dc90 tx=0x7fe1080047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9693228f5139d1fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe0f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:42:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/3970141753 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108010af0 con 0x564834dc5680 2023-11-06T01:42:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/3970141753 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe108010c50 con 0x564834dc5680 2023-11-06T01:42:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe125c86b80 1 -- 172.21.15.53:0/3970141753 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564834d41cb0 con 0x564834dc5680 2023-11-06T01:42:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/3970141753 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe108010e10 con 0x564834dc5680 2023-11-06T01:42:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe125c86b80 10 client.26011 fetch_fsmap learned FSMap version 42 2023-11-06T01:42:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.773+0000 7fe125c86b80 1 -- 172.21.15.53:0/3970141753 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fe1141081f0 con 0x564834dc5680 2023-11-06T01:42:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.774+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/3970141753 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564834d41cb0 con 0x564834dc5680 2023-11-06T01:42:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.774+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/3970141753 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe10800f600 con 0x564834dc5680 2023-11-06T01:42:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.774+0000 7fe125c86b80 10 client.26011 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:42:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.774+0000 7fe125c86b80 10 client.26011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.774+0000 7fe125c86b80 20 client.26011 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.774+0000 7fe125c86b80 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564834ddb3c0 0x564834ddd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.774+0000 7fe125c86b80 4 client.26011 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:42:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.774+0000 7fe125c86b80 1 -- 172.21.15.53:0/3970141753 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564834d94db0 con 0x564834ddb3c0 2023-11-06T01:42:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.774+0000 7fe112ffd700 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564834ddb3c0 0x564834ddd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.775+0000 7fe0f2ffd700 10 client.26011 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.775+0000 7fe112ffd700 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564834ddb3c0 0x564834ddd7e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc55883a7572c2e8 server_cookie=38542bae5aafa905 in_seq=0 out_seq=0 2023-11-06T01:42:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.775+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/3970141753 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564834d94db0 con 0x564834ddb3c0 2023-11-06T01:42:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.775+0000 7fe0f2ffd700 10 client.26011 handle_command_reply: tid=0 2023-11-06T01:42:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.775+0000 7fe125c86b80 1 client.26011 shutdown 2023-11-06T01:42:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe125c86b80 1 -- 172.21.15.53:0/3970141753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 msgr2=0x564834dcf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe125c86b80 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834dcf430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe10800dc90 tx=0x7fe1080047c0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe125c86b80 1 -- 172.21.15.53:0/3970141753 shutdown_connections 2023-11-06T01:42:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe125c86b80 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564834ddb3c0 0x564834ddd7e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe125c86b80 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc6ed0 0x564834dd69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe0f2ffd700 0 client.26011 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe125c86b80 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834dcf430 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe125c86b80 1 --2- 172.21.15.53:0/3970141753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5fe0 0x564834dcf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe125c86b80 1 -- 172.21.15.53:0/3970141753 >> 172.21.15.53:0/3970141753 conn(0x564834d25070 msgr2=0x564834dc03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe125c86b80 1 -- 172.21.15.53:0/3970141753 shutdown_connections 2023-11-06T01:42:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe125c86b80 1 -- 172.21.15.53:0/3970141753 wait complete. 2023-11-06T01:42:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.776+0000 7fe125c86b80 20 client.26011 trim_cache size 0 max 16384 2023-11-06T01:42:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.797+0000 7fe125c86b80 1 Processor -- start 2023-11-06T01:42:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.797+0000 7fe125c86b80 1 -- start start 2023-11-06T01:42:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.797+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5680 0x564834dd4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.797+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5fe0 0x564834dca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.797+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 0x564834dca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.797+0000 7fe125c86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834db87a0 con 0x564834dc5fe0 2023-11-06T01:42:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.797+0000 7fe125c86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834ddaf50 con 0x564834dc5680 2023-11-06T01:42:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.797+0000 7fe125c86b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834dce8e0 con 0x564834dc6ed0 2023-11-06T01:42:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe112ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5fe0 0x564834dca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe112ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5fe0 0x564834dca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42232/0 (socket says 172.21.15.53:42232) 2023-11-06T01:42:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe112ffd700 1 -- 172.21.15.53:0/2890156229 learned_addr learned my addr 172.21.15.53:0/2890156229 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe113fff700 1 --2- 172.21.15.53:0/2890156229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 0x564834dca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe1137fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5680 0x564834dd4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe112ffd700 1 -- 172.21.15.53:0/2890156229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 msgr2=0x564834dca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe112ffd700 1 --2- 172.21.15.53:0/2890156229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 0x564834dca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe112ffd700 1 -- 172.21.15.53:0/2890156229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5680 msgr2=0x564834dd4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe112ffd700 1 --2- 172.21.15.53:0/2890156229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5680 0x564834dd4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe112ffd700 1 -- 172.21.15.53:0/2890156229 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564834b2dc00 con 0x564834dc5fe0 2023-11-06T01:42:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe113fff700 1 --2- 172.21.15.53:0/2890156229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 0x564834dca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:42:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe1137fe700 1 --2- 172.21.15.53:0/2890156229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5680 0x564834dd4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:42:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe112ffd700 1 --2- 172.21.15.53:0/2890156229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5fe0 0x564834dca120 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe1040990e0 tx=0x7fe104078040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9406ffe2626856 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe0fa7fc700 1 -- 172.21.15.53:0/2890156229 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1040b0070 con 0x564834dc5fe0 2023-11-06T01:42:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe0fa7fc700 1 -- 172.21.15.53:0/2890156229 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1040840b0 con 0x564834dc5fe0 2023-11-06T01:42:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.798+0000 7fe0fa7fc700 1 -- 172.21.15.53:0/2890156229 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1040acea0 con 0x564834dc5fe0 2023-11-06T01:42:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.799+0000 7fe125c86b80 1 -- 172.21.15.53:0/2890156229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5fe0 msgr2=0x564834dca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.799+0000 7fe125c86b80 1 --2- 172.21.15.53:0/2890156229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5fe0 0x564834dca120 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe1040990e0 tx=0x7fe104078040 comp rx=0 tx=0).stop 2023-11-06T01:42:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.799+0000 7fe125c86b80 1 -- 172.21.15.53:0/2890156229 shutdown_connections 2023-11-06T01:42:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.799+0000 7fe125c86b80 1 --2- 172.21.15.53:0/2890156229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc5680 0x564834dd4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.799+0000 7fe125c86b80 1 --2- 172.21.15.53:0/2890156229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc6ed0 0x564834dca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.799+0000 7fe125c86b80 1 --2- 172.21.15.53:0/2890156229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5fe0 0x564834dca120 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.799+0000 7fe125c86b80 1 -- 172.21.15.53:0/2890156229 >> 172.21.15.53:0/2890156229 conn(0x564834d25070 msgr2=0x564834dc1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.799+0000 7fe125c86b80 1 -- 172.21.15.53:0/2890156229 shutdown_connections 2023-11-06T01:42:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.799+0000 7fe125c86b80 1 -- 172.21.15.53:0/2890156229 wait complete. 2023-11-06T01:42:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.800+0000 7fe125c86b80 1 -- start start 2023-11-06T01:42:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.800+0000 7fe125c86b80 1 Processor -- start 2023-11-06T01:42:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.800+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834e250a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.800+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5fe0 0x564834e255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.800+0000 7fe125c86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc6ed0 0x564834e2bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe125c86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834e2c460 con 0x564834dc5fe0 2023-11-06T01:42:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe125c86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834e2c5a0 con 0x564834dc6ed0 2023-11-06T01:42:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe125c86b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564834e2c760 con 0x564834dc5680 2023-11-06T01:42:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe125c86b80 10 client.0 _ll_register_callbacks cb 0x564834c00a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe1137fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834e250a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe1137fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834e250a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56478/0 (socket says 172.21.15.53:56478) 2023-11-06T01:42:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe1137fe700 1 -- 172.21.15.53:0/2879155526 learned_addr learned my addr 172.21.15.53:0/2879155526 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe1137fe700 1 -- 172.21.15.53:0/2879155526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc6ed0 msgr2=0x564834e2bf20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:42:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe1137fe700 1 --2- 172.21.15.53:0/2879155526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc6ed0 0x564834e2bf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe1137fe700 1 -- 172.21.15.53:0/2879155526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5fe0 msgr2=0x564834e255e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe1137fe700 1 --2- 172.21.15.53:0/2879155526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5fe0 0x564834e255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.801+0000 7fe1137fe700 1 -- 172.21.15.53:0/2879155526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1141081f0 con 0x564834dc5680 2023-11-06T01:42:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.802+0000 7fe1137fe700 1 --2- 172.21.15.53:0/2879155526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834e250a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe10800f960 tx=0x7fe10800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5acf7a0a6b6fd5e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.802+0000 7fe0f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:42:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.802+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/2879155526 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800f5c0 con 0x564834dc5680 2023-11-06T01:42:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.802+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/2879155526 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe10800f720 con 0x564834dc5680 2023-11-06T01:42:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.802+0000 7fe125c86b80 1 -- 172.21.15.53:0/2879155526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe10c01dd60 con 0x564834dc5680 2023-11-06T01:42:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.803+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/2879155526 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe108010740 con 0x564834dc5680 2023-11-06T01:42:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.803+0000 7fe125c86b80 10 client.26017 fetch_fsmap learned FSMap version 42 2023-11-06T01:42:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.803+0000 7fe125c86b80 1 -- 172.21.15.53:0/2879155526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x564834b44d80 con 0x564834dc5680 2023-11-06T01:42:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.804+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/2879155526 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe10c01dd60 con 0x564834dc5680 2023-11-06T01:42:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.804+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/2879155526 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe10801cc60 con 0x564834dc5680 2023-11-06T01:42:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.804+0000 7fe125c86b80 10 client.26017 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:42:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.804+0000 7fe125c86b80 10 client.26017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.804+0000 7fe125c86b80 20 client.26017 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.804+0000 7fe125c86b80 1 --2- 172.21.15.53:0/2879155526 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564834e2e250 0x564834e30670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.804+0000 7fe125c86b80 4 client.26017 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:42:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.804+0000 7fe125c86b80 1 -- 172.21.15.53:0/2879155526 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564834d0fce0 con 0x564834e2e250 2023-11-06T01:42:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.804+0000 7fe112ffd700 1 --2- 172.21.15.53:0/2879155526 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564834e2e250 0x564834e30670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.804+0000 7fe0f2ffd700 10 client.26017 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.805+0000 7fe112ffd700 1 --2- 172.21.15.53:0/2879155526 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564834e2e250 0x564834e30670 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd58e45ef89213ff server_cookie=81b766ea58afce3d in_seq=0 out_seq=0 2023-11-06T01:42:09.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:09 smithi053 ceph-mon[99472]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 775 op/s 2023-11-06T01:42:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe0f2ffd700 1 -- 172.21.15.53:0/2879155526 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564834d0fce0 con 0x564834e2e250 2023-11-06T01:42:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe0f2ffd700 10 client.26017 handle_command_reply: tid=0 2023-11-06T01:42:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 1 client.26017 shutdown 2023-11-06T01:42:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 1 -- 172.21.15.53:0/2879155526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 msgr2=0x564834e250a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 1 --2- 172.21.15.53:0/2879155526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834e250a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe10800f960 tx=0x7fe10800b8b0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 1 -- 172.21.15.53:0/2879155526 shutdown_connections 2023-11-06T01:42:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 1 --2- 172.21.15.53:0/2879155526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564834dc6ed0 0x564834e2bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 1 --2- 172.21.15.53:0/2879155526 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564834e2e250 0x564834e30670 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 1 --2- 172.21.15.53:0/2879155526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564834dc5680 0x564834e250a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe0f2ffd700 0 client.26017 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 1 --2- 172.21.15.53:0/2879155526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564834dc5fe0 0x564834e255e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 1 -- 172.21.15.53:0/2879155526 >> 172.21.15.53:0/2879155526 conn(0x564834d25070 msgr2=0x564834dca120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 1 -- 172.21.15.53:0/2879155526 shutdown_connections 2023-11-06T01:42:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 1 -- 172.21.15.53:0/2879155526 wait complete. 2023-11-06T01:42:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.882+0000 7fe125c86b80 20 client.26017 trim_cache size 0 max 16384 2023-11-06T01:42:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:42:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:42:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:42:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:42:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.883+0000 7fe119af6700 1 -- 172.21.15.53:0/3704851599 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0fc0a0620 msgr2=0x7fe0fc0a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.883+0000 7fe119af6700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0fc0a0620 0x7fe0fc0a2ae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe108005950 tx=0x7fe1080058e0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.883+0000 7fe119af6700 1 -- 172.21.15.53:0/3704851599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe114100650 msgr2=0x7fe11410cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.883+0000 7fe119af6700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe114100650 0x7fe11410cbc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe10400e9a0 tx=0x7fe10400ecb0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.883+0000 7fe119af6700 1 -- 172.21.15.53:0/3704851599 shutdown_connections 2023-11-06T01:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.883+0000 7fe119af6700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0fc0a0620 0x7fe0fc0a2ae0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.883+0000 7fe119af6700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe114100ff0 0x7fe11410f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.883+0000 7fe119af6700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1140ff460 0x7fe11410a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.883+0000 7fe119af6700 1 --2- 172.21.15.53:0/3704851599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe114100650 0x7fe11410cbc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.883+0000 7fe119af6700 1 -- 172.21.15.53:0/3704851599 >> 172.21.15.53:0/3704851599 conn(0x7fe1140fa9b0 msgr2=0x7fe1140fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.884+0000 7fe119af6700 1 -- 172.21.15.53:0/3704851599 shutdown_connections 2023-11-06T01:42:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:09.884+0000 7fe119af6700 1 -- 172.21.15.53:0/3704851599 wait complete. 2023-11-06T01:42:09.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:09 smithi062 ceph-mon[103924]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 775 op/s 2023-11-06T01:42:11.849 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:11 smithi053 ceph-mon[99472]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 769 op/s 2023-11-06T01:42:11.903 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:11 smithi062 ceph-mon[103924]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 769 op/s 2023-11-06T01:42:11.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:11 smithi163 ceph-mon[104416]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 769 op/s 2023-11-06T01:42:12.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 475.0 s, 1233.8 tps, lat 80.914 ms stddev 97.734 2023-11-06T01:42:13.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:13 smithi053 ceph-mon[99472]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 858 op/s 2023-11-06T01:42:13.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:13 smithi062 ceph-mon[103924]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 858 op/s 2023-11-06T01:42:13.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:13 smithi163 ceph-mon[104416]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 858 op/s 2023-11-06T01:42:15.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:15 smithi053 ceph-mon[99472]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 773 op/s 2023-11-06T01:42:15.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:15 smithi062 ceph-mon[103924]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 773 op/s 2023-11-06T01:42:15.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:15 smithi163 ceph-mon[104416]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 773 op/s 2023-11-06T01:42:17.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 480.0 s, 1776.2 tps, lat 55.682 ms stddev 59.690 2023-11-06T01:42:17.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:17 smithi053 ceph-mon[99472]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 747 op/s 2023-11-06T01:42:17.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:17 smithi062 ceph-mon[103924]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 747 op/s 2023-11-06T01:42:17.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:17 smithi163 ceph-mon[104416]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 747 op/s 2023-11-06T01:42:19.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:19 smithi053 ceph-mon[99472]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 806 op/s 2023-11-06T01:42:19.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:19 smithi062 ceph-mon[103924]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 806 op/s 2023-11-06T01:42:19.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:19 smithi163 ceph-mon[104416]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 806 op/s 2023-11-06T01:42:21.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:21 smithi053 ceph-mon[99472]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 767 op/s 2023-11-06T01:42:21.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:21 smithi062 ceph-mon[103924]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 767 op/s 2023-11-06T01:42:21.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:21 smithi163 ceph-mon[104416]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 767 op/s 2023-11-06T01:42:22.184 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 485.0 s, 1975.4 tps, lat 50.624 ms stddev 60.357 2023-11-06T01:42:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.062+0000 7fa059067700 1 -- 172.21.15.53:0/415611523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa054100e90 msgr2=0x7fa0541012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.062+0000 7fa059067700 1 --2- 172.21.15.53:0/415611523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa054100e90 0x7fa0541012f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa0480099c0 tx=0x7fa048009cd0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.062+0000 7fa059067700 1 -- 172.21.15.53:0/415611523 shutdown_connections 2023-11-06T01:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.062+0000 7fa059067700 1 --2- 172.21.15.53:0/415611523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0540ffca0 0x7fa0541000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.062+0000 7fa059067700 1 --2- 172.21.15.53:0/415611523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa054101830 0x7fa0541080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.062+0000 7fa059067700 1 --2- 172.21.15.53:0/415611523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa054100e90 0x7fa0541012f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.062+0000 7fa059067700 1 -- 172.21.15.53:0/415611523 >> 172.21.15.53:0/415611523 conn(0x7fa0540fb230 msgr2=0x7fa0540fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.063+0000 7fa059067700 1 -- 172.21.15.53:0/415611523 shutdown_connections 2023-11-06T01:42:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.063+0000 7fa059067700 1 -- 172.21.15.53:0/415611523 wait complete. 2023-11-06T01:42:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.063+0000 7fa059067700 1 Processor -- start 2023-11-06T01:42:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.064+0000 7fa059067700 1 -- start start 2023-11-06T01:42:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.064+0000 7fa059067700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540ffca0 0x7fa05419d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.064+0000 7fa059067700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa054100e90 0x7fa05419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.064+0000 7fa059067700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa054101830 0x7fa05419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.064+0000 7fa059067700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05419ebe0 con 0x7fa054101830 2023-11-06T01:42:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.064+0000 7fa059067700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05419ed50 con 0x7fa054100e90 2023-11-06T01:42:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.064+0000 7fa059067700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05419ef10 con 0x7fa0540ffca0 2023-11-06T01:42:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.064+0000 7fa05359e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa054101830 0x7fa05419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.064+0000 7fa05359e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa054101830 0x7fa05419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42250/0 (socket says 172.21.15.53:42250) 2023-11-06T01:42:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.064+0000 7fa05359e700 1 -- 172.21.15.53:0/4222719195 learned_addr learned my addr 172.21.15.53:0/4222719195 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.065+0000 7fa05359e700 1 -- 172.21.15.53:0/4222719195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540ffca0 msgr2=0x7fa05419d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.065+0000 7fa052d9d700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540ffca0 0x7fa05419d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.065+0000 7fa05259c700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa054100e90 0x7fa05419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.065+0000 7fa05359e700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540ffca0 0x7fa05419d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.065+0000 7fa05359e700 1 -- 172.21.15.53:0/4222719195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa054100e90 msgr2=0x7fa05419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.065+0000 7fa05359e700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa054100e90 0x7fa05419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.065+0000 7fa05359e700 1 -- 172.21.15.53:0/4222719195 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0480095f0 con 0x7fa054101830 2023-11-06T01:42:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.065+0000 7fa05259c700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa054100e90 0x7fa05419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:42:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.065+0000 7fa052d9d700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540ffca0 0x7fa05419d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.065+0000 7fa05359e700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa054101830 0x7fa05419e580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00ef30 tx=0x7fa04c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.065+0000 7fa03bfff700 1 -- 172.21.15.53:0/4222719195 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c018070 con 0x7fa054101830 2023-11-06T01:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.066+0000 7fa059067700 1 -- 172.21.15.53:0/4222719195 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05419f190 con 0x7fa054101830 2023-11-06T01:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.066+0000 7fa03bfff700 1 -- 172.21.15.53:0/4222719195 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa04c00f040 con 0x7fa054101830 2023-11-06T01:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.066+0000 7fa03bfff700 1 -- 172.21.15.53:0/4222719195 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c004020 con 0x7fa054101830 2023-11-06T01:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.066+0000 7fa059067700 1 -- 172.21.15.53:0/4222719195 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0541a59b0 con 0x7fa054101830 2023-11-06T01:42:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.069+0000 7fa03bfff700 1 -- 172.21.15.53:0/4222719195 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa04c004ad0 con 0x7fa054101830 2023-11-06T01:42:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.069+0000 7fa0651f7b80 1 Processor -- start 2023-11-06T01:42:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.069+0000 7fa0651f7b80 1 -- start start 2023-11-06T01:42:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.069+0000 7fa03bfff700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa03c0a0550 0x7fa03c0a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.069+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.069+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9fe0 0x558c963ea8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.069+0000 7fa03bfff700 1 -- 172.21.15.53:0/4222719195 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa04c0cb640 con 0x7fa054101830 2023-11-06T01:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.069+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963eb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c963419b0 con 0x558c963e9680 2023-11-06T01:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9633f370 con 0x558c963e9fe0 2023-11-06T01:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9633ac60 con 0x558c963eaed0 2023-11-06T01:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa05259c700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa03c0a0550 0x7fa03c0a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa05259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9fe0 0x558c963ea8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963e9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42258/0 (socket says 172.21.15.53:42258) 2023-11-06T01:42:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa052d9d700 1 -- 172.21.15.53:0/1210819603 learned_addr learned my addr 172.21.15.53:0/1210819603 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa05359e700 1 --2- 172.21.15.53:0/1210819603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa05259c700 1 -- 172.21.15.53:0/1210819603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 msgr2=0x558c963eb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa05259c700 1 --2- 172.21.15.53:0/1210819603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963eb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa05259c700 1 -- 172.21.15.53:0/1210819603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 msgr2=0x558c963e9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa05259c700 1 --2- 172.21.15.53:0/1210819603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963e9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa05259c700 1 -- 172.21.15.53:0/1210819603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c96168d80 con 0x558c963e9fe0 2023-11-06T01:42:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.070+0000 7fa05259c700 1 --2- 172.21.15.53:0/1210819603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9fe0 0x558c963ea8f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa048006010 tx=0x7fa04800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3774050b661f1401 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.071+0000 7fa0397fa700 1 -- 172.21.15.53:0/1210819603 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04802b030 con 0x558c963e9fe0 2023-11-06T01:42:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.071+0000 7fa0397fa700 1 -- 172.21.15.53:0/1210819603 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa04800fd00 con 0x558c963e9fe0 2023-11-06T01:42:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.071+0000 7fa0397fa700 1 -- 172.21.15.53:0/1210819603 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048026da0 con 0x558c963e9fe0 2023-11-06T01:42:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.072+0000 7fa0651f7b80 1 -- 172.21.15.53:0/1210819603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9fe0 msgr2=0x558c963ea8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.072+0000 7fa05259c700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa03c0a0550 0x7fa03c0a2a10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa04802b000 tx=0x7fa048017780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.072+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/1210819603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9fe0 0x558c963ea8f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa048006010 tx=0x7fa04800be60 comp rx=0 tx=0).stop 2023-11-06T01:42:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.072+0000 7fa0651f7b80 1 -- 172.21.15.53:0/1210819603 shutdown_connections 2023-11-06T01:42:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.072+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/1210819603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963eb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.072+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/1210819603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963e9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.072+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/1210819603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9fe0 0x558c963ea8f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.072+0000 7fa0651f7b80 1 -- 172.21.15.53:0/1210819603 >> 172.21.15.53:0/1210819603 conn(0x558c96349070 msgr2=0x558c963e6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.072+0000 7fa0651f7b80 1 -- 172.21.15.53:0/1210819603 shutdown_connections 2023-11-06T01:42:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.072+0000 7fa0651f7b80 1 -- 172.21.15.53:0/1210819603 wait complete. 2023-11-06T01:42:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa0651f7b80 1 -- start start 2023-11-06T01:42:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa0651f7b80 1 Processor -- start 2023-11-06T01:42:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9680 0x558c963f3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9fe0 0x558c963f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963fa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c962d74b0 con 0x558c963e9fe0 2023-11-06T01:42:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c962d7150 con 0x558c963e9680 2023-11-06T01:42:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c961be8c0 con 0x558c963eaed0 2023-11-06T01:42:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa0651f7b80 10 client.0 _ll_register_callbacks cb 0x558c96224a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa05259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9fe0 0x558c963f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa05259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9fe0 0x558c963f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42266/0 (socket says 172.21.15.53:42266) 2023-11-06T01:42:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9680 0x558c963f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa05259c700 1 -- 172.21.15.53:0/4103406147 learned_addr learned my addr 172.21.15.53:0/4103406147 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.073+0000 7fa05359e700 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963fa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa05259c700 1 -- 172.21.15.53:0/4103406147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 msgr2=0x558c963fa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa05259c700 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963fa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa05259c700 1 -- 172.21.15.53:0/4103406147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9680 msgr2=0x558c963f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa05259c700 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9680 0x558c963f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa05259c700 1 -- 172.21.15.53:0/4103406147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa054108a30 con 0x558c963e9fe0 2023-11-06T01:42:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa052d9d700 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9680 0x558c963f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:42:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa05259c700 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9fe0 0x558c963f3960 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa04802c040 tx=0x7fa048009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361e202f8f2e570a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa036ff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:42:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa036ff5700 1 -- 172.21.15.53:0/4103406147 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04800fe70 con 0x558c963e9fe0 2023-11-06T01:42:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa036ff5700 1 -- 172.21.15.53:0/4103406147 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa04802c070 con 0x558c963e9fe0 2023-11-06T01:42:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa0651f7b80 1 -- 172.21.15.53:0/4103406147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c96365cb0 con 0x558c963e9fe0 2023-11-06T01:42:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa036ff5700 1 -- 172.21.15.53:0/4103406147 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0480263f0 con 0x558c963e9fe0 2023-11-06T01:42:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa0651f7b80 10 client.16386 fetch_fsmap learned FSMap version 42 2023-11-06T01:42:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa0651f7b80 1 -- 172.21.15.53:0/4103406147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558c96151c00 con 0x558c963e9fe0 2023-11-06T01:42:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.074+0000 7fa036ff5700 1 -- 172.21.15.53:0/4103406147 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa04806d080 con 0x558c963e9fe0 2023-11-06T01:42:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.075+0000 7fa036ff5700 1 -- 172.21.15.53:0/4103406147 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa048067030 con 0x558c963e9fe0 2023-11-06T01:42:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.075+0000 7fa0651f7b80 10 client.16386 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:42:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.075+0000 7fa0651f7b80 10 client.16386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.075+0000 7fa0651f7b80 20 client.16386 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.075+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558c963fedf0 0x558c96401210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.075+0000 7fa0651f7b80 4 client.16386 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:42:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.075+0000 7fa0651f7b80 1 -- 172.21.15.53:0/4103406147 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c963b8db0 con 0x558c963fedf0 2023-11-06T01:42:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.075+0000 7fa052d9d700 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558c963fedf0 0x558c96401210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.075+0000 7fa036ff5700 10 client.16386 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.075+0000 7fa052d9d700 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558c963fedf0 0x558c96401210 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d865d557a99f21e6 server_cookie=b36801de95082b08 in_seq=0 out_seq=0 2023-11-06T01:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa036ff5700 1 -- 172.21.15.53:0/4103406147 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558c963b8db0 con 0x558c963fedf0 2023-11-06T01:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa036ff5700 10 client.16386 handle_command_reply: tid=0 2023-11-06T01:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 1 client.16386 shutdown 2023-11-06T01:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 1 -- 172.21.15.53:0/4103406147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9fe0 msgr2=0x558c963f3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9fe0 0x558c963f3960 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa04802c040 tx=0x7fa048009cd0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 1 -- 172.21.15.53:0/4103406147 shutdown_connections 2023-11-06T01:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9680 0x558c963f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558c963fedf0 0x558c96401210 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963fa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/4103406147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9fe0 0x558c963f3960 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa036ff5700 0 client.16386 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 1 -- 172.21.15.53:0/4103406147 >> 172.21.15.53:0/4103406147 conn(0x558c96349070 msgr2=0x558c963e43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 1 -- 172.21.15.53:0/4103406147 shutdown_connections 2023-11-06T01:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 1 -- 172.21.15.53:0/4103406147 wait complete. 2023-11-06T01:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.076+0000 7fa0651f7b80 20 client.16386 trim_cache size 0 max 16384 2023-11-06T01:42:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.095+0000 7fa0651f7b80 1 Processor -- start 2023-11-06T01:42:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.095+0000 7fa0651f7b80 1 -- start start 2023-11-06T01:42:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.095+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.095+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9fe0 0x558c963eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.095+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963e5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.095+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c963f1d70 con 0x558c963e9680 2023-11-06T01:42:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.095+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c963f42c0 con 0x558c963e9fe0 2023-11-06T01:42:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.095+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c963f40a0 con 0x558c963eaed0 2023-11-06T01:42:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42272/0 (socket says 172.21.15.53:42272) 2023-11-06T01:42:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa052d9d700 1 -- 172.21.15.53:0/1672017010 learned_addr learned my addr 172.21.15.53:0/1672017010 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa05359e700 1 --2- 172.21.15.53:0/1672017010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963e5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa052d9d700 1 -- 172.21.15.53:0/1672017010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 msgr2=0x558c963e5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa052d9d700 1 --2- 172.21.15.53:0/1672017010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963e5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa052d9d700 1 -- 172.21.15.53:0/1672017010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9fe0 msgr2=0x558c963eb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa052d9d700 1 --2- 172.21.15.53:0/1672017010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9fe0 0x558c963eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa052d9d700 1 -- 172.21.15.53:0/1672017010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c9618f1f0 con 0x558c963e9680 2023-11-06T01:42:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa052d9d700 1 --2- 172.21.15.53:0/1672017010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa044009490 tx=0x7fa04401db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cfbff35f4a9007 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa039ffb700 1 -- 172.21.15.53:0/1672017010 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044017040 con 0x558c963e9680 2023-11-06T01:42:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa039ffb700 1 -- 172.21.15.53:0/1672017010 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa04401d330 con 0x558c963e9680 2023-11-06T01:42:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.096+0000 7fa039ffb700 1 -- 172.21.15.53:0/1672017010 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044021970 con 0x558c963e9680 2023-11-06T01:42:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.097+0000 7fa0651f7b80 1 -- 172.21.15.53:0/1672017010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 msgr2=0x558c963f8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.097+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/1672017010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa044009490 tx=0x7fa04401db80 comp rx=0 tx=0).stop 2023-11-06T01:42:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.097+0000 7fa0651f7b80 1 -- 172.21.15.53:0/1672017010 shutdown_connections 2023-11-06T01:42:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.097+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/1672017010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963eaed0 0x558c963e5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.097+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/1672017010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.097+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/1672017010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963e9fe0 0x558c963eb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.097+0000 7fa0651f7b80 1 -- 172.21.15.53:0/1672017010 >> 172.21.15.53:0/1672017010 conn(0x558c96349070 msgr2=0x558c963f21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.097+0000 7fa0651f7b80 1 -- 172.21.15.53:0/1672017010 shutdown_connections 2023-11-06T01:42:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.097+0000 7fa0651f7b80 1 -- 172.21.15.53:0/1672017010 wait complete. 2023-11-06T01:42:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.097+0000 7fa0651f7b80 1 -- start start 2023-11-06T01:42:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa0651f7b80 1 Processor -- start 2023-11-06T01:42:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963e9fe0 0x558c9644c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa0651f7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963eaed0 0x558c9644ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9644f620 con 0x558c963e9680 2023-11-06T01:42:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9644f790 con 0x558c963eaed0 2023-11-06T01:42:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa0651f7b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9644f950 con 0x558c963e9fe0 2023-11-06T01:42:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa0651f7b80 10 client.0 _ll_register_callbacks cb 0x7fa048056040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42278/0 (socket says 172.21.15.53:42278) 2023-11-06T01:42:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa052d9d700 1 -- 172.21.15.53:0/2673547467 learned_addr learned my addr 172.21.15.53:0/2673547467 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa05259c700 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963e9fe0 0x558c9644c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa05359e700 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963eaed0 0x558c9644ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa052d9d700 1 -- 172.21.15.53:0/2673547467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963e9fe0 msgr2=0x558c9644c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa052d9d700 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963e9fe0 0x558c9644c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa052d9d700 1 -- 172.21.15.53:0/2673547467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963eaed0 msgr2=0x558c9644ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa052d9d700 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963eaed0 0x558c9644ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa052d9d700 1 -- 172.21.15.53:0/2673547467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa044009dc0 con 0x558c963e9680 2023-11-06T01:42:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa052d9d700 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa044000c00 tx=0x7fa04401d290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2e6603af9e9a33 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.098+0000 7fa036ff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:42:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.099+0000 7fa036ff5700 1 -- 172.21.15.53:0/2673547467 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044017040 con 0x558c963e9680 2023-11-06T01:42:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.099+0000 7fa036ff5700 1 -- 172.21.15.53:0/2673547467 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa044005550 con 0x558c963e9680 2023-11-06T01:42:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.099+0000 7fa0651f7b80 1 -- 172.21.15.53:0/2673547467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa04800fd00 con 0x558c963e9680 2023-11-06T01:42:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.099+0000 7fa036ff5700 1 -- 172.21.15.53:0/2673547467 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa044005710 con 0x558c963e9680 2023-11-06T01:42:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.099+0000 7fa0651f7b80 10 client.16392 fetch_fsmap learned FSMap version 42 2023-11-06T01:42:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.099+0000 7fa0651f7b80 1 -- 172.21.15.53:0/2673547467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558c96167250 con 0x558c963e9680 2023-11-06T01:42:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.100+0000 7fa036ff5700 1 -- 172.21.15.53:0/2673547467 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa04800fd00 con 0x558c963e9680 2023-11-06T01:42:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.101+0000 7fa036ff5700 1 -- 172.21.15.53:0/2673547467 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa044021bf0 con 0x558c963e9680 2023-11-06T01:42:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.101+0000 7fa0651f7b80 10 client.16392 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:42:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.101+0000 7fa0651f7b80 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.101+0000 7fa0651f7b80 20 client.16392 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.101+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558c964512d0 0x558c964536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.101+0000 7fa0651f7b80 4 client.16392 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:42:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.101+0000 7fa0651f7b80 1 -- 172.21.15.53:0/2673547467 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558c96333ce0 con 0x558c964512d0 2023-11-06T01:42:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.101+0000 7fa05259c700 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558c964512d0 0x558c964536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.101+0000 7fa036ff5700 10 client.16392 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.101+0000 7fa05259c700 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558c964512d0 0x558c964536f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc2538596e9ae48d server_cookie=4e6b95f7b8dbac9 in_seq=0 out_seq=0 2023-11-06T01:42:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa036ff5700 1 -- 172.21.15.53:0/2673547467 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558c96333ce0 con 0x558c964512d0 2023-11-06T01:42:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa036ff5700 10 client.16392 handle_command_reply: tid=0 2023-11-06T01:42:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa0651f7b80 1 client.16392 shutdown 2023-11-06T01:42:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa0651f7b80 1 -- 172.21.15.53:0/2673547467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 msgr2=0x558c963f8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa044000c00 tx=0x7fa04401d290 comp rx=0 tx=0).stop 2023-11-06T01:42:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa0651f7b80 1 -- 172.21.15.53:0/2673547467 shutdown_connections 2023-11-06T01:42:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c963eaed0 0x558c9644ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558c964512d0 0x558c964536f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c963e9680 0x558c963f8950 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa0651f7b80 1 --2- 172.21.15.53:0/2673547467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558c963e9fe0 0x558c9644c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa036ff5700 0 client.16392 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.154+0000 7fa0651f7b80 1 -- 172.21.15.53:0/2673547467 >> 172.21.15.53:0/2673547467 conn(0x558c96349070 msgr2=0x558c963e5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa0651f7b80 1 -- 172.21.15.53:0/2673547467 shutdown_connections 2023-11-06T01:42:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa0651f7b80 1 -- 172.21.15.53:0/2673547467 wait complete. 2023-11-06T01:42:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa0651f7b80 20 client.16392 trim_cache size 0 max 16384 2023-11-06T01:42:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:42:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:42:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:42:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:42:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa059067700 1 -- 172.21.15.53:0/4222719195 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa03c0a0550 msgr2=0x7fa03c0a2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa059067700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa03c0a0550 0x7fa03c0a2a10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa04802b000 tx=0x7fa048017780 comp rx=0 tx=0).stop 2023-11-06T01:42:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa059067700 1 -- 172.21.15.53:0/4222719195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa054101830 msgr2=0x7fa05419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa059067700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa054101830 0x7fa05419e580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00ef30 tx=0x7fa04c00c450 comp rx=0 tx=0).stop 2023-11-06T01:42:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa059067700 1 -- 172.21.15.53:0/4222719195 shutdown_connections 2023-11-06T01:42:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa059067700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa03c0a0550 0x7fa03c0a2a10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa059067700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa054101830 0x7fa05419e580 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa059067700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540ffca0 0x7fa05419d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa059067700 1 --2- 172.21.15.53:0/4222719195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa054100e90 0x7fa05419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa059067700 1 -- 172.21.15.53:0/4222719195 >> 172.21.15.53:0/4222719195 conn(0x7fa0540fb230 msgr2=0x7fa054104380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.155+0000 7fa059067700 1 -- 172.21.15.53:0/4222719195 shutdown_connections 2023-11-06T01:42:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:23.156+0000 7fa059067700 1 -- 172.21.15.53:0/4222719195 wait complete. 2023-11-06T01:42:23.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:23 smithi062 ceph-mon[103924]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 841 op/s 2023-11-06T01:42:23.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:23 smithi163 ceph-mon[104416]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 841 op/s 2023-11-06T01:42:24.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:23 smithi053 ceph-mon[99472]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 841 op/s 2023-11-06T01:42:25.896 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:25 smithi062 ceph-mon[103924]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 710 op/s 2023-11-06T01:42:25.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:25 smithi163 ceph-mon[104416]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 710 op/s 2023-11-06T01:42:26.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:25 smithi053 ceph-mon[99472]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 710 op/s 2023-11-06T01:42:27.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 490.0 s, 1266.7 tps, lat 76.842 ms stddev 149.616 2023-11-06T01:42:27.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:27 smithi062 ceph-mon[103924]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 696 op/s 2023-11-06T01:42:27.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:27 smithi163 ceph-mon[104416]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 696 op/s 2023-11-06T01:42:28.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:27 smithi053 ceph-mon[99472]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 696 op/s 2023-11-06T01:42:29.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:29 smithi062 ceph-mon[103924]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2023-11-06T01:42:29.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:29 smithi163 ceph-mon[104416]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2023-11-06T01:42:30.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:29 smithi053 ceph-mon[99472]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2023-11-06T01:42:31.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:31 smithi062 ceph-mon[103924]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 MiB/s wr, 1.57k op/s 2023-11-06T01:42:31.993 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:31 smithi163 ceph-mon[104416]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 MiB/s wr, 1.57k op/s 2023-11-06T01:42:32.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:31 smithi053 ceph-mon[99472]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 MiB/s wr, 1.57k op/s 2023-11-06T01:42:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 495.0 s, 1830.1 tps, lat 56.430 ms stddev 86.787 2023-11-06T01:42:33.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:33 smithi062 ceph-mon[103924]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 38 MiB/s wr, 1.99k op/s 2023-11-06T01:42:33.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:33 smithi163 ceph-mon[104416]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 38 MiB/s wr, 1.99k op/s 2023-11-06T01:42:34.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:33 smithi053 ceph-mon[99472]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 38 MiB/s wr, 1.99k op/s 2023-11-06T01:42:35.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:35 smithi062 ceph-mon[103924]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 2.01k op/s 2023-11-06T01:42:35.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:35 smithi163 ceph-mon[104416]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 2.01k op/s 2023-11-06T01:42:36.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:35 smithi053 ceph-mon[99472]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 2.01k op/s 2023-11-06T01:42:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.311+0000 7fe088c92700 1 -- 172.21.15.53:0/1261812104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0840ffc90 msgr2=0x7fe0841000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.311+0000 7fe088c92700 1 --2- 172.21.15.53:0/1261812104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0840ffc90 0x7fe0841000b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00b0a0 tx=0x7fe07c00b3b0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.311+0000 7fe088c92700 1 -- 172.21.15.53:0/1261812104 shutdown_connections 2023-11-06T01:42:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.311+0000 7fe088c92700 1 --2- 172.21.15.53:0/1261812104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe084101820 0x7fe0841080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.311+0000 7fe088c92700 1 --2- 172.21.15.53:0/1261812104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0840ffc90 0x7fe0841000b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.311+0000 7fe088c92700 1 --2- 172.21.15.53:0/1261812104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe084100e80 0x7fe0841012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.311+0000 7fe088c92700 1 -- 172.21.15.53:0/1261812104 >> 172.21.15.53:0/1261812104 conn(0x7fe0840fb1e0 msgr2=0x7fe0840fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.311+0000 7fe088c92700 1 -- 172.21.15.53:0/1261812104 shutdown_connections 2023-11-06T01:42:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.311+0000 7fe088c92700 1 -- 172.21.15.53:0/1261812104 wait complete. 2023-11-06T01:42:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.312+0000 7fe088c92700 1 Processor -- start 2023-11-06T01:42:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.312+0000 7fe088c92700 1 -- start start 2023-11-06T01:42:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe088c92700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0840ffc90 0x7fe084198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe088c92700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe084100e80 0x7fe084199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe088c92700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe084101820 0x7fe08419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe088c92700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08419a960 con 0x7fe0840ffc90 2023-11-06T01:42:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe088c92700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08419aad0 con 0x7fe084100e80 2023-11-06T01:42:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe088c92700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08419ac90 con 0x7fe084101820 2023-11-06T01:42:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe08259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0840ffc90 0x7fe084198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe08259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0840ffc90 0x7fe084198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42292/0 (socket says 172.21.15.53:42292) 2023-11-06T01:42:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe08259c700 1 -- 172.21.15.53:0/3515232001 learned_addr learned my addr 172.21.15.53:0/3515232001 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe081d9b700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe084100e80 0x7fe084199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe082d9d700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe084101820 0x7fe08419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe08259c700 1 -- 172.21.15.53:0/3515232001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe084101820 msgr2=0x7fe08419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe08259c700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe084101820 0x7fe08419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe08259c700 1 -- 172.21.15.53:0/3515232001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe084100e80 msgr2=0x7fe084199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe08259c700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe084100e80 0x7fe084199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe08259c700 1 -- 172.21.15.53:0/3515232001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c009d00 con 0x7fe0840ffc90 2023-11-06T01:42:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.313+0000 7fe08259c700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0840ffc90 0x7fe084198f40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00b780 tx=0x7fe07c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.314+0000 7fe0737fe700 1 -- 172.21.15.53:0/3515232001 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00e040 con 0x7fe0840ffc90 2023-11-06T01:42:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.314+0000 7fe0737fe700 1 -- 172.21.15.53:0/3515232001 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe07c004e60 con 0x7fe0840ffc90 2023-11-06T01:42:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.314+0000 7fe0737fe700 1 -- 172.21.15.53:0/3515232001 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c005280 con 0x7fe0840ffc90 2023-11-06T01:42:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.314+0000 7fe088c92700 1 -- 172.21.15.53:0/3515232001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08419af10 con 0x7fe0840ffc90 2023-11-06T01:42:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.314+0000 7fe088c92700 1 -- 172.21.15.53:0/3515232001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0841a1730 con 0x7fe0840ffc90 2023-11-06T01:42:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.316+0000 7fe0737fe700 1 -- 172.21.15.53:0/3515232001 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe07c0054a0 con 0x7fe0840ffc90 2023-11-06T01:42:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.316+0000 7fe0737fe700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe06c0a05a0 0x7fe06c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.317+0000 7fe0737fe700 1 -- 172.21.15.53:0/3515232001 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe07c0ccfe0 con 0x7fe0840ffc90 2023-11-06T01:42:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.317+0000 7fe081d9b700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe06c0a05a0 0x7fe06c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.317+0000 7fe081d9b700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe06c0a05a0 0x7fe06c0a2a60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe074005950 tx=0x7fe07400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.317+0000 7fe094e22b80 1 Processor -- start 2023-11-06T01:42:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.318+0000 7fe094e22b80 1 -- start start 2023-11-06T01:42:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.318+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfd680 0x556db7dfdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.318+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7dfe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.318+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 0x556db7dff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.318+0000 7fe094e22b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7d559b0 con 0x556db7dfeed0 2023-11-06T01:42:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.318+0000 7fe094e22b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7d53370 con 0x556db7dfd680 2023-11-06T01:42:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.318+0000 7fe094e22b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7d4ec60 con 0x556db7dfdfe0 2023-11-06T01:42:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.319+0000 7fe081d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7dfe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.319+0000 7fe081d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7dfe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56544/0 (socket says 172.21.15.53:56544) 2023-11-06T01:42:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.319+0000 7fe081d9b700 1 -- 172.21.15.53:0/558200762 learned_addr learned my addr 172.21.15.53:0/558200762 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe08259c700 1 --2- 172.21.15.53:0/558200762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfd680 0x556db7dfdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe082d9d700 1 --2- 172.21.15.53:0/558200762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 0x556db7dff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe081d9b700 1 -- 172.21.15.53:0/558200762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfd680 msgr2=0x556db7dfdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe081d9b700 1 --2- 172.21.15.53:0/558200762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfd680 0x556db7dfdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe081d9b700 1 -- 172.21.15.53:0/558200762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 msgr2=0x556db7dff350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe081d9b700 1 --2- 172.21.15.53:0/558200762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 0x556db7dff350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe081d9b700 1 -- 172.21.15.53:0/558200762 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556db7b7cd80 con 0x556db7dfdfe0 2023-11-06T01:42:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe082d9d700 1 --2- 172.21.15.53:0/558200762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 0x556db7dff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:42:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe081d9b700 1 --2- 172.21.15.53:0/558200762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7dfe8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe074010fa0 tx=0x7fe074013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7206cf5e748009 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe070ff9700 1 -- 172.21.15.53:0/558200762 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07401e620 con 0x556db7dfdfe0 2023-11-06T01:42:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe070ff9700 1 -- 172.21.15.53:0/558200762 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe074015070 con 0x556db7dfdfe0 2023-11-06T01:42:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.320+0000 7fe070ff9700 1 -- 172.21.15.53:0/558200762 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07401ca20 con 0x556db7dfdfe0 2023-11-06T01:42:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.321+0000 7fe094e22b80 1 -- 172.21.15.53:0/558200762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 msgr2=0x556db7dfe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.321+0000 7fe094e22b80 1 --2- 172.21.15.53:0/558200762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7dfe8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe074010fa0 tx=0x7fe074013000 comp rx=0 tx=0).stop 2023-11-06T01:42:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.321+0000 7fe094e22b80 1 -- 172.21.15.53:0/558200762 shutdown_connections 2023-11-06T01:42:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.321+0000 7fe094e22b80 1 --2- 172.21.15.53:0/558200762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfd680 0x556db7dfdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.321+0000 7fe094e22b80 1 --2- 172.21.15.53:0/558200762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 0x556db7dff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.321+0000 7fe094e22b80 1 --2- 172.21.15.53:0/558200762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7dfe8f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.321+0000 7fe094e22b80 1 -- 172.21.15.53:0/558200762 >> 172.21.15.53:0/558200762 conn(0x556db7d5d070 msgr2=0x556db7dfa040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.321+0000 7fe094e22b80 1 -- 172.21.15.53:0/558200762 shutdown_connections 2023-11-06T01:42:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe094e22b80 1 -- 172.21.15.53:0/558200762 wait complete. 2023-11-06T01:42:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe094e22b80 1 -- start start 2023-11-06T01:42:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe094e22b80 1 Processor -- start 2023-11-06T01:42:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfd680 0x556db7e07430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfdfe0 0x556db7e07970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfeed0 0x556db7e0e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe094e22b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7ceb4b0 con 0x556db7dfdfe0 2023-11-06T01:42:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe094e22b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7ceb150 con 0x556db7dfeed0 2023-11-06T01:42:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe094e22b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7bd28c0 con 0x556db7dfd680 2023-11-06T01:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe094e22b80 10 client.0 _ll_register_callbacks cb 0x556db7c38a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe081d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfdfe0 0x556db7e07970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe081d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfdfe0 0x556db7e07970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42308/0 (socket says 172.21.15.53:42308) 2023-11-06T01:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe081d9b700 1 -- 172.21.15.53:0/4004880352 learned_addr learned my addr 172.21.15.53:0/4004880352 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe08259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfd680 0x556db7e07430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.322+0000 7fe082d9d700 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfeed0 0x556db7e0e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe08259c700 1 -- 172.21.15.53:0/4004880352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfeed0 msgr2=0x556db7e0e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe08259c700 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfeed0 0x556db7e0e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe08259c700 1 -- 172.21.15.53:0/4004880352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfdfe0 msgr2=0x556db7e07970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe08259c700 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfdfe0 0x556db7e07970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe08259c700 1 -- 172.21.15.53:0/4004880352 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe074010c50 con 0x556db7dfd680 2023-11-06T01:42:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe081d9b700 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfdfe0 0x556db7e07970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:42:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe08259c700 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfd680 0x556db7e07430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe07c007a90 tx=0x7fe07c09a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e468677776df4a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe069ffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:42:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe069ffb700 1 -- 172.21.15.53:0/4004880352 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c09af50 con 0x556db7dfd680 2023-11-06T01:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe094e22b80 1 -- 172.21.15.53:0/4004880352 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556db7d79cb0 con 0x556db7dfd680 2023-11-06T01:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe069ffb700 1 -- 172.21.15.53:0/4004880352 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe07c09a850 con 0x556db7dfd680 2023-11-06T01:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe069ffb700 1 -- 172.21.15.53:0/4004880352 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe07c09e680 con 0x556db7dfd680 2023-11-06T01:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe094e22b80 10 client.26041 fetch_fsmap learned FSMap version 42 2023-11-06T01:42:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.323+0000 7fe094e22b80 1 -- 172.21.15.53:0/4004880352 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x556db7b65c00 con 0x556db7dfd680 2023-11-06T01:42:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.324+0000 7fe069ffb700 1 -- 172.21.15.53:0/4004880352 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe07c064020 con 0x556db7dfd680 2023-11-06T01:42:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.324+0000 7fe069ffb700 1 -- 172.21.15.53:0/4004880352 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe07c0a23a0 con 0x556db7dfd680 2023-11-06T01:42:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.324+0000 7fe094e22b80 10 client.26041 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:42:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.324+0000 7fe094e22b80 10 client.26041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.324+0000 7fe094e22b80 20 client.26041 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.324+0000 7fe094e22b80 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556db7e13060 0x556db7e15480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.324+0000 7fe094e22b80 4 client.26041 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.324+0000 7fe094e22b80 1 -- 172.21.15.53:0/4004880352 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556db7dccdb0 con 0x556db7e13060 2023-11-06T01:42:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.324+0000 7fe081d9b700 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556db7e13060 0x556db7e15480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.324+0000 7fe081d9b700 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556db7e13060 0x556db7e15480 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1b87b7e26c35df server_cookie=b373b5de1585f16f in_seq=0 out_seq=0 2023-11-06T01:42:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.324+0000 7fe069ffb700 10 client.26041 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe069ffb700 1 -- 172.21.15.53:0/4004880352 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556db7dccdb0 con 0x556db7e13060 2023-11-06T01:42:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe069ffb700 10 client.26041 handle_command_reply: tid=0 2023-11-06T01:42:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe094e22b80 1 client.26041 shutdown 2023-11-06T01:42:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe094e22b80 1 -- 172.21.15.53:0/4004880352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfd680 msgr2=0x556db7e07430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe094e22b80 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfd680 0x556db7e07430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe07c007a90 tx=0x7fe07c09a620 comp rx=0 tx=0).stop 2023-11-06T01:42:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe094e22b80 1 -- 172.21.15.53:0/4004880352 shutdown_connections 2023-11-06T01:42:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe094e22b80 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfeed0 0x556db7e0e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe094e22b80 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556db7e13060 0x556db7e15480 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe094e22b80 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfd680 0x556db7e07430 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe069ffb700 0 client.26041 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe094e22b80 1 --2- 172.21.15.53:0/4004880352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfdfe0 0x556db7e07970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe094e22b80 1 -- 172.21.15.53:0/4004880352 >> 172.21.15.53:0/4004880352 conn(0x556db7d5d070 msgr2=0x556db7df83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe094e22b80 1 -- 172.21.15.53:0/4004880352 shutdown_connections 2023-11-06T01:42:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.325+0000 7fe094e22b80 1 -- 172.21.15.53:0/4004880352 wait complete. 2023-11-06T01:42:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.326+0000 7fe094e22b80 20 client.26041 trim_cache size 0 max 16384 2023-11-06T01:42:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe094e22b80 1 Processor -- start 2023-11-06T01:42:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe094e22b80 1 -- start start 2023-11-06T01:42:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfd680 0x556db7df7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfdfe0 0x556db7e0e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 0x556db7df9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe094e22b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7e068e0 con 0x556db7dfeed0 2023-11-06T01:42:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe094e22b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7df3500 con 0x556db7dfdfe0 2023-11-06T01:42:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe094e22b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7dfe6e0 con 0x556db7dfd680 2023-11-06T01:42:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe082d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 0x556db7df9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe08259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfd680 0x556db7df7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe082d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 0x556db7df9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42318/0 (socket says 172.21.15.53:42318) 2023-11-06T01:42:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.343+0000 7fe082d9d700 1 -- 172.21.15.53:0/3084389607 learned_addr learned my addr 172.21.15.53:0/3084389607 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.344+0000 7fe081d9b700 1 --2- 172.21.15.53:0/3084389607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfdfe0 0x556db7e0e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.344+0000 7fe082d9d700 1 -- 172.21.15.53:0/3084389607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfd680 msgr2=0x556db7df7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.344+0000 7fe082d9d700 1 --2- 172.21.15.53:0/3084389607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfd680 0x556db7df7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.344+0000 7fe082d9d700 1 -- 172.21.15.53:0/3084389607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfdfe0 msgr2=0x556db7e0e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.344+0000 7fe082d9d700 1 --2- 172.21.15.53:0/3084389607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfdfe0 0x556db7e0e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.344+0000 7fe082d9d700 1 -- 172.21.15.53:0/3084389607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556db7ba31f0 con 0x556db7dfeed0 2023-11-06T01:42:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.344+0000 7fe082d9d700 1 --2- 172.21.15.53:0/3084389607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 0x556db7df9c20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe07800dcc0 tx=0x7fe078009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da53d25a00e24c74 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.344+0000 7fe0717fa700 1 -- 172.21.15.53:0/3084389607 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe078012030 con 0x556db7dfeed0 2023-11-06T01:42:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.344+0000 7fe0717fa700 1 -- 172.21.15.53:0/3084389607 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe078013040 con 0x556db7dfeed0 2023-11-06T01:42:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.344+0000 7fe0717fa700 1 -- 172.21.15.53:0/3084389607 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07801ad60 con 0x556db7dfeed0 2023-11-06T01:42:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.345+0000 7fe094e22b80 1 -- 172.21.15.53:0/3084389607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 msgr2=0x556db7df9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.345+0000 7fe094e22b80 1 --2- 172.21.15.53:0/3084389607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 0x556db7df9c20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe07800dcc0 tx=0x7fe078009260 comp rx=0 tx=0).stop 2023-11-06T01:42:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.345+0000 7fe094e22b80 1 -- 172.21.15.53:0/3084389607 shutdown_connections 2023-11-06T01:42:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.345+0000 7fe094e22b80 1 --2- 172.21.15.53:0/3084389607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfeed0 0x556db7df9c20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.345+0000 7fe094e22b80 1 --2- 172.21.15.53:0/3084389607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfd680 0x556db7df7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.345+0000 7fe094e22b80 1 --2- 172.21.15.53:0/3084389607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfdfe0 0x556db7e0e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.345+0000 7fe094e22b80 1 -- 172.21.15.53:0/3084389607 >> 172.21.15.53:0/3084389607 conn(0x556db7d5d070 msgr2=0x556db7e06170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.345+0000 7fe094e22b80 1 -- 172.21.15.53:0/3084389607 shutdown_connections 2023-11-06T01:42:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.345+0000 7fe094e22b80 1 -- 172.21.15.53:0/3084389607 wait complete. 2023-11-06T01:42:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.345+0000 7fe094e22b80 1 -- start start 2023-11-06T01:42:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.345+0000 7fe094e22b80 1 Processor -- start 2023-11-06T01:42:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfd680 0x556db7e0e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7e0e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe094e22b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfeed0 0x556db7e0caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe094e22b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7e0d140 con 0x556db7dfd680 2023-11-06T01:42:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe094e22b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7e0d2b0 con 0x556db7dfeed0 2023-11-06T01:42:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe094e22b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db7e0d470 con 0x556db7dfdfe0 2023-11-06T01:42:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe094e22b80 10 client.0 _ll_register_callbacks cb 0x7fe07c096b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe081d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7e0e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe081d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7e0e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56562/0 (socket says 172.21.15.53:56562) 2023-11-06T01:42:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe081d9b700 1 -- 172.21.15.53:0/3221054797 learned_addr learned my addr 172.21.15.53:0/3221054797 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe08259c700 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfd680 0x556db7e0e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe081d9b700 1 -- 172.21.15.53:0/3221054797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfeed0 msgr2=0x556db7e0caa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:42:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe081d9b700 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfeed0 0x556db7e0caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe081d9b700 1 -- 172.21.15.53:0/3221054797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfd680 msgr2=0x556db7e0e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe081d9b700 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfd680 0x556db7e0e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe081d9b700 1 -- 172.21.15.53:0/3221054797 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07800d9a0 con 0x556db7dfdfe0 2023-11-06T01:42:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.346+0000 7fe08259c700 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfd680 0x556db7e0e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:42:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe081d9b700 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7e0e8e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe074011ab0 tx=0x7fe074010b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aee67ce4808bc825 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe069ffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:42:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe069ffb700 1 -- 172.21.15.53:0/3221054797 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07402d040 con 0x556db7dfdfe0 2023-11-06T01:42:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe069ffb700 1 -- 172.21.15.53:0/3221054797 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe074017070 con 0x556db7dfdfe0 2023-11-06T01:42:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe094e22b80 1 -- 172.21.15.53:0/3221054797 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe074015070 con 0x556db7dfdfe0 2023-11-06T01:42:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe069ffb700 1 -- 172.21.15.53:0/3221054797 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe074005680 con 0x556db7dfdfe0 2023-11-06T01:42:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe094e22b80 10 client.26047 fetch_fsmap learned FSMap version 42 2023-11-06T01:42:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe094e22b80 1 -- 172.21.15.53:0/3221054797 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x556db7b7b250 con 0x556db7dfdfe0 2023-11-06T01:42:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe069ffb700 1 -- 172.21.15.53:0/3221054797 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe074015070 con 0x556db7dfdfe0 2023-11-06T01:42:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe069ffb700 1 -- 172.21.15.53:0/3221054797 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe074004bd0 con 0x556db7dfdfe0 2023-11-06T01:42:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe094e22b80 10 client.26047 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:42:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe094e22b80 10 client.26047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe094e22b80 20 client.26047 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe094e22b80 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556db7e65f60 0x556db7e68380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe094e22b80 4 client.26047 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:42:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.347+0000 7fe094e22b80 1 -- 172.21.15.53:0/3221054797 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556db7d47ce0 con 0x556db7e65f60 2023-11-06T01:42:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.348+0000 7fe08259c700 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556db7e65f60 0x556db7e68380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.348+0000 7fe08259c700 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556db7e65f60 0x556db7e68380 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39a847dbc7a2ff7 server_cookie=e76f4eb75b78256d in_seq=0 out_seq=0 2023-11-06T01:42:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.348+0000 7fe069ffb700 10 client.26047 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.357+0000 7fe069ffb700 1 -- 172.21.15.53:0/3221054797 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556db7d47ce0 con 0x556db7e65f60 2023-11-06T01:42:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.357+0000 7fe069ffb700 10 client.26047 handle_command_reply: tid=0 2023-11-06T01:42:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 1 client.26047 shutdown 2023-11-06T01:42:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 1 -- 172.21.15.53:0/3221054797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 msgr2=0x556db7e0e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7e0e8e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe074011ab0 tx=0x7fe074010b70 comp rx=0 tx=0).stop 2023-11-06T01:42:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 1 -- 172.21.15.53:0/3221054797 shutdown_connections 2023-11-06T01:42:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556db7dfeed0 0x556db7e0caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x556db7e65f60 0x556db7e68380 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556db7dfd680 0x556db7e0e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 1 --2- 172.21.15.53:0/3221054797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556db7dfdfe0 0x556db7e0e8e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 1 -- 172.21.15.53:0/3221054797 >> 172.21.15.53:0/3221054797 conn(0x556db7d5d070 msgr2=0x556db7e05200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe069ffb700 0 client.26047 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 1 -- 172.21.15.53:0/3221054797 shutdown_connections 2023-11-06T01:42:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 1 -- 172.21.15.53:0/3221054797 wait complete. 2023-11-06T01:42:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.358+0000 7fe094e22b80 20 client.26047 trim_cache size 0 max 16384 2023-11-06T01:42:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:42:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:42:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:42:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:42:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 -- 172.21.15.53:0/3515232001 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe06c0a05a0 msgr2=0x7fe06c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe06c0a05a0 0x7fe06c0a2a60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe074005950 tx=0x7fe07400b410 comp rx=0 tx=0).stop 2023-11-06T01:42:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 -- 172.21.15.53:0/3515232001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0840ffc90 msgr2=0x7fe084198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0840ffc90 0x7fe084198f40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00b780 tx=0x7fe07c00bd60 comp rx=0 tx=0).stop 2023-11-06T01:42:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 -- 172.21.15.53:0/3515232001 shutdown_connections 2023-11-06T01:42:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe06c0a05a0 0x7fe06c0a2a60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe084101820 0x7fe08419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0840ffc90 0x7fe084198f40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 --2- 172.21.15.53:0/3515232001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe084100e80 0x7fe084199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 -- 172.21.15.53:0/3515232001 >> 172.21.15.53:0/3515232001 conn(0x7fe0840fb1e0 msgr2=0x7fe0841043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 -- 172.21.15.53:0/3515232001 shutdown_connections 2023-11-06T01:42:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:36.359+0000 7fe088c92700 1 -- 172.21.15.53:0/3515232001 wait complete. 2023-11-06T01:42:37.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 500.0 s, 1826.3 tps, lat 54.690 ms stddev 63.669 2023-11-06T01:42:37.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:37 smithi163 ceph-mon[104416]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 MiB/s wr, 1.98k op/s 2023-11-06T01:42:38.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:37 smithi053 ceph-mon[99472]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 MiB/s wr, 1.98k op/s 2023-11-06T01:42:38.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:37 smithi062 ceph-mon[103924]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 MiB/s wr, 1.98k op/s 2023-11-06T01:42:38.361 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:42:38.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.614+0000 7fda4e102700 1 -- 172.21.15.53:0/1228244985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4810a700 msgr2=0x7fda4810cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:38.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.614+0000 7fda4e102700 1 --2- 172.21.15.53:0/1228244985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4810a700 0x7fda4810cb90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009990 tx=0x7fda3c009ca0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.623+0000 7fda4e102700 1 -- 172.21.15.53:0/1228244985 shutdown_connections 2023-11-06T01:42:38.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.623+0000 7fda4e102700 1 --2- 172.21.15.53:0/1228244985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda48107d90 0x7fda4810a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.623+0000 7fda4e102700 1 --2- 172.21.15.53:0/1228244985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda4810d0d0 0x7fda4810d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.623+0000 7fda4e102700 1 --2- 172.21.15.53:0/1228244985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4810a700 0x7fda4810cb90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.623+0000 7fda4e102700 1 -- 172.21.15.53:0/1228244985 >> 172.21.15.53:0/1228244985 conn(0x7fda4806db40 msgr2=0x7fda4806ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:38.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.624+0000 7fda4e102700 1 -- 172.21.15.53:0/1228244985 shutdown_connections 2023-11-06T01:42:38.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.624+0000 7fda4e102700 1 -- 172.21.15.53:0/1228244985 wait complete. 2023-11-06T01:42:38.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.624+0000 7fda4e102700 1 Processor -- start 2023-11-06T01:42:38.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda4e102700 1 -- start start 2023-11-06T01:42:38.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda4e102700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda48107d90 0x7fda481a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda4e102700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda4810a700 0x7fda481a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda4e102700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4810d0d0 0x7fda481ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda4e102700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda481ad5d0 con 0x7fda4810d0d0 2023-11-06T01:42:38.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda4e102700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda481ad710 con 0x7fda4810a700 2023-11-06T01:42:38.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda4e102700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda481ad8a0 con 0x7fda48107d90 2023-11-06T01:42:38.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda47fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4810d0d0 0x7fda481ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda46ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda4810a700 0x7fda481a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda47fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4810d0d0 0x7fda481ad000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42334/0 (socket says 172.21.15.53:42334) 2023-11-06T01:42:38.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda47fff700 1 -- 172.21.15.53:0/3431307651 learned_addr learned my addr 172.21.15.53:0/3431307651 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:38.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda46ffd700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda4810a700 0x7fda481a5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53770/0 (socket says 172.21.15.53:53770) 2023-11-06T01:42:38.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda477fe700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda48107d90 0x7fda481a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda47fff700 1 -- 172.21.15.53:0/3431307651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda48107d90 msgr2=0x7fda481a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:38.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.625+0000 7fda47fff700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda48107d90 0x7fda481a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.626+0000 7fda47fff700 1 -- 172.21.15.53:0/3431307651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda4810a700 msgr2=0x7fda481a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:38.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.626+0000 7fda47fff700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda4810a700 0x7fda481a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.626+0000 7fda47fff700 1 -- 172.21.15.53:0/3431307651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c0095f0 con 0x7fda4810d0d0 2023-11-06T01:42:38.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.626+0000 7fda47fff700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4810d0d0 0x7fda481ad000 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fda4000b970 tx=0x7fda4000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:38.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.626+0000 7fda44ff9700 1 -- 172.21.15.53:0/3431307651 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4000c930 con 0x7fda4810d0d0 2023-11-06T01:42:38.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.626+0000 7fda44ff9700 1 -- 172.21.15.53:0/3431307651 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda40014440 con 0x7fda4810d0d0 2023-11-06T01:42:38.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.626+0000 7fda4e102700 1 -- 172.21.15.53:0/3431307651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda481adb80 con 0x7fda4810d0d0 2023-11-06T01:42:38.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.626+0000 7fda44ff9700 1 -- 172.21.15.53:0/3431307651 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda400075b0 con 0x7fda4810d0d0 2023-11-06T01:42:38.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.626+0000 7fda4e102700 1 -- 172.21.15.53:0/3431307651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda481ae040 con 0x7fda4810d0d0 2023-11-06T01:42:38.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.629+0000 7fda44ff9700 1 -- 172.21.15.53:0/3431307651 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fda400145b0 con 0x7fda4810d0d0 2023-11-06T01:42:38.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.629+0000 7fda44ff9700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda300a0520 0x7fda300a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.630+0000 7fda477fe700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda300a0520 0x7fda300a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.630+0000 7fda44ff9700 1 -- 172.21.15.53:0/3431307651 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fda400cafd0 con 0x7fda4810d0d0 2023-11-06T01:42:38.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.630+0000 7fda477fe700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda300a0520 0x7fda300a29e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fda38006fd0 tx=0x7fda38009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:38.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.631+0000 7fda5a292b80 1 Processor -- start 2023-11-06T01:42:38.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.631+0000 7fda5a292b80 1 -- start start 2023-11-06T01:42:38.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.631+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c6850f0 0x55c90c685530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.631+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c685a70 0x55c90c683e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.631+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c684870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.631+0000 7fda5a292b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c5dba10 con 0x55c90c685a70 2023-11-06T01:42:38.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.631+0000 7fda5a292b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c5d93d0 con 0x55c90c6843f0 2023-11-06T01:42:38.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.631+0000 7fda5a292b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c5d4cc0 con 0x55c90c6850f0 2023-11-06T01:42:38.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.632+0000 7fda47fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c684870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.632+0000 7fda47fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c684870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53778/0 (socket says 172.21.15.53:53778) 2023-11-06T01:42:38.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.632+0000 7fda47fff700 1 -- 172.21.15.53:0/1996191903 learned_addr learned my addr 172.21.15.53:0/1996191903 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:38.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.632+0000 7fda46ffd700 1 --2- 172.21.15.53:0/1996191903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c685a70 0x55c90c683e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.632+0000 7fda47fff700 1 -- 172.21.15.53:0/1996191903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c6850f0 msgr2=0x55c90c685530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:38.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.632+0000 7fda47fff700 1 --2- 172.21.15.53:0/1996191903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c6850f0 0x55c90c685530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.632+0000 7fda47fff700 1 -- 172.21.15.53:0/1996191903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c685a70 msgr2=0x55c90c683e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:38.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.632+0000 7fda47fff700 1 --2- 172.21.15.53:0/1996191903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c685a70 0x55c90c683e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.632+0000 7fda47fff700 1 -- 172.21.15.53:0/1996191903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c90c402de0 con 0x55c90c6843f0 2023-11-06T01:42:38.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.632+0000 7fda47fff700 1 --2- 172.21.15.53:0/1996191903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c684870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fda400985b0 tx=0x7fda40098500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a4f4ad10e35e63c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:38.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.635+0000 7fda2dffb700 1 -- 172.21.15.53:0/1996191903 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4009c6e0 con 0x55c90c6843f0 2023-11-06T01:42:38.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.635+0000 7fda2dffb700 1 -- 172.21.15.53:0/1996191903 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda400a0920 con 0x55c90c6843f0 2023-11-06T01:42:38.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.635+0000 7fda2dffb700 1 -- 172.21.15.53:0/1996191903 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda400a0490 con 0x55c90c6843f0 2023-11-06T01:42:38.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.635+0000 7fda5a292b80 1 -- 172.21.15.53:0/1996191903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 msgr2=0x55c90c684870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:38.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.635+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1996191903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c684870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fda400985b0 tx=0x7fda40098500 comp rx=0 tx=0).stop 2023-11-06T01:42:38.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.636+0000 7fda5a292b80 1 -- 172.21.15.53:0/1996191903 shutdown_connections 2023-11-06T01:42:38.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.636+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1996191903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c684870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.636+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1996191903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c6850f0 0x55c90c685530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.636+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1996191903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c685a70 0x55c90c683e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.636+0000 7fda5a292b80 1 -- 172.21.15.53:0/1996191903 >> 172.21.15.53:0/1996191903 conn(0x55c90c5e30d0 msgr2=0x55c90c67e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:38.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.636+0000 7fda5a292b80 1 -- 172.21.15.53:0/1996191903 shutdown_connections 2023-11-06T01:42:38.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.636+0000 7fda5a292b80 1 -- 172.21.15.53:0/1996191903 wait complete. 2023-11-06T01:42:38.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.636+0000 7fda5a292b80 1 -- start start 2023-11-06T01:42:38.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.636+0000 7fda5a292b80 1 Processor -- start 2023-11-06T01:42:38.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.636+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c6843f0 0x55c90c68da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c68df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c685a70 0x55c90c694fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda5a292b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c571510 con 0x55c90c6850f0 2023-11-06T01:42:38.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda5a292b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c5711b0 con 0x55c90c685a70 2023-11-06T01:42:38.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda5a292b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c458920 con 0x55c90c6843f0 2023-11-06T01:42:38.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda5a292b80 10 client.0 _ll_register_callbacks cb 0x55c90c4bea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:38.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda46ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c68df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda477fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c6843f0 0x55c90c68da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda477fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c6843f0 0x55c90c68da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56584/0 (socket says 172.21.15.53:56584) 2023-11-06T01:42:38.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda46ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c68df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42346/0 (socket says 172.21.15.53:42346) 2023-11-06T01:42:38.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda477fe700 1 -- 172.21.15.53:0/1493256372 learned_addr learned my addr 172.21.15.53:0/1493256372 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:38.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda47fff700 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c685a70 0x55c90c694fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda477fe700 1 -- 172.21.15.53:0/1493256372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c685a70 msgr2=0x55c90c694fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:38.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda477fe700 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c685a70 0x55c90c694fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda477fe700 1 -- 172.21.15.53:0/1493256372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 msgr2=0x55c90c68df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:38.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda477fe700 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c68df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda477fe700 1 -- 172.21.15.53:0/1493256372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda481ae040 con 0x55c90c6843f0 2023-11-06T01:42:38.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda46ffd700 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c68df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:42:38.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda477fe700 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c6843f0 0x55c90c68da30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda38015730 tx=0x7fda38015760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a857e011a255d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:38.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.637+0000 7fda26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:42:38.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda26ffd700 1 -- 172.21.15.53:0/1493256372 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3800ee70 con 0x55c90c6843f0 2023-11-06T01:42:38.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda26ffd700 1 -- 172.21.15.53:0/1493256372 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fda38015d70 con 0x55c90c6843f0 2023-11-06T01:42:38.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda5a292b80 1 -- 172.21.15.53:0/1493256372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c90c5ffd10 con 0x55c90c6843f0 2023-11-06T01:42:38.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda26ffd700 1 -- 172.21.15.53:0/1493256372 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda38013450 con 0x55c90c6843f0 2023-11-06T01:42:38.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda5a292b80 10 client.26053 fetch_fsmap learned FSMap version 42 2023-11-06T01:42:38.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda5a292b80 1 -- 172.21.15.53:0/1493256372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c90c3ebc60 con 0x55c90c6843f0 2023-11-06T01:42:38.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda26ffd700 1 -- 172.21.15.53:0/1493256372 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda3802e020 con 0x55c90c6843f0 2023-11-06T01:42:38.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda26ffd700 1 -- 172.21.15.53:0/1493256372 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda3801e070 con 0x55c90c6843f0 2023-11-06T01:42:38.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda5a292b80 10 client.26053 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:42:38.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda5a292b80 10 client.26053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:38.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda5a292b80 20 client.26053 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:38.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c90c6998f0 0x55c90c69bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda5a292b80 4 client.26053 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:42:38.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda5a292b80 1 -- 172.21.15.53:0/1493256372 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c90c652e10 con 0x55c90c6998f0 2023-11-06T01:42:38.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.638+0000 7fda46ffd700 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c90c6998f0 0x55c90c69bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.639+0000 7fda26ffd700 10 client.26053 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.639+0000 7fda46ffd700 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c90c6998f0 0x55c90c69bd10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d718daa1e6bfc7 server_cookie=5ddf787cfd423205 in_seq=0 out_seq=0 2023-11-06T01:42:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.639+0000 7fda26ffd700 1 -- 172.21.15.53:0/1493256372 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c90c652e10 con 0x55c90c6998f0 2023-11-06T01:42:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.639+0000 7fda26ffd700 10 client.26053 handle_command_reply: tid=0 2023-11-06T01:42:38.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.639+0000 7fda5a292b80 1 client.26053 shutdown 2023-11-06T01:42:38.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda5a292b80 1 -- 172.21.15.53:0/1493256372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c6843f0 msgr2=0x55c90c68da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:38.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c6843f0 0x55c90c68da30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda38015730 tx=0x7fda38015760 comp rx=0 tx=0).stop 2023-11-06T01:42:38.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda5a292b80 1 -- 172.21.15.53:0/1493256372 shutdown_connections 2023-11-06T01:42:38.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c685a70 0x55c90c694fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c90c6998f0 0x55c90c69bd10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c6843f0 0x55c90c68da30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1493256372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c68df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda26ffd700 0 client.26053 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:38.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda5a292b80 1 -- 172.21.15.53:0/1493256372 >> 172.21.15.53:0/1493256372 conn(0x55c90c5e30d0 msgr2=0x55c90c67e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:38.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda5a292b80 1 -- 172.21.15.53:0/1493256372 shutdown_connections 2023-11-06T01:42:38.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda5a292b80 1 -- 172.21.15.53:0/1493256372 wait complete. 2023-11-06T01:42:38.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.640+0000 7fda5a292b80 20 client.26053 trim_cache size 0 max 16384 2023-11-06T01:42:38.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.663+0000 7fda5a292b80 1 Processor -- start 2023-11-06T01:42:38.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.663+0000 7fda5a292b80 1 -- start start 2023-11-06T01:42:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.663+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c683e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.663+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c692f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.663+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 0x55c90c68d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.663+0000 7fda5a292b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c68cee0 con 0x55c90c6850f0 2023-11-06T01:42:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.663+0000 7fda5a292b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c680470 con 0x55c90c6843f0 2023-11-06T01:42:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.663+0000 7fda5a292b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c67f5f0 con 0x55c90c685a70 2023-11-06T01:42:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.663+0000 7fda477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c683e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.663+0000 7fda47fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 0x55c90c68d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda47fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 0x55c90c68d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56594/0 (socket says 172.21.15.53:56594) 2023-11-06T01:42:38.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda47fff700 1 -- 172.21.15.53:0/1183724049 learned_addr learned my addr 172.21.15.53:0/1183724049 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:38.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda46ffd700 1 --2- 172.21.15.53:0/1183724049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c692f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda47fff700 1 -- 172.21.15.53:0/1183724049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 msgr2=0x55c90c683e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:38.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda47fff700 1 --2- 172.21.15.53:0/1183724049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c683e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda47fff700 1 -- 172.21.15.53:0/1183724049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 msgr2=0x55c90c692f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:38.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda47fff700 1 --2- 172.21.15.53:0/1183724049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c692f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda47fff700 1 -- 172.21.15.53:0/1183724049 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c90c429250 con 0x55c90c685a70 2023-11-06T01:42:38.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda46ffd700 1 --2- 172.21.15.53:0/1183724049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c692f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:42:38.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda47fff700 1 --2- 172.21.15.53:0/1183724049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 0x55c90c68d7e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fda40014860 tx=0x7fda4009b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c40c67650486063 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:38.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda2e7fc700 1 -- 172.21.15.53:0/1183724049 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4009c6e0 con 0x55c90c685a70 2023-11-06T01:42:38.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda2e7fc700 1 -- 172.21.15.53:0/1183724049 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda40096c50 con 0x55c90c685a70 2023-11-06T01:42:38.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.664+0000 7fda2e7fc700 1 -- 172.21.15.53:0/1183724049 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda400a06f0 con 0x55c90c685a70 2023-11-06T01:42:38.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.665+0000 7fda5a292b80 1 -- 172.21.15.53:0/1183724049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 msgr2=0x55c90c68d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:38.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.665+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1183724049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 0x55c90c68d7e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fda40014860 tx=0x7fda4009b040 comp rx=0 tx=0).stop 2023-11-06T01:42:38.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.665+0000 7fda5a292b80 1 -- 172.21.15.53:0/1183724049 shutdown_connections 2023-11-06T01:42:38.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.665+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1183724049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c683e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.665+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1183724049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 0x55c90c68d7e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.665+0000 7fda5a292b80 1 --2- 172.21.15.53:0/1183724049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c692f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.665+0000 7fda5a292b80 1 -- 172.21.15.53:0/1183724049 >> 172.21.15.53:0/1183724049 conn(0x55c90c5e30d0 msgr2=0x55c90c6958a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:38.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.665+0000 7fda5a292b80 1 -- 172.21.15.53:0/1183724049 shutdown_connections 2023-11-06T01:42:38.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.665+0000 7fda5a292b80 1 -- 172.21.15.53:0/1183724049 wait complete. 2023-11-06T01:42:38.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.665+0000 7fda5a292b80 1 -- start start 2023-11-06T01:42:38.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda5a292b80 1 Processor -- start 2023-11-06T01:42:38.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c692b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c691140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda5a292b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 0x55c90c691680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda5a292b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c693350 con 0x55c90c6850f0 2023-11-06T01:42:38.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda5a292b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c691bf0 con 0x55c90c6843f0 2023-11-06T01:42:38.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda5a292b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c691db0 con 0x55c90c685a70 2023-11-06T01:42:38.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda5a292b80 10 client.0 _ll_register_callbacks cb 0x7fda3800eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:38.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c692b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda46ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c691140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda47fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 0x55c90c691680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c692b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53796/0 (socket says 172.21.15.53:53796) 2023-11-06T01:42:38.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda477fe700 1 -- 172.21.15.53:0/2397461186 learned_addr learned my addr 172.21.15.53:0/2397461186 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:38.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda46ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c691140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42358/0 (socket says 172.21.15.53:42358) 2023-11-06T01:42:38.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda47fff700 1 -- 172.21.15.53:0/2397461186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 msgr2=0x55c90c692b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:38.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda47fff700 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c692b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda47fff700 1 -- 172.21.15.53:0/2397461186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 msgr2=0x55c90c691140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:38.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda47fff700 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c691140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda47fff700 1 -- 172.21.15.53:0/2397461186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c90c402de0 con 0x55c90c685a70 2023-11-06T01:42:38.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.666+0000 7fda477fe700 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c692b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:42:38.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.667+0000 7fda47fff700 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 0x55c90c691680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fda4000bfd0 tx=0x7fda40009970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c9611b2ae712cb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:38.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.667+0000 7fda26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:42:38.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.667+0000 7fda26ffd700 1 -- 172.21.15.53:0/2397461186 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4000cc90 con 0x55c90c685a70 2023-11-06T01:42:38.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.667+0000 7fda26ffd700 1 -- 172.21.15.53:0/2397461186 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fda4009c6e0 con 0x55c90c685a70 2023-11-06T01:42:38.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.667+0000 7fda5a292b80 1 -- 172.21.15.53:0/2397461186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda400a0920 con 0x55c90c685a70 2023-11-06T01:42:38.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.667+0000 7fda26ffd700 1 -- 172.21.15.53:0/2397461186 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda4000cdf0 con 0x55c90c685a70 2023-11-06T01:42:38.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.667+0000 7fda5a292b80 10 client.26059 fetch_fsmap learned FSMap version 42 2023-11-06T01:42:38.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.667+0000 7fda5a292b80 1 -- 172.21.15.53:0/2397461186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c90c4012b0 con 0x55c90c685a70 2023-11-06T01:42:38.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.667+0000 7fda26ffd700 1 -- 172.21.15.53:0/2397461186 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda400a0920 con 0x55c90c685a70 2023-11-06T01:42:38.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.669+0000 7fda26ffd700 1 -- 172.21.15.53:0/2397461186 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda40097a10 con 0x55c90c685a70 2023-11-06T01:42:38.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.669+0000 7fda5a292b80 10 client.26059 fetch_fsmap finished waiting for FSMap version 42 2023-11-06T01:42:38.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.669+0000 7fda5a292b80 10 client.26059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:38.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.669+0000 7fda5a292b80 20 client.26059 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:38.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.669+0000 7fda5a292b80 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c90c6ec5f0 0x55c90c6eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:38.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.669+0000 7fda5a292b80 4 client.26059 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:42:38.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.669+0000 7fda5a292b80 1 -- 172.21.15.53:0/2397461186 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c90c5cdd40 con 0x55c90c6ec5f0 2023-11-06T01:42:38.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.670+0000 7fda477fe700 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c90c6ec5f0 0x55c90c6eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:38.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.670+0000 7fda26ffd700 10 client.26059 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:38.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.670+0000 7fda477fe700 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c90c6ec5f0 0x55c90c6eea10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b60ebce6aaef97b6 server_cookie=28f027a1b9b3c679 in_seq=0 out_seq=0 2023-11-06T01:42:38.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.670+0000 7fda26ffd700 1 -- 172.21.15.53:0/2397461186 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c90c5cdd40 con 0x55c90c6ec5f0 2023-11-06T01:42:38.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.670+0000 7fda26ffd700 10 client.26059 handle_command_reply: tid=0 2023-11-06T01:42:38.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.670+0000 7fda5a292b80 1 client.26059 shutdown 2023-11-06T01:42:38.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda5a292b80 1 -- 172.21.15.53:0/2397461186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 msgr2=0x55c90c691680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:38.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda5a292b80 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 0x55c90c691680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fda4000bfd0 tx=0x7fda40009970 comp rx=0 tx=0).stop 2023-11-06T01:42:38.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda5a292b80 1 -- 172.21.15.53:0/2397461186 shutdown_connections 2023-11-06T01:42:38.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda5a292b80 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c90c6843f0 0x55c90c692b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda5a292b80 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c90c6ec5f0 0x55c90c6eea10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda5a292b80 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c90c685a70 0x55c90c691680 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda26ffd700 0 client.26059 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:38.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda5a292b80 1 --2- 172.21.15.53:0/2397461186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c90c6850f0 0x55c90c691140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda5a292b80 1 -- 172.21.15.53:0/2397461186 >> 172.21.15.53:0/2397461186 conn(0x55c90c5e30d0 msgr2=0x55c90c68d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:38.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda5a292b80 1 -- 172.21.15.53:0/2397461186 shutdown_connections 2023-11-06T01:42:38.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda5a292b80 1 -- 172.21.15.53:0/2397461186 wait complete. 2023-11-06T01:42:38.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda5a292b80 20 client.26059 trim_cache size 0 max 16384 2023-11-06T01:42:38.688 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:42:38.688 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:42:38.688 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:42:38.689 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:42:38.689 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:42:38.689 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:42:38.689 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:42:38.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:42:38.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:42:38.690 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:42:38.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda4e102700 1 -- 172.21.15.53:0/3431307651 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda300a0520 msgr2=0x7fda300a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:38.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda4e102700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda300a0520 0x7fda300a29e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fda38006fd0 tx=0x7fda38009190 comp rx=0 tx=0).stop 2023-11-06T01:42:38.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.671+0000 7fda4e102700 1 -- 172.21.15.53:0/3431307651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4810d0d0 msgr2=0x7fda481ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:38.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.672+0000 7fda4e102700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4810d0d0 0x7fda481ad000 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fda4000b970 tx=0x7fda4000bc80 comp rx=0 tx=0).stop 2023-11-06T01:42:38.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.672+0000 7fda4e102700 1 -- 172.21.15.53:0/3431307651 shutdown_connections 2023-11-06T01:42:38.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.672+0000 7fda4e102700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda300a0520 0x7fda300a29e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.672+0000 7fda4e102700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4810d0d0 0x7fda481ad000 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.672+0000 7fda4e102700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda48107d90 0x7fda481a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.672+0000 7fda4e102700 1 --2- 172.21.15.53:0/3431307651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda4810a700 0x7fda481a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:38.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.672+0000 7fda4e102700 1 -- 172.21.15.53:0/3431307651 >> 172.21.15.53:0/3431307651 conn(0x7fda4806db40 msgr2=0x7fda48108a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:38.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.672+0000 7fda4e102700 1 -- 172.21.15.53:0/3431307651 shutdown_connections 2023-11-06T01:42:38.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:42:38.672+0000 7fda4e102700 1 -- 172.21.15.53:0/3431307651 wait complete. 2023-11-06T01:42:38.694 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:42:39.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:39 smithi163 ceph-mon[104416]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 2.01k op/s 2023-11-06T01:42:40.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:39 smithi053 ceph-mon[99472]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 2.01k op/s 2023-11-06T01:42:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:39 smithi062 ceph-mon[103924]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 2.01k op/s 2023-11-06T01:42:40.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:40 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:42:41.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:40 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:42:41.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:40 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:42:41.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:41 smithi163 ceph-mon[104416]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 1.55k op/s 2023-11-06T01:42:42.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:41 smithi053 ceph-mon[99472]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 1.55k op/s 2023-11-06T01:42:42.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:41 smithi062 ceph-mon[103924]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 1.55k op/s 2023-11-06T01:42:42.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 505.0 s, 1670.4 tps, lat 60.450 ms stddev 67.176 2023-11-06T01:42:43.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:43 smithi163 ceph-mon[104416]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 1.09k op/s 2023-11-06T01:42:44.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:43 smithi053 ceph-mon[99472]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 1.09k op/s 2023-11-06T01:42:44.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:43 smithi062 ceph-mon[103924]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 1.09k op/s 2023-11-06T01:42:45.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:45 smithi163 ceph-mon[104416]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 765 op/s 2023-11-06T01:42:46.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:45 smithi053 ceph-mon[99472]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 765 op/s 2023-11-06T01:42:46.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:45 smithi062 ceph-mon[103924]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 765 op/s 2023-11-06T01:42:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 510.0 s, 1737.4 tps, lat 57.175 ms stddev 61.475 2023-11-06T01:42:47.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:47 smithi053 ceph-mon[99472]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 625 op/s 2023-11-06T01:42:47.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:47 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:42:47.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:47 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:42:47.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:47 smithi062 ceph-mon[103924]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 625 op/s 2023-11-06T01:42:47.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:47 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:42:47.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:47 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:42:47.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:47 smithi163 ceph-mon[104416]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 625 op/s 2023-11-06T01:42:47.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:47 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:42:47.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:47 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:42:48.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:48 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:42:48.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:48 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:42:48.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:48 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:42:48.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:48 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:42:48.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:48 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:42:48.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:48 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:42:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.545+0000 7f4773fe0700 1 -- 172.21.15.53:0/4027997598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f476c101fa0 msgr2=0x7f476c102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.545+0000 7f4773fe0700 1 --2- 172.21.15.53:0/4027997598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f476c101fa0 0x7f476c102400 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f475c00faf0 tx=0x7f475c00fe00 comp rx=0 tx=0).stop 2023-11-06T01:42:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.546+0000 7f4773fe0700 1 -- 172.21.15.53:0/4027997598 shutdown_connections 2023-11-06T01:42:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.546+0000 7f4773fe0700 1 --2- 172.21.15.53:0/4027997598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f476c102940 0x7f476c1091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.546+0000 7f4773fe0700 1 --2- 172.21.15.53:0/4027997598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f476c100db0 0x7f476c1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.546+0000 7f4773fe0700 1 --2- 172.21.15.53:0/4027997598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f476c101fa0 0x7f476c102400 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.546+0000 7f4773fe0700 1 -- 172.21.15.53:0/4027997598 >> 172.21.15.53:0/4027997598 conn(0x7f476c0fc320 msgr2=0x7f476c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.549+0000 7f4773fe0700 1 -- 172.21.15.53:0/4027997598 shutdown_connections 2023-11-06T01:42:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.549+0000 7f4773fe0700 1 -- 172.21.15.53:0/4027997598 wait complete. 2023-11-06T01:42:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.549+0000 7f4773fe0700 1 Processor -- start 2023-11-06T01:42:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.549+0000 7f4773fe0700 1 -- start start 2023-11-06T01:42:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f4773fe0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f476c100db0 0x7f476c1949c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f4773fe0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f476c101fa0 0x7f476c194f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f4773fe0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f476c102940 0x7f476c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f4773fe0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1963e0 con 0x7f476c101fa0 2023-11-06T01:42:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f4773fe0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c196550 con 0x7f476c100db0 2023-11-06T01:42:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f4773fe0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c196710 con 0x7f476c102940 2023-11-06T01:42:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f477257d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f476c102940 0x7f476c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f477257d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f476c102940 0x7f476c195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56616/0 (socket says 172.21.15.53:56616) 2023-11-06T01:42:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f477257d700 1 -- 172.21.15.53:0/3228806304 learned_addr learned my addr 172.21.15.53:0/3228806304 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f477257d700 1 -- 172.21.15.53:0/3228806304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f476c100db0 msgr2=0x7f476c1949c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-06T01:42:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f477257d700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f476c100db0 0x7f476c1949c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f477257d700 1 -- 172.21.15.53:0/3228806304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f476c101fa0 msgr2=0x7f476c194f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f477257d700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f476c101fa0 0x7f476c194f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.550+0000 7f477257d700 1 -- 172.21.15.53:0/3228806304 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4768009520 con 0x7f476c102940 2023-11-06T01:42:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.551+0000 7f477257d700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f476c102940 0x7f476c195d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4764015a10 tx=0x7f4764015dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.551+0000 7f4762ffd700 1 -- 172.21.15.53:0/3228806304 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764013c90 con 0x7f476c102940 2023-11-06T01:42:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.551+0000 7f4762ffd700 1 -- 172.21.15.53:0/3228806304 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4764013df0 con 0x7f476c102940 2023-11-06T01:42:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.551+0000 7f4762ffd700 1 -- 172.21.15.53:0/3228806304 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764021440 con 0x7f476c102940 2023-11-06T01:42:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.552+0000 7f4773fe0700 1 -- 172.21.15.53:0/3228806304 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c00f750 con 0x7f476c102940 2023-11-06T01:42:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.552+0000 7f4773fe0700 1 -- 172.21.15.53:0/3228806304 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c19cfd0 con 0x7f476c102940 2023-11-06T01:42:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.556+0000 7f4780170b80 1 Processor -- start 2023-11-06T01:42:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.557+0000 7f4762ffd700 1 -- 172.21.15.53:0/3228806304 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4764003a00 con 0x7f476c102940 2023-11-06T01:42:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.558+0000 7f4762ffd700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47580a06f0 0x7f47580a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.558+0000 7f477157b700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47580a06f0 0x7f47580a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.558+0000 7f4762ffd700 1 -- 172.21.15.53:0/3228806304 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f47640d46c0 con 0x7f476c102940 2023-11-06T01:42:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.559+0000 7f4780170b80 1 -- start start 2023-11-06T01:42:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.559+0000 7f4780170b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 0x55b040378aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.559+0000 7f4780170b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378fe0 0x55b0403798f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.559+0000 7f4780170b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 0x55b04037a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.559+0000 7f4780170b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0402d09b0 con 0x55b040378fe0 2023-11-06T01:42:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.559+0000 7f4780170b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0402ce370 con 0x55b040379ed0 2023-11-06T01:42:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.559+0000 7f4780170b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0402c9c60 con 0x55b040378680 2023-11-06T01:42:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.559+0000 7f477257d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 0x55b04037a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.559+0000 7f4771d7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 0x55b040378aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f477257d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 0x55b04037a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53820/0 (socket says 172.21.15.53:53820) 2023-11-06T01:42:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f4771d7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 0x55b040378aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56620/0 (socket says 172.21.15.53:56620) 2023-11-06T01:42:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f477257d700 1 -- 172.21.15.53:0/2450864645 learned_addr learned my addr 172.21.15.53:0/2450864645 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f477157b700 1 --2- 172.21.15.53:0/2450864645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378fe0 0x55b0403798f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f477157b700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47580a06f0 0x7f47580a2bb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f475c005850 tx=0x7f475c00ff90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f477257d700 1 -- 172.21.15.53:0/2450864645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 msgr2=0x55b040378aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f477257d700 1 --2- 172.21.15.53:0/2450864645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 0x55b040378aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f477257d700 1 -- 172.21.15.53:0/2450864645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378fe0 msgr2=0x55b0403798f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f477257d700 1 --2- 172.21.15.53:0/2450864645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378fe0 0x55b0403798f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f477257d700 1 -- 172.21.15.53:0/2450864645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0400f7d80 con 0x55b040379ed0 2023-11-06T01:42:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f477257d700 1 --2- 172.21.15.53:0/2450864645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 0x55b04037a350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f47640a1e30 tx=0x7f47640043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a0f4f69de45e5f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.560+0000 7f4757fff700 1 -- 172.21.15.53:0/2450864645 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640a1800 con 0x55b040379ed0 2023-11-06T01:42:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.561+0000 7f4757fff700 1 -- 172.21.15.53:0/2450864645 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47640a25a0 con 0x55b040379ed0 2023-11-06T01:42:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.561+0000 7f4757fff700 1 -- 172.21.15.53:0/2450864645 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764003d40 con 0x55b040379ed0 2023-11-06T01:42:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.562+0000 7f4780170b80 1 -- 172.21.15.53:0/2450864645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 msgr2=0x55b04037a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.562+0000 7f4780170b80 1 --2- 172.21.15.53:0/2450864645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 0x55b04037a350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f47640a1e30 tx=0x7f47640043b0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.562+0000 7f4780170b80 1 -- 172.21.15.53:0/2450864645 shutdown_connections 2023-11-06T01:42:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.562+0000 7f4780170b80 1 --2- 172.21.15.53:0/2450864645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 0x55b04037a350 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.562+0000 7f4780170b80 1 --2- 172.21.15.53:0/2450864645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 0x55b040378aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.562+0000 7f4780170b80 1 --2- 172.21.15.53:0/2450864645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378fe0 0x55b0403798f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.562+0000 7f4780170b80 1 -- 172.21.15.53:0/2450864645 >> 172.21.15.53:0/2450864645 conn(0x55b0402d8070 msgr2=0x55b040375040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.562+0000 7f4780170b80 1 -- 172.21.15.53:0/2450864645 shutdown_connections 2023-11-06T01:42:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.562+0000 7f4780170b80 1 -- 172.21.15.53:0/2450864645 wait complete. 2023-11-06T01:42:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f4780170b80 1 -- start start 2023-11-06T01:42:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f4780170b80 1 Processor -- start 2023-11-06T01:42:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f4780170b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378680 0x55b040382540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f4780170b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 0x55b040382a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f4780170b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040379ed0 0x55b040389ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.573 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f4780170b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0402664b0 con 0x55b040378680 2023-11-06T01:42:49.573 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f4780170b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b040266150 con 0x55b040378fe0 2023-11-06T01:42:49.573 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f4780170b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04014d8c0 con 0x55b040379ed0 2023-11-06T01:42:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f4780170b80 10 client.0 _ll_register_callbacks cb 0x55b0401b3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f477157b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 0x55b040382a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f477257d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040379ed0 0x55b040389ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f477257d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040379ed0 0x55b040389ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56630/0 (socket says 172.21.15.53:56630) 2023-11-06T01:42:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f477157b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 0x55b040382a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53826/0 (socket says 172.21.15.53:53826) 2023-11-06T01:42:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f477257d700 1 -- 172.21.15.53:0/2546029800 learned_addr learned my addr 172.21.15.53:0/2546029800 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.563+0000 7f4771d7c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378680 0x55b040382540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.564+0000 7f477257d700 1 -- 172.21.15.53:0/2546029800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 msgr2=0x55b040382a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.564+0000 7f477257d700 1 --2- 172.21.15.53:0/2546029800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 0x55b040382a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.564+0000 7f477257d700 1 -- 172.21.15.53:0/2546029800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378680 msgr2=0x55b040382540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.564+0000 7f477257d700 1 --2- 172.21.15.53:0/2546029800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378680 0x55b040382540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.564+0000 7f477257d700 1 -- 172.21.15.53:0/2546029800 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c19cfd0 con 0x55b040379ed0 2023-11-06T01:42:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.564+0000 7f477257d700 1 --2- 172.21.15.53:0/2546029800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040379ed0 0x55b040389ac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47640121f0 tx=0x7f47640a0240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50935bbeb2c37c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.564+0000 7f47557fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:42:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.565+0000 7f47557fa700 1 -- 172.21.15.53:0/2546029800 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640a1ea0 con 0x55b040379ed0 2023-11-06T01:42:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.565+0000 7f47557fa700 1 -- 172.21.15.53:0/2546029800 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f47640a5cd0 con 0x55b040379ed0 2023-11-06T01:42:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.565+0000 7f4780170b80 1 -- 172.21.15.53:0/2546029800 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0402f4cb0 con 0x55b040379ed0 2023-11-06T01:42:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.565+0000 7f47557fa700 1 -- 172.21.15.53:0/2546029800 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47640d2250 con 0x55b040379ed0 2023-11-06T01:42:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.565+0000 7f4780170b80 10 client.26071 fetch_fsmap learned FSMap version 43 2023-11-06T01:42:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.565+0000 7f4780170b80 1 -- 172.21.15.53:0/2546029800 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55b0400e0c00 con 0x55b040379ed0 2023-11-06T01:42:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.565+0000 7f47557fa700 1 -- 172.21.15.53:0/2546029800 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0402f4cb0 con 0x55b040379ed0 2023-11-06T01:42:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.565+0000 7f47557fa700 1 -- 172.21.15.53:0/2546029800 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f47640a9a80 con 0x55b040379ed0 2023-11-06T01:42:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.565+0000 7f4780170b80 10 client.26071 fetch_fsmap finished waiting for FSMap version 43 2023-11-06T01:42:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.565+0000 7f4780170b80 10 client.26071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.566+0000 7f4780170b80 20 client.26071 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.566+0000 7f4780170b80 1 --2- 172.21.15.53:0/2546029800 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b04038de90 0x55b0403902b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.566+0000 7f4780170b80 4 client.26071 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:42:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.566+0000 7f4780170b80 1 -- 172.21.15.53:0/2546029800 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b040347db0 con 0x55b04038de90 2023-11-06T01:42:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.566+0000 7f4771d7c700 1 --2- 172.21.15.53:0/2546029800 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b04038de90 0x55b0403902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.567+0000 7f4771d7c700 1 --2- 172.21.15.53:0/2546029800 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b04038de90 0x55b0403902b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d719eb0ef0ca0c77 server_cookie=324b2317fcae23d3 in_seq=0 out_seq=0 2023-11-06T01:42:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.567+0000 7f47557fa700 10 client.26071 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f47557fa700 1 -- 172.21.15.53:0/2546029800 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b040347db0 con 0x55b04038de90 2023-11-06T01:42:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f47557fa700 10 client.26071 handle_command_reply: tid=0 2023-11-06T01:42:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 1 client.26071 shutdown 2023-11-06T01:42:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 1 -- 172.21.15.53:0/2546029800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040379ed0 msgr2=0x55b040389ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 1 --2- 172.21.15.53:0/2546029800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040379ed0 0x55b040389ac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47640121f0 tx=0x7f47640a0240 comp rx=0 tx=0).stop 2023-11-06T01:42:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 1 -- 172.21.15.53:0/2546029800 shutdown_connections 2023-11-06T01:42:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 1 --2- 172.21.15.53:0/2546029800 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b04038de90 0x55b0403902b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 1 --2- 172.21.15.53:0/2546029800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040379ed0 0x55b040389ac0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f47557fa700 0 client.26071 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 1 --2- 172.21.15.53:0/2546029800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378680 0x55b040382540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 1 --2- 172.21.15.53:0/2546029800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 0x55b040382a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 1 -- 172.21.15.53:0/2546029800 >> 172.21.15.53:0/2546029800 conn(0x55b0402d8070 msgr2=0x55b0403733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 1 -- 172.21.15.53:0/2546029800 shutdown_connections 2023-11-06T01:42:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 1 -- 172.21.15.53:0/2546029800 wait complete. 2023-11-06T01:42:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.568+0000 7f4780170b80 20 client.26071 trim_cache size 0 max 16384 2023-11-06T01:42:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.589+0000 7f4780170b80 1 Processor -- start 2023-11-06T01:42:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.589+0000 7f4780170b80 1 -- start start 2023-11-06T01:42:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.589+0000 7f4780170b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378680 0x55b0403879d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.589+0000 7f4780170b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378fe0 0x55b04037a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.589+0000 7f4780170b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 0x55b0403811f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.589+0000 7f4780170b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04036b7a0 con 0x55b040378680 2023-11-06T01:42:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.589+0000 7f4780170b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0403819f0 con 0x55b040379ed0 2023-11-06T01:42:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.589+0000 7f4780170b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04036e530 con 0x55b040378fe0 2023-11-06T01:42:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f4771d7c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378680 0x55b0403879d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f4771d7c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378680 0x55b0403879d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42394/0 (socket says 172.21.15.53:42394) 2023-11-06T01:42:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f477157b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378fe0 0x55b04037a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f477257d700 1 --2- 172.21.15.53:0/2748547456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 0x55b0403811f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f4771d7c700 1 -- 172.21.15.53:0/2748547456 learned_addr learned my addr 172.21.15.53:0/2748547456 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f477157b700 1 -- 172.21.15.53:0/2748547456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 msgr2=0x55b0403811f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f477157b700 1 --2- 172.21.15.53:0/2748547456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 0x55b0403811f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f477157b700 1 -- 172.21.15.53:0/2748547456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378680 msgr2=0x55b0403879d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f477157b700 1 --2- 172.21.15.53:0/2748547456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378680 0x55b0403879d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f477157b700 1 -- 172.21.15.53:0/2748547456 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b04011e1f0 con 0x55b040378fe0 2023-11-06T01:42:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f477257d700 1 --2- 172.21.15.53:0/2748547456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 0x55b0403811f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:42:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f477157b700 1 --2- 172.21.15.53:0/2748547456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378fe0 0x55b04037a310 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f475c00fa40 tx=0x7f475c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=715fdeaad4a99af8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f4760ff9700 1 -- 172.21.15.53:0/2748547456 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c003ea0 con 0x55b040378fe0 2023-11-06T01:42:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f4760ff9700 1 -- 172.21.15.53:0/2748547456 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f475c02b040 con 0x55b040378fe0 2023-11-06T01:42:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.590+0000 7f4760ff9700 1 -- 172.21.15.53:0/2748547456 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c019d10 con 0x55b040378fe0 2023-11-06T01:42:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.591+0000 7f4780170b80 1 -- 172.21.15.53:0/2748547456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378fe0 msgr2=0x55b04037a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.591+0000 7f4780170b80 1 --2- 172.21.15.53:0/2748547456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378fe0 0x55b04037a310 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f475c00fa40 tx=0x7f475c01c000 comp rx=0 tx=0).stop 2023-11-06T01:42:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.591+0000 7f4780170b80 1 -- 172.21.15.53:0/2748547456 shutdown_connections 2023-11-06T01:42:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.591+0000 7f4780170b80 1 --2- 172.21.15.53:0/2748547456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040379ed0 0x55b0403811f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.591+0000 7f4780170b80 1 --2- 172.21.15.53:0/2748547456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040378680 0x55b0403879d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.591+0000 7f4780170b80 1 --2- 172.21.15.53:0/2748547456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378fe0 0x55b04037a310 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.591+0000 7f4780170b80 1 -- 172.21.15.53:0/2748547456 >> 172.21.15.53:0/2748547456 conn(0x55b0402d8070 msgr2=0x55b040374c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.592+0000 7f4780170b80 1 -- 172.21.15.53:0/2748547456 shutdown_connections 2023-11-06T01:42:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.592+0000 7f4780170b80 1 -- 172.21.15.53:0/2748547456 wait complete. 2023-11-06T01:42:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.592+0000 7f4780170b80 1 -- start start 2023-11-06T01:42:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.592+0000 7f4780170b80 1 Processor -- start 2023-11-06T01:42:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f4780170b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 0x55b0403d7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f4780170b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 0x55b0403d80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f4780170b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040379ed0 0x55b0403def90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f4780170b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0403df4d0 con 0x55b040379ed0 2023-11-06T01:42:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f4780170b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0403df640 con 0x55b040378fe0 2023-11-06T01:42:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f4780170b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0403df800 con 0x55b040378680 2023-11-06T01:42:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f4780170b80 10 client.0 _ll_register_callbacks cb 0x7f475c005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:42:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f4771d7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 0x55b0403d7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f477157b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 0x55b0403d80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f4771d7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 0x55b0403d7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56640/0 (socket says 172.21.15.53:56640) 2023-11-06T01:42:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f4771d7c700 1 -- 172.21.15.53:0/3752857320 learned_addr learned my addr 172.21.15.53:0/3752857320 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:42:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f477257d700 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040379ed0 0x55b0403def90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.603 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:49 smithi053 ceph-mon[99472]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 653 op/s 2023-11-06T01:42:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f477157b700 1 -- 172.21.15.53:0/3752857320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 msgr2=0x55b0403d7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f477157b700 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 0x55b0403d7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f477157b700 1 -- 172.21.15.53:0/3752857320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040379ed0 msgr2=0x55b0403def90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:42:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f477157b700 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040379ed0 0x55b0403def90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f477157b700 1 -- 172.21.15.53:0/3752857320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0400e0c00 con 0x55b040378fe0 2023-11-06T01:42:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f477257d700 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040379ed0 0x55b0403def90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:42:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f477157b700 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 0x55b0403d80b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f475c003870 tx=0x7f475c009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231dd19f68e39c47 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:42:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.593+0000 7f47557fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:42:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f47557fa700 1 -- 172.21.15.53:0/3752857320 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c003ea0 con 0x55b040378fe0 2023-11-06T01:42:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f4780170b80 1 -- 172.21.15.53:0/3752857320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47640a25a0 con 0x55b040378fe0 2023-11-06T01:42:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f47557fa700 1 -- 172.21.15.53:0/3752857320 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f475c004000 con 0x55b040378fe0 2023-11-06T01:42:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f47557fa700 1 -- 172.21.15.53:0/3752857320 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f475c019e60 con 0x55b040378fe0 2023-11-06T01:42:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f4780170b80 10 client.25997 fetch_fsmap learned FSMap version 43 2023-11-06T01:42:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f4780170b80 1 -- 172.21.15.53:0/3752857320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f475c01f050 con 0x55b040378fe0 2023-11-06T01:42:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f47557fa700 1 -- 172.21.15.53:0/3752857320 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f475c047080 con 0x55b040378fe0 2023-11-06T01:42:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f47557fa700 1 -- 172.21.15.53:0/3752857320 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f475c041030 con 0x55b040378fe0 2023-11-06T01:42:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f4780170b80 10 client.25997 fetch_fsmap finished waiting for FSMap version 43 2023-11-06T01:42:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f4780170b80 10 client.25997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:42:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f4780170b80 20 client.25997 populate_metadata read hostname 'smithi053' 2023-11-06T01:42:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f4780170b80 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b0403e0ef0 0x55b0403e32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:42:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f4780170b80 4 client.25997 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:42:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.594+0000 7f4780170b80 1 -- 172.21.15.53:0/3752857320 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b0402c2ce0 con 0x55b0403e0ef0 2023-11-06T01:42:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.597+0000 7f4771d7c700 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b0403e0ef0 0x55b0403e32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:42:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.598+0000 7f47557fa700 10 client.25997 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.598+0000 7f4771d7c700 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b0403e0ef0 0x55b0403e32e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d72907cbb0dd1229 server_cookie=3a2511d8c6d9f2ab in_seq=0 out_seq=0 2023-11-06T01:42:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f47557fa700 1 -- 172.21.15.53:0/3752857320 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b0402c2ce0 con 0x55b0403e0ef0 2023-11-06T01:42:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f47557fa700 10 client.25997 handle_command_reply: tid=0 2023-11-06T01:42:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f4780170b80 1 client.25997 shutdown 2023-11-06T01:42:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f4780170b80 1 -- 172.21.15.53:0/3752857320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 msgr2=0x55b0403d80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f4780170b80 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 0x55b0403d80b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f475c003870 tx=0x7f475c009eb0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f4780170b80 1 -- 172.21.15.53:0/3752857320 shutdown_connections 2023-11-06T01:42:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f4780170b80 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b0403e0ef0 0x55b0403e32e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f4780170b80 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b040379ed0 0x55b0403def90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f47557fa700 0 client.25997 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:42:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f4780170b80 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b040378680 0x55b0403d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f4780170b80 1 --2- 172.21.15.53:0/3752857320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b040378fe0 0x55b0403d80b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.608+0000 7f4780170b80 1 -- 172.21.15.53:0/3752857320 >> 172.21.15.53:0/3752857320 conn(0x55b0402d8070 msgr2=0x55b0403811f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.609+0000 7f4780170b80 1 -- 172.21.15.53:0/3752857320 shutdown_connections 2023-11-06T01:42:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.609+0000 7f4780170b80 1 -- 172.21.15.53:0/3752857320 wait complete. 2023-11-06T01:42:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.609+0000 7f4780170b80 20 client.25997 trim_cache size 0 max 16384 2023-11-06T01:42:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:42:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:42:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:42:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:42:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.610+0000 7f4773fe0700 1 -- 172.21.15.53:0/3228806304 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47580a06f0 msgr2=0x7f47580a2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.610+0000 7f4773fe0700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47580a06f0 0x7f47580a2bb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f475c005850 tx=0x7f475c00ff90 comp rx=0 tx=0).stop 2023-11-06T01:42:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.610+0000 7f4773fe0700 1 -- 172.21.15.53:0/3228806304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f476c102940 msgr2=0x7f476c195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:42:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.610+0000 7f4773fe0700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f476c102940 0x7f476c195d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4764015a10 tx=0x7f4764015dd0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.610+0000 7f4773fe0700 1 -- 172.21.15.53:0/3228806304 shutdown_connections 2023-11-06T01:42:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.610+0000 7f4773fe0700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47580a06f0 0x7f47580a2bb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.610+0000 7f4773fe0700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f476c100db0 0x7f476c1949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.610+0000 7f4773fe0700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f476c102940 0x7f476c195d80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.610+0000 7f4773fe0700 1 --2- 172.21.15.53:0/3228806304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f476c101fa0 0x7f476c194f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:42:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.610+0000 7f4773fe0700 1 -- 172.21.15.53:0/3228806304 >> 172.21.15.53:0/3228806304 conn(0x7f476c0fc320 msgr2=0x7f476c1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:42:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.611+0000 7f4773fe0700 1 -- 172.21.15.53:0/3228806304 shutdown_connections 2023-11-06T01:42:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:42:49.611+0000 7f4773fe0700 1 -- 172.21.15.53:0/3228806304 wait complete. 2023-11-06T01:42:49.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:49 smithi062 ceph-mon[103924]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 653 op/s 2023-11-06T01:42:49.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:49 smithi163 ceph-mon[104416]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 653 op/s 2023-11-06T01:42:51.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:51 smithi053 ceph-mon[99472]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 634 op/s 2023-11-06T01:42:51.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:51 smithi062 ceph-mon[103924]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 634 op/s 2023-11-06T01:42:51.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:51 smithi163 ceph-mon[104416]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 634 op/s 2023-11-06T01:42:52.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 515.0 s, 1737.6 tps, lat 57.564 ms stddev 67.482 2023-11-06T01:42:53.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:53 smithi053 ceph-mon[99472]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 616 op/s 2023-11-06T01:42:53.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:53 smithi062 ceph-mon[103924]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 616 op/s 2023-11-06T01:42:53.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:53 smithi163 ceph-mon[104416]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 616 op/s 2023-11-06T01:42:55.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:55 smithi053 ceph-mon[99472]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 721 op/s 2023-11-06T01:42:55.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:55 smithi062 ceph-mon[103924]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 721 op/s 2023-11-06T01:42:55.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:55 smithi163 ceph-mon[104416]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 721 op/s 2023-11-06T01:42:57.185 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 520.0 s, 1779.4 tps, lat 56.025 ms stddev 63.517 2023-11-06T01:42:57.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:57 smithi163 ceph-mon[104416]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 621 op/s 2023-11-06T01:42:57.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:57 smithi053 ceph-mon[99472]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 621 op/s 2023-11-06T01:42:57.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:57 smithi062 ceph-mon[103924]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 621 op/s 2023-11-06T01:42:59.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:42:59 smithi163 ceph-mon[104416]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 656 op/s 2023-11-06T01:42:59.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:42:59 smithi053 ceph-mon[99472]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 656 op/s 2023-11-06T01:42:59.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:42:59 smithi062 ceph-mon[103924]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 656 op/s 2023-11-06T01:43:01.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:01 smithi163 ceph-mon[104416]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.3 MiB/s wr, 661 op/s 2023-11-06T01:43:01.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:01 smithi053 ceph-mon[99472]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.3 MiB/s wr, 661 op/s 2023-11-06T01:43:01.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:01 smithi062 ceph-mon[103924]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.3 MiB/s wr, 661 op/s 2023-11-06T01:43:02.189 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 525.0 s, 2018.9 tps, lat 49.882 ms stddev 56.779 2023-11-06T01:43:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.771+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/806339882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2dc102960 msgr2=0x7fd2dc109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.771+0000 7fd2e1e5b700 1 --2- 172.21.15.53:0/806339882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2dc102960 0x7fd2dc109210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400b0a0 tx=0x7fd2d400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.771+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/806339882 shutdown_connections 2023-11-06T01:43:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.771+0000 7fd2e1e5b700 1 --2- 172.21.15.53:0/806339882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2dc102960 0x7fd2dc109210 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.771+0000 7fd2e1e5b700 1 --2- 172.21.15.53:0/806339882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc100dd0 0x7fd2dc1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.771+0000 7fd2e1e5b700 1 --2- 172.21.15.53:0/806339882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2dc101fc0 0x7fd2dc102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.771+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/806339882 >> 172.21.15.53:0/806339882 conn(0x7fd2dc0fc340 msgr2=0x7fd2dc0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.771+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/806339882 shutdown_connections 2023-11-06T01:43:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.771+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/806339882 wait complete. 2023-11-06T01:43:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2e1e5b700 1 Processor -- start 2023-11-06T01:43:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2e1e5b700 1 -- start start 2023-11-06T01:43:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2e1e5b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2dc100dd0 0x7fd2dc0717c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2e1e5b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc101fc0 0x7fd2dc071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2e1e5b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2dc102960 0x7fd2dc1a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2e1e5b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc1a6ff0 con 0x7fd2dc101fc0 2023-11-06T01:43:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2e1e5b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc1a7130 con 0x7fd2dc100dd0 2023-11-06T01:43:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2e1e5b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc1a72f0 con 0x7fd2dc102960 2023-11-06T01:43:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2daffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc101fc0 0x7fd2dc071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2db7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2dc100dd0 0x7fd2dc0717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2dbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2dc102960 0x7fd2dc1a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2db7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2dc100dd0 0x7fd2dc0717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53852/0 (socket says 172.21.15.53:53852) 2023-11-06T01:43:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2dbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2dc102960 0x7fd2dc1a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56656/0 (socket says 172.21.15.53:56656) 2023-11-06T01:43:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.772+0000 7fd2db7fe700 1 -- 172.21.15.53:0/1629597710 learned_addr learned my addr 172.21.15.53:0/1629597710 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2db7fe700 1 -- 172.21.15.53:0/1629597710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2dc102960 msgr2=0x7fd2dc1a6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2db7fe700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2dc102960 0x7fd2dc1a6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2db7fe700 1 -- 172.21.15.53:0/1629597710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc101fc0 msgr2=0x7fd2dc071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2db7fe700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc101fc0 0x7fd2dc071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2db7fe700 1 -- 172.21.15.53:0/1629597710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d4009d00 con 0x7fd2dc100dd0 2023-11-06T01:43:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2dbfff700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2dc102960 0x7fd2dc1a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2db7fe700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2dc100dd0 0x7fd2dc0717c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00d980 tx=0x7fd2cc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2d8ff9700 1 -- 172.21.15.53:0/1629597710 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc004020 con 0x7fd2dc100dd0 2023-11-06T01:43:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/1629597710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2dc1a7570 con 0x7fd2dc100dd0 2023-11-06T01:43:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2d8ff9700 1 -- 172.21.15.53:0/1629597710 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2cc00f690 con 0x7fd2dc100dd0 2023-11-06T01:43:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2d8ff9700 1 -- 172.21.15.53:0/1629597710 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc003a60 con 0x7fd2dc100dd0 2023-11-06T01:43:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.773+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/1629597710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc1a7a80 con 0x7fd2dc100dd0 2023-11-06T01:43:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.777+0000 7fd2d8ff9700 1 -- 172.21.15.53:0/1629597710 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fd2cc00fb00 con 0x7fd2dc100dd0 2023-11-06T01:43:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.777+0000 7fd2d8ff9700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd2c40a0520 0x7fd2c40a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2d8ff9700 1 -- 172.21.15.53:0/1629597710 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd2cc0cb550 con 0x7fd2dc100dd0 2023-11-06T01:43:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2daffd700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd2c40a0520 0x7fd2c40a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2edfebb80 1 Processor -- start 2023-11-06T01:43:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2edfebb80 1 -- start start 2023-11-06T01:43:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7680 0x55d43dff7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7fe0 0x55d43dff88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43dff9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43df4f9b0 con 0x55d43dff7680 2023-11-06T01:43:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43df4d370 con 0x55d43dff8ed0 2023-11-06T01:43:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43df48c60 con 0x55d43dff7fe0 2023-11-06T01:43:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43dff9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43dff9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53860/0 (socket says 172.21.15.53:53860) 2023-11-06T01:43:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2dbfff700 1 -- 172.21.15.53:0/2310104453 learned_addr learned my addr 172.21.15.53:0/2310104453 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.778+0000 7fd2daffd700 1 --2- 172.21.15.53:0/2310104453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7fe0 0x55d43dff88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.779+0000 7fd2daffd700 1 -- 172.21.15.53:0/2310104453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 msgr2=0x55d43dff9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.779+0000 7fd2daffd700 1 --2- 172.21.15.53:0/2310104453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43dff9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.779+0000 7fd2db7fe700 1 --2- 172.21.15.53:0/2310104453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7680 0x55d43dff7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.779+0000 7fd2daffd700 1 -- 172.21.15.53:0/2310104453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7680 msgr2=0x55d43dff7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.779+0000 7fd2daffd700 1 --2- 172.21.15.53:0/2310104453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7680 0x55d43dff7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.779+0000 7fd2daffd700 1 -- 172.21.15.53:0/2310104453 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d43dd76d80 con 0x55d43dff7fe0 2023-11-06T01:43:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.779+0000 7fd2db7fe700 1 --2- 172.21.15.53:0/2310104453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7680 0x55d43dff7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:43:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.779+0000 7fd2daffd700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd2c40a0520 0x7fd2c40a29e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009cb0 tx=0x7fd2d0011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.779+0000 7fd2daffd700 1 --2- 172.21.15.53:0/2310104453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7fe0 0x55d43dff88f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0011b20 tx=0x7fd2d001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c5c81b2891e4107 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.779+0000 7fd2c1ffb700 1 -- 172.21.15.53:0/2310104453 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d001ed10 con 0x55d43dff7fe0 2023-11-06T01:43:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.780+0000 7fd2c1ffb700 1 -- 172.21.15.53:0/2310104453 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2d001ee70 con 0x55d43dff7fe0 2023-11-06T01:43:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.780+0000 7fd2c1ffb700 1 -- 172.21.15.53:0/2310104453 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d002b8e0 con 0x55d43dff7fe0 2023-11-06T01:43:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.780+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2310104453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7fe0 msgr2=0x55d43dff88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.780+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/2310104453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7fe0 0x55d43dff88f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0011b20 tx=0x7fd2d001e000 comp rx=0 tx=0).stop 2023-11-06T01:43:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.780+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2310104453 shutdown_connections 2023-11-06T01:43:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.780+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/2310104453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43dff9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.780+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/2310104453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7680 0x55d43dff7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/2310104453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7fe0 0x55d43dff88f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2310104453 >> 172.21.15.53:0/2310104453 conn(0x55d43df57070 msgr2=0x55d43dff4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2310104453 shutdown_connections 2023-11-06T01:43:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2310104453 wait complete. 2023-11-06T01:43:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 -- start start 2023-11-06T01:43:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 Processor -- start 2023-11-06T01:43:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7680 0x55d43e001510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e001a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43e008a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43dee54b0 con 0x55d43dff7fe0 2023-11-06T01:43:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43dee5150 con 0x55d43dff8ed0 2023-11-06T01:43:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43ddcc8c0 con 0x55d43dff7680 2023-11-06T01:43:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.781+0000 7fd2edfebb80 10 client.0 _ll_register_callbacks cb 0x55d43de32a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2daffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e001a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2db7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7680 0x55d43e001510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2daffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e001a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42428/0 (socket says 172.21.15.53:42428) 2023-11-06T01:43:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2db7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7680 0x55d43e001510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56666/0 (socket says 172.21.15.53:56666) 2023-11-06T01:43:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2daffd700 1 -- 172.21.15.53:0/529204464 learned_addr learned my addr 172.21.15.53:0/529204464 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2dbfff700 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43e008a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2daffd700 1 -- 172.21.15.53:0/529204464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7680 msgr2=0x55d43e001510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2daffd700 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7680 0x55d43e001510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2daffd700 1 -- 172.21.15.53:0/529204464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 msgr2=0x55d43e008a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2daffd700 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43e008a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2daffd700 1 -- 172.21.15.53:0/529204464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d0009e10 con 0x55d43dff7fe0 2023-11-06T01:43:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2db7fe700 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7680 0x55d43e001510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2daffd700 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e001a50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00057e0 tx=0x7fd2d0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33aee2796e752e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2baffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:43:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2baffd700 1 -- 172.21.15.53:0/529204464 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0031070 con 0x55d43dff7fe0 2023-11-06T01:43:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2edfebb80 1 -- 172.21.15.53:0/529204464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d43df73cb0 con 0x55d43dff7fe0 2023-11-06T01:43:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.782+0000 7fd2baffd700 1 -- 172.21.15.53:0/529204464 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd2d0027030 con 0x55d43dff7fe0 2023-11-06T01:43:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.783+0000 7fd2baffd700 1 -- 172.21.15.53:0/529204464 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2d001ec50 con 0x55d43dff7fe0 2023-11-06T01:43:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.783+0000 7fd2edfebb80 10 client.16449 fetch_fsmap learned FSMap version 43 2023-11-06T01:43:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.783+0000 7fd2edfebb80 1 -- 172.21.15.53:0/529204464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55d43dd5fc00 con 0x55d43dff7fe0 2023-11-06T01:43:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.783+0000 7fd2baffd700 1 -- 172.21.15.53:0/529204464 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d43df73cb0 con 0x55d43dff7fe0 2023-11-06T01:43:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.783+0000 7fd2baffd700 1 -- 172.21.15.53:0/529204464 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7fd2d002ba30 con 0x55d43dff7fe0 2023-11-06T01:43:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.783+0000 7fd2edfebb80 10 client.16449 fetch_fsmap finished waiting for FSMap version 43 2023-11-06T01:43:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.783+0000 7fd2edfebb80 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.783+0000 7fd2edfebb80 20 client.16449 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.783+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d43e00d180 0x55d43e00f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.783+0000 7fd2edfebb80 4 client.16449 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:43:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.783+0000 7fd2edfebb80 1 -- 172.21.15.53:0/529204464 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d43dfc6db0 con 0x55d43e00d180 2023-11-06T01:43:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.784+0000 7fd2db7fe700 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d43e00d180 0x55d43e00f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.785+0000 7fd2baffd700 10 client.16449 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.785+0000 7fd2db7fe700 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d43e00d180 0x55d43e00f5a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f582310896be5a9 server_cookie=b84a2db73ad8e7c5 in_seq=0 out_seq=0 2023-11-06T01:43:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.785+0000 7fd2baffd700 1 -- 172.21.15.53:0/529204464 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d43dfc6db0 con 0x55d43e00d180 2023-11-06T01:43:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.785+0000 7fd2baffd700 10 client.16449 handle_command_reply: tid=0 2023-11-06T01:43:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 1 client.16449 shutdown 2023-11-06T01:43:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 1 -- 172.21.15.53:0/529204464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 msgr2=0x55d43e001a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e001a50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00057e0 tx=0x7fd2d0005810 comp rx=0 tx=0).stop 2023-11-06T01:43:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 1 -- 172.21.15.53:0/529204464 shutdown_connections 2023-11-06T01:43:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43e008a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d43e00d180 0x55d43e00f5a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7680 0x55d43e001510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/529204464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e001a50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 1 -- 172.21.15.53:0/529204464 >> 172.21.15.53:0/529204464 conn(0x55d43df57070 msgr2=0x55d43dff23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2baffd700 0 client.16449 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 1 -- 172.21.15.53:0/529204464 shutdown_connections 2023-11-06T01:43:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 1 -- 172.21.15.53:0/529204464 wait complete. 2023-11-06T01:43:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.786+0000 7fd2edfebb80 20 client.16449 trim_cache size 0 max 16384 2023-11-06T01:43:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.822+0000 7fd2edfebb80 1 Processor -- start 2023-11-06T01:43:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.822+0000 7fd2edfebb80 1 -- start start 2023-11-06T01:43:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.822+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7680 0x55d43e006a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.822+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff7fe0 0x55d43e00f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.822+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff8ed0 0x55d43dffc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.822+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43dffff30 con 0x55d43dff7680 2023-11-06T01:43:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.822+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43dfea9f0 con 0x55d43dff7fe0 2023-11-06T01:43:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.822+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43e0009c0 con 0x55d43dff8ed0 2023-11-06T01:43:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.823+0000 7fd2dbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff8ed0 0x55d43dffc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.823+0000 7fd2dbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff8ed0 0x55d43dffc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56674/0 (socket says 172.21.15.53:56674) 2023-11-06T01:43:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.823+0000 7fd2dbfff700 1 -- 172.21.15.53:0/1513762676 learned_addr learned my addr 172.21.15.53:0/1513762676 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.826+0000 7fd2db7fe700 1 --2- 172.21.15.53:0/1513762676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7680 0x55d43e006a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.827+0000 7fd2daffd700 1 --2- 172.21.15.53:0/1513762676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff7fe0 0x55d43e00f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.831+0000 7fd2dbfff700 1 -- 172.21.15.53:0/1513762676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff7fe0 msgr2=0x55d43e00f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.831+0000 7fd2dbfff700 1 --2- 172.21.15.53:0/1513762676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff7fe0 0x55d43e00f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.831+0000 7fd2dbfff700 1 -- 172.21.15.53:0/1513762676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7680 msgr2=0x55d43e006a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.831+0000 7fd2dbfff700 1 --2- 172.21.15.53:0/1513762676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7680 0x55d43e006a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.831+0000 7fd2dbfff700 1 -- 172.21.15.53:0/1513762676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d43dd9d1f0 con 0x55d43dff8ed0 2023-11-06T01:43:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.831+0000 7fd2dbfff700 1 --2- 172.21.15.53:0/1513762676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff8ed0 0x55d43dffc0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009500 tx=0x7fd2d4009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b72f023b01bd9f25 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.832+0000 7fd2c27fc700 1 -- 172.21.15.53:0/1513762676 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d400e040 con 0x55d43dff8ed0 2023-11-06T01:43:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.832+0000 7fd2c27fc700 1 -- 172.21.15.53:0/1513762676 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2d4003c00 con 0x55d43dff8ed0 2023-11-06T01:43:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.832+0000 7fd2c27fc700 1 -- 172.21.15.53:0/1513762676 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4040480 con 0x55d43dff8ed0 2023-11-06T01:43:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.833+0000 7fd2edfebb80 1 -- 172.21.15.53:0/1513762676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff8ed0 msgr2=0x55d43dffc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.833+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/1513762676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff8ed0 0x55d43dffc0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009500 tx=0x7fd2d4009530 comp rx=0 tx=0).stop 2023-11-06T01:43:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.849+0000 7fd2edfebb80 1 -- 172.21.15.53:0/1513762676 shutdown_connections 2023-11-06T01:43:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.849+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/1513762676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff8ed0 0x55d43dffc0a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.849+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/1513762676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7680 0x55d43e006a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.849+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/1513762676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff7fe0 0x55d43e00f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.849+0000 7fd2edfebb80 1 -- 172.21.15.53:0/1513762676 >> 172.21.15.53:0/1513762676 conn(0x55d43df57070 msgr2=0x55d43e000140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.852+0000 7fd2edfebb80 1 -- 172.21.15.53:0/1513762676 shutdown_connections 2023-11-06T01:43:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.859+0000 7fd2edfebb80 1 -- 172.21.15.53:0/1513762676 wait complete. 2023-11-06T01:43:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.859+0000 7fd2edfebb80 1 -- start start 2023-11-06T01:43:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.859+0000 7fd2edfebb80 1 Processor -- start 2023-11-06T01:43:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.860+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7680 0x55d43e002c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.860+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e0031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.860+0000 7fd2edfebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43e005d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.860+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43e0062c0 con 0x55d43dff7fe0 2023-11-06T01:43:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.860+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43e006430 con 0x55d43dff8ed0 2023-11-06T01:43:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.860+0000 7fd2edfebb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43e0065a0 con 0x55d43dff7680 2023-11-06T01:43:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.860+0000 7fd2edfebb80 10 client.0 _ll_register_callbacks cb 0x7fd2d0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.860+0000 7fd2daffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e0031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.860+0000 7fd2daffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e0031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42440/0 (socket says 172.21.15.53:42440) 2023-11-06T01:43:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.860+0000 7fd2daffd700 1 -- 172.21.15.53:0/2130655450 learned_addr learned my addr 172.21.15.53:0/2130655450 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.861+0000 7fd2daffd700 1 -- 172.21.15.53:0/2130655450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7680 msgr2=0x55d43e002c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:43:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.861+0000 7fd2daffd700 1 --2- 172.21.15.53:0/2130655450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7680 0x55d43e002c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.861+0000 7fd2daffd700 1 -- 172.21.15.53:0/2130655450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 msgr2=0x55d43e005d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:43:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.861+0000 7fd2daffd700 1 --2- 172.21.15.53:0/2130655450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43e005d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.861+0000 7fd2daffd700 1 -- 172.21.15.53:0/2130655450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc109b60 con 0x55d43dff7fe0 2023-11-06T01:43:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.862+0000 7fd2daffd700 1 --2- 172.21.15.53:0/2130655450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e0031b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd2d001ee40 tx=0x7fd2d001ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1172f074dc7d5a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.862+0000 7fd2baffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:43:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.862+0000 7fd2baffd700 1 -- 172.21.15.53:0/2130655450 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0031030 con 0x55d43dff7fe0 2023-11-06T01:43:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.862+0000 7fd2baffd700 1 -- 172.21.15.53:0/2130655450 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd2d0032070 con 0x55d43dff7fe0 2023-11-06T01:43:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.863+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2130655450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2d001ee70 con 0x55d43dff7fe0 2023-11-06T01:43:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.863+0000 7fd2baffd700 1 -- 172.21.15.53:0/2130655450 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2d001ec50 con 0x55d43dff7fe0 2023-11-06T01:43:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.863+0000 7fd2edfebb80 10 client.16455 fetch_fsmap learned FSMap version 43 2023-11-06T01:43:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.863+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2130655450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55d43dd75250 con 0x55d43dff7fe0 2023-11-06T01:43:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.863+0000 7fd2baffd700 1 -- 172.21.15.53:0/2130655450 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2d001ee70 con 0x55d43dff7fe0 2023-11-06T01:43:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.863+0000 7fd2baffd700 1 -- 172.21.15.53:0/2130655450 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7fd2d002b9a0 con 0x55d43dff7fe0 2023-11-06T01:43:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.863+0000 7fd2edfebb80 10 client.16455 fetch_fsmap finished waiting for FSMap version 43 2023-11-06T01:43:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.863+0000 7fd2edfebb80 10 client.16455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.864+0000 7fd2edfebb80 20 client.16455 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.864+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/2130655450 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d43e05b990 0x55d43e05de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.864+0000 7fd2edfebb80 4 client.16455 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:43:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.864+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2130655450 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d43df41ce0 con 0x55d43e05b990 2023-11-06T01:43:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.864+0000 7fd2db7fe700 1 --2- 172.21.15.53:0/2130655450 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d43e05b990 0x55d43e05de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.865+0000 7fd2db7fe700 1 --2- 172.21.15.53:0/2130655450 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d43e05b990 0x55d43e05de30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c79f94be5d5bbe5 server_cookie=59e35323cf42d735 in_seq=0 out_seq=0 2023-11-06T01:43:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.865+0000 7fd2baffd700 10 client.16455 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2baffd700 1 -- 172.21.15.53:0/2130655450 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d43df41ce0 con 0x55d43e05b990 2023-11-06T01:43:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2baffd700 10 client.16455 handle_command_reply: tid=0 2023-11-06T01:43:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2edfebb80 1 client.16455 shutdown 2023-11-06T01:43:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2130655450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 msgr2=0x55d43e0031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/2130655450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e0031b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd2d001ee40 tx=0x7fd2d001ea90 comp rx=0 tx=0).stop 2023-11-06T01:43:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2130655450 shutdown_connections 2023-11-06T01:43:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/2130655450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d43dff8ed0 0x55d43e005d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/2130655450 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d43e05b990 0x55d43e05de30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/2130655450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d43dff7680 0x55d43e002c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2baffd700 0 client.16455 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2edfebb80 1 --2- 172.21.15.53:0/2130655450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d43dff7fe0 0x55d43e0031b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.902+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2130655450 >> 172.21.15.53:0/2130655450 conn(0x55d43df57070 msgr2=0x55d43e00f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.903+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2130655450 shutdown_connections 2023-11-06T01:43:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.903+0000 7fd2edfebb80 1 -- 172.21.15.53:0/2130655450 wait complete. 2023-11-06T01:43:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2edfebb80 20 client.16455 trim_cache size 0 max 16384 2023-11-06T01:43:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:43:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:43:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:43:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:43:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/1629597710 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd2c40a0520 msgr2=0x7fd2c40a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd2c40a0520 0x7fd2c40a29e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009cb0 tx=0x7fd2d0011720 comp rx=0 tx=0).stop 2023-11-06T01:43:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/1629597710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2dc100dd0 msgr2=0x7fd2dc0717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2dc100dd0 0x7fd2dc0717c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00d980 tx=0x7fd2cc00dc90 comp rx=0 tx=0).stop 2023-11-06T01:43:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/1629597710 shutdown_connections 2023-11-06T01:43:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fd2c40a0520 0x7fd2c40a29e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2dc100dd0 0x7fd2dc0717c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2dc102960 0x7fd2dc1a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 --2- 172.21.15.53:0/1629597710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2dc101fc0 0x7fd2dc071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/1629597710 >> 172.21.15.53:0/1629597710 conn(0x7fd2dc0fc340 msgr2=0x7fd2dc0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/1629597710 shutdown_connections 2023-11-06T01:43:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:02.905+0000 7fd2e1e5b700 1 -- 172.21.15.53:0/1629597710 wait complete. 2023-11-06T01:43:03.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:03 smithi163 ceph-mon[104416]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 10 MiB/s wr, 702 op/s 2023-11-06T01:43:03.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:03 smithi053 ceph-mon[99472]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 10 MiB/s wr, 702 op/s 2023-11-06T01:43:03.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:03 smithi062 ceph-mon[103924]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 10 MiB/s wr, 702 op/s 2023-11-06T01:43:05.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:05 smithi163 ceph-mon[104416]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 MiB/s wr, 798 op/s 2023-11-06T01:43:05.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:05 smithi053 ceph-mon[99472]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 MiB/s wr, 798 op/s 2023-11-06T01:43:05.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:05 smithi062 ceph-mon[103924]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 MiB/s wr, 798 op/s 2023-11-06T01:43:07.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 530.0 s, 2029.9 tps, lat 48.926 ms stddev 60.130 2023-11-06T01:43:07.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:07 smithi053 ceph-mon[99472]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.7 MiB/s wr, 726 op/s 2023-11-06T01:43:07.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:07 smithi062 ceph-mon[103924]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.7 MiB/s wr, 726 op/s 2023-11-06T01:43:07.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:07 smithi163 ceph-mon[104416]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.7 MiB/s wr, 726 op/s 2023-11-06T01:43:08.695 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:43:08.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.939+0000 7fb2fea2f700 1 -- 172.21.15.53:0/2107195265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f810d0d0 msgr2=0x7fb2f810d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:08.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.939+0000 7fb2fea2f700 1 --2- 172.21.15.53:0/2107195265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f810d0d0 0x7fb2f810d550 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009990 tx=0x7fb2e0009ca0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.939+0000 7fb2fea2f700 1 -- 172.21.15.53:0/2107195265 shutdown_connections 2023-11-06T01:43:08.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.939+0000 7fb2fea2f700 1 --2- 172.21.15.53:0/2107195265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2f8107d90 0x7fb2f810a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.939+0000 7fb2fea2f700 1 --2- 172.21.15.53:0/2107195265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f810d0d0 0x7fb2f810d550 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.939+0000 7fb2fea2f700 1 --2- 172.21.15.53:0/2107195265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2f810a700 0x7fb2f810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.939+0000 7fb2fea2f700 1 -- 172.21.15.53:0/2107195265 >> 172.21.15.53:0/2107195265 conn(0x7fb2f806db40 msgr2=0x7fb2f806ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:08.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.940+0000 7fb2fea2f700 1 -- 172.21.15.53:0/2107195265 shutdown_connections 2023-11-06T01:43:08.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.940+0000 7fb2fea2f700 1 -- 172.21.15.53:0/2107195265 wait complete. 2023-11-06T01:43:08.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.940+0000 7fb2fea2f700 1 Processor -- start 2023-11-06T01:43:08.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.940+0000 7fb2fea2f700 1 -- start start 2023-11-06T01:43:08.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.941+0000 7fb2fea2f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2f8107d90 0x7fb2f8077e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.941+0000 7fb2fea2f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f810a700 0x7fb2f8078390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.941+0000 7fb2fea2f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2f810d0d0 0x7fb2f807f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.941+0000 7fb2fea2f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f807f9d0 con 0x7fb2f810a700 2023-11-06T01:43:08.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.941+0000 7fb2fea2f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f807fb40 con 0x7fb2f8107d90 2023-11-06T01:43:08.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.941+0000 7fb2fea2f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f807fd00 con 0x7fb2f810d0d0 2023-11-06T01:43:08.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.942+0000 7fb2f77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f810a700 0x7fb2f8078390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:08.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.942+0000 7fb2f77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f810a700 0x7fb2f8078390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42452/0 (socket says 172.21.15.53:42452) 2023-11-06T01:43:08.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.942+0000 7fb2f77fe700 1 -- 172.21.15.53:0/3349427924 learned_addr learned my addr 172.21.15.53:0/3349427924 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:08.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.942+0000 7fb2f77fe700 1 -- 172.21.15.53:0/3349427924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2f810d0d0 msgr2=0x7fb2f807f400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-06T01:43:08.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.942+0000 7fb2f77fe700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2f810d0d0 0x7fb2f807f400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.942+0000 7fb2f77fe700 1 -- 172.21.15.53:0/3349427924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2f8107d90 msgr2=0x7fb2f8077e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-06T01:43:08.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.942+0000 7fb2f77fe700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2f8107d90 0x7fb2f8077e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.942+0000 7fb2f77fe700 1 -- 172.21.15.53:0/3349427924 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e00095f0 con 0x7fb2f810a700 2023-11-06T01:43:08.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.942+0000 7fb2f77fe700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f810a700 0x7fb2f8078390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb2e804f550 tx=0x7fb2e804f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.943+0000 7fb2f57fa700 1 -- 172.21.15.53:0/3349427924 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8055830 con 0x7fb2f810a700 2023-11-06T01:43:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.943+0000 7fb2f57fa700 1 -- 172.21.15.53:0/3349427924 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2e8055e50 con 0x7fb2f810a700 2023-11-06T01:43:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.943+0000 7fb2f57fa700 1 -- 172.21.15.53:0/3349427924 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80532e0 con 0x7fb2f810a700 2023-11-06T01:43:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.943+0000 7fb2fea2f700 1 -- 172.21.15.53:0/3349427924 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f807ff80 con 0x7fb2f810a700 2023-11-06T01:43:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.943+0000 7fb2fea2f700 1 -- 172.21.15.53:0/3349427924 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f80804c0 con 0x7fb2f810a700 2023-11-06T01:43:08.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.947+0000 7fb30abbfb80 1 Processor -- start 2023-11-06T01:43:08.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.948+0000 7fb2f57fa700 1 -- 172.21.15.53:0/3349427924 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fb2e8055990 con 0x7fb2f810a700 2023-11-06T01:43:08.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.949+0000 7fb2f57fa700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb2e40a05f0 0x7fb2e40a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.949+0000 7fb2f57fa700 1 -- 172.21.15.53:0/3349427924 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb2e810f440 con 0x7fb2f810a700 2023-11-06T01:43:08.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.952+0000 7fb30abbfb80 1 -- start start 2023-11-06T01:43:08.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.952+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c20f0 0x55ac799c2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.952+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c2a70 0x55ac799c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.952+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c13f0 0x55ac799c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.952+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac79918a10 con 0x55ac799c2a70 2023-11-06T01:43:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.952+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac799163d0 con 0x55ac799c13f0 2023-11-06T01:43:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.952+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac79911cc0 con 0x55ac799c20f0 2023-11-06T01:43:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.952+0000 7fb2f77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c2a70 0x55ac799c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.953+0000 7fb2f77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c2a70 0x55ac799c0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42460/0 (socket says 172.21.15.53:42460) 2023-11-06T01:43:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.953+0000 7fb2f77fe700 1 -- 172.21.15.53:0/2312199747 learned_addr learned my addr 172.21.15.53:0/2312199747 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.953+0000 7fb2fcfcc700 1 --2- 172.21.15.53:0/2312199747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c13f0 0x55ac799c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.953+0000 7fb2f77fe700 1 -- 172.21.15.53:0/2312199747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c20f0 msgr2=0x55ac799c2530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:43:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.953+0000 7fb2f77fe700 1 --2- 172.21.15.53:0/2312199747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c20f0 0x55ac799c2530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.953+0000 7fb2f77fe700 1 -- 172.21.15.53:0/2312199747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c13f0 msgr2=0x55ac799c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.953+0000 7fb2f77fe700 1 --2- 172.21.15.53:0/2312199747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c13f0 0x55ac799c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.953+0000 7fb2f77fe700 1 -- 172.21.15.53:0/2312199747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac7973fde0 con 0x55ac799c2a70 2023-11-06T01:43:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.953+0000 7fb2f77fe700 1 --2- 172.21.15.53:0/2312199747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c2a70 0x55ac799c0e60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80dcc10 tx=0x7fb2e80dc8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecaff23874103e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:08.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb2ee7fc700 1 -- 172.21.15.53:0/2312199747 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80e0ab0 con 0x55ac799c2a70 2023-11-06T01:43:08.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb2ee7fc700 1 -- 172.21.15.53:0/2312199747 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2e80e4cf0 con 0x55ac799c2a70 2023-11-06T01:43:08.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb2ee7fc700 1 -- 172.21.15.53:0/2312199747 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80fc310 con 0x55ac799c2a70 2023-11-06T01:43:08.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb30abbfb80 1 -- 172.21.15.53:0/2312199747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c2a70 msgr2=0x55ac799c0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:08.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/2312199747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c2a70 0x55ac799c0e60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80dcc10 tx=0x7fb2e80dc8c0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb30abbfb80 1 -- 172.21.15.53:0/2312199747 shutdown_connections 2023-11-06T01:43:08.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/2312199747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c13f0 0x55ac799c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/2312199747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c20f0 0x55ac799c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/2312199747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c2a70 0x55ac799c0e60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb30abbfb80 1 -- 172.21.15.53:0/2312199747 >> 172.21.15.53:0/2312199747 conn(0x55ac799200d0 msgr2=0x55ac799bb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:08.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb30abbfb80 1 -- 172.21.15.53:0/2312199747 shutdown_connections 2023-11-06T01:43:08.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.954+0000 7fb30abbfb80 1 -- 172.21.15.53:0/2312199747 wait complete. 2023-11-06T01:43:08.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.955+0000 7fb30abbfb80 1 -- start start 2023-11-06T01:43:08.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.955+0000 7fb30abbfb80 1 Processor -- start 2023-11-06T01:43:08.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.955+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c13f0 0x55ac799dba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c20f0 0x55ac799dbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac799e3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac798ae510 con 0x55ac799c13f0 2023-11-06T01:43:08.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac798ae1b0 con 0x55ac799c2a70 2023-11-06T01:43:08.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac79795920 con 0x55ac799c20f0 2023-11-06T01:43:08.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb30abbfb80 10 client.0 _ll_register_callbacks cb 0x55ac797fba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:08.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2f7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c13f0 0x55ac799dba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:08.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2f7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c13f0 0x55ac799dba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42464/0 (socket says 172.21.15.53:42464) 2023-11-06T01:43:08.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2fcfcc700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb2e40a05f0 0x7fb2e40a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:08.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2f7fff700 1 -- 172.21.15.53:0/1024627757 learned_addr learned my addr 172.21.15.53:0/1024627757 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:08.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2fcfcc700 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac799e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:08.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2f77fe700 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c20f0 0x55ac799dbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:08.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2f7fff700 1 -- 172.21.15.53:0/1024627757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c20f0 msgr2=0x55ac799dbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:08.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2f7fff700 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c20f0 0x55ac799dbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2f7fff700 1 -- 172.21.15.53:0/1024627757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 msgr2=0x55ac799e3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:08.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2f7fff700 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac799e3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2f7fff700 1 -- 172.21.15.53:0/1024627757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f80804c0 con 0x55ac799c13f0 2023-11-06T01:43:08.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2fcfcc700 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac799e3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:43:08.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.956+0000 7fb2fcfcc700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb2e40a05f0 0x7fb2e40a2ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000b5c0 tx=0x7fb2e000b9d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:08.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.957+0000 7fb2f7fff700 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c13f0 0x55ac799dba80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000b300 tx=0x7fb2f000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c15bceefdf7ab04 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:08.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.957+0000 7fb2d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:43:08.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.957+0000 7fb2d77fe700 1 -- 172.21.15.53:0/1024627757 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000e070 con 0x55ac799c13f0 2023-11-06T01:43:08.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.957+0000 7fb2d77fe700 1 -- 172.21.15.53:0/1024627757 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb2f000be10 con 0x55ac799c13f0 2023-11-06T01:43:08.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.957+0000 7fb30abbfb80 1 -- 172.21.15.53:0/1024627757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac7993cd10 con 0x55ac799c13f0 2023-11-06T01:43:08.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.957+0000 7fb2d77fe700 1 -- 172.21.15.53:0/1024627757 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2f00097a0 con 0x55ac799c13f0 2023-11-06T01:43:08.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.957+0000 7fb30abbfb80 10 client.16467 fetch_fsmap learned FSMap version 43 2023-11-06T01:43:08.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.957+0000 7fb30abbfb80 1 -- 172.21.15.53:0/1024627757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55ac79728c60 con 0x55ac799c13f0 2023-11-06T01:43:08.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.957+0000 7fb2d77fe700 1 -- 172.21.15.53:0/1024627757 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f0021020 con 0x55ac799c13f0 2023-11-06T01:43:08.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.958+0000 7fb2d77fe700 1 -- 172.21.15.53:0/1024627757 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7fb2f00127e0 con 0x55ac799c13f0 2023-11-06T01:43:08.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.958+0000 7fb30abbfb80 10 client.16467 fetch_fsmap finished waiting for FSMap version 43 2023-11-06T01:43:08.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.958+0000 7fb30abbfb80 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:08.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.958+0000 7fb30abbfb80 20 client.16467 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:08.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.958+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ac799e78c0 0x55ac799e9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.958+0000 7fb30abbfb80 4 client.16467 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:43:08.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.958+0000 7fb30abbfb80 1 -- 172.21.15.53:0/1024627757 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac7998fe10 con 0x55ac799e78c0 2023-11-06T01:43:08.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.958+0000 7fb2f77fe700 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ac799e78c0 0x55ac799e9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:08.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.958+0000 7fb2f77fe700 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ac799e78c0 0x55ac799e9ce0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b289d2d8253410 server_cookie=61a47df2e6012bff in_seq=0 out_seq=0 2023-11-06T01:43:08.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.958+0000 7fb2d77fe700 10 client.16467 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:08.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.959+0000 7fb2d77fe700 1 -- 172.21.15.53:0/1024627757 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ac7998fe10 con 0x55ac799e78c0 2023-11-06T01:43:08.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.959+0000 7fb2d77fe700 10 client.16467 handle_command_reply: tid=0 2023-11-06T01:43:08.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.959+0000 7fb30abbfb80 1 client.16467 shutdown 2023-11-06T01:43:08.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.959+0000 7fb30abbfb80 1 -- 172.21.15.53:0/1024627757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c13f0 msgr2=0x55ac799dba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:08.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.959+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c13f0 0x55ac799dba80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000b300 tx=0x7fb2f000b6c0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.959+0000 7fb30abbfb80 1 -- 172.21.15.53:0/1024627757 shutdown_connections 2023-11-06T01:43:08.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.959+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac799e3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.959+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ac799e78c0 0x55ac799e9ce0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.960+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c13f0 0x55ac799dba80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.960+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/1024627757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c20f0 0x55ac799dbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.960+0000 7fb2d77fe700 0 client.16467 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:08.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.960+0000 7fb30abbfb80 1 -- 172.21.15.53:0/1024627757 >> 172.21.15.53:0/1024627757 conn(0x55ac799200d0 msgr2=0x55ac799bb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:08.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.960+0000 7fb30abbfb80 1 -- 172.21.15.53:0/1024627757 shutdown_connections 2023-11-06T01:43:08.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.960+0000 7fb30abbfb80 1 -- 172.21.15.53:0/1024627757 wait complete. 2023-11-06T01:43:08.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.960+0000 7fb30abbfb80 20 client.16467 trim_cache size 0 max 16384 2023-11-06T01:43:08.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.985+0000 7fb30abbfb80 1 Processor -- start 2023-11-06T01:43:08.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.986+0000 7fb30abbfb80 1 -- start start 2023-11-06T01:43:08.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.987+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c13f0 0x55ac799c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.987+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c20f0 0x55ac799e0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.987+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac799db830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:08.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.987+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac799bc810 con 0x55ac799c20f0 2023-11-06T01:43:08.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.987+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac799e4cc0 con 0x55ac799c2a70 2023-11-06T01:43:08.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.987+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac799c8f50 con 0x55ac799c13f0 2023-11-06T01:43:08.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.987+0000 7fb2fcfcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac799db830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:08.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.987+0000 7fb2f7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c13f0 0x55ac799c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:08.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.987+0000 7fb2fcfcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac799db830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53912/0 (socket says 172.21.15.53:53912) 2023-11-06T01:43:08.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.987+0000 7fb2fcfcc700 1 -- 172.21.15.53:0/2033127070 learned_addr learned my addr 172.21.15.53:0/2033127070 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:08.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.988+0000 7fb2fcfcc700 1 -- 172.21.15.53:0/2033127070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c13f0 msgr2=0x55ac799c0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:08.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.988+0000 7fb2fcfcc700 1 --2- 172.21.15.53:0/2033127070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c13f0 0x55ac799c0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.988+0000 7fb2fcfcc700 1 -- 172.21.15.53:0/2033127070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c20f0 msgr2=0x55ac799e0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:08.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.988+0000 7fb2fcfcc700 1 --2- 172.21.15.53:0/2033127070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c20f0 0x55ac799e0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.988+0000 7fb2fcfcc700 1 -- 172.21.15.53:0/2033127070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac79766250 con 0x55ac799c2a70 2023-11-06T01:43:08.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.988+0000 7fb2f7fff700 1 --2- 172.21.15.53:0/2033127070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c13f0 0x55ac799c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:08.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.988+0000 7fb2fcfcc700 1 --2- 172.21.15.53:0/2033127070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac799db830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009990 tx=0x7fb2e0004000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bb57719fe68723e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:08.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.988+0000 7fb2eeffd700 1 -- 172.21.15.53:0/2033127070 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0005410 con 0x55ac799c2a70 2023-11-06T01:43:08.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.988+0000 7fb2eeffd700 1 -- 172.21.15.53:0/2033127070 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2e001e070 con 0x55ac799c2a70 2023-11-06T01:43:08.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.988+0000 7fb2eeffd700 1 -- 172.21.15.53:0/2033127070 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0027420 con 0x55ac799c2a70 2023-11-06T01:43:08.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.989+0000 7fb30abbfb80 1 -- 172.21.15.53:0/2033127070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 msgr2=0x55ac799db830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:08.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.989+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/2033127070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac799db830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009990 tx=0x7fb2e0004000 comp rx=0 tx=0).stop 2023-11-06T01:43:08.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.989+0000 7fb30abbfb80 1 -- 172.21.15.53:0/2033127070 shutdown_connections 2023-11-06T01:43:08.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.989+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/2033127070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac799db830 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.989+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/2033127070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c13f0 0x55ac799c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.989+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/2033127070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c20f0 0x55ac799e0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:08.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.989+0000 7fb30abbfb80 1 -- 172.21.15.53:0/2033127070 >> 172.21.15.53:0/2033127070 conn(0x55ac799200d0 msgr2=0x55ac799c5450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:08.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.989+0000 7fb30abbfb80 1 -- 172.21.15.53:0/2033127070 shutdown_connections 2023-11-06T01:43:08.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.989+0000 7fb30abbfb80 1 -- 172.21.15.53:0/2033127070 wait complete. 2023-11-06T01:43:08.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.990+0000 7fb30abbfb80 1 -- start start 2023-11-06T01:43:09.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.990+0000 7fb30abbfb80 1 Processor -- start 2023-11-06T01:43:09.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.990+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c13f0 0x55ac79a31b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:09.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.990+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c20f0 0x55ac79a32080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:09.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.990+0000 7fb30abbfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac79a390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:09.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.990+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac79a39600 con 0x55ac799c20f0 2023-11-06T01:43:09.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.990+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac79a39740 con 0x55ac799c2a70 2023-11-06T01:43:09.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.990+0000 7fb30abbfb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac79a398d0 con 0x55ac799c13f0 2023-11-06T01:43:09.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.990+0000 7fb30abbfb80 10 client.0 _ll_register_callbacks cb 0x7fb2f0009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:09.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.991+0000 7fb2f77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c20f0 0x55ac79a32080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:09.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.991+0000 7fb2fcfcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac79a390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:09.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.991+0000 7fb2fcfcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac79a390c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53918/0 (socket says 172.21.15.53:53918) 2023-11-06T01:43:09.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.991+0000 7fb2fcfcc700 1 -- 172.21.15.53:0/3272717642 learned_addr learned my addr 172.21.15.53:0/3272717642 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:09.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.991+0000 7fb2f7fff700 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c13f0 0x55ac79a31b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:09.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.991+0000 7fb2fcfcc700 1 -- 172.21.15.53:0/3272717642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c13f0 msgr2=0x55ac79a31b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:09.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.991+0000 7fb2fcfcc700 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c13f0 0x55ac79a31b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:09.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.991+0000 7fb2fcfcc700 1 -- 172.21.15.53:0/3272717642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c20f0 msgr2=0x55ac79a32080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:09.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.991+0000 7fb2fcfcc700 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c20f0 0x55ac79a32080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:09.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.991+0000 7fb2fcfcc700 1 -- 172.21.15.53:0/3272717642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac79728c60 con 0x55ac799c2a70 2023-11-06T01:43:09.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.992+0000 7fb2fcfcc700 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac79a390c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000be40 tx=0x7fb2e0005a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d73c9ebe3ccd424 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:09.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.992+0000 7fb2d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:43:09.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.992+0000 7fb2d77fe700 1 -- 172.21.15.53:0/3272717642 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0038070 con 0x55ac799c2a70 2023-11-06T01:43:09.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.992+0000 7fb30abbfb80 1 -- 172.21.15.53:0/3272717642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2e80e4cf0 con 0x55ac799c2a70 2023-11-06T01:43:09.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.992+0000 7fb2d77fe700 1 -- 172.21.15.53:0/3272717642 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb2e000f460 con 0x55ac799c2a70 2023-11-06T01:43:09.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.992+0000 7fb2d77fe700 1 -- 172.21.15.53:0/3272717642 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2e000f660 con 0x55ac799c2a70 2023-11-06T01:43:09.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.992+0000 7fb30abbfb80 10 client.26024 fetch_fsmap learned FSMap version 43 2023-11-06T01:43:09.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.992+0000 7fb30abbfb80 1 -- 172.21.15.53:0/3272717642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fb2f810e0d0 con 0x55ac799c2a70 2023-11-06T01:43:09.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.992+0000 7fb2d77fe700 1 -- 172.21.15.53:0/3272717642 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2e80e4cf0 con 0x55ac799c2a70 2023-11-06T01:43:09.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.993+0000 7fb2d77fe700 1 -- 172.21.15.53:0/3272717642 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7fb2e00327a0 con 0x55ac799c2a70 2023-11-06T01:43:09.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.993+0000 7fb30abbfb80 10 client.26024 fetch_fsmap finished waiting for FSMap version 43 2023-11-06T01:43:09.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.993+0000 7fb30abbfb80 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:09.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.993+0000 7fb30abbfb80 20 client.26024 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:09.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.993+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ac79a3afc0 0x55ac79a3d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:09.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.993+0000 7fb30abbfb80 4 client.26024 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:43:09.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.993+0000 7fb30abbfb80 1 -- 172.21.15.53:0/3272717642 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ac7990ad40 con 0x55ac79a3afc0 2023-11-06T01:43:09.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.993+0000 7fb2f7fff700 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ac79a3afc0 0x55ac79a3d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:09.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.993+0000 7fb2d77fe700 10 client.26024 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:09.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.993+0000 7fb2f7fff700 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ac79a3afc0 0x55ac79a3d3e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5849e434552435d6 server_cookie=7cf9dc287be425a5 in_seq=0 out_seq=0 2023-11-06T01:43:09.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb2d77fe700 1 -- 172.21.15.53:0/3272717642 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ac7990ad40 con 0x55ac79a3afc0 2023-11-06T01:43:09.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb2d77fe700 10 client.26024 handle_command_reply: tid=0 2023-11-06T01:43:09.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 1 client.26024 shutdown 2023-11-06T01:43:09.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 1 -- 172.21.15.53:0/3272717642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 msgr2=0x55ac79a390c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:09.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac79a390c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000be40 tx=0x7fb2e0005a10 comp rx=0 tx=0).stop 2023-11-06T01:43:09.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 1 -- 172.21.15.53:0/3272717642 shutdown_connections 2023-11-06T01:43:09.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac799c2a70 0x55ac79a390c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:09.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ac79a3afc0 0x55ac79a3d3e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:09.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ac799c13f0 0x55ac79a31b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:09.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 1 --2- 172.21.15.53:0/3272717642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac799c20f0 0x55ac79a32080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:09.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb2d77fe700 0 client.26024 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:09.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 1 -- 172.21.15.53:0/3272717642 >> 172.21.15.53:0/3272717642 conn(0x55ac799200d0 msgr2=0x55ac799db830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:09.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 1 -- 172.21.15.53:0/3272717642 shutdown_connections 2023-11-06T01:43:09.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 1 -- 172.21.15.53:0/3272717642 wait complete. 2023-11-06T01:43:09.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.994+0000 7fb30abbfb80 20 client.26024 trim_cache size 0 max 16384 2023-11-06T01:43:09.011 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:43:09.011 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:43:09.011 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:43:09.011 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:43:09.012 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:43:09.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:43:09.012 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:43:09.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:43:09.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:43:09.012 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:43:09.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 -- 172.21.15.53:0/3349427924 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb2e40a05f0 msgr2=0x7fb2e40a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:09.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb2e40a05f0 0x7fb2e40a2ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000b5c0 tx=0x7fb2e000b9d0 comp rx=0 tx=0).stop 2023-11-06T01:43:09.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 -- 172.21.15.53:0/3349427924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f810a700 msgr2=0x7fb2f8078390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:09.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f810a700 0x7fb2f8078390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb2e804f550 tx=0x7fb2e804f910 comp rx=0 tx=0).stop 2023-11-06T01:43:09.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 -- 172.21.15.53:0/3349427924 shutdown_connections 2023-11-06T01:43:09.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb2e40a05f0 0x7fb2e40a2ab0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:09.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2f8107d90 0x7fb2f8077e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:09.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2f810d0d0 0x7fb2f807f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:09.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 --2- 172.21.15.53:0/3349427924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f810a700 0x7fb2f8078390 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:09.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 -- 172.21.15.53:0/3349427924 >> 172.21.15.53:0/3349427924 conn(0x7fb2f806db40 msgr2=0x7fb2f8108ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:09.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 -- 172.21.15.53:0/3349427924 shutdown_connections 2023-11-06T01:43:09.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:08.995+0000 7fb2fea2f700 1 -- 172.21.15.53:0/3349427924 wait complete. 2023-11-06T01:43:09.017 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:43:09.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:09 smithi053 ceph-mon[99472]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 MiB/s wr, 800 op/s 2023-11-06T01:43:09.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:09 smithi062 ceph-mon[103924]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 MiB/s wr, 800 op/s 2023-11-06T01:43:09.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:09 smithi163 ceph-mon[104416]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 MiB/s wr, 800 op/s 2023-11-06T01:43:11.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:11 smithi053 ceph-mon[99472]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 797 op/s 2023-11-06T01:43:11.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:11 smithi062 ceph-mon[103924]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 797 op/s 2023-11-06T01:43:11.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:11 smithi163 ceph-mon[104416]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 797 op/s 2023-11-06T01:43:12.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 535.0 s, 1210.0 tps, lat 82.597 ms stddev 92.559 2023-11-06T01:43:13.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:13 smithi053 ceph-mon[99472]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 836 op/s 2023-11-06T01:43:13.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:13 smithi062 ceph-mon[103924]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 836 op/s 2023-11-06T01:43:13.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:13 smithi163 ceph-mon[104416]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 836 op/s 2023-11-06T01:43:15.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:15 smithi053 ceph-mon[99472]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 911 op/s 2023-11-06T01:43:15.901 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:15 smithi062 ceph-mon[103924]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 911 op/s 2023-11-06T01:43:15.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:15 smithi163 ceph-mon[104416]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 911 op/s 2023-11-06T01:43:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.047+0000 7f38f7b9d700 1 -- 172.21.15.53:0/1553815595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f0106b30 msgr2=0x7f38f0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.047+0000 7f38f7b9d700 1 --2- 172.21.15.53:0/1553815595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f0106b30 0x7f38f0108f20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f38e800b0a0 tx=0x7f38e800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.047+0000 7f38f7b9d700 1 -- 172.21.15.53:0/1553815595 shutdown_connections 2023-11-06T01:43:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.047+0000 7f38f7b9d700 1 --2- 172.21.15.53:0/1553815595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f0106b30 0x7f38f0108f20 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.047+0000 7f38f7b9d700 1 --2- 172.21.15.53:0/1553815595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f01018d0 0x7f38f0103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.047+0000 7f38f7b9d700 1 --2- 172.21.15.53:0/1553815595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0104200 0x7f38f01065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.047+0000 7f38f7b9d700 1 -- 172.21.15.53:0/1553815595 >> 172.21.15.53:0/1553815595 conn(0x7f38f00fb240 msgr2=0x7f38f00fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.047+0000 7f38f7b9d700 1 -- 172.21.15.53:0/1553815595 shutdown_connections 2023-11-06T01:43:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.047+0000 7f38f7b9d700 1 -- 172.21.15.53:0/1553815595 wait complete. 2023-11-06T01:43:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f7b9d700 1 Processor -- start 2023-11-06T01:43:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f7b9d700 1 -- start start 2023-11-06T01:43:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f7b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f01018d0 0x7f38f00ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f7b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0104200 0x7f38f00ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f7b9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f0106b30 0x7f38f0100300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f7b9d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f01a0bd0 con 0x7f38f0106b30 2023-11-06T01:43:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f7b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f01a0d10 con 0x7f38f0104200 2023-11-06T01:43:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f7b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f01a0e50 con 0x7f38f01018d0 2023-11-06T01:43:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f5939700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f01018d0 0x7f38f00ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f5939700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f01018d0 0x7f38f00ff880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56732/0 (socket says 172.21.15.53:56732) 2023-11-06T01:43:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f5939700 1 -- 172.21.15.53:0/2200797721 learned_addr learned my addr 172.21.15.53:0/2200797721 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f5138700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0104200 0x7f38f00ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.048+0000 7f38f613a700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f0106b30 0x7f38f0100300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.049+0000 7f38f5939700 1 -- 172.21.15.53:0/2200797721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0104200 msgr2=0x7f38f00ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.049+0000 7f38f5939700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0104200 0x7f38f00ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.049+0000 7f38f5939700 1 -- 172.21.15.53:0/2200797721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f0106b30 msgr2=0x7f38f0100300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.049+0000 7f38f5939700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f0106b30 0x7f38f0100300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.049+0000 7f38f5939700 1 -- 172.21.15.53:0/2200797721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e8009d00 con 0x7f38f01018d0 2023-11-06T01:43:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.049+0000 7f38f5939700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f01018d0 0x7f38f00ff880 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00e9a0 tx=0x7f38ec00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.049+0000 7f38e6ffd700 1 -- 172.21.15.53:0/2200797721 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec009790 con 0x7f38f01018d0 2023-11-06T01:43:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.049+0000 7f38e6ffd700 1 -- 172.21.15.53:0/2200797721 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38ec010430 con 0x7f38f01018d0 2023-11-06T01:43:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.049+0000 7f38f7b9d700 1 -- 172.21.15.53:0/2200797721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f01a10e0 con 0x7f38f01018d0 2023-11-06T01:43:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.049+0000 7f38e6ffd700 1 -- 172.21.15.53:0/2200797721 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec018680 con 0x7f38f01018d0 2023-11-06T01:43:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.049+0000 7f38f7b9d700 1 -- 172.21.15.53:0/2200797721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f01a15f0 con 0x7f38f01018d0 2023-11-06T01:43:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.054+0000 7f3903d2db80 1 Processor -- start 2023-11-06T01:43:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f3903d2db80 1 -- start start 2023-11-06T01:43:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72dd0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0fe0 0x561e72dd18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 0x561e72dd2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f3903d2db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72d289b0 con 0x561e72dd1ed0 2023-11-06T01:43:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f3903d2db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72d26370 con 0x561e72dd0680 2023-11-06T01:43:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f3903d2db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72d21c60 con 0x561e72dd0fe0 2023-11-06T01:43:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f38f613a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72dd0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f38f5138700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 0x561e72dd2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f38f613a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72dd0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53936/0 (socket says 172.21.15.53:53936) 2023-11-06T01:43:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f38f613a700 1 -- 172.21.15.53:0/4129594499 learned_addr learned my addr 172.21.15.53:0/4129594499 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f38f5138700 1 -- 172.21.15.53:0/4129594499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0fe0 msgr2=0x561e72dd18f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:43:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f38f5138700 1 --2- 172.21.15.53:0/4129594499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0fe0 0x561e72dd18f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f38f5138700 1 -- 172.21.15.53:0/4129594499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 msgr2=0x561e72dd0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f38f5138700 1 --2- 172.21.15.53:0/4129594499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72dd0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f38f5138700 1 -- 172.21.15.53:0/4129594499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e72b4fd80 con 0x561e72dd1ed0 2023-11-06T01:43:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.055+0000 7f38f613a700 1 --2- 172.21.15.53:0/4129594499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72dd0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.056+0000 7f38f5138700 1 --2- 172.21.15.53:0/4129594499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 0x561e72dd2350 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009c80 tx=0x7f38e0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae3df32211828fe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.056+0000 7f38e6ffd700 1 -- 172.21.15.53:0/2200797721 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f38ec0105a0 con 0x7f38f01018d0 2023-11-06T01:43:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.056+0000 7f38dbfff700 1 -- 172.21.15.53:0/4129594499 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0011cb0 con 0x561e72dd1ed0 2023-11-06T01:43:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.056+0000 7f38dbfff700 1 -- 172.21.15.53:0/4129594499 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e0011e10 con 0x561e72dd1ed0 2023-11-06T01:43:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.056+0000 7f38dbfff700 1 -- 172.21.15.53:0/4129594499 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0005280 con 0x561e72dd1ed0 2023-11-06T01:43:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.056+0000 7f38e6ffd700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f38dc0a0520 0x7f38dc0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f38e6ffd700 1 -- 172.21.15.53:0/2200797721 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f38ec014070 con 0x7f38f01018d0 2023-11-06T01:43:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f38f613a700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f38dc0a0520 0x7f38dc0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f3903d2db80 1 -- 172.21.15.53:0/4129594499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 msgr2=0x561e72dd2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f3903d2db80 1 --2- 172.21.15.53:0/4129594499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 0x561e72dd2350 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009c80 tx=0x7f38e0009f90 comp rx=0 tx=0).stop 2023-11-06T01:43:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f3903d2db80 1 -- 172.21.15.53:0/4129594499 shutdown_connections 2023-11-06T01:43:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f3903d2db80 1 --2- 172.21.15.53:0/4129594499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72dd0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f3903d2db80 1 --2- 172.21.15.53:0/4129594499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 0x561e72dd2350 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f3903d2db80 1 --2- 172.21.15.53:0/4129594499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0fe0 0x561e72dd18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f3903d2db80 1 -- 172.21.15.53:0/4129594499 >> 172.21.15.53:0/4129594499 conn(0x561e72d30070 msgr2=0x561e72dcd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f3903d2db80 1 -- 172.21.15.53:0/4129594499 shutdown_connections 2023-11-06T01:43:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f3903d2db80 1 -- 172.21.15.53:0/4129594499 wait complete. 2023-11-06T01:43:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.057+0000 7f38f613a700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f38dc0a0520 0x7f38dc0a29e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f38e8015000 tx=0x7f38e800b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.058+0000 7f3903d2db80 1 -- start start 2023-11-06T01:43:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.058+0000 7f3903d2db80 1 Processor -- start 2023-11-06T01:43:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.058+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0680 0x561e72dda460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.058+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0fe0 0x561e72dda9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.058+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 0x561e72de19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.058+0000 7f3903d2db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72cbe4b0 con 0x561e72dd1ed0 2023-11-06T01:43:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.058+0000 7f3903d2db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72cbe150 con 0x561e72dd0fe0 2023-11-06T01:43:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.058+0000 7f3903d2db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72ba58c0 con 0x561e72dd0680 2023-11-06T01:43:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.058+0000 7f3903d2db80 10 client.0 _ll_register_callbacks cb 0x561e72c0ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.058+0000 7f38f613a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 0x561e72de19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f5138700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0fe0 0x561e72dda9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f5939700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0680 0x561e72dda460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f613a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 0x561e72de19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42508/0 (socket says 172.21.15.53:42508) 2023-11-06T01:43:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f613a700 1 -- 172.21.15.53:0/1326397551 learned_addr learned my addr 172.21.15.53:0/1326397551 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f5138700 1 -- 172.21.15.53:0/1326397551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0680 msgr2=0x561e72dda460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f5138700 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0680 0x561e72dda460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f5138700 1 -- 172.21.15.53:0/1326397551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 msgr2=0x561e72de19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f5138700 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 0x561e72de19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f5138700 1 -- 172.21.15.53:0/1326397551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f0109b10 con 0x561e72dd0fe0 2023-11-06T01:43:16.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f5939700 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0680 0x561e72dda460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:16.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f613a700 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 0x561e72de19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:16.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38f5138700 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0fe0 0x561e72dda9a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38e0000c00 tx=0x7f38e0005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a84da68162a91fc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:16.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:43:16.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38d97fa700 1 -- 172.21.15.53:0/1326397551 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0004260 con 0x561e72dd0fe0 2023-11-06T01:43:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f38d97fa700 1 -- 172.21.15.53:0/1326397551 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f38e00043c0 con 0x561e72dd0fe0 2023-11-06T01:43:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.059+0000 7f3903d2db80 1 -- 172.21.15.53:0/1326397551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e72d4ccb0 con 0x561e72dd0fe0 2023-11-06T01:43:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f38d97fa700 1 -- 172.21.15.53:0/1326397551 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38e0004580 con 0x561e72dd0fe0 2023-11-06T01:43:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f3903d2db80 10 client.26036 fetch_fsmap learned FSMap version 43 2023-11-06T01:43:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f3903d2db80 1 -- 172.21.15.53:0/1326397551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f38e00098e0 con 0x561e72dd0fe0 2023-11-06T01:43:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f38d97fa700 1 -- 172.21.15.53:0/1326397551 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e72d4ccb0 con 0x561e72dd0fe0 2023-11-06T01:43:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f38d97fa700 1 -- 172.21.15.53:0/1326397551 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f38e00052e0 con 0x561e72dd0fe0 2023-11-06T01:43:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f3903d2db80 10 client.26036 fetch_fsmap finished waiting for FSMap version 43 2023-11-06T01:43:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f3903d2db80 10 client.26036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f3903d2db80 20 client.26036 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561e72de6220 0x561e72de8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f3903d2db80 4 client.26036 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:43:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f3903d2db80 1 -- 172.21.15.53:0/1326397551 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e72d9fdb0 con 0x561e72de6220 2023-11-06T01:43:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.060+0000 7f38f5939700 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561e72de6220 0x561e72de8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.061+0000 7f38d97fa700 10 client.26036 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.061+0000 7f38f5939700 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561e72de6220 0x561e72de8640 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d40c8a5db9f9b8 server_cookie=70e43f8cee8141cd in_seq=0 out_seq=0 2023-11-06T01:43:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.061+0000 7f38d97fa700 1 -- 172.21.15.53:0/1326397551 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561e72d9fdb0 con 0x561e72de6220 2023-11-06T01:43:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.061+0000 7f38d97fa700 10 client.26036 handle_command_reply: tid=0 2023-11-06T01:43:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.061+0000 7f3903d2db80 1 client.26036 shutdown 2023-11-06T01:43:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.061+0000 7f3903d2db80 1 -- 172.21.15.53:0/1326397551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0fe0 msgr2=0x561e72dda9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.061+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0fe0 0x561e72dda9a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38e0000c00 tx=0x7f38e0005e40 comp rx=0 tx=0).stop 2023-11-06T01:43:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.062+0000 7f3903d2db80 1 -- 172.21.15.53:0/1326397551 shutdown_connections 2023-11-06T01:43:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.062+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561e72de6220 0x561e72de8640 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.062+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd1ed0 0x561e72de19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.062+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0680 0x561e72dda460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.062+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1326397551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0fe0 0x561e72dda9a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.062+0000 7f3903d2db80 1 -- 172.21.15.53:0/1326397551 >> 172.21.15.53:0/1326397551 conn(0x561e72d30070 msgr2=0x561e72dcb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.062+0000 7f38d97fa700 0 client.26036 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.063+0000 7f3903d2db80 1 -- 172.21.15.53:0/1326397551 shutdown_connections 2023-11-06T01:43:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.063+0000 7f3903d2db80 1 -- 172.21.15.53:0/1326397551 wait complete. 2023-11-06T01:43:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.064+0000 7f3903d2db80 20 client.26036 trim_cache size 0 max 16384 2023-11-06T01:43:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.096+0000 7f3903d2db80 1 Processor -- start 2023-11-06T01:43:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.104+0000 7f3903d2db80 1 -- start start 2023-11-06T01:43:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.104+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0680 0x561e72ddf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.104+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd0fe0 0x561e72dda460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.104+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd1ed0 0x561e72dda9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.104+0000 7f3903d2db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72dd9910 con 0x561e72dd0fe0 2023-11-06T01:43:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.104+0000 7f3903d2db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72dc6500 con 0x561e72dd1ed0 2023-11-06T01:43:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.104+0000 7f3903d2db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72dd9fd0 con 0x561e72dd0680 2023-11-06T01:43:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.108+0000 7f38f5138700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd0fe0 0x561e72dda460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.108+0000 7f38f5138700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd0fe0 0x561e72dda460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42516/0 (socket says 172.21.15.53:42516) 2023-11-06T01:43:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.108+0000 7f38f5138700 1 -- 172.21.15.53:0/2494585072 learned_addr learned my addr 172.21.15.53:0/2494585072 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.110+0000 7f38f5939700 1 --2- 172.21.15.53:0/2494585072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0680 0x561e72ddf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.118+0000 7f38f5138700 1 -- 172.21.15.53:0/2494585072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0680 msgr2=0x561e72ddf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.118+0000 7f38f5138700 1 --2- 172.21.15.53:0/2494585072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0680 0x561e72ddf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.118+0000 7f38f5138700 1 -- 172.21.15.53:0/2494585072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd1ed0 msgr2=0x561e72dda9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.118+0000 7f38f5138700 1 --2- 172.21.15.53:0/2494585072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd1ed0 0x561e72dda9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.118+0000 7f38f5138700 1 -- 172.21.15.53:0/2494585072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e72b38c00 con 0x561e72dd0fe0 2023-11-06T01:43:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.131+0000 7f38f5138700 1 --2- 172.21.15.53:0/2494585072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd0fe0 0x561e72dda460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f38e0011de0 tx=0x7f38e0004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c272c7027106e485 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.143+0000 7f38e4daa700 1 -- 172.21.15.53:0/2494585072 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0022550 con 0x561e72dd0fe0 2023-11-06T01:43:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.143+0000 7f38e4daa700 1 -- 172.21.15.53:0/2494585072 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e0015070 con 0x561e72dd0fe0 2023-11-06T01:43:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.143+0000 7f38e4daa700 1 -- 172.21.15.53:0/2494585072 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0004d70 con 0x561e72dd0fe0 2023-11-06T01:43:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.144+0000 7f3903d2db80 1 -- 172.21.15.53:0/2494585072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd0fe0 msgr2=0x561e72dda460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.144+0000 7f3903d2db80 1 --2- 172.21.15.53:0/2494585072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd0fe0 0x561e72dda460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f38e0011de0 tx=0x7f38e0004c30 comp rx=0 tx=0).stop 2023-11-06T01:43:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.150+0000 7f3903d2db80 1 -- 172.21.15.53:0/2494585072 shutdown_connections 2023-11-06T01:43:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.150+0000 7f3903d2db80 1 --2- 172.21.15.53:0/2494585072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd1ed0 0x561e72dda9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.150+0000 7f3903d2db80 1 --2- 172.21.15.53:0/2494585072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd0680 0x561e72ddf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.150+0000 7f3903d2db80 1 --2- 172.21.15.53:0/2494585072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd0fe0 0x561e72dda460 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.150+0000 7f3903d2db80 1 -- 172.21.15.53:0/2494585072 >> 172.21.15.53:0/2494585072 conn(0x561e72d30070 msgr2=0x561e72dca660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.152+0000 7f3903d2db80 1 -- 172.21.15.53:0/2494585072 shutdown_connections 2023-11-06T01:43:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.152+0000 7f3903d2db80 1 -- 172.21.15.53:0/2494585072 wait complete. 2023-11-06T01:43:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.152+0000 7f3903d2db80 1 -- start start 2023-11-06T01:43:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.152+0000 7f3903d2db80 1 Processor -- start 2023-11-06T01:43:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.153+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72e31ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.153+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd0fe0 0x561e72e34440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.153+0000 7f3903d2db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd1ed0 0x561e72e36990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.153+0000 7f3903d2db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72e37080 con 0x561e72dd0fe0 2023-11-06T01:43:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.153+0000 7f3903d2db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72e371f0 con 0x561e72dd0680 2023-11-06T01:43:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.153+0000 7f3903d2db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e72e373b0 con 0x561e72dd1ed0 2023-11-06T01:43:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.153+0000 7f3903d2db80 10 client.0 _ll_register_callbacks cb 0x7f38e0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.153+0000 7f38f5939700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72e31ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.153+0000 7f38f5939700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72e31ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:53956/0 (socket says 172.21.15.53:53956) 2023-11-06T01:43:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.153+0000 7f38f5939700 1 -- 172.21.15.53:0/1728003209 learned_addr learned my addr 172.21.15.53:0/1728003209 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.154+0000 7f38f5939700 1 -- 172.21.15.53:0/1728003209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd1ed0 msgr2=0x561e72e36990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:43:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.154+0000 7f38f5939700 1 --2- 172.21.15.53:0/1728003209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd1ed0 0x561e72e36990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.154+0000 7f38f5939700 1 -- 172.21.15.53:0/1728003209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd0fe0 msgr2=0x561e72e34440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.154+0000 7f38f5939700 1 --2- 172.21.15.53:0/1728003209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd0fe0 0x561e72e34440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.154+0000 7f38f5939700 1 -- 172.21.15.53:0/1728003209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e00098e0 con 0x561e72dd0680 2023-11-06T01:43:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.157+0000 7f38f5939700 1 --2- 172.21.15.53:0/1728003209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72e31ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0c96a0 tx=0x7f38ec010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac7732cad9863aa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.157+0000 7f38d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:43:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.157+0000 7f38d97fa700 1 -- 172.21.15.53:0/1728003209 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0993b0 con 0x561e72dd0680 2023-11-06T01:43:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.157+0000 7f38d97fa700 1 -- 172.21.15.53:0/1728003209 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f38ec09d1e0 con 0x561e72dd0680 2023-11-06T01:43:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.158+0000 7f3903d2db80 1 -- 172.21.15.53:0/1728003209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38e0011e10 con 0x561e72dd0680 2023-11-06T01:43:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.158+0000 7f38d97fa700 1 -- 172.21.15.53:0/1728003209 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38ec0a1420 con 0x561e72dd0680 2023-11-06T01:43:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.160+0000 7f3903d2db80 10 client.26039 fetch_fsmap learned FSMap version 43 2023-11-06T01:43:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.160+0000 7f3903d2db80 1 -- 172.21.15.53:0/1728003209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561e72b761f0 con 0x561e72dd0680 2023-11-06T01:43:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.160+0000 7f38d97fa700 1 -- 172.21.15.53:0/1728003209 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e0011e10 con 0x561e72dd0680 2023-11-06T01:43:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.161+0000 7f38d97fa700 1 -- 172.21.15.53:0/1728003209 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f38ec064f60 con 0x561e72dd0680 2023-11-06T01:43:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.161+0000 7f3903d2db80 10 client.26039 fetch_fsmap finished waiting for FSMap version 43 2023-11-06T01:43:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.161+0000 7f3903d2db80 10 client.26039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.161+0000 7f3903d2db80 20 client.26039 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.161+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1728003209 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561e72e38dd0 0x561e72e3b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.161+0000 7f3903d2db80 4 client.26039 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:43:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.161+0000 7f3903d2db80 1 -- 172.21.15.53:0/1728003209 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561e72d1ace0 con 0x561e72e38dd0 2023-11-06T01:43:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.161+0000 7f38f5138700 1 --2- 172.21.15.53:0/1728003209 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561e72e38dd0 0x561e72e3b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.162+0000 7f38f5138700 1 --2- 172.21.15.53:0/1728003209 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561e72e38dd0 0x561e72e3b1f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e67e92f965cfa58a server_cookie=a10ff90fbb139f61 in_seq=0 out_seq=0 2023-11-06T01:43:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.162+0000 7f38d97fa700 10 client.26039 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.174+0000 7f38d97fa700 1 -- 172.21.15.53:0/1728003209 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561e72d1ace0 con 0x561e72e38dd0 2023-11-06T01:43:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.174+0000 7f38d97fa700 10 client.26039 handle_command_reply: tid=0 2023-11-06T01:43:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.174+0000 7f3903d2db80 1 client.26039 shutdown 2023-11-06T01:43:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.176+0000 7f3903d2db80 1 -- 172.21.15.53:0/1728003209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 msgr2=0x561e72e31ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.176+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1728003209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72e31ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0c96a0 tx=0x7f38ec010c80 comp rx=0 tx=0).stop 2023-11-06T01:43:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.176+0000 7f3903d2db80 1 -- 172.21.15.53:0/1728003209 shutdown_connections 2023-11-06T01:43:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.176+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1728003209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e72dd0680 0x561e72e31ef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.176+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1728003209 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561e72e38dd0 0x561e72e3b1f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.176+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1728003209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e72dd1ed0 0x561e72e36990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.176+0000 7f3903d2db80 1 --2- 172.21.15.53:0/1728003209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e72dd0fe0 0x561e72e34440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.176+0000 7f38d97fa700 0 client.26039 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.176+0000 7f3903d2db80 1 -- 172.21.15.53:0/1728003209 >> 172.21.15.53:0/1728003209 conn(0x561e72d30070 msgr2=0x561e72dda400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.176+0000 7f3903d2db80 1 -- 172.21.15.53:0/1728003209 shutdown_connections 2023-11-06T01:43:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.176+0000 7f3903d2db80 1 -- 172.21.15.53:0/1728003209 wait complete. 2023-11-06T01:43:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f3903d2db80 20 client.26039 trim_cache size 0 max 16384 2023-11-06T01:43:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:43:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:43:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:43:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:43:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 -- 172.21.15.53:0/2200797721 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f38dc0a0520 msgr2=0x7f38dc0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f38dc0a0520 0x7f38dc0a29e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f38e8015000 tx=0x7f38e800b720 comp rx=0 tx=0).stop 2023-11-06T01:43:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 -- 172.21.15.53:0/2200797721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f01018d0 msgr2=0x7f38f00ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f01018d0 0x7f38f00ff880 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00e9a0 tx=0x7f38ec00ecb0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 -- 172.21.15.53:0/2200797721 shutdown_connections 2023-11-06T01:43:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f38dc0a0520 0x7f38dc0a29e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f0106b30 0x7f38f0100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f01018d0 0x7f38f00ff880 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 --2- 172.21.15.53:0/2200797721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0104200 0x7f38f00ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 -- 172.21.15.53:0/2200797721 >> 172.21.15.53:0/2200797721 conn(0x7f38f00fb240 msgr2=0x7f38f0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 -- 172.21.15.53:0/2200797721 shutdown_connections 2023-11-06T01:43:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:16.177+0000 7f38f7b9d700 1 -- 172.21.15.53:0/2200797721 wait complete. 2023-11-06T01:43:17.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 540.0 s, 1619.2 tps, lat 61.455 ms stddev 70.461 2023-11-06T01:43:17.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:17 smithi053 ceph-mon[99472]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 792 op/s 2023-11-06T01:43:17.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:17 smithi062 ceph-mon[103924]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 792 op/s 2023-11-06T01:43:17.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:17 smithi163 ceph-mon[104416]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 792 op/s 2023-11-06T01:43:19.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:19 smithi053 ceph-mon[99472]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 820 op/s 2023-11-06T01:43:19.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:19 smithi062 ceph-mon[103924]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 820 op/s 2023-11-06T01:43:19.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:19 smithi163 ceph-mon[104416]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 820 op/s 2023-11-06T01:43:20.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:20 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:43:20.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:20 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:43:21.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:20 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:43:21.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:21 smithi062 ceph-mon[103924]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 785 op/s 2023-11-06T01:43:21.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:21 smithi163 ceph-mon[104416]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 785 op/s 2023-11-06T01:43:22.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:21 smithi053 ceph-mon[99472]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 785 op/s 2023-11-06T01:43:22.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 545.0 s, 1581.2 tps, lat 63.575 ms stddev 74.280 2023-11-06T01:43:23.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:23 smithi062 ceph-mon[103924]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 747 op/s 2023-11-06T01:43:23.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:23 smithi163 ceph-mon[104416]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 747 op/s 2023-11-06T01:43:24.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:23 smithi053 ceph-mon[99472]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 747 op/s 2023-11-06T01:43:25.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:25 smithi062 ceph-mon[103924]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 784 op/s 2023-11-06T01:43:25.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:25 smithi163 ceph-mon[104416]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 784 op/s 2023-11-06T01:43:26.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:25 smithi053 ceph-mon[99472]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 784 op/s 2023-11-06T01:43:27.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 550.0 s, 1460.6 tps, lat 68.890 ms stddev 75.843 2023-11-06T01:43:27.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:27 smithi062 ceph-mon[103924]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 624 op/s 2023-11-06T01:43:27.992 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:27 smithi163 ceph-mon[104416]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 624 op/s 2023-11-06T01:43:28.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:27 smithi053 ceph-mon[99472]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 624 op/s 2023-11-06T01:43:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.355+0000 7f23d818e700 1 -- 172.21.15.53:0/3953969852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d0104b60 msgr2=0x7f23d010b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.355+0000 7f23d818e700 1 --2- 172.21.15.53:0/3953969852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d0104b60 0x7f23d010b410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f23c800b0a0 tx=0x7f23c800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.356+0000 7f23d818e700 1 -- 172.21.15.53:0/3953969852 shutdown_connections 2023-11-06T01:43:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.356+0000 7f23d818e700 1 --2- 172.21.15.53:0/3953969852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d0104b60 0x7f23d010b410 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.356+0000 7f23d818e700 1 --2- 172.21.15.53:0/3953969852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d0102fd0 0x7f23d01033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.356+0000 7f23d818e700 1 --2- 172.21.15.53:0/3953969852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23d01041c0 0x7f23d0104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.356+0000 7f23d818e700 1 -- 172.21.15.53:0/3953969852 >> 172.21.15.53:0/3953969852 conn(0x7f23d00fe540 msgr2=0x7f23d0100980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.356+0000 7f23d818e700 1 -- 172.21.15.53:0/3953969852 shutdown_connections 2023-11-06T01:43:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.356+0000 7f23d818e700 1 -- 172.21.15.53:0/3953969852 wait complete. 2023-11-06T01:43:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.356+0000 7f23d818e700 1 Processor -- start 2023-11-06T01:43:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d818e700 1 -- start start 2023-11-06T01:43:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d818e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23d0102fd0 0x7f23d0194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d818e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d01041c0 0x7f23d0194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d818e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d0104b60 0x7f23d0195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d818e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d01964a0 con 0x7f23d01041c0 2023-11-06T01:43:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d818e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0196610 con 0x7f23d0102fd0 2023-11-06T01:43:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d818e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d01967d0 con 0x7f23d0104b60 2023-11-06T01:43:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d5729700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d01041c0 0x7f23d0194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d672b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d0104b60 0x7f23d0195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d5729700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d01041c0 0x7f23d0194f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42532/0 (socket says 172.21.15.53:42532) 2023-11-06T01:43:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d5729700 1 -- 172.21.15.53:0/3561063270 learned_addr learned my addr 172.21.15.53:0/3561063270 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23d0102fd0 0x7f23d0194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d672b700 1 -- 172.21.15.53:0/3561063270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23d0102fd0 msgr2=0x7f23d0194a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d672b700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23d0102fd0 0x7f23d0194a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d672b700 1 -- 172.21.15.53:0/3561063270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d01041c0 msgr2=0x7f23d0194f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d672b700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d01041c0 0x7f23d0194f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.357+0000 7f23d672b700 1 -- 172.21.15.53:0/3561063270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c8009d00 con 0x7f23d0104b60 2023-11-06T01:43:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.358+0000 7f23d672b700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d0104b60 0x7f23d0195e40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009fd0 tx=0x7f23c800bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.363+0000 7f23c6ffd700 1 -- 172.21.15.53:0/3561063270 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c800e040 con 0x7f23d0104b60 2023-11-06T01:43:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.363+0000 7f23c6ffd700 1 -- 172.21.15.53:0/3561063270 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23c8004e60 con 0x7f23d0104b60 2023-11-06T01:43:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.363+0000 7f23c6ffd700 1 -- 172.21.15.53:0/3561063270 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c80052a0 con 0x7f23d0104b60 2023-11-06T01:43:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.363+0000 7f23d818e700 1 -- 172.21.15.53:0/3561063270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d0196a50 con 0x7f23d0104b60 2023-11-06T01:43:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.363+0000 7f23d818e700 1 -- 172.21.15.53:0/3561063270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d019b280 con 0x7f23d0104b60 2023-11-06T01:43:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.366+0000 7f23c6ffd700 1 -- 172.21.15.53:0/3561063270 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f23c8005400 con 0x7f23d0104b60 2023-11-06T01:43:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.366+0000 7f23c6ffd700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f23bc0a0670 0x7f23bc0a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.367+0000 7f23c6ffd700 1 -- 172.21.15.53:0/3561063270 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f23c80cce00 con 0x7f23d0104b60 2023-11-06T01:43:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.367+0000 7f23e431eb80 1 Processor -- start 2023-11-06T01:43:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.367+0000 7f23e431eb80 1 -- start start 2023-11-06T01:43:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.367+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732053680 0x55d732053aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.367+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 0x55d7320548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.367+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 0x55d732055350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.367+0000 7f23e431eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d731fab9b0 con 0x55d732054ed0 2023-11-06T01:43:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.367+0000 7f23e431eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d731fa9370 con 0x55d732053680 2023-11-06T01:43:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.367+0000 7f23e431eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d731fa4c60 con 0x55d732053fe0 2023-11-06T01:43:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f23bc0a0670 0x7f23bc0a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d672b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 0x55d732055350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d672b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 0x55d732055350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42542/0 (socket says 172.21.15.53:42542) 2023-11-06T01:43:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d672b700 1 -- 172.21.15.53:0/1126646800 learned_addr learned my addr 172.21.15.53:0/1126646800 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/1126646800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732053680 0x55d732053aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d5729700 1 --2- 172.21.15.53:0/1126646800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 0x55d7320548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d5729700 1 -- 172.21.15.53:0/1126646800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732053680 msgr2=0x55d732053aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d5729700 1 --2- 172.21.15.53:0/1126646800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732053680 0x55d732053aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d5729700 1 -- 172.21.15.53:0/1126646800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 msgr2=0x55d732055350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d5729700 1 --2- 172.21.15.53:0/1126646800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 0x55d732055350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d5729700 1 -- 172.21.15.53:0/1126646800 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d731dd2d80 con 0x55d732053fe0 2023-11-06T01:43:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d672b700 1 --2- 172.21.15.53:0/1126646800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 0x55d732055350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/1126646800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732053680 0x55d732053aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f23bc0a0670 0x7f23bc0a2b30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f23d01947a0 tx=0x7f23c0009ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.368+0000 7f23d5729700 1 --2- 172.21.15.53:0/1126646800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 0x55d7320548f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009d10 tx=0x7f23cc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=847aa5c90e4c1a89 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.369+0000 7f23bbfff700 1 -- 172.21.15.53:0/1126646800 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc004230 con 0x55d732053fe0 2023-11-06T01:43:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.369+0000 7f23bbfff700 1 -- 172.21.15.53:0/1126646800 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23cc004390 con 0x55d732053fe0 2023-11-06T01:43:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.369+0000 7f23bbfff700 1 -- 172.21.15.53:0/1126646800 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0196d0 con 0x55d732053fe0 2023-11-06T01:43:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.369+0000 7f23e431eb80 1 -- 172.21.15.53:0/1126646800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 msgr2=0x55d7320548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.369+0000 7f23e431eb80 1 --2- 172.21.15.53:0/1126646800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 0x55d7320548f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009d10 tx=0x7f23cc011720 comp rx=0 tx=0).stop 2023-11-06T01:43:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.369+0000 7f23e431eb80 1 -- 172.21.15.53:0/1126646800 shutdown_connections 2023-11-06T01:43:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.369+0000 7f23e431eb80 1 --2- 172.21.15.53:0/1126646800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732053680 0x55d732053aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.369+0000 7f23e431eb80 1 --2- 172.21.15.53:0/1126646800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 0x55d732055350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.369+0000 7f23e431eb80 1 --2- 172.21.15.53:0/1126646800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 0x55d7320548f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.369+0000 7f23e431eb80 1 -- 172.21.15.53:0/1126646800 >> 172.21.15.53:0/1126646800 conn(0x55d731fb3070 msgr2=0x55d732050040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.370+0000 7f23e431eb80 1 -- 172.21.15.53:0/1126646800 shutdown_connections 2023-11-06T01:43:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.370+0000 7f23e431eb80 1 -- 172.21.15.53:0/1126646800 wait complete. 2023-11-06T01:43:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.370+0000 7f23e431eb80 1 -- start start 2023-11-06T01:43:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.370+0000 7f23e431eb80 1 Processor -- start 2023-11-06T01:43:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d73205d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053fe0 0x55d73205d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732054ed0 0x55d7320649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23e431eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d731f414b0 con 0x55d732053fe0 2023-11-06T01:43:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23e431eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d731f41150 con 0x55d732054ed0 2023-11-06T01:43:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23e431eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d731e288c0 con 0x55d732053680 2023-11-06T01:43:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23e431eb80 10 client.0 _ll_register_callbacks cb 0x55d731e8ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23d5f2a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d73205d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23d5f2a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d73205d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56786/0 (socket says 172.21.15.53:56786) 2023-11-06T01:43:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23d5f2a700 1 -- 172.21.15.53:0/1538713729 learned_addr learned my addr 172.21.15.53:0/1538713729 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23d5729700 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053fe0 0x55d73205d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23d672b700 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732054ed0 0x55d7320649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23d5f2a700 1 -- 172.21.15.53:0/1538713729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732054ed0 msgr2=0x55d7320649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732054ed0 0x55d7320649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23d5f2a700 1 -- 172.21.15.53:0/1538713729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053fe0 msgr2=0x55d73205d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053fe0 0x55d73205d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23d5f2a700 1 -- 172.21.15.53:0/1538713729 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c001a040 con 0x55d732053680 2023-11-06T01:43:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d73205d420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f23c0012300 tx=0x7f23c0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8090fa3859f54b9b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.371+0000 7f23b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:43:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.372+0000 7f23b97fa700 1 -- 172.21.15.53:0/1538713729 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0011440 con 0x55d732053680 2023-11-06T01:43:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.372+0000 7f23b97fa700 1 -- 172.21.15.53:0/1538713729 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f23c00115a0 con 0x55d732053680 2023-11-06T01:43:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.372+0000 7f23e431eb80 1 -- 172.21.15.53:0/1538713729 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d731fcfcb0 con 0x55d732053680 2023-11-06T01:43:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.372+0000 7f23b97fa700 1 -- 172.21.15.53:0/1538713729 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23c0011760 con 0x55d732053680 2023-11-06T01:43:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.372+0000 7f23e431eb80 10 client.26122 fetch_fsmap learned FSMap version 44 2023-11-06T01:43:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.372+0000 7f23e431eb80 1 -- 172.21.15.53:0/1538713729 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f23d019b280 con 0x55d732053680 2023-11-06T01:43:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.374+0000 7f23b97fa700 1 -- 172.21.15.53:0/1538713729 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d731fcfcb0 con 0x55d732053680 2023-11-06T01:43:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.374+0000 7f23b97fa700 1 -- 172.21.15.53:0/1538713729 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f23c0031020 con 0x55d732053680 2023-11-06T01:43:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.374+0000 7f23e431eb80 10 client.26122 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:43:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.374+0000 7f23e431eb80 10 client.26122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.374+0000 7f23e431eb80 20 client.26122 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.374+0000 7f23e431eb80 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7320691d0 0x55d73206b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.374+0000 7f23e431eb80 4 client.26122 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:43:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.374+0000 7f23e431eb80 1 -- 172.21.15.53:0/1538713729 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d732022db0 con 0x55d7320691d0 2023-11-06T01:43:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.374+0000 7f23d5729700 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7320691d0 0x55d73206b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.374+0000 7f23b97fa700 10 client.26122 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.374+0000 7f23d5729700 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7320691d0 0x55d73206b5f0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e62a68c43c0781f server_cookie=1886d4ab15cd553d in_seq=0 out_seq=0 2023-11-06T01:43:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23b97fa700 1 -- 172.21.15.53:0/1538713729 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d732022db0 con 0x55d7320691d0 2023-11-06T01:43:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23b97fa700 10 client.26122 handle_command_reply: tid=0 2023-11-06T01:43:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 1 client.26122 shutdown 2023-11-06T01:43:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 1 -- 172.21.15.53:0/1538713729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 msgr2=0x55d73205d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d73205d420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f23c0012300 tx=0x7f23c0009260 comp rx=0 tx=0).stop 2023-11-06T01:43:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 1 -- 172.21.15.53:0/1538713729 shutdown_connections 2023-11-06T01:43:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732054ed0 0x55d7320649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7320691d0 0x55d73206b5f0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d73205d420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 1 --2- 172.21.15.53:0/1538713729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053fe0 0x55d73205d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23b97fa700 0 client.26122 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 1 -- 172.21.15.53:0/1538713729 >> 172.21.15.53:0/1538713729 conn(0x55d731fb3070 msgr2=0x55d73204e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 1 -- 172.21.15.53:0/1538713729 shutdown_connections 2023-11-06T01:43:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 1 -- 172.21.15.53:0/1538713729 wait complete. 2023-11-06T01:43:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.375+0000 7f23e431eb80 20 client.26122 trim_cache size 0 max 16384 2023-11-06T01:43:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23e431eb80 1 Processor -- start 2023-11-06T01:43:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23e431eb80 1 -- start start 2023-11-06T01:43:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053680 0x55d732062950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 0x55d7320580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732054ed0 0x55d73204fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23e431eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73205cf90 con 0x55d732053680 2023-11-06T01:43:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23e431eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73205e5a0 con 0x55d732054ed0 2023-11-06T01:43:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23e431eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73204f820 con 0x55d732053fe0 2023-11-06T01:43:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23d5f2a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053680 0x55d732062950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23d5f2a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053680 0x55d732062950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42554/0 (socket says 172.21.15.53:42554) 2023-11-06T01:43:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23d5f2a700 1 -- 172.21.15.53:0/834279217 learned_addr learned my addr 172.21.15.53:0/834279217 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23d5729700 1 --2- 172.21.15.53:0/834279217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 0x55d7320580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.391+0000 7f23d672b700 1 --2- 172.21.15.53:0/834279217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732054ed0 0x55d73204fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.392+0000 7f23d5729700 1 -- 172.21.15.53:0/834279217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732054ed0 msgr2=0x55d73204fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.392+0000 7f23d5729700 1 --2- 172.21.15.53:0/834279217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732054ed0 0x55d73204fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.392+0000 7f23d5729700 1 -- 172.21.15.53:0/834279217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053680 msgr2=0x55d732062950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.392+0000 7f23d5729700 1 --2- 172.21.15.53:0/834279217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053680 0x55d732062950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.392+0000 7f23d5729700 1 -- 172.21.15.53:0/834279217 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc009970 con 0x55d732053fe0 2023-11-06T01:43:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.392+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/834279217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053680 0x55d732062950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.392+0000 7f23d672b700 1 --2- 172.21.15.53:0/834279217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732054ed0 0x55d73204fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:43:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.392+0000 7f23d5729700 1 --2- 172.21.15.53:0/834279217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 0x55d7320580a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f23cc000c00 tx=0x7f23cc009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8f6d04063ec92c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.392+0000 7f23c4ff9700 1 -- 172.21.15.53:0/834279217 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0040f0 con 0x55d732053fe0 2023-11-06T01:43:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.392+0000 7f23c4ff9700 1 -- 172.21.15.53:0/834279217 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23cc005510 con 0x55d732053fe0 2023-11-06T01:43:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.392+0000 7f23c4ff9700 1 -- 172.21.15.53:0/834279217 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc003d60 con 0x55d732053fe0 2023-11-06T01:43:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.393+0000 7f23e431eb80 1 -- 172.21.15.53:0/834279217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 msgr2=0x55d7320580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.393+0000 7f23e431eb80 1 --2- 172.21.15.53:0/834279217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 0x55d7320580a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f23cc000c00 tx=0x7f23cc009f40 comp rx=0 tx=0).stop 2023-11-06T01:43:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.393+0000 7f23e431eb80 1 -- 172.21.15.53:0/834279217 shutdown_connections 2023-11-06T01:43:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.393+0000 7f23e431eb80 1 --2- 172.21.15.53:0/834279217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732054ed0 0x55d73204fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.393+0000 7f23e431eb80 1 --2- 172.21.15.53:0/834279217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732053680 0x55d732062950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.393+0000 7f23e431eb80 1 --2- 172.21.15.53:0/834279217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053fe0 0x55d7320580a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.393+0000 7f23e431eb80 1 -- 172.21.15.53:0/834279217 >> 172.21.15.53:0/834279217 conn(0x55d731fb3070 msgr2=0x55d73205c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.393+0000 7f23e431eb80 1 -- 172.21.15.53:0/834279217 shutdown_connections 2023-11-06T01:43:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.393+0000 7f23e431eb80 1 -- 172.21.15.53:0/834279217 wait complete. 2023-11-06T01:43:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.393+0000 7f23e431eb80 1 -- start start 2023-11-06T01:43:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.393+0000 7f23e431eb80 1 Processor -- start 2023-11-06T01:43:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d7320b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732053fe0 0x55d7320b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23e431eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 0x55d7320ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23e431eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7320ba700 con 0x55d732054ed0 2023-11-06T01:43:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23e431eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7320ba870 con 0x55d732053fe0 2023-11-06T01:43:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23e431eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7320baa30 con 0x55d732053680 2023-11-06T01:43:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23e431eb80 10 client.0 _ll_register_callbacks cb 0x55d731e8ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23d5f2a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d7320b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23d5f2a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d7320b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56800/0 (socket says 172.21.15.53:56800) 2023-11-06T01:43:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23d5f2a700 1 -- 172.21.15.53:0/2208199696 learned_addr learned my addr 172.21.15.53:0/2208199696 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23d672b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 0x55d7320ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23d5729700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732053fe0 0x55d7320b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23d5f2a700 1 -- 172.21.15.53:0/2208199696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732053fe0 msgr2=0x55d7320b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/2208199696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732053fe0 0x55d7320b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23d5f2a700 1 -- 172.21.15.53:0/2208199696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 msgr2=0x55d7320ba1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/2208199696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 0x55d7320ba1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23d5f2a700 1 -- 172.21.15.53:0/2208199696 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d019b280 con 0x55d732053680 2023-11-06T01:43:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23d5f2a700 1 --2- 172.21.15.53:0/2208199696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d7320b2f10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f23c0029a10 tx=0x7f23c0010f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eb294e3c1d11da4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:43:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.394+0000 7f23b97fa700 1 -- 172.21.15.53:0/2208199696 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c00122d0 con 0x55d732053680 2023-11-06T01:43:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23b97fa700 1 -- 172.21.15.53:0/2208199696 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f23c0029430 con 0x55d732053680 2023-11-06T01:43:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23e431eb80 1 -- 172.21.15.53:0/2208199696 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23cc004390 con 0x55d732053680 2023-11-06T01:43:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23b97fa700 1 -- 172.21.15.53:0/2208199696 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23c00295f0 con 0x55d732053680 2023-11-06T01:43:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23e431eb80 10 client.26128 fetch_fsmap learned FSMap version 44 2023-11-06T01:43:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23e431eb80 1 -- 172.21.15.53:0/2208199696 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f23cc01e040 con 0x55d732053680 2023-11-06T01:43:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23b97fa700 1 -- 172.21.15.53:0/2208199696 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23cc004390 con 0x55d732053680 2023-11-06T01:43:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23b97fa700 1 -- 172.21.15.53:0/2208199696 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f23c0046050 con 0x55d732053680 2023-11-06T01:43:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23e431eb80 10 client.26128 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:43:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23e431eb80 10 client.26128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23e431eb80 20 client.26128 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23e431eb80 1 --2- 172.21.15.53:0/2208199696 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7320bc260 0x55d7320be680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23e431eb80 4 client.26128 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:43:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23e431eb80 1 -- 172.21.15.53:0/2208199696 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d731f9dce0 con 0x55d7320bc260 2023-11-06T01:43:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.395+0000 7f23d5729700 1 --2- 172.21.15.53:0/2208199696 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7320bc260 0x55d7320be680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.396+0000 7f23b97fa700 10 client.26128 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.396+0000 7f23d5729700 1 --2- 172.21.15.53:0/2208199696 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7320bc260 0x55d7320be680 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87cfc246927a92bb server_cookie=8a4e50be8df72bfa in_seq=0 out_seq=0 2023-11-06T01:43:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23b97fa700 1 -- 172.21.15.53:0/2208199696 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d731f9dce0 con 0x55d7320bc260 2023-11-06T01:43:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23b97fa700 10 client.26128 handle_command_reply: tid=0 2023-11-06T01:43:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23e431eb80 1 client.26128 shutdown 2023-11-06T01:43:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23e431eb80 1 -- 172.21.15.53:0/2208199696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 msgr2=0x55d7320b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23e431eb80 1 --2- 172.21.15.53:0/2208199696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d7320b2f10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f23c0029a10 tx=0x7f23c0010f70 comp rx=0 tx=0).stop 2023-11-06T01:43:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23e431eb80 1 -- 172.21.15.53:0/2208199696 shutdown_connections 2023-11-06T01:43:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23e431eb80 1 --2- 172.21.15.53:0/2208199696 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7320bc260 0x55d7320be680 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23e431eb80 1 --2- 172.21.15.53:0/2208199696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d732054ed0 0x55d7320ba1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23e431eb80 1 --2- 172.21.15.53:0/2208199696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d732053680 0x55d7320b2f10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23b97fa700 0 client.26128 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23e431eb80 1 --2- 172.21.15.53:0/2208199696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d732053fe0 0x55d7320b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23e431eb80 1 -- 172.21.15.53:0/2208199696 >> 172.21.15.53:0/2208199696 conn(0x55d731fb3070 msgr2=0x55d7320580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23e431eb80 1 -- 172.21.15.53:0/2208199696 shutdown_connections 2023-11-06T01:43:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.428+0000 7f23e431eb80 1 -- 172.21.15.53:0/2208199696 wait complete. 2023-11-06T01:43:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23e431eb80 20 client.26128 trim_cache size 0 max 16384 2023-11-06T01:43:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:43:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:43:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:43:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:43:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 -- 172.21.15.53:0/3561063270 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f23bc0a0670 msgr2=0x7f23bc0a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f23bc0a0670 0x7f23bc0a2b30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f23d01947a0 tx=0x7f23c0009ae0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 -- 172.21.15.53:0/3561063270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d0104b60 msgr2=0x7f23d0195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d0104b60 0x7f23d0195e40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009fd0 tx=0x7f23c800bce0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 -- 172.21.15.53:0/3561063270 shutdown_connections 2023-11-06T01:43:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f23bc0a0670 0x7f23bc0a2b30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23d0102fd0 0x7f23d0194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d0104b60 0x7f23d0195e40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 --2- 172.21.15.53:0/3561063270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d01041c0 0x7f23d0194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 -- 172.21.15.53:0/3561063270 >> 172.21.15.53:0/3561063270 conn(0x7f23d00fe540 msgr2=0x7f23d00771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 -- 172.21.15.53:0/3561063270 shutdown_connections 2023-11-06T01:43:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:29.429+0000 7f23d818e700 1 -- 172.21.15.53:0/3561063270 wait complete. 2023-11-06T01:43:29.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:29 smithi163 ceph-mon[104416]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 688 op/s 2023-11-06T01:43:30.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:29 smithi053 ceph-mon[99472]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 688 op/s 2023-11-06T01:43:30.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:29 smithi062 ceph-mon[103924]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 688 op/s 2023-11-06T01:43:31.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:31 smithi163 ceph-mon[104416]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 869 op/s 2023-11-06T01:43:32.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:31 smithi053 ceph-mon[99472]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 869 op/s 2023-11-06T01:43:32.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:31 smithi062 ceph-mon[103924]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 869 op/s 2023-11-06T01:43:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 555.0 s, 1300.0 tps, lat 76.927 ms stddev 141.095 2023-11-06T01:43:33.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:33 smithi163 ceph-mon[104416]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.21k op/s 2023-11-06T01:43:34.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:33 smithi053 ceph-mon[99472]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.21k op/s 2023-11-06T01:43:34.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:33 smithi062 ceph-mon[103924]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.21k op/s 2023-11-06T01:43:35.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:35 smithi163 ceph-mon[104416]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.94k op/s 2023-11-06T01:43:36.095 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:35 smithi053 ceph-mon[99472]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.94k op/s 2023-11-06T01:43:36.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:35 smithi062 ceph-mon[103924]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.94k op/s 2023-11-06T01:43:37.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 560.0 s, 1766.6 tps, lat 56.454 ms stddev 63.812 2023-11-06T01:43:37.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:37 smithi163 ceph-mon[104416]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 1.91k op/s 2023-11-06T01:43:38.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:37 smithi053 ceph-mon[99472]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 1.91k op/s 2023-11-06T01:43:38.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:37 smithi062 ceph-mon[103924]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 1.91k op/s 2023-11-06T01:43:39.017 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:43:39.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.200+0000 7f4b687cb700 1 -- 172.21.15.53:0/3973863885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b60104950 msgr2=0x7f4b60104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:39.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.200+0000 7f4b687cb700 1 --2- 172.21.15.53:0/3973863885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b60104950 0x7f4b60104dd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800b0a0 tx=0x7f4b5800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.200+0000 7f4b687cb700 1 -- 172.21.15.53:0/3973863885 shutdown_connections 2023-11-06T01:43:39.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.200+0000 7f4b687cb700 1 --2- 172.21.15.53:0/3973863885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b60104950 0x7f4b60104dd0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.200+0000 7f4b687cb700 1 --2- 172.21.15.53:0/3973863885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b600ff610 0x7f4b60101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.200+0000 7f4b687cb700 1 --2- 172.21.15.53:0/3973863885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b60101f80 0x7f4b60104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.200+0000 7f4b687cb700 1 -- 172.21.15.53:0/3973863885 >> 172.21.15.53:0/3973863885 conn(0x7f4b600fb1f0 msgr2=0x7f4b600fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:39.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.201+0000 7f4b687cb700 1 -- 172.21.15.53:0/3973863885 shutdown_connections 2023-11-06T01:43:39.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.201+0000 7f4b687cb700 1 -- 172.21.15.53:0/3973863885 wait complete. 2023-11-06T01:43:39.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.201+0000 7f4b687cb700 1 Processor -- start 2023-11-06T01:43:39.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b687cb700 1 -- start start 2023-11-06T01:43:39.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b687cb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b600ff610 0x7f4b6019d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b687cb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b60101f80 0x7f4b6019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b687cb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b60104950 0x7f4b601a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b687cb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601a4d00 con 0x7f4b60104950 2023-11-06T01:43:39.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b687cb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601a4e40 con 0x7f4b60101f80 2023-11-06T01:43:39.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b687cb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601a4fd0 con 0x7f4b600ff610 2023-11-06T01:43:39.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b66567700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b600ff610 0x7f4b6019d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b66d68700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b60104950 0x7f4b601a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b66567700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b600ff610 0x7f4b6019d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56824/0 (socket says 172.21.15.53:56824) 2023-11-06T01:43:39.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b66567700 1 -- 172.21.15.53:0/4022178736 learned_addr learned my addr 172.21.15.53:0/4022178736 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:39.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b65d66700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b60101f80 0x7f4b6019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b66567700 1 -- 172.21.15.53:0/4022178736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b60101f80 msgr2=0x7f4b6019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:39.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b66567700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b60101f80 0x7f4b6019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b66567700 1 -- 172.21.15.53:0/4022178736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b60104950 msgr2=0x7f4b601a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:39.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b66567700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b60104950 0x7f4b601a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.202+0000 7f4b66567700 1 -- 172.21.15.53:0/4022178736 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50009520 con 0x7f4b600ff610 2023-11-06T01:43:39.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.203+0000 7f4b66567700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b600ff610 0x7f4b6019d180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00d950 tx=0x7f4b5c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:39.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.203+0000 7f4b577fe700 1 -- 172.21.15.53:0/4022178736 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00f890 con 0x7f4b600ff610 2023-11-06T01:43:39.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.203+0000 7f4b687cb700 1 -- 172.21.15.53:0/4022178736 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b58009d00 con 0x7f4b600ff610 2023-11-06T01:43:39.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.203+0000 7f4b577fe700 1 -- 172.21.15.53:0/4022178736 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5c010460 con 0x7f4b600ff610 2023-11-06T01:43:39.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.203+0000 7f4b577fe700 1 -- 172.21.15.53:0/4022178736 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00e610 con 0x7f4b600ff610 2023-11-06T01:43:39.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.203+0000 7f4b687cb700 1 -- 172.21.15.53:0/4022178736 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b601a5680 con 0x7f4b600ff610 2023-11-06T01:43:39.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.205+0000 7f4b577fe700 1 -- 172.21.15.53:0/4022178736 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4b5c00f9f0 con 0x7f4b600ff610 2023-11-06T01:43:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.206+0000 7f4b577fe700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b4c0a0520 0x7f4b4c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.206+0000 7f4b577fe700 1 -- 172.21.15.53:0/4022178736 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4b5c0cbb30 con 0x7f4b600ff610 2023-11-06T01:43:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.207+0000 7f4b65d66700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b4c0a0520 0x7f4b4c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.207+0000 7f4b7495bb80 1 Processor -- start 2023-11-06T01:43:39.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.207+0000 7f4b7495bb80 1 -- start start 2023-11-06T01:43:39.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.207+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.207+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d1a70 0x5585515cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.207+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.207+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558551527a10 con 0x5585515d03f0 2023-11-06T01:43:39.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.207+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585515253d0 con 0x5585515d10f0 2023-11-06T01:43:39.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.207+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558551520cc0 con 0x5585515d1a70 2023-11-06T01:43:39.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.207+0000 7f4b65d66700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b4c0a0520 0x7f4b4c0a29e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009df0 tx=0x7f4b50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:39.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b66d68700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b66567700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b66d68700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42594/0 (socket says 172.21.15.53:42594) 2023-11-06T01:43:39.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b66567700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54030/0 (socket says 172.21.15.53:54030) 2023-11-06T01:43:39.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b66d68700 1 -- 172.21.15.53:0/303051729 learned_addr learned my addr 172.21.15.53:0/303051729 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:39.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b65d66700 1 --2- 172.21.15.53:0/303051729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d1a70 0x5585515cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b66567700 1 -- 172.21.15.53:0/303051729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d1a70 msgr2=0x5585515cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:39.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b66567700 1 --2- 172.21.15.53:0/303051729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d1a70 0x5585515cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b66567700 1 -- 172.21.15.53:0/303051729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 msgr2=0x5585515d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:39.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b66567700 1 --2- 172.21.15.53:0/303051729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b66567700 1 -- 172.21.15.53:0/303051729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55855134ede0 con 0x5585515d10f0 2023-11-06T01:43:39.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b66567700 1 --2- 172.21.15.53:0/303051729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d1530 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0c95b0 tx=0x7f4b5c099a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d1ef18fa08f97af server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:39.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b54ff9700 1 -- 172.21.15.53:0/303051729 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c09d140 con 0x5585515d10f0 2023-11-06T01:43:39.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b54ff9700 1 -- 172.21.15.53:0/303051729 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5c0a1380 con 0x5585515d10f0 2023-11-06T01:43:39.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.208+0000 7f4b54ff9700 1 -- 172.21.15.53:0/303051729 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00fca0 con 0x5585515d10f0 2023-11-06T01:43:39.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.209+0000 7f4b7495bb80 1 -- 172.21.15.53:0/303051729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 msgr2=0x5585515d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:39.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.209+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/303051729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d1530 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0c95b0 tx=0x7f4b5c099a90 comp rx=0 tx=0).stop 2023-11-06T01:43:39.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.209+0000 7f4b7495bb80 1 -- 172.21.15.53:0/303051729 shutdown_connections 2023-11-06T01:43:39.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.209+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/303051729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d1530 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.209+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/303051729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.209+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/303051729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d1a70 0x5585515cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.209+0000 7f4b7495bb80 1 -- 172.21.15.53:0/303051729 >> 172.21.15.53:0/303051729 conn(0x55855152f0d0 msgr2=0x5585515ca940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:39.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.209+0000 7f4b7495bb80 1 -- 172.21.15.53:0/303051729 shutdown_connections 2023-11-06T01:43:39.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.209+0000 7f4b7495bb80 1 -- 172.21.15.53:0/303051729 wait complete. 2023-11-06T01:43:39.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b7495bb80 1 -- start start 2023-11-06T01:43:39.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b7495bb80 1 Processor -- start 2023-11-06T01:43:39.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515d9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d1a70 0x5585515e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585514bd510 con 0x5585515d03f0 2023-11-06T01:43:39.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585514bd1b0 con 0x5585515d10f0 2023-11-06T01:43:39.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585513a4920 con 0x5585515d1a70 2023-11-06T01:43:39.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b7495bb80 10 client.0 _ll_register_callbacks cb 0x55855140aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:39.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b66567700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515d9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b65d66700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b65d66700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54040/0 (socket says 172.21.15.53:54040) 2023-11-06T01:43:39.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b65d66700 1 -- 172.21.15.53:0/3911848169 learned_addr learned my addr 172.21.15.53:0/3911848169 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:39.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.210+0000 7f4b66d68700 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d1a70 0x5585515e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b65d66700 1 -- 172.21.15.53:0/3911848169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d1a70 msgr2=0x5585515e0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:39.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b65d66700 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d1a70 0x5585515e0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b65d66700 1 -- 172.21.15.53:0/3911848169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 msgr2=0x5585515d9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:39.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b65d66700 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515d9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b65d66700 1 -- 172.21.15.53:0/3911848169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b601a5680 con 0x5585515d10f0 2023-11-06T01:43:39.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b65d66700 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d9e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000ecf0 tx=0x7f4b500047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a56de6d48ee62dce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:39.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:43:39.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b49ffb700 1 -- 172.21.15.53:0/3911848169 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50022070 con 0x5585515d10f0 2023-11-06T01:43:39.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b7495bb80 1 -- 172.21.15.53:0/3911848169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55855154bd10 con 0x5585515d10f0 2023-11-06T01:43:39.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b49ffb700 1 -- 172.21.15.53:0/3911848169 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4b5001dbb0 con 0x5585515d10f0 2023-11-06T01:43:39.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b49ffb700 1 -- 172.21.15.53:0/3911848169 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b5001ddb0 con 0x5585515d10f0 2023-11-06T01:43:39.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b7495bb80 10 client.26066 fetch_fsmap learned FSMap version 44 2023-11-06T01:43:39.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.211+0000 7f4b7495bb80 1 -- 172.21.15.53:0/3911848169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x558551337c60 con 0x5585515d10f0 2023-11-06T01:43:39.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.212+0000 7f4b49ffb700 1 -- 172.21.15.53:0/3911848169 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55855154bd10 con 0x5585515d10f0 2023-11-06T01:43:39.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.212+0000 7f4b49ffb700 1 -- 172.21.15.53:0/3911848169 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f4b50003c50 con 0x5585515d10f0 2023-11-06T01:43:39.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.212+0000 7f4b7495bb80 10 client.26066 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:43:39.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.212+0000 7f4b7495bb80 10 client.26066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:39.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.212+0000 7f4b7495bb80 20 client.26066 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:39.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.212+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5585515e56a0 0x5585515e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.212+0000 7f4b7495bb80 4 client.26066 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:43:39.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.212+0000 7f4b7495bb80 1 -- 172.21.15.53:0/3911848169 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55855159ee10 con 0x5585515e56a0 2023-11-06T01:43:39.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.212+0000 7f4b66567700 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5585515e56a0 0x5585515e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.214+0000 7f4b49ffb700 10 client.26066 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:39.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.214+0000 7f4b66567700 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5585515e56a0 0x5585515e7ac0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2e86c4c86b8d9f server_cookie=bdd3ffb9db26e9ac in_seq=0 out_seq=0 2023-11-06T01:43:39.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b49ffb700 1 -- 172.21.15.53:0/3911848169 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55855159ee10 con 0x5585515e56a0 2023-11-06T01:43:39.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b49ffb700 10 client.26066 handle_command_reply: tid=0 2023-11-06T01:43:39.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 1 client.26066 shutdown 2023-11-06T01:43:39.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 1 -- 172.21.15.53:0/3911848169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 msgr2=0x5585515d9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:39.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d9e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000ecf0 tx=0x7f4b500047c0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 1 -- 172.21.15.53:0/3911848169 shutdown_connections 2023-11-06T01:43:39.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5585515e56a0 0x5585515e7ac0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d1a70 0x5585515e0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515d9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/3911848169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d10f0 0x5585515d9e60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b49ffb700 0 client.26066 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:39.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 1 -- 172.21.15.53:0/3911848169 >> 172.21.15.53:0/3911848169 conn(0x55855152f0d0 msgr2=0x5585515ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:39.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 1 -- 172.21.15.53:0/3911848169 shutdown_connections 2023-11-06T01:43:39.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 1 -- 172.21.15.53:0/3911848169 wait complete. 2023-11-06T01:43:39.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.215+0000 7f4b7495bb80 20 client.26066 trim_cache size 0 max 16384 2023-11-06T01:43:39.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b7495bb80 1 Processor -- start 2023-11-06T01:43:39.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b7495bb80 1 -- start start 2023-11-06T01:43:39.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515dee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d10f0 0x5585515d8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d1a70 0x5585515e1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855154bd10 con 0x5585515d03f0 2023-11-06T01:43:39.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585515dab10 con 0x5585515d1a70 2023-11-06T01:43:39.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585515da870 con 0x5585515d10f0 2023-11-06T01:43:39.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b66567700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515dee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b66567700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515dee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42608/0 (socket says 172.21.15.53:42608) 2023-11-06T01:43:39.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b66567700 1 -- 172.21.15.53:0/1200964278 learned_addr learned my addr 172.21.15.53:0/1200964278 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:39.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b66d68700 1 --2- 172.21.15.53:0/1200964278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d1a70 0x5585515e1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b66567700 1 -- 172.21.15.53:0/1200964278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d10f0 msgr2=0x5585515d8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:39.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b66567700 1 --2- 172.21.15.53:0/1200964278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d10f0 0x5585515d8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.237+0000 7f4b66567700 1 -- 172.21.15.53:0/1200964278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d1a70 msgr2=0x5585515e1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:39.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.238+0000 7f4b66567700 1 --2- 172.21.15.53:0/1200964278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d1a70 0x5585515e1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.238+0000 7f4b66567700 1 -- 172.21.15.53:0/1200964278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558551375250 con 0x5585515d03f0 2023-11-06T01:43:39.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.238+0000 7f4b66567700 1 --2- 172.21.15.53:0/1200964278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515dee50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c010fd0 tx=0x7f4b5c095050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896d1e2e0ffe9bce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:39.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.238+0000 7f4b557fa700 1 -- 172.21.15.53:0/1200964278 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c095ba0 con 0x5585515d03f0 2023-11-06T01:43:39.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.238+0000 7f4b557fa700 1 -- 172.21.15.53:0/1200964278 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5c0976b0 con 0x5585515d03f0 2023-11-06T01:43:39.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.238+0000 7f4b557fa700 1 -- 172.21.15.53:0/1200964278 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00ff00 con 0x5585515d03f0 2023-11-06T01:43:39.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 -- 172.21.15.53:0/1200964278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 msgr2=0x5585515dee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:39.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/1200964278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515dee50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c010fd0 tx=0x7f4b5c095050 comp rx=0 tx=0).stop 2023-11-06T01:43:39.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 -- 172.21.15.53:0/1200964278 shutdown_connections 2023-11-06T01:43:39.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/1200964278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d1a70 0x5585515e1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/1200964278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d03f0 0x5585515dee50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/1200964278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d10f0 0x5585515d8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 -- 172.21.15.53:0/1200964278 >> 172.21.15.53:0/1200964278 conn(0x55855152f0d0 msgr2=0x5585515ca910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:39.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 -- 172.21.15.53:0/1200964278 shutdown_connections 2023-11-06T01:43:39.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 -- 172.21.15.53:0/1200964278 wait complete. 2023-11-06T01:43:39.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 -- start start 2023-11-06T01:43:39.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 Processor -- start 2023-11-06T01:43:39.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d03f0 0x558551631370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.239+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d10f0 0x5585516338c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b7495bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d1a70 0x558551635f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558551636640 con 0x5585515d1a70 2023-11-06T01:43:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585516367b0 con 0x5585515d03f0 2023-11-06T01:43:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b7495bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558551636970 con 0x5585515d10f0 2023-11-06T01:43:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b7495bb80 10 client.0 _ll_register_callbacks cb 0x7f4b5000e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:39.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b66567700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d03f0 0x558551631370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b66567700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d03f0 0x558551631370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54052/0 (socket says 172.21.15.53:54052) 2023-11-06T01:43:39.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b65d66700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d10f0 0x5585516338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b66567700 1 -- 172.21.15.53:0/4061728528 learned_addr learned my addr 172.21.15.53:0/4061728528 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:39.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b66d68700 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d1a70 0x558551635f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b65d66700 1 -- 172.21.15.53:0/4061728528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d03f0 msgr2=0x558551631370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b65d66700 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d03f0 0x558551631370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b65d66700 1 -- 172.21.15.53:0/4061728528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d1a70 msgr2=0x558551635f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:39.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b65d66700 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d1a70 0x558551635f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b65d66700 1 -- 172.21.15.53:0/4061728528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55855134ede0 con 0x5585515d10f0 2023-11-06T01:43:39.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b65d66700 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d10f0 0x5585516338c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b50003fd0 tx=0x7f4b50027b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb2101eadb899240 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:39.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:43:39.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b49ffb700 1 -- 172.21.15.53:0/4061728528 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50022070 con 0x5585515d10f0 2023-11-06T01:43:39.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b49ffb700 1 -- 172.21.15.53:0/4061728528 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4b500092d0 con 0x5585515d10f0 2023-11-06T01:43:39.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.240+0000 7f4b7495bb80 1 -- 172.21.15.53:0/4061728528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b5c0a1380 con 0x5585515d10f0 2023-11-06T01:43:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b49ffb700 1 -- 172.21.15.53:0/4061728528 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b50009b60 con 0x5585515d10f0 2023-11-06T01:43:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b7495bb80 10 client.26143 fetch_fsmap learned FSMap version 44 2023-11-06T01:43:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b7495bb80 1 -- 172.21.15.53:0/4061728528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55855134d2b0 con 0x5585515d10f0 2023-11-06T01:43:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b49ffb700 1 -- 172.21.15.53:0/4061728528 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5c0a1380 con 0x5585515d10f0 2023-11-06T01:43:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b49ffb700 1 -- 172.21.15.53:0/4061728528 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f4b500037a0 con 0x5585515d10f0 2023-11-06T01:43:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b7495bb80 10 client.26143 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:43:39.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b7495bb80 10 client.26143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:39.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b7495bb80 20 client.26143 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:39.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558551638340 0x55855163a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:39.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b7495bb80 4 client.26143 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:43:39.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b7495bb80 1 -- 172.21.15.53:0/4061728528 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558551519d40 con 0x558551638340 2023-11-06T01:43:39.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.241+0000 7f4b66567700 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558551638340 0x55855163a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:39.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.242+0000 7f4b66567700 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558551638340 0x55855163a760 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=635f84869a7267b server_cookie=1f90ba702f416f3 in_seq=0 out_seq=0 2023-11-06T01:43:39.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.242+0000 7f4b49ffb700 10 client.26143 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:39.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.242+0000 7f4b49ffb700 1 -- 172.21.15.53:0/4061728528 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558551519d40 con 0x558551638340 2023-11-06T01:43:39.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.242+0000 7f4b49ffb700 10 client.26143 handle_command_reply: tid=0 2023-11-06T01:43:39.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.242+0000 7f4b7495bb80 1 client.26143 shutdown 2023-11-06T01:43:39.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b7495bb80 1 -- 172.21.15.53:0/4061728528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d10f0 msgr2=0x5585516338c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:39.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d10f0 0x5585516338c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b50003fd0 tx=0x7f4b50027b70 comp rx=0 tx=0).stop 2023-11-06T01:43:39.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b7495bb80 1 -- 172.21.15.53:0/4061728528 shutdown_connections 2023-11-06T01:43:39.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585515d03f0 0x558551631370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558551638340 0x55855163a760 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585515d1a70 0x558551635f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b7495bb80 1 --2- 172.21.15.53:0/4061728528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585515d10f0 0x5585516338c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b7495bb80 1 -- 172.21.15.53:0/4061728528 >> 172.21.15.53:0/4061728528 conn(0x55855152f0d0 msgr2=0x5585515d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:39.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b49ffb700 0 client.26143 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:39.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b7495bb80 1 -- 172.21.15.53:0/4061728528 shutdown_connections 2023-11-06T01:43:39.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b7495bb80 1 -- 172.21.15.53:0/4061728528 wait complete. 2023-11-06T01:43:39.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.243+0000 7f4b7495bb80 20 client.26143 trim_cache size 0 max 16384 2023-11-06T01:43:39.273 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:43:39.274 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:43:39.274 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:43:39.275 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:43:39.275 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:43:39.275 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:43:39.276 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:43:39.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:43:39.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:43:39.276 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:43:39.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 -- 172.21.15.53:0/4022178736 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b4c0a0520 msgr2=0x7f4b4c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:39.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b4c0a0520 0x7f4b4c0a29e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009df0 tx=0x7f4b50009260 comp rx=0 tx=0).stop 2023-11-06T01:43:39.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 -- 172.21.15.53:0/4022178736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b600ff610 msgr2=0x7f4b6019d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:39.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b600ff610 0x7f4b6019d180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00d950 tx=0x7f4b5c00dc60 comp rx=0 tx=0).stop 2023-11-06T01:43:39.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 -- 172.21.15.53:0/4022178736 shutdown_connections 2023-11-06T01:43:39.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4b4c0a0520 0x7f4b4c0a29e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b60104950 0x7f4b601a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b600ff610 0x7f4b6019d180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 --2- 172.21.15.53:0/4022178736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b60101f80 0x7f4b6019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:39.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 -- 172.21.15.53:0/4022178736 >> 172.21.15.53:0/4022178736 conn(0x7f4b600fb1f0 msgr2=0x7f4b601002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:39.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 -- 172.21.15.53:0/4022178736 shutdown_connections 2023-11-06T01:43:39.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:43:39.244+0000 7f4b687cb700 1 -- 172.21.15.53:0/4022178736 wait complete. 2023-11-06T01:43:39.281 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:43:40.096 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:39 smithi053 ceph-mon[99472]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.96k op/s 2023-11-06T01:43:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:39 smithi062 ceph-mon[103924]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.96k op/s 2023-11-06T01:43:40.238 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:39 smithi163 ceph-mon[104416]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.96k op/s 2023-11-06T01:43:42.099 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:41 smithi053 ceph-mon[99472]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.94k op/s 2023-11-06T01:43:42.145 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:41 smithi062 ceph-mon[103924]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.94k op/s 2023-11-06T01:43:42.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 565.0 s, 1898.4 tps, lat 52.229 ms stddev 62.939 2023-11-06T01:43:42.238 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:41 smithi163 ceph-mon[104416]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.94k op/s 2023-11-06T01:43:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.611+0000 7f89203e7700 1 -- 172.21.15.53:0/1435733918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8918101810 msgr2=0x7f8918103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.611+0000 7f89203e7700 1 --2- 172.21.15.53:0/1435733918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8918101810 0x7f8918103c00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8914009990 tx=0x7f8914009ca0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.611+0000 7f89203e7700 1 -- 172.21.15.53:0/1435733918 shutdown_connections 2023-11-06T01:43:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.611+0000 7f89203e7700 1 --2- 172.21.15.53:0/1435733918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8918106a70 0x7f8918108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.611+0000 7f89203e7700 1 --2- 172.21.15.53:0/1435733918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8918101810 0x7f8918103c00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.611+0000 7f89203e7700 1 --2- 172.21.15.53:0/1435733918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8918104140 0x7f8918106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.611+0000 7f89203e7700 1 -- 172.21.15.53:0/1435733918 >> 172.21.15.53:0/1435733918 conn(0x7f89180fb1e0 msgr2=0x7f89180fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.611+0000 7f89203e7700 1 -- 172.21.15.53:0/1435733918 shutdown_connections 2023-11-06T01:43:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.611+0000 7f89203e7700 1 -- 172.21.15.53:0/1435733918 wait complete. 2023-11-06T01:43:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f89203e7700 1 Processor -- start 2023-11-06T01:43:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f89203e7700 1 -- start start 2023-11-06T01:43:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f89203e7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8918101810 0x7f891819d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f89203e7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8918104140 0x7f891819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f89203e7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8918106a70 0x7f891819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f89203e7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891819eb70 con 0x7f8918104140 2023-11-06T01:43:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f89203e7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891819ece0 con 0x7f8918101810 2023-11-06T01:43:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f89203e7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891819eea0 con 0x7f8918106a70 2023-11-06T01:43:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f891d982700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8918104140 0x7f891819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f891d982700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8918104140 0x7f891819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42630/0 (socket says 172.21.15.53:42630) 2023-11-06T01:43:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f891d982700 1 -- 172.21.15.53:0/4120378605 learned_addr learned my addr 172.21.15.53:0/4120378605 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f891e183700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8918101810 0x7f891819d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f891e984700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8918106a70 0x7f891819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f891d982700 1 -- 172.21.15.53:0/4120378605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8918106a70 msgr2=0x7f891819e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f891d982700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8918106a70 0x7f891819e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f891d982700 1 -- 172.21.15.53:0/4120378605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8918101810 msgr2=0x7f891819d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f891d982700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8918101810 0x7f891819d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f891d982700 1 -- 172.21.15.53:0/4120378605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89140095f0 con 0x7f8918104140 2023-11-06T01:43:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.612+0000 7f891e183700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8918101810 0x7f891819d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.613+0000 7f891d982700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8918104140 0x7f891819d690 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f890800e9a0 tx=0x7f890800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.613+0000 7f890f7fe700 1 -- 172.21.15.53:0/4120378605 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908009790 con 0x7f8918104140 2023-11-06T01:43:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.613+0000 7f890f7fe700 1 -- 172.21.15.53:0/4120378605 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8908010430 con 0x7f8918104140 2023-11-06T01:43:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.613+0000 7f890f7fe700 1 -- 172.21.15.53:0/4120378605 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908018680 con 0x7f8918104140 2023-11-06T01:43:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.613+0000 7f89203e7700 1 -- 172.21.15.53:0/4120378605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891819f180 con 0x7f8918104140 2023-11-06T01:43:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.613+0000 7f89203e7700 1 -- 172.21.15.53:0/4120378605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89181a5940 con 0x7f8918104140 2023-11-06T01:43:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.616+0000 7f890f7fe700 1 -- 172.21.15.53:0/4120378605 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f89080105a0 con 0x7f8918104140 2023-11-06T01:43:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.616+0000 7f890f7fe700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89040a0520 0x7f89040a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.617+0000 7f891e183700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89040a0520 0x7f89040a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.617+0000 7f890f7fe700 1 -- 172.21.15.53:0/4120378605 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8908014070 con 0x7f8918104140 2023-11-06T01:43:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.617+0000 7f891e183700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89040a0520 0x7f89040a29e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f891400b720 tx=0x7f891400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f892c577b80 1 Processor -- start 2023-11-06T01:43:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f892c577b80 1 -- start start 2023-11-06T01:43:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f892c577b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39680 0x56236ec39aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f892c577b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39fe0 0x56236ec3a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f892c577b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 0x56236ec3b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f892c577b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236eb919b0 con 0x56236ec3aed0 2023-11-06T01:43:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f892c577b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236eb8f370 con 0x56236ec39680 2023-11-06T01:43:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f892c577b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236eb8ac60 con 0x56236ec39fe0 2023-11-06T01:43:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f891e183700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39680 0x56236ec39aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f891d982700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39fe0 0x56236ec3a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f891e984700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 0x56236ec3b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f891e183700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39680 0x56236ec39aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54074/0 (socket says 172.21.15.53:54074) 2023-11-06T01:43:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f891d982700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39fe0 0x56236ec3a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56878/0 (socket says 172.21.15.53:56878) 2023-11-06T01:43:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.618+0000 7f891e183700 1 -- 172.21.15.53:0/2001418932 learned_addr learned my addr 172.21.15.53:0/2001418932 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.619+0000 7f891e183700 1 -- 172.21.15.53:0/2001418932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39fe0 msgr2=0x56236ec3a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.619+0000 7f891e183700 1 --2- 172.21.15.53:0/2001418932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39fe0 0x56236ec3a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.619+0000 7f891e183700 1 -- 172.21.15.53:0/2001418932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 msgr2=0x56236ec3b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.619+0000 7f891e183700 1 --2- 172.21.15.53:0/2001418932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 0x56236ec3b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.619+0000 7f891e183700 1 -- 172.21.15.53:0/2001418932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236e9b8d80 con 0x56236ec39680 2023-11-06T01:43:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.619+0000 7f891d982700 1 --2- 172.21.15.53:0/2001418932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39fe0 0x56236ec3a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.619+0000 7f891e183700 1 --2- 172.21.15.53:0/2001418932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39680 0x56236ec39aa0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8914009990 tx=0x7f8914020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4fbafbedbb2f52b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.619+0000 7f890cdaa700 1 -- 172.21.15.53:0/2001418932 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891402b070 con 0x56236ec39680 2023-11-06T01:43:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.619+0000 7f890cdaa700 1 -- 172.21.15.53:0/2001418932 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f891401e040 con 0x56236ec39680 2023-11-06T01:43:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.619+0000 7f890cdaa700 1 -- 172.21.15.53:0/2001418932 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891400fa50 con 0x56236ec39680 2023-11-06T01:43:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.620+0000 7f892c577b80 1 -- 172.21.15.53:0/2001418932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39680 msgr2=0x56236ec39aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.620+0000 7f892c577b80 1 --2- 172.21.15.53:0/2001418932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39680 0x56236ec39aa0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8914009990 tx=0x7f8914020000 comp rx=0 tx=0).stop 2023-11-06T01:43:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.620+0000 7f892c577b80 1 -- 172.21.15.53:0/2001418932 shutdown_connections 2023-11-06T01:43:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.620+0000 7f892c577b80 1 --2- 172.21.15.53:0/2001418932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39680 0x56236ec39aa0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.620+0000 7f892c577b80 1 --2- 172.21.15.53:0/2001418932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 0x56236ec3b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.620+0000 7f892c577b80 1 --2- 172.21.15.53:0/2001418932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39fe0 0x56236ec3a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.620+0000 7f892c577b80 1 -- 172.21.15.53:0/2001418932 >> 172.21.15.53:0/2001418932 conn(0x56236eb99070 msgr2=0x56236ec36040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.622+0000 7f892c577b80 1 -- 172.21.15.53:0/2001418932 shutdown_connections 2023-11-06T01:43:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.624+0000 7f892c577b80 1 -- 172.21.15.53:0/2001418932 wait complete. 2023-11-06T01:43:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.624+0000 7f892c577b80 1 -- start start 2023-11-06T01:43:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.624+0000 7f892c577b80 1 Processor -- start 2023-11-06T01:43:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.624+0000 7f892c577b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39680 0x56236ec433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.624+0000 7f892c577b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39fe0 0x56236ec438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.624+0000 7f892c577b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec3aed0 0x56236ec4a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.624+0000 7f892c577b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236eb274b0 con 0x56236ec39680 2023-11-06T01:43:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f892c577b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236eb27150 con 0x56236ec39fe0 2023-11-06T01:43:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f892c577b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236ea0e8c0 con 0x56236ec3aed0 2023-11-06T01:43:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f891d982700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39fe0 0x56236ec438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f891e183700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39680 0x56236ec433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f892c577b80 10 client.0 _ll_register_callbacks cb 0x56236ea74a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f891e183700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39680 0x56236ec433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42642/0 (socket says 172.21.15.53:42642) 2023-11-06T01:43:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f891d982700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39fe0 0x56236ec438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54082/0 (socket says 172.21.15.53:54082) 2023-11-06T01:43:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f891e183700 1 -- 172.21.15.53:0/966128539 learned_addr learned my addr 172.21.15.53:0/966128539 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f891e183700 1 -- 172.21.15.53:0/966128539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec3aed0 msgr2=0x56236ec4a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f891e183700 1 --2- 172.21.15.53:0/966128539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec3aed0 0x56236ec4a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f891e183700 1 -- 172.21.15.53:0/966128539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39fe0 msgr2=0x56236ec438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f891e183700 1 --2- 172.21.15.53:0/966128539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39fe0 0x56236ec438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f891e183700 1 -- 172.21.15.53:0/966128539 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89181a5940 con 0x56236ec39680 2023-11-06T01:43:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f891e183700 1 --2- 172.21.15.53:0/966128539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39680 0x56236ec433a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f89140098e0 tx=0x7f8914005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd07d752e797fd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f8901ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:43:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f8901ffb700 1 -- 172.21.15.53:0/966128539 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891402b040 con 0x56236ec39680 2023-11-06T01:43:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f8901ffb700 1 -- 172.21.15.53:0/966128539 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8914004d70 con 0x56236ec39680 2023-11-06T01:43:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.625+0000 7f892c577b80 1 -- 172.21.15.53:0/966128539 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56236ebb5cb0 con 0x56236ec39680 2023-11-06T01:43:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f8901ffb700 1 -- 172.21.15.53:0/966128539 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8914038410 con 0x56236ec39680 2023-11-06T01:43:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f892c577b80 10 client.16533 fetch_fsmap learned FSMap version 44 2023-11-06T01:43:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f892c577b80 1 -- 172.21.15.53:0/966128539 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f8918109ad0 con 0x56236ec39680 2023-11-06T01:43:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f8901ffb700 1 -- 172.21.15.53:0/966128539 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56236ebb5cb0 con 0x56236ec39680 2023-11-06T01:43:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f8901ffb700 1 -- 172.21.15.53:0/966128539 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8914040020 con 0x56236ec39680 2023-11-06T01:43:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f892c577b80 10 client.16533 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:43:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f892c577b80 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f892c577b80 20 client.16533 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f892c577b80 1 --2- 172.21.15.53:0/966128539 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56236ec4ecc0 0x56236ec510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f892c577b80 4 client.16533 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:43:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f892c577b80 1 -- 172.21.15.53:0/966128539 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56236ec08db0 con 0x56236ec4ecc0 2023-11-06T01:43:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.626+0000 7f891d982700 1 --2- 172.21.15.53:0/966128539 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56236ec4ecc0 0x56236ec510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f8901ffb700 10 client.16533 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f891d982700 1 --2- 172.21.15.53:0/966128539 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56236ec4ecc0 0x56236ec510b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a13ddfdb457278e3 server_cookie=c1e939e4660dbf4c in_seq=0 out_seq=0 2023-11-06T01:43:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f8901ffb700 1 -- 172.21.15.53:0/966128539 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56236ec08db0 con 0x56236ec4ecc0 2023-11-06T01:43:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f8901ffb700 10 client.16533 handle_command_reply: tid=0 2023-11-06T01:43:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f892c577b80 1 client.16533 shutdown 2023-11-06T01:43:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f892c577b80 1 -- 172.21.15.53:0/966128539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39680 msgr2=0x56236ec433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f892c577b80 1 --2- 172.21.15.53:0/966128539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39680 0x56236ec433a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f89140098e0 tx=0x7f8914005700 comp rx=0 tx=0).stop 2023-11-06T01:43:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f892c577b80 1 -- 172.21.15.53:0/966128539 shutdown_connections 2023-11-06T01:43:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f892c577b80 1 --2- 172.21.15.53:0/966128539 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56236ec4ecc0 0x56236ec510b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f892c577b80 1 --2- 172.21.15.53:0/966128539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec3aed0 0x56236ec4a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f8901ffb700 0 client.16533 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f892c577b80 1 --2- 172.21.15.53:0/966128539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39680 0x56236ec433a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f892c577b80 1 --2- 172.21.15.53:0/966128539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39fe0 0x56236ec438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.627+0000 7f892c577b80 1 -- 172.21.15.53:0/966128539 >> 172.21.15.53:0/966128539 conn(0x56236eb99070 msgr2=0x56236ec343c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.628+0000 7f892c577b80 1 -- 172.21.15.53:0/966128539 shutdown_connections 2023-11-06T01:43:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.628+0000 7f892c577b80 1 -- 172.21.15.53:0/966128539 wait complete. 2023-11-06T01:43:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.628+0000 7f892c577b80 20 client.16533 trim_cache size 0 max 16384 2023-11-06T01:43:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f892c577b80 1 Processor -- start 2023-11-06T01:43:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f892c577b80 1 -- start start 2023-11-06T01:43:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f892c577b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec488d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f892c577b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39fe0 0x56236ec42a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f892c577b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 0x56236ec3e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f892c577b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236ec39280 con 0x56236ec3aed0 2023-11-06T01:43:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f892c577b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236ec2c7a0 con 0x56236ec39fe0 2023-11-06T01:43:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f892c577b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236ec42850 con 0x56236ec39680 2023-11-06T01:43:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f891e984700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 0x56236ec3e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f891e183700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f891d982700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39fe0 0x56236ec42a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f891e984700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 0x56236ec3e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42654/0 (socket says 172.21.15.53:42654) 2023-11-06T01:43:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f891e183700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56890/0 (socket says 172.21.15.53:56890) 2023-11-06T01:43:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.649+0000 7f891e984700 1 -- 172.21.15.53:0/889997142 learned_addr learned my addr 172.21.15.53:0/889997142 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.650+0000 7f891e183700 1 -- 172.21.15.53:0/889997142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39fe0 msgr2=0x56236ec42a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.650+0000 7f891e183700 1 --2- 172.21.15.53:0/889997142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39fe0 0x56236ec42a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.650+0000 7f891e183700 1 -- 172.21.15.53:0/889997142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 msgr2=0x56236ec3e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.650+0000 7f891e183700 1 --2- 172.21.15.53:0/889997142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 0x56236ec3e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.650+0000 7f891e183700 1 -- 172.21.15.53:0/889997142 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236e9a1c00 con 0x56236ec39680 2023-11-06T01:43:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.650+0000 7f891e984700 1 --2- 172.21.15.53:0/889997142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 0x56236ec3e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.650+0000 7f891e183700 1 --2- 172.21.15.53:0/889997142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec488d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f891400bf40 tx=0x7f8914009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a156106a2d49af52 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.650+0000 7f890d5ab700 1 -- 172.21.15.53:0/889997142 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891402b070 con 0x56236ec39680 2023-11-06T01:43:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.650+0000 7f890d5ab700 1 -- 172.21.15.53:0/889997142 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8914005a00 con 0x56236ec39680 2023-11-06T01:43:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.650+0000 7f890d5ab700 1 -- 172.21.15.53:0/889997142 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914042680 con 0x56236ec39680 2023-11-06T01:43:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.651+0000 7f892c577b80 1 -- 172.21.15.53:0/889997142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 msgr2=0x56236ec488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.651+0000 7f892c577b80 1 --2- 172.21.15.53:0/889997142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec488d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f891400bf40 tx=0x7f8914009ca0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.651+0000 7f892c577b80 1 -- 172.21.15.53:0/889997142 shutdown_connections 2023-11-06T01:43:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.651+0000 7f892c577b80 1 --2- 172.21.15.53:0/889997142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec3aed0 0x56236ec3e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.651+0000 7f892c577b80 1 --2- 172.21.15.53:0/889997142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec488d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.651+0000 7f892c577b80 1 --2- 172.21.15.53:0/889997142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec39fe0 0x56236ec42a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.651+0000 7f892c577b80 1 -- 172.21.15.53:0/889997142 >> 172.21.15.53:0/889997142 conn(0x56236eb99070 msgr2=0x56236ec420b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.651+0000 7f892c577b80 1 -- 172.21.15.53:0/889997142 shutdown_connections 2023-11-06T01:43:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.651+0000 7f892c577b80 1 -- 172.21.15.53:0/889997142 wait complete. 2023-11-06T01:43:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.651+0000 7f892c577b80 1 -- start start 2023-11-06T01:43:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.651+0000 7f892c577b80 1 Processor -- start 2023-11-06T01:43:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.652+0000 7f892c577b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec9a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.652+0000 7f892c577b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39fe0 0x56236ec9ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.652+0000 7f892c577b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec3aed0 0x56236ec9f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f892c577b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236ec9f750 con 0x56236ec39fe0 2023-11-06T01:43:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f892c577b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236ec9f890 con 0x56236ec3aed0 2023-11-06T01:43:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f892c577b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236ec9fa20 con 0x56236ec39680 2023-11-06T01:43:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f892c577b80 10 client.0 _ll_register_callbacks cb 0x7f8910009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f891e183700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec9a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f891d982700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39fe0 0x56236ec9ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f891e183700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec9a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56896/0 (socket says 172.21.15.53:56896) 2023-11-06T01:43:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f891d982700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39fe0 0x56236ec9ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42658/0 (socket says 172.21.15.53:42658) 2023-11-06T01:43:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f891e183700 1 -- 172.21.15.53:0/1551252209 learned_addr learned my addr 172.21.15.53:0/1551252209 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f891e183700 1 -- 172.21.15.53:0/1551252209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec3aed0 msgr2=0x56236ec9f210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:43:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f891e183700 1 --2- 172.21.15.53:0/1551252209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec3aed0 0x56236ec9f210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f891e183700 1 -- 172.21.15.53:0/1551252209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39fe0 msgr2=0x56236ec9ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f891e183700 1 --2- 172.21.15.53:0/1551252209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39fe0 0x56236ec9ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.653+0000 7f891e183700 1 -- 172.21.15.53:0/1551252209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8910014320 con 0x56236ec39680 2023-11-06T01:43:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.654+0000 7f891e183700 1 --2- 172.21.15.53:0/1551252209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec9a770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f891400fbf0 tx=0x7f8914024840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e321f94c8a67f0bb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.654+0000 7f8901ffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:43:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.654+0000 7f8901ffb700 1 -- 172.21.15.53:0/1551252209 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891402b040 con 0x56236ec39680 2023-11-06T01:43:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f8901ffb700 1 -- 172.21.15.53:0/1551252209 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8914042420 con 0x56236ec39680 2023-11-06T01:43:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f892c577b80 1 -- 172.21.15.53:0/1551252209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f891401e040 con 0x56236ec39680 2023-11-06T01:43:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f8901ffb700 1 -- 172.21.15.53:0/1551252209 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8914024410 con 0x56236ec39680 2023-11-06T01:43:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f892c577b80 10 client.26158 fetch_fsmap learned FSMap version 44 2023-11-06T01:43:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f892c577b80 1 -- 172.21.15.53:0/1551252209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f8918109ad0 con 0x56236ec39680 2023-11-06T01:43:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f8901ffb700 1 -- 172.21.15.53:0/1551252209 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f891401e040 con 0x56236ec39680 2023-11-06T01:43:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f8901ffb700 1 -- 172.21.15.53:0/1551252209 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8914040070 con 0x56236ec39680 2023-11-06T01:43:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f892c577b80 10 client.26158 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:43:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f892c577b80 10 client.26158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f892c577b80 20 client.26158 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f892c577b80 1 --2- 172.21.15.53:0/1551252209 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56236eca0f70 0x56236eca3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f892c577b80 4 client.26158 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:43:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f892c577b80 1 -- 172.21.15.53:0/1551252209 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56236eb83ce0 con 0x56236eca0f70 2023-11-06T01:43:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.655+0000 7f891d982700 1 --2- 172.21.15.53:0/1551252209 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56236eca0f70 0x56236eca3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.656+0000 7f8901ffb700 10 client.26158 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.656+0000 7f891d982700 1 --2- 172.21.15.53:0/1551252209 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56236eca0f70 0x56236eca3360 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379397aa17469172 server_cookie=51bd2f5c6fed60ae in_seq=0 out_seq=0 2023-11-06T01:43:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f8901ffb700 1 -- 172.21.15.53:0/1551252209 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56236eb83ce0 con 0x56236eca0f70 2023-11-06T01:43:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f8901ffb700 10 client.26158 handle_command_reply: tid=0 2023-11-06T01:43:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 1 client.26158 shutdown 2023-11-06T01:43:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 1 -- 172.21.15.53:0/1551252209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 msgr2=0x56236ec9a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 1 --2- 172.21.15.53:0/1551252209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec9a770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f891400fbf0 tx=0x7f8914024840 comp rx=0 tx=0).stop 2023-11-06T01:43:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 1 -- 172.21.15.53:0/1551252209 shutdown_connections 2023-11-06T01:43:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 1 --2- 172.21.15.53:0/1551252209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236ec3aed0 0x56236ec9f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 1 --2- 172.21.15.53:0/1551252209 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56236eca0f70 0x56236eca3360 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 1 --2- 172.21.15.53:0/1551252209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56236ec39680 0x56236ec9a770 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 1 --2- 172.21.15.53:0/1551252209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56236ec39fe0 0x56236ec9ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f8901ffb700 0 client.26158 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 1 -- 172.21.15.53:0/1551252209 >> 172.21.15.53:0/1551252209 conn(0x56236eb99070 msgr2=0x56236ec42800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 1 -- 172.21.15.53:0/1551252209 shutdown_connections 2023-11-06T01:43:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 1 -- 172.21.15.53:0/1551252209 wait complete. 2023-11-06T01:43:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.664+0000 7f892c577b80 20 client.26158 trim_cache size 0 max 16384 2023-11-06T01:43:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:43:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:43:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:43:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:43:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 -- 172.21.15.53:0/4120378605 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89040a0520 msgr2=0x7f89040a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89040a0520 0x7f89040a29e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f891400b720 tx=0x7f891400b610 comp rx=0 tx=0).stop 2023-11-06T01:43:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 -- 172.21.15.53:0/4120378605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8918104140 msgr2=0x7f891819d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8918104140 0x7f891819d690 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f890800e9a0 tx=0x7f890800ecb0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 -- 172.21.15.53:0/4120378605 shutdown_connections 2023-11-06T01:43:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f89040a0520 0x7f89040a29e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8918101810 0x7f891819d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8918106a70 0x7f891819e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 --2- 172.21.15.53:0/4120378605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8918104140 0x7f891819d690 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 -- 172.21.15.53:0/4120378605 >> 172.21.15.53:0/4120378605 conn(0x7f89180fb1e0 msgr2=0x7f8918102480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 -- 172.21.15.53:0/4120378605 shutdown_connections 2023-11-06T01:43:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:42.665+0000 7f89203e7700 1 -- 172.21.15.53:0/4120378605 wait complete. 2023-11-06T01:43:44.094 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:43 smithi053 ceph-mon[99472]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 MiB/s wr, 1.70k op/s 2023-11-06T01:43:44.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:43 smithi062 ceph-mon[103924]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 MiB/s wr, 1.70k op/s 2023-11-06T01:43:44.238 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:43 smithi163 ceph-mon[104416]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 MiB/s wr, 1.70k op/s 2023-11-06T01:43:46.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:45 smithi053 ceph-mon[99472]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 1.41k op/s 2023-11-06T01:43:46.153 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:45 smithi062 ceph-mon[103924]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 1.41k op/s 2023-11-06T01:43:46.238 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:45 smithi163 ceph-mon[104416]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 1.41k op/s 2023-11-06T01:43:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 570.0 s, 2112.3 tps, lat 47.674 ms stddev 53.497 2023-11-06T01:43:47.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:47 smithi053 ceph-mon[99472]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 698 op/s 2023-11-06T01:43:47.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:47 smithi062 ceph-mon[103924]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 698 op/s 2023-11-06T01:43:47.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:47 smithi163 ceph-mon[104416]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 698 op/s 2023-11-06T01:43:48.599 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:48 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:43:48.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:48 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:43:48.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:48 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:43:49.599 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:49 smithi053 ceph-mon[99472]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 672 op/s 2023-11-06T01:43:49.600 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:49 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:43:49.600 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:49 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:43:49.600 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:49 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:43:49.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:49 smithi062 ceph-mon[103924]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 672 op/s 2023-11-06T01:43:49.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:49 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:43:49.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:49 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:43:49.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:49 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:43:49.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:49 smithi163 ceph-mon[104416]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 672 op/s 2023-11-06T01:43:49.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:49 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:43:49.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:49 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:43:49.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:49 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:43:51.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:51 smithi053 ceph-mon[99472]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 654 op/s 2023-11-06T01:43:51.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:51 smithi062 ceph-mon[103924]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 654 op/s 2023-11-06T01:43:51.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:51 smithi163 ceph-mon[104416]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 654 op/s 2023-11-06T01:43:52.186 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 575.0 s, 1509.7 tps, lat 66.469 ms stddev 72.112 2023-11-06T01:43:53.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:53 smithi053 ceph-mon[99472]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 635 op/s 2023-11-06T01:43:53.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:53 smithi062 ceph-mon[103924]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 635 op/s 2023-11-06T01:43:53.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:53 smithi163 ceph-mon[104416]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 635 op/s 2023-11-06T01:43:55.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:55 smithi053 ceph-mon[99472]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 MiB/s wr, 692 op/s 2023-11-06T01:43:55.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:55 smithi062 ceph-mon[103924]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 MiB/s wr, 692 op/s 2023-11-06T01:43:55.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:55 smithi163 ceph-mon[104416]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 MiB/s wr, 692 op/s 2023-11-06T01:43:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.797+0000 7f4ecec3c700 1 -- 172.21.15.53:0/1739649885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec80ffe90 msgr2=0x7f4ec8100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.797+0000 7f4ecec3c700 1 --2- 172.21.15.53:0/1739649885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec80ffe90 0x7f4ec8100310 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009990 tx=0x7f4eb4009ca0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.797+0000 7f4ecec3c700 1 -- 172.21.15.53:0/1739649885 shutdown_connections 2023-11-06T01:43:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.797+0000 7f4ecec3c700 1 --2- 172.21.15.53:0/1739649885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ec8100850 0x7f4ec81080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.797+0000 7f4ecec3c700 1 --2- 172.21.15.53:0/1739649885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8101240 0x7f4ec8101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.797+0000 7f4ecec3c700 1 --2- 172.21.15.53:0/1739649885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec80ffe90 0x7f4ec8100310 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.797+0000 7f4ecec3c700 1 -- 172.21.15.53:0/1739649885 >> 172.21.15.53:0/1739649885 conn(0x7f4ec80fa960 msgr2=0x7f4ec80fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.797+0000 7f4ecec3c700 1 -- 172.21.15.53:0/1739649885 shutdown_connections 2023-11-06T01:43:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.797+0000 7f4ecec3c700 1 -- 172.21.15.53:0/1739649885 wait complete. 2023-11-06T01:43:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecec3c700 1 Processor -- start 2023-11-06T01:43:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecec3c700 1 -- start start 2023-11-06T01:43:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecec3c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80ffe90 0x7f4ec81997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecec3c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ec8100850 0x7f4ec819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecec3c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8101240 0x7f4ec819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecec3c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec819e860 con 0x7f4ec8101240 2023-11-06T01:43:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecec3c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec819e9a0 con 0x7f4ec8100850 2023-11-06T01:43:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecec3c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec819eb60 con 0x7f4ec80ffe90 2023-11-06T01:43:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecd1d9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8101240 0x7f4ec819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecd1d9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8101240 0x7f4ec819e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42676/0 (socket says 172.21.15.53:42676) 2023-11-06T01:43:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecd1d9700 1 -- 172.21.15.53:0/3329284612 learned_addr learned my addr 172.21.15.53:0/3329284612 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ec8100850 0x7f4ec819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.798+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80ffe90 0x7f4ec81997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.799+0000 7f4ecd1d9700 1 -- 172.21.15.53:0/3329284612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80ffe90 msgr2=0x7f4ec81997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.799+0000 7f4ecd1d9700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80ffe90 0x7f4ec81997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.799+0000 7f4ecd1d9700 1 -- 172.21.15.53:0/3329284612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ec8100850 msgr2=0x7f4ec819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.799+0000 7f4ecd1d9700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ec8100850 0x7f4ec819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.799+0000 7f4ecd1d9700 1 -- 172.21.15.53:0/3329284612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb40095f0 con 0x7f4ec8101240 2023-11-06T01:43:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.799+0000 7f4ecd1d9700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8101240 0x7f4ec819e290 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000ef30 tx=0x7f4ec000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.799+0000 7f4ec5ffb700 1 -- 172.21.15.53:0/3329284612 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0018070 con 0x7f4ec8101240 2023-11-06T01:43:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.799+0000 7f4ec5ffb700 1 -- 172.21.15.53:0/3329284612 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec000f040 con 0x7f4ec8101240 2023-11-06T01:43:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.799+0000 7f4ec5ffb700 1 -- 172.21.15.53:0/3329284612 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000cd00 con 0x7f4ec8101240 2023-11-06T01:43:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.799+0000 7f4ecec3c700 1 -- 172.21.15.53:0/3329284612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec819ede0 con 0x7f4ec8101240 2023-11-06T01:43:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.799+0000 7f4ecec3c700 1 -- 172.21.15.53:0/3329284612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec819f210 con 0x7f4ec8101240 2023-11-06T01:43:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.801+0000 7f4ec5ffb700 1 -- 172.21.15.53:0/3329284612 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4ec0003a00 con 0x7f4ec8101240 2023-11-06T01:43:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.802+0000 7f4ec5ffb700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4eb80a0520 0x7f4eb80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.802+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4eb80a0520 0x7f4eb80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.802+0000 7f4ec5ffb700 1 -- 172.21.15.53:0/3329284612 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4ec00cb460 con 0x7f4ec8101240 2023-11-06T01:43:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.802+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4eb80a0520 0x7f4eb80a29e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80ff840 tx=0x7f4ebc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.803+0000 7f4edadccb80 1 Processor -- start 2023-11-06T01:43:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.803+0000 7f4edadccb80 1 -- start start 2023-11-06T01:43:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.803+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b8fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.803+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b908d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.803+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081b912e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.803+0000 7f4edadccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d081ae79b0 con 0x55d081b8ff70 2023-11-06T01:43:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.803+0000 7f4edadccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d081ae5370 con 0x55d081b8f610 2023-11-06T01:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.803+0000 7f4edadccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d081ae0c60 con 0x55d081b90e60 2023-11-06T01:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.804+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b908d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.804+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b908d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42680/0 (socket says 172.21.15.53:42680) 2023-11-06T01:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.804+0000 7f4ec7fff700 1 -- 172.21.15.53:0/2672716215 learned_addr learned my addr 172.21.15.53:0/2672716215 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.804+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/2672716215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b8fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.804+0000 7f4ec7fff700 1 -- 172.21.15.53:0/2672716215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 msgr2=0x55d081b912e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.804+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/2672716215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081b912e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.804+0000 7f4ec7fff700 1 -- 172.21.15.53:0/2672716215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 msgr2=0x55d081b8fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.804+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/2672716215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b8fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.804+0000 7f4ec7fff700 1 -- 172.21.15.53:0/2672716215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d08190ed80 con 0x55d081b8ff70 2023-11-06T01:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.805+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/2672716215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b908d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009990 tx=0x7f4eb4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e8afd9653fcea2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.806+0000 7f4eb2ffd700 1 -- 172.21.15.53:0/2672716215 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb401d050 con 0x55d081b8ff70 2023-11-06T01:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.806+0000 7f4eb2ffd700 1 -- 172.21.15.53:0/2672716215 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb401e040 con 0x55d081b8ff70 2023-11-06T01:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.806+0000 7f4eb2ffd700 1 -- 172.21.15.53:0/2672716215 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4023d60 con 0x55d081b8ff70 2023-11-06T01:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.807+0000 7f4edadccb80 1 -- 172.21.15.53:0/2672716215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 msgr2=0x55d081b908d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.807+0000 7f4edadccb80 1 --2- 172.21.15.53:0/2672716215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b908d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009990 tx=0x7f4eb4005e80 comp rx=0 tx=0).stop 2023-11-06T01:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.807+0000 7f4edadccb80 1 -- 172.21.15.53:0/2672716215 shutdown_connections 2023-11-06T01:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.807+0000 7f4edadccb80 1 --2- 172.21.15.53:0/2672716215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b8fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.807+0000 7f4edadccb80 1 --2- 172.21.15.53:0/2672716215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081b912e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.807+0000 7f4edadccb80 1 --2- 172.21.15.53:0/2672716215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b908d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.807+0000 7f4edadccb80 1 -- 172.21.15.53:0/2672716215 >> 172.21.15.53:0/2672716215 conn(0x55d081aef070 msgr2=0x55d081b8c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.807+0000 7f4edadccb80 1 -- 172.21.15.53:0/2672716215 shutdown_connections 2023-11-06T01:43:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.807+0000 7f4edadccb80 1 -- 172.21.15.53:0/2672716215 wait complete. 2023-11-06T01:43:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.808+0000 7f4edadccb80 1 -- start start 2023-11-06T01:43:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.808+0000 7f4edadccb80 1 Processor -- start 2023-11-06T01:43:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.808+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b994b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.808+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b99a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.808+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081ba0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.808+0000 7f4edadccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d081a7d4b0 con 0x55d081b8ff70 2023-11-06T01:43:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.808+0000 7f4edadccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d081a7d150 con 0x55d081b8f610 2023-11-06T01:43:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.808+0000 7f4edadccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0819648c0 con 0x55d081b90e60 2023-11-06T01:43:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.808+0000 7f4edadccb80 10 client.0 _ll_register_callbacks cb 0x55d0819caa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.817+0000 7f4ecd1d9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081ba0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.817+0000 7f4ecd1d9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081ba0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56928/0 (socket says 172.21.15.53:56928) 2023-11-06T01:43:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.817+0000 7f4ecd1d9700 1 -- 172.21.15.53:0/2609002536 learned_addr learned my addr 172.21.15.53:0/2609002536 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.819+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.820+0000 7f4ecc9d8700 1 -- 172.21.15.53:0/2609002536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 msgr2=0x55d081ba0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.820+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081ba0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.820+0000 7f4ecc9d8700 1 -- 172.21.15.53:0/2609002536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 msgr2=0x55d081b99a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.820+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b99a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.820+0000 7f4ecc9d8700 1 -- 172.21.15.53:0/2609002536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec8108a20 con 0x55d081b8f610 2023-11-06T01:43:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.839+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b994b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0117d0 tx=0x7f4ebc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355e5f10b7ede7f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.839+0000 7f4ea7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:43:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.850+0000 7f4ea7fff700 1 -- 172.21.15.53:0/2609002536 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc004220 con 0x55d081b8f610 2023-11-06T01:43:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.850+0000 7f4ea7fff700 1 -- 172.21.15.53:0/2609002536 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4ebc004380 con 0x55d081b8f610 2023-11-06T01:43:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.852+0000 7f4edadccb80 1 -- 172.21.15.53:0/2609002536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d081b0bcb0 con 0x55d081b8f610 2023-11-06T01:43:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.871+0000 7f4ea7fff700 1 -- 172.21.15.53:0/2609002536 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ebc004540 con 0x55d081b8f610 2023-11-06T01:43:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.871+0000 7f4edadccb80 10 client.26099 fetch_fsmap learned FSMap version 44 2023-11-06T01:43:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.871+0000 7f4edadccb80 1 -- 172.21.15.53:0/2609002536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55d0818f7c00 con 0x55d081b8f610 2023-11-06T01:43:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.871+0000 7f4ea7fff700 1 -- 172.21.15.53:0/2609002536 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d081b0bcb0 con 0x55d081b8f610 2023-11-06T01:43:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.872+0000 7f4ea7fff700 1 -- 172.21.15.53:0/2609002536 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f4ebc003ae0 con 0x55d081b8f610 2023-11-06T01:43:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.872+0000 7f4edadccb80 10 client.26099 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:43:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.872+0000 7f4edadccb80 10 client.26099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.872+0000 7f4edadccb80 20 client.26099 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.872+0000 7f4edadccb80 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d081bc7480 0x55d081bc98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.872+0000 7f4edadccb80 4 client.26099 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:43:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.872+0000 7f4edadccb80 1 -- 172.21.15.53:0/2609002536 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d081b5edb0 con 0x55d081bc7480 2023-11-06T01:43:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.874+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d081bc7480 0x55d081bc98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.877+0000 7f4ea7fff700 10 client.26099 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.877+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d081bc7480 0x55d081bc98a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c77e95fd715b70e6 server_cookie=af815376e91ac69d in_seq=0 out_seq=0 2023-11-06T01:43:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4ea7fff700 1 -- 172.21.15.53:0/2609002536 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d081b5edb0 con 0x55d081bc7480 2023-11-06T01:43:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4ea7fff700 10 client.26099 handle_command_reply: tid=0 2023-11-06T01:43:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 1 client.26099 shutdown 2023-11-06T01:43:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 1 -- 172.21.15.53:0/2609002536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 msgr2=0x55d081b994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b994b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0117d0 tx=0x7f4ebc0047c0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 1 -- 172.21.15.53:0/2609002536 shutdown_connections 2023-11-06T01:43:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b994b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d081bc7480 0x55d081bc98a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081ba0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4ea7fff700 0 client.26099 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 1 --2- 172.21.15.53:0/2609002536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b99a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 1 -- 172.21.15.53:0/2609002536 >> 172.21.15.53:0/2609002536 conn(0x55d081aef070 msgr2=0x55d081b8a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 1 -- 172.21.15.53:0/2609002536 shutdown_connections 2023-11-06T01:43:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 1 -- 172.21.15.53:0/2609002536 wait complete. 2023-11-06T01:43:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.879+0000 7f4edadccb80 20 client.26099 trim_cache size 0 max 16384 2023-11-06T01:43:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4edadccb80 1 Processor -- start 2023-11-06T01:43:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4edadccb80 1 -- start start 2023-11-06T01:43:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b9ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081b98bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4edadccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d081b87710 con 0x55d081b8ff70 2023-11-06T01:43:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4edadccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d081b827a0 con 0x55d081b8f610 2023-11-06T01:43:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4edadccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d081b989f0 con 0x55d081b90e60 2023-11-06T01:43:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b91260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42694/0 (socket says 172.21.15.53:42694) 2023-11-06T01:43:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4ec7fff700 1 -- 172.21.15.53:0/1143967819 learned_addr learned my addr 172.21.15.53:0/1143967819 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/1143967819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b9ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.902+0000 7f4ecd1d9700 1 --2- 172.21.15.53:0/1143967819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081b98bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.903+0000 7f4ec7fff700 1 -- 172.21.15.53:0/1143967819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 msgr2=0x55d081b98bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.903+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/1143967819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081b98bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.903+0000 7f4ec7fff700 1 -- 172.21.15.53:0/1143967819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 msgr2=0x55d081b9ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.903+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/1143967819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b9ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.903+0000 7f4ec7fff700 1 -- 172.21.15.53:0/1143967819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0819351f0 con 0x55d081b8ff70 2023-11-06T01:43:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.903+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/1143967819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b91260 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400b620 tx=0x7f4eb4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c60b2ac2f6350b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.903+0000 7f4eb37fe700 1 -- 172.21.15.53:0/1143967819 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4023580 con 0x55d081b8ff70 2023-11-06T01:43:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.903+0000 7f4eb37fe700 1 -- 172.21.15.53:0/1143967819 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb400bd30 con 0x55d081b8ff70 2023-11-06T01:43:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.903+0000 7f4eb37fe700 1 -- 172.21.15.53:0/1143967819 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4005570 con 0x55d081b8ff70 2023-11-06T01:43:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.904+0000 7f4edadccb80 1 -- 172.21.15.53:0/1143967819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 msgr2=0x55d081b91260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.904+0000 7f4edadccb80 1 --2- 172.21.15.53:0/1143967819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b91260 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400b620 tx=0x7f4eb4009310 comp rx=0 tx=0).stop 2023-11-06T01:43:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.904+0000 7f4edadccb80 1 -- 172.21.15.53:0/1143967819 shutdown_connections 2023-11-06T01:43:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.904+0000 7f4edadccb80 1 --2- 172.21.15.53:0/1143967819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b8f610 0x55d081b9ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.904+0000 7f4edadccb80 1 --2- 172.21.15.53:0/1143967819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b90e60 0x55d081b98bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.904+0000 7f4edadccb80 1 --2- 172.21.15.53:0/1143967819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8ff70 0x55d081b91260 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.904+0000 7f4edadccb80 1 -- 172.21.15.53:0/1143967819 >> 172.21.15.53:0/1143967819 conn(0x55d081aef070 msgr2=0x55d081b8bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.904+0000 7f4edadccb80 1 -- 172.21.15.53:0/1143967819 shutdown_connections 2023-11-06T01:43:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.904+0000 7f4edadccb80 1 -- 172.21.15.53:0/1143967819 wait complete. 2023-11-06T01:43:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.904+0000 7f4edadccb80 1 -- start start 2023-11-06T01:43:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.904+0000 7f4edadccb80 1 Processor -- start 2023-11-06T01:43:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8f610 0x55d081b9e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b8ff70 0x55d081c14f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4edadccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b90e60 0x55d081c17520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4edadccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d081c17c30 con 0x55d081b8f610 2023-11-06T01:43:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4edadccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d081c17da0 con 0x55d081b90e60 2023-11-06T01:43:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4edadccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d081c17f60 con 0x55d081b8ff70 2023-11-06T01:43:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4edadccb80 10 client.0 _ll_register_callbacks cb 0x7f4ebc011e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:43:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ecc9d8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8f610 0x55d081b9e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b8ff70 0x55d081c14f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ecc9d8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8f610 0x55d081b9e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42698/0 (socket says 172.21.15.53:42698) 2023-11-06T01:43:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b8ff70 0x55d081c14f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56940/0 (socket says 172.21.15.53:56940) 2023-11-06T01:43:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ecc9d8700 1 -- 172.21.15.53:0/4032480260 learned_addr learned my addr 172.21.15.53:0/4032480260 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:43:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ecd1d9700 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b90e60 0x55d081c17520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ecc9d8700 1 -- 172.21.15.53:0/4032480260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b8ff70 msgr2=0x55d081c14f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b8ff70 0x55d081c14f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ecc9d8700 1 -- 172.21.15.53:0/4032480260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b90e60 msgr2=0x55d081c17520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:43:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b90e60 0x55d081c17520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ecc9d8700 1 -- 172.21.15.53:0/4032480260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d08190ed80 con 0x55d081b8f610 2023-11-06T01:43:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b8ff70 0x55d081c14f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:43:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ecc9d8700 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8f610 0x55d081b9e990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc01edc0 tx=0x7f4ebc026470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2e36da8b7078e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:43:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.905+0000 7f4ea7fff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:43:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4ea7fff700 1 -- 172.21.15.53:0/4032480260 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc026540 con 0x55d081b8f610 2023-11-06T01:43:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4ea7fff700 1 -- 172.21.15.53:0/4032480260 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4ebc004830 con 0x55d081b8f610 2023-11-06T01:43:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4edadccb80 1 -- 172.21.15.53:0/4032480260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4eb401e040 con 0x55d081b8f610 2023-11-06T01:43:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4ea7fff700 1 -- 172.21.15.53:0/4032480260 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ebc0049f0 con 0x55d081b8f610 2023-11-06T01:43:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4edadccb80 10 client.16554 fetch_fsmap learned FSMap version 44 2023-11-06T01:43:55.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4edadccb80 1 -- 172.21.15.53:0/4032480260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55d08190d250 con 0x55d081b8f610 2023-11-06T01:43:55.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4ea7fff700 1 -- 172.21.15.53:0/4032480260 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb401e040 con 0x55d081b8f610 2023-11-06T01:43:55.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4ea7fff700 1 -- 172.21.15.53:0/4032480260 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f4ebc042020 con 0x55d081b8f610 2023-11-06T01:43:55.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4edadccb80 10 client.16554 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:43:55.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4edadccb80 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:43:55.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4edadccb80 20 client.16554 populate_metadata read hostname 'smithi053' 2023-11-06T01:43:55.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4edadccb80 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d081c19900 0x55d081c1bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:43:55.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4edadccb80 4 client.16554 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:43:55.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4edadccb80 1 -- 172.21.15.53:0/4032480260 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d081ad9ce0 con 0x55d081c19900 2023-11-06T01:43:55.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.906+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d081c19900 0x55d081c1bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:43:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.907+0000 7f4ea7fff700 10 client.16554 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.907+0000 7f4ec7fff700 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d081c19900 0x55d081c1bd20 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e054ad6e2d40798 server_cookie=7a403dc1362da414 in_seq=0 out_seq=0 2023-11-06T01:43:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.919+0000 7f4ea7fff700 1 -- 172.21.15.53:0/4032480260 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d081ad9ce0 con 0x55d081c19900 2023-11-06T01:43:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.919+0000 7f4ea7fff700 10 client.16554 handle_command_reply: tid=0 2023-11-06T01:43:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.919+0000 7f4edadccb80 1 client.16554 shutdown 2023-11-06T01:43:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4edadccb80 1 -- 172.21.15.53:0/4032480260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8f610 msgr2=0x55d081b9e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4edadccb80 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8f610 0x55d081b9e990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc01edc0 tx=0x7f4ebc026470 comp rx=0 tx=0).stop 2023-11-06T01:43:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4edadccb80 1 -- 172.21.15.53:0/4032480260 shutdown_connections 2023-11-06T01:43:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4edadccb80 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d081b90e60 0x55d081c17520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4edadccb80 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d081c19900 0x55d081c1bd20 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4edadccb80 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d081b8f610 0x55d081b9e990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4edadccb80 1 --2- 172.21.15.53:0/4032480260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d081b8ff70 0x55d081c14f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4ea7fff700 0 client.16554 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:43:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4edadccb80 1 -- 172.21.15.53:0/4032480260 >> 172.21.15.53:0/4032480260 conn(0x55d081aef070 msgr2=0x55d081b8bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4edadccb80 1 -- 172.21.15.53:0/4032480260 shutdown_connections 2023-11-06T01:43:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4edadccb80 1 -- 172.21.15.53:0/4032480260 wait complete. 2023-11-06T01:43:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.920+0000 7f4edadccb80 20 client.16554 trim_cache size 0 max 16384 2023-11-06T01:43:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:43:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:43:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:43:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:43:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 -- 172.21.15.53:0/3329284612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4eb80a0520 msgr2=0x7f4eb80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4eb80a0520 0x7f4eb80a29e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80ff840 tx=0x7f4ebc009260 comp rx=0 tx=0).stop 2023-11-06T01:43:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 -- 172.21.15.53:0/3329284612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8101240 msgr2=0x7f4ec819e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:43:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8101240 0x7f4ec819e290 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000ef30 tx=0x7f4ec000c3a0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 -- 172.21.15.53:0/3329284612 shutdown_connections 2023-11-06T01:43:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4eb80a0520 0x7f4eb80a29e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8101240 0x7f4ec819e290 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80ffe90 0x7f4ec81997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 --2- 172.21.15.53:0/3329284612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ec8100850 0x7f4ec819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:43:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 -- 172.21.15.53:0/3329284612 >> 172.21.15.53:0/3329284612 conn(0x7f4ec80fa960 msgr2=0x7f4ec80fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:43:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 -- 172.21.15.53:0/3329284612 shutdown_connections 2023-11-06T01:43:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:43:55.922+0000 7f4ecec3c700 1 -- 172.21.15.53:0/3329284612 wait complete. 2023-11-06T01:43:57.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 580.0 s, 1719.5 tps, lat 58.140 ms stddev 61.281 2023-11-06T01:43:57.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:57 smithi053 ceph-mon[99472]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 649 op/s 2023-11-06T01:43:57.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:57 smithi062 ceph-mon[103924]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 649 op/s 2023-11-06T01:43:57.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:57 smithi163 ceph-mon[104416]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 649 op/s 2023-11-06T01:43:59.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:43:59 smithi053 ceph-mon[99472]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 MiB/s wr, 632 op/s 2023-11-06T01:43:59.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:43:59 smithi062 ceph-mon[103924]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 MiB/s wr, 632 op/s 2023-11-06T01:43:59.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:43:59 smithi163 ceph-mon[104416]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 MiB/s wr, 632 op/s 2023-11-06T01:44:01.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:01 smithi053 ceph-mon[99472]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.1 MiB/s wr, 637 op/s 2023-11-06T01:44:01.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:01 smithi062 ceph-mon[103924]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.1 MiB/s wr, 637 op/s 2023-11-06T01:44:01.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:01 smithi163 ceph-mon[104416]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.1 MiB/s wr, 637 op/s 2023-11-06T01:44:02.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 585.0 s, 1639.4 tps, lat 61.123 ms stddev 65.583 2023-11-06T01:44:03.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:03 smithi062 ceph-mon[103924]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 MiB/s wr, 651 op/s 2023-11-06T01:44:03.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:03 smithi163 ceph-mon[104416]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 MiB/s wr, 651 op/s 2023-11-06T01:44:03.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:03 smithi053 ceph-mon[99472]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 MiB/s wr, 651 op/s 2023-11-06T01:44:05.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:05 smithi062 ceph-mon[103924]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.9 MiB/s wr, 727 op/s 2023-11-06T01:44:05.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:05 smithi163 ceph-mon[104416]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.9 MiB/s wr, 727 op/s 2023-11-06T01:44:05.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:05 smithi053 ceph-mon[99472]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.9 MiB/s wr, 727 op/s 2023-11-06T01:44:07.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 590.0 s, 1918.6 tps, lat 51.164 ms stddev 58.211 2023-11-06T01:44:07.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:07 smithi062 ceph-mon[103924]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 MiB/s wr, 705 op/s 2023-11-06T01:44:07.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:07 smithi163 ceph-mon[104416]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 MiB/s wr, 705 op/s 2023-11-06T01:44:07.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:07 smithi053 ceph-mon[99472]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 MiB/s wr, 705 op/s 2023-11-06T01:44:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.091+0000 7f56c53de700 1 -- 172.21.15.53:0/2718276558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c00ffc90 msgr2=0x7f56c01000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.091+0000 7f56c53de700 1 --2- 172.21.15.53:0/2718276558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c00ffc90 0x7f56c01000b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f56b00099c0 tx=0x7f56b0009cd0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.091+0000 7f56c53de700 1 -- 172.21.15.53:0/2718276558 shutdown_connections 2023-11-06T01:44:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.091+0000 7f56c53de700 1 --2- 172.21.15.53:0/2718276558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c0101820 0x7f56c01080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.091+0000 7f56c53de700 1 --2- 172.21.15.53:0/2718276558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c00ffc90 0x7f56c01000b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.091+0000 7f56c53de700 1 --2- 172.21.15.53:0/2718276558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c0100e80 0x7f56c01012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.091+0000 7f56c53de700 1 -- 172.21.15.53:0/2718276558 >> 172.21.15.53:0/2718276558 conn(0x7f56c00fb1e0 msgr2=0x7f56c00fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.091+0000 7f56c53de700 1 -- 172.21.15.53:0/2718276558 shutdown_connections 2023-11-06T01:44:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.091+0000 7f56c53de700 1 -- 172.21.15.53:0/2718276558 wait complete. 2023-11-06T01:44:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.092+0000 7f56c53de700 1 Processor -- start 2023-11-06T01:44:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.092+0000 7f56c53de700 1 -- start start 2023-11-06T01:44:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.092+0000 7f56c53de700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c00ffc90 0x7f56c0196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.092+0000 7f56c53de700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c0100e80 0x7f56c0197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.092+0000 7f56c53de700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c0101820 0x7f56c0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.092+0000 7f56c53de700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c0198660 con 0x7f56c0101820 2023-11-06T01:44:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.092+0000 7f56c53de700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c01987d0 con 0x7f56c00ffc90 2023-11-06T01:44:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.092+0000 7f56c53de700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c0198990 con 0x7f56c0100e80 2023-11-06T01:44:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56be7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c0100e80 0x7f56c0197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56bf7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c0101820 0x7f56c0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56be7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c0100e80 0x7f56c0197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56954/0 (socket says 172.21.15.53:56954) 2023-11-06T01:44:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56be7fc700 1 -- 172.21.15.53:0/2338000015 learned_addr learned my addr 172.21.15.53:0/2338000015 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56beffd700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c00ffc90 0x7f56c0196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56be7fc700 1 -- 172.21.15.53:0/2338000015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c00ffc90 msgr2=0x7f56c0196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56be7fc700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c00ffc90 0x7f56c0196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56be7fc700 1 -- 172.21.15.53:0/2338000015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c0101820 msgr2=0x7f56c0197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56be7fc700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c0101820 0x7f56c0197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56be7fc700 1 -- 172.21.15.53:0/2338000015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4009520 con 0x7f56c0100e80 2023-11-06T01:44:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56be7fc700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c0100e80 0x7f56c0197220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f56b400eb30 tx=0x7f56b400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56a7fff700 1 -- 172.21.15.53:0/2338000015 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4009880 con 0x7f56c0100e80 2023-11-06T01:44:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56c53de700 1 -- 172.21.15.53:0/2338000015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b00095f0 con 0x7f56c0100e80 2023-11-06T01:44:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56c53de700 1 -- 172.21.15.53:0/2338000015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c0102280 con 0x7f56c0100e80 2023-11-06T01:44:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56a7fff700 1 -- 172.21.15.53:0/2338000015 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b400ce10 con 0x7f56c0100e80 2023-11-06T01:44:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.093+0000 7f56a7fff700 1 -- 172.21.15.53:0/2338000015 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4003cb0 con 0x7f56c0100e80 2023-11-06T01:44:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.108+0000 7f56d156eb80 1 Processor -- start 2023-11-06T01:44:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.112+0000 7f56a7fff700 1 -- 172.21.15.53:0/2338000015 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f56b400c950 con 0x7f56c0100e80 2023-11-06T01:44:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.112+0000 7f56a7fff700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56a80a05a0 0x7f56a80a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.113+0000 7f56a7fff700 1 -- 172.21.15.53:0/2338000015 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f56b4014070 con 0x7f56c0100e80 2023-11-06T01:44:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.114+0000 7f56d156eb80 1 -- start start 2023-11-06T01:44:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.114+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf47e90 0x558f1bf482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.114+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 0x558f1bf49100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.114+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf496e0 0x558f1bf49b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.114+0000 7f56d156eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1be979b0 con 0x558f1bf496e0 2023-11-06T01:44:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.114+0000 7f56d156eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1be95370 con 0x558f1bf487f0 2023-11-06T01:44:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.114+0000 7f56d156eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1be90c60 con 0x558f1bf47e90 2023-11-06T01:44:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.117+0000 7f56beffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf47e90 0x558f1bf482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.117+0000 7f56beffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf47e90 0x558f1bf482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56960/0 (socket says 172.21.15.53:56960) 2023-11-06T01:44:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.117+0000 7f56beffd700 1 -- 172.21.15.53:0/3420576768 learned_addr learned my addr 172.21.15.53:0/3420576768 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.118+0000 7f56beffd700 1 -- 172.21.15.53:0/3420576768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 msgr2=0x558f1bf49100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:44:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.118+0000 7f56beffd700 1 --2- 172.21.15.53:0/3420576768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 0x558f1bf49100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.118+0000 7f56beffd700 1 -- 172.21.15.53:0/3420576768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf496e0 msgr2=0x558f1bf49b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:44:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.118+0000 7f56beffd700 1 --2- 172.21.15.53:0/3420576768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf496e0 0x558f1bf49b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.118+0000 7f56beffd700 1 -- 172.21.15.53:0/3420576768 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f1bcbed80 con 0x558f1bf47e90 2023-11-06T01:44:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.118+0000 7f56beffd700 1 --2- 172.21.15.53:0/3420576768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf47e90 0x558f1bf482b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f56b0005950 tx=0x7f56b0005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c2721808975c55b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.118+0000 7f56a57fa700 1 -- 172.21.15.53:0/3420576768 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b001d030 con 0x558f1bf47e90 2023-11-06T01:44:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.119+0000 7f56a57fa700 1 -- 172.21.15.53:0/3420576768 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b001e040 con 0x558f1bf47e90 2023-11-06T01:44:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.119+0000 7f56a57fa700 1 -- 172.21.15.53:0/3420576768 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0003b50 con 0x558f1bf47e90 2023-11-06T01:44:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.119+0000 7f56d156eb80 1 -- 172.21.15.53:0/3420576768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf47e90 msgr2=0x558f1bf482b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.119+0000 7f56d156eb80 1 --2- 172.21.15.53:0/3420576768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf47e90 0x558f1bf482b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f56b0005950 tx=0x7f56b0005c00 comp rx=0 tx=0).stop 2023-11-06T01:44:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.124+0000 7f56bf7fe700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56a80a05a0 0x7f56a80a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.134+0000 7f56d156eb80 1 -- 172.21.15.53:0/3420576768 shutdown_connections 2023-11-06T01:44:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.134+0000 7f56d156eb80 1 --2- 172.21.15.53:0/3420576768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf496e0 0x558f1bf49b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.134+0000 7f56d156eb80 1 --2- 172.21.15.53:0/3420576768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf47e90 0x558f1bf482b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.134+0000 7f56d156eb80 1 --2- 172.21.15.53:0/3420576768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 0x558f1bf49100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.134+0000 7f56d156eb80 1 -- 172.21.15.53:0/3420576768 >> 172.21.15.53:0/3420576768 conn(0x558f1be9f070 msgr2=0x558f1bf3c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.134+0000 7f56bf7fe700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56a80a05a0 0x7f56a80a2a60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x558f1bf48ec0 tx=0x7f56b800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.141+0000 7f56d156eb80 1 -- 172.21.15.53:0/3420576768 shutdown_connections 2023-11-06T01:44:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.145+0000 7f56d156eb80 1 -- 172.21.15.53:0/3420576768 wait complete. 2023-11-06T01:44:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.145+0000 7f56d156eb80 1 -- start start 2023-11-06T01:44:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.145+0000 7f56d156eb80 1 Processor -- start 2023-11-06T01:44:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.145+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf47e90 0x558f1bf40b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.145+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf487f0 0x558f1bf410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.145+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf496e0 0x558f1bf46100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.145+0000 7f56d156eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1be2d4b0 con 0x558f1bf47e90 2023-11-06T01:44:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.145+0000 7f56d156eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1be2d150 con 0x558f1bf496e0 2023-11-06T01:44:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.145+0000 7f56d156eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1bd148c0 con 0x558f1bf487f0 2023-11-06T01:44:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.145+0000 7f56d156eb80 10 client.0 _ll_register_callbacks cb 0x558f1bd7aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.146+0000 7f56beffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf47e90 0x558f1bf40b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.146+0000 7f56beffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf47e90 0x558f1bf40b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42722/0 (socket says 172.21.15.53:42722) 2023-11-06T01:44:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.146+0000 7f56beffd700 1 -- 172.21.15.53:0/2789604700 learned_addr learned my addr 172.21.15.53:0/2789604700 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.146+0000 7f56be7fc700 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf487f0 0x558f1bf410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.146+0000 7f56beffd700 1 -- 172.21.15.53:0/2789604700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf487f0 msgr2=0x558f1bf410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.146+0000 7f56beffd700 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf487f0 0x558f1bf410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.146+0000 7f56beffd700 1 -- 172.21.15.53:0/2789604700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf496e0 msgr2=0x558f1bf46100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:44:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.146+0000 7f56beffd700 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf496e0 0x558f1bf46100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.146+0000 7f56beffd700 1 -- 172.21.15.53:0/2789604700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c0108a20 con 0x558f1bf47e90 2023-11-06T01:44:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.147+0000 7f56be7fc700 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf487f0 0x558f1bf410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.147+0000 7f56beffd700 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf47e90 0x558f1bf40b90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f56b000b580 tx=0x7f56b0003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d6f2429c1cd14e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.147+0000 7f56a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:44:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.147+0000 7f56a2ff5700 1 -- 172.21.15.53:0/2789604700 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b001d070 con 0x558f1bf47e90 2023-11-06T01:44:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.147+0000 7f56a2ff5700 1 -- 172.21.15.53:0/2789604700 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f56b0005250 con 0x558f1bf47e90 2023-11-06T01:44:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.148+0000 7f56d156eb80 1 -- 172.21.15.53:0/2789604700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f1bebbcb0 con 0x558f1bf47e90 2023-11-06T01:44:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.148+0000 7f56a2ff5700 1 -- 172.21.15.53:0/2789604700 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56b0005410 con 0x558f1bf47e90 2023-11-06T01:44:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.148+0000 7f56d156eb80 10 client.16563 fetch_fsmap learned FSMap version 44 2023-11-06T01:44:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.148+0000 7f56d156eb80 1 -- 172.21.15.53:0/2789604700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x558f1bca7c00 con 0x558f1bf47e90 2023-11-06T01:44:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.149+0000 7f56a2ff5700 1 -- 172.21.15.53:0/2789604700 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f1bebbcb0 con 0x558f1bf47e90 2023-11-06T01:44:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.149+0000 7f56a2ff5700 1 -- 172.21.15.53:0/2789604700 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f56b0033070 con 0x558f1bf47e90 2023-11-06T01:44:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.149+0000 7f56d156eb80 10 client.16563 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:44:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.149+0000 7f56d156eb80 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.149+0000 7f56d156eb80 20 client.16563 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.149+0000 7f56d156eb80 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558f1bf65cf0 0x558f1bf680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.149+0000 7f56d156eb80 4 client.16563 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.149+0000 7f56d156eb80 1 -- 172.21.15.53:0/2789604700 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f1bf0edb0 con 0x558f1bf65cf0 2023-11-06T01:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.149+0000 7f56be7fc700 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558f1bf65cf0 0x558f1bf680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.150+0000 7f56a2ff5700 10 client.16563 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.150+0000 7f56be7fc700 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558f1bf65cf0 0x558f1bf680e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52bdb104fca91e4 server_cookie=4d6fa663046c17a5 in_seq=0 out_seq=0 2023-11-06T01:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.150+0000 7f56a2ff5700 1 -- 172.21.15.53:0/2789604700 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558f1bf0edb0 con 0x558f1bf65cf0 2023-11-06T01:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.150+0000 7f56a2ff5700 10 client.16563 handle_command_reply: tid=0 2023-11-06T01:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.150+0000 7f56d156eb80 1 client.16563 shutdown 2023-11-06T01:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.150+0000 7f56d156eb80 1 -- 172.21.15.53:0/2789604700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf47e90 msgr2=0x558f1bf40b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.150+0000 7f56d156eb80 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf47e90 0x558f1bf40b90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f56b000b580 tx=0x7f56b0003b70 comp rx=0 tx=0).stop 2023-11-06T01:44:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.151+0000 7f56d156eb80 1 -- 172.21.15.53:0/2789604700 shutdown_connections 2023-11-06T01:44:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.151+0000 7f56d156eb80 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf496e0 0x558f1bf46100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.151+0000 7f56d156eb80 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558f1bf65cf0 0x558f1bf680e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.151+0000 7f56d156eb80 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf47e90 0x558f1bf40b90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.151+0000 7f56d156eb80 1 --2- 172.21.15.53:0/2789604700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf487f0 0x558f1bf410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.151+0000 7f56d156eb80 1 -- 172.21.15.53:0/2789604700 >> 172.21.15.53:0/2789604700 conn(0x558f1be9f070 msgr2=0x558f1bf3a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.151+0000 7f56a2ff5700 0 client.16563 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.151+0000 7f56d156eb80 1 -- 172.21.15.53:0/2789604700 shutdown_connections 2023-11-06T01:44:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.151+0000 7f56d156eb80 1 -- 172.21.15.53:0/2789604700 wait complete. 2023-11-06T01:44:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.152+0000 7f56d156eb80 20 client.16563 trim_cache size 0 max 16384 2023-11-06T01:44:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.170+0000 7f56d156eb80 1 Processor -- start 2023-11-06T01:44:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.171+0000 7f56d156eb80 1 -- start start 2023-11-06T01:44:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.171+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 0x558f1bf6ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.171+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bf755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.171+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf49ac0 0x558f1bf40240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.171+0000 7f56d156eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1bf47b50 con 0x558f1bf49ac0 2023-11-06T01:44:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.171+0000 7f56d156eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1bf40070 con 0x558f1bf487f0 2023-11-06T01:44:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.171+0000 7f56d156eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1bf0edb0 con 0x558f1bf496e0 2023-11-06T01:44:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56be7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bf755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56be7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bf755c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56970/0 (socket says 172.21.15.53:56970) 2023-11-06T01:44:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56be7fc700 1 -- 172.21.15.53:0/3501608423 learned_addr learned my addr 172.21.15.53:0/3501608423 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56bf7fe700 1 --2- 172.21.15.53:0/3501608423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf49ac0 0x558f1bf40240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56be7fc700 1 -- 172.21.15.53:0/3501608423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 msgr2=0x558f1bf6ca40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:44:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56be7fc700 1 --2- 172.21.15.53:0/3501608423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 0x558f1bf6ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56be7fc700 1 -- 172.21.15.53:0/3501608423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf49ac0 msgr2=0x558f1bf40240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56be7fc700 1 --2- 172.21.15.53:0/3501608423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf49ac0 0x558f1bf40240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56be7fc700 1 -- 172.21.15.53:0/3501608423 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f1bce51f0 con 0x558f1bf496e0 2023-11-06T01:44:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56bf7fe700 1 --2- 172.21.15.53:0/3501608423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf49ac0 0x558f1bf40240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:44:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56be7fc700 1 --2- 172.21.15.53:0/3501608423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bf755c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f56b40992b0 tx=0x7f56b400cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3069e3402e10b72 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56a5ffb700 1 -- 172.21.15.53:0/3501608423 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b409d3e0 con 0x558f1bf496e0 2023-11-06T01:44:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56a5ffb700 1 -- 172.21.15.53:0/3501608423 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b40a1620 con 0x558f1bf496e0 2023-11-06T01:44:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.172+0000 7f56a5ffb700 1 -- 172.21.15.53:0/3501608423 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b401fea0 con 0x558f1bf496e0 2023-11-06T01:44:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.173+0000 7f56d156eb80 1 -- 172.21.15.53:0/3501608423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 msgr2=0x558f1bf755c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.173+0000 7f56d156eb80 1 --2- 172.21.15.53:0/3501608423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bf755c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f56b40992b0 tx=0x7f56b400cf80 comp rx=0 tx=0).stop 2023-11-06T01:44:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.174+0000 7f56d156eb80 1 -- 172.21.15.53:0/3501608423 shutdown_connections 2023-11-06T01:44:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.174+0000 7f56d156eb80 1 --2- 172.21.15.53:0/3501608423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 0x558f1bf6ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.174+0000 7f56d156eb80 1 --2- 172.21.15.53:0/3501608423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf49ac0 0x558f1bf40240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.174+0000 7f56d156eb80 1 --2- 172.21.15.53:0/3501608423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bf755c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.174+0000 7f56d156eb80 1 -- 172.21.15.53:0/3501608423 >> 172.21.15.53:0/3501608423 conn(0x558f1be9f070 msgr2=0x558f1bf3f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.174+0000 7f56d156eb80 1 -- 172.21.15.53:0/3501608423 shutdown_connections 2023-11-06T01:44:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.174+0000 7f56d156eb80 1 -- 172.21.15.53:0/3501608423 wait complete. 2023-11-06T01:44:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.174+0000 7f56d156eb80 1 -- start start 2023-11-06T01:44:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.174+0000 7f56d156eb80 1 Processor -- start 2023-11-06T01:44:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 0x558f1bf735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bfc6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56d156eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf49ac0 0x558f1bfc9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56d156eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1bfc9a30 con 0x558f1bf49ac0 2023-11-06T01:44:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56d156eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1bfc9ba0 con 0x558f1bf487f0 2023-11-06T01:44:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56d156eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1bfc9d60 con 0x558f1bf496e0 2023-11-06T01:44:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56d156eb80 10 client.0 _ll_register_callbacks cb 0x7f56b0009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56be7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bfc6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56bf7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf49ac0 0x558f1bfc9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56be7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bfc6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:56980/0 (socket says 172.21.15.53:56980) 2023-11-06T01:44:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56be7fc700 1 -- 172.21.15.53:0/799980489 learned_addr learned my addr 172.21.15.53:0/799980489 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56beffd700 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 0x558f1bf735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56be7fc700 1 -- 172.21.15.53:0/799980489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 msgr2=0x558f1bf735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56be7fc700 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 0x558f1bf735b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56be7fc700 1 -- 172.21.15.53:0/799980489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf49ac0 msgr2=0x558f1bfc9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56be7fc700 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf49ac0 0x558f1bfc9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.175+0000 7f56be7fc700 1 -- 172.21.15.53:0/799980489 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c0102280 con 0x558f1bf496e0 2023-11-06T01:44:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56be7fc700 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bfc6d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f56b400eb00 tx=0x7f56b400cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38e979506c920072 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:44:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56a2ff5700 1 -- 172.21.15.53:0/799980489 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b409d3e0 con 0x558f1bf496e0 2023-11-06T01:44:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56a2ff5700 1 -- 172.21.15.53:0/799980489 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f56b4003e10 con 0x558f1bf496e0 2023-11-06T01:44:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56d156eb80 1 -- 172.21.15.53:0/799980489 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56b001e040 con 0x558f1bf496e0 2023-11-06T01:44:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56a2ff5700 1 -- 172.21.15.53:0/799980489 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56b409d070 con 0x558f1bf496e0 2023-11-06T01:44:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56d156eb80 10 client.26191 fetch_fsmap learned FSMap version 44 2023-11-06T01:44:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56d156eb80 1 -- 172.21.15.53:0/799980489 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x558f1bcbd250 con 0x558f1bf496e0 2023-11-06T01:44:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56a2ff5700 1 -- 172.21.15.53:0/799980489 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b001e040 con 0x558f1bf496e0 2023-11-06T01:44:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56a2ff5700 1 -- 172.21.15.53:0/799980489 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f56b4099810 con 0x558f1bf496e0 2023-11-06T01:44:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56d156eb80 10 client.26191 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:44:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56d156eb80 10 client.26191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56d156eb80 20 client.26191 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.176+0000 7f56d156eb80 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558f1bfcb510 0x558f1bfcd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.177+0000 7f56d156eb80 4 client.26191 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:44:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.177+0000 7f56d156eb80 1 -- 172.21.15.53:0/799980489 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558f1be89ce0 con 0x558f1bfcb510 2023-11-06T01:44:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.177+0000 7f56beffd700 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558f1bfcb510 0x558f1bfcd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.177+0000 7f56beffd700 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558f1bfcb510 0x558f1bfcd930 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec0ccdcbc9066396 server_cookie=b425c294e57f440c in_seq=0 out_seq=0 2023-11-06T01:44:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.177+0000 7f56a2ff5700 10 client.26191 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56a2ff5700 1 -- 172.21.15.53:0/799980489 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558f1be89ce0 con 0x558f1bfcb510 2023-11-06T01:44:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56a2ff5700 10 client.26191 handle_command_reply: tid=0 2023-11-06T01:44:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 1 client.26191 shutdown 2023-11-06T01:44:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 1 -- 172.21.15.53:0/799980489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 msgr2=0x558f1bfc6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bfc6d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f56b400eb00 tx=0x7f56b400cc00 comp rx=0 tx=0).stop 2023-11-06T01:44:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 1 -- 172.21.15.53:0/799980489 shutdown_connections 2023-11-06T01:44:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f1bf487f0 0x558f1bf735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558f1bfcb510 0x558f1bfcd930 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1bf49ac0 0x558f1bfc9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 1 --2- 172.21.15.53:0/799980489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558f1bf496e0 0x558f1bfc6d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56a2ff5700 0 client.26191 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 1 -- 172.21.15.53:0/799980489 >> 172.21.15.53:0/799980489 conn(0x558f1be9f070 msgr2=0x558f1bf4c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 1 -- 172.21.15.53:0/799980489 shutdown_connections 2023-11-06T01:44:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 1 -- 172.21.15.53:0/799980489 wait complete. 2023-11-06T01:44:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.188+0000 7f56d156eb80 20 client.26191 trim_cache size 0 max 16384 2023-11-06T01:44:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:44:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:44:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:44:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:44:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.189+0000 7f56c53de700 1 -- 172.21.15.53:0/2338000015 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56a80a05a0 msgr2=0x7f56a80a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.189+0000 7f56c53de700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56a80a05a0 0x7f56a80a2a60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x558f1bf48ec0 tx=0x7f56b800d040 comp rx=0 tx=0).stop 2023-11-06T01:44:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.189+0000 7f56c53de700 1 -- 172.21.15.53:0/2338000015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c0100e80 msgr2=0x7f56c0197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.189+0000 7f56c53de700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c0100e80 0x7f56c0197220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f56b400eb30 tx=0x7f56b400ee40 comp rx=0 tx=0).stop 2023-11-06T01:44:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.189+0000 7f56c53de700 1 -- 172.21.15.53:0/2338000015 shutdown_connections 2023-11-06T01:44:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.189+0000 7f56c53de700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f56a80a05a0 0x7f56a80a2a60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.189+0000 7f56c53de700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c00ffc90 0x7f56c0196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.189+0000 7f56c53de700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c0101820 0x7f56c0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.189+0000 7f56c53de700 1 --2- 172.21.15.53:0/2338000015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56c0100e80 0x7f56c0197220 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.189+0000 7f56c53de700 1 -- 172.21.15.53:0/2338000015 >> 172.21.15.53:0/2338000015 conn(0x7f56c00fb1e0 msgr2=0x7f56c00681a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.189+0000 7f56c53de700 1 -- 172.21.15.53:0/2338000015 shutdown_connections 2023-11-06T01:44:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:09.190+0000 7f56c53de700 1 -- 172.21.15.53:0/2338000015 wait complete. 2023-11-06T01:44:09.282 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:44:09.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.556+0000 7ff825a08700 1 -- 172.21.15.53:0/2707503474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff820101860 msgr2=0x7ff820103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.556+0000 7ff825a08700 1 --2- 172.21.15.53:0/2707503474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff820101860 0x7ff820103c50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff810009990 tx=0x7ff810009ca0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.556+0000 7ff825a08700 1 -- 172.21.15.53:0/2707503474 shutdown_connections 2023-11-06T01:44:09.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.556+0000 7ff825a08700 1 --2- 172.21.15.53:0/2707503474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff820106ac0 0x7ff820108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.556+0000 7ff825a08700 1 --2- 172.21.15.53:0/2707503474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff820101860 0x7ff820103c50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.556+0000 7ff825a08700 1 --2- 172.21.15.53:0/2707503474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff820104190 0x7ff820106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.556+0000 7ff825a08700 1 -- 172.21.15.53:0/2707503474 >> 172.21.15.53:0/2707503474 conn(0x7ff8200fb1f0 msgr2=0x7ff8200fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.556+0000 7ff825a08700 1 -- 172.21.15.53:0/2707503474 shutdown_connections 2023-11-06T01:44:09.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.556+0000 7ff825a08700 1 -- 172.21.15.53:0/2707503474 wait complete. 2023-11-06T01:44:09.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.556+0000 7ff825a08700 1 Processor -- start 2023-11-06T01:44:09.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff825a08700 1 -- start start 2023-11-06T01:44:09.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff825a08700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff820101860 0x7ff8201011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff825a08700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff820104190 0x7ff8200ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff825a08700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff820106ac0 0x7ff8200ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff825a08700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201004f0 con 0x7ff820101860 2023-11-06T01:44:09.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff825a08700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff820100660 con 0x7ff820104190 2023-11-06T01:44:09.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff825a08700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201a0b90 con 0x7ff820106ac0 2023-11-06T01:44:09.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff81effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff820101860 0x7ff8201011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff81effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff820101860 0x7ff8201011d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42748/0 (socket says 172.21.15.53:42748) 2023-11-06T01:44:09.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff81effd700 1 -- 172.21.15.53:0/1082939980 learned_addr learned my addr 172.21.15.53:0/1082939980 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:09.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff81e7fc700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff820104190 0x7ff8200ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff81effd700 1 -- 172.21.15.53:0/1082939980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff820106ac0 msgr2=0x7ff8200ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff81effd700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff820106ac0 0x7ff8200ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff81effd700 1 -- 172.21.15.53:0/1082939980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff820104190 msgr2=0x7ff8200ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff81effd700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff820104190 0x7ff8200ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff81effd700 1 -- 172.21.15.53:0/1082939980 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100095f0 con 0x7ff820101860 2023-11-06T01:44:09.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.557+0000 7ff81effd700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff820101860 0x7ff8201011d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff810004730 tx=0x7ff810004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.558+0000 7ff824a06700 1 -- 172.21.15.53:0/1082939980 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81001d070 con 0x7ff820101860 2023-11-06T01:44:09.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.558+0000 7ff825a08700 1 -- 172.21.15.53:0/1082939980 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8201a0e10 con 0x7ff820101860 2023-11-06T01:44:09.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.558+0000 7ff824a06700 1 -- 172.21.15.53:0/1082939980 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff810004c90 con 0x7ff820101860 2023-11-06T01:44:09.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.558+0000 7ff824a06700 1 -- 172.21.15.53:0/1082939980 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8100054e0 con 0x7ff820101860 2023-11-06T01:44:09.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.558+0000 7ff825a08700 1 -- 172.21.15.53:0/1082939980 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8201a1250 con 0x7ff820101860 2023-11-06T01:44:09.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.560+0000 7ff824a06700 1 -- 172.21.15.53:0/1082939980 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff81000fbf0 con 0x7ff820101860 2023-11-06T01:44:09.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.560+0000 7ff824a06700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8080a0520 0x7ff8080a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.560+0000 7ff824a06700 1 -- 172.21.15.53:0/1082939980 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff8100ccc60 con 0x7ff820101860 2023-11-06T01:44:09.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.560+0000 7ff81e7fc700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8080a0520 0x7ff8080a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff81e7fc700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8080a0520 0x7ff8080a29e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff814009a10 tx=0x7ff814009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff831b98b80 1 Processor -- start 2023-11-06T01:44:09.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff831b98b80 1 -- start start 2023-11-06T01:44:09.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f635e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635ea70 0x5564f635ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f635d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff831b98b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f62b4a10 con 0x5564f635e0f0 2023-11-06T01:44:09.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff831b98b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f62b23d0 con 0x5564f635d3f0 2023-11-06T01:44:09.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff831b98b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f62adcc0 con 0x5564f635ea70 2023-11-06T01:44:09.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff81effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f635e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff81effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f635e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42756/0 (socket says 172.21.15.53:42756) 2023-11-06T01:44:09.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff81effd700 1 -- 172.21.15.53:0/3078495135 learned_addr learned my addr 172.21.15.53:0/3078495135 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:09.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.561+0000 7ff81f7fe700 1 --2- 172.21.15.53:0/3078495135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f635d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.562+0000 7ff81effd700 1 -- 172.21.15.53:0/3078495135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635ea70 msgr2=0x5564f635ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.562+0000 7ff81effd700 1 --2- 172.21.15.53:0/3078495135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635ea70 0x5564f635ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.562+0000 7ff81effd700 1 -- 172.21.15.53:0/3078495135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 msgr2=0x5564f635d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.562+0000 7ff81effd700 1 --2- 172.21.15.53:0/3078495135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f635d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.562+0000 7ff81effd700 1 -- 172.21.15.53:0/3078495135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564f60dbde0 con 0x5564f635e0f0 2023-11-06T01:44:09.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.562+0000 7ff81f7fe700 1 --2- 172.21.15.53:0/3078495135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f635d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:09.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.562+0000 7ff81effd700 1 --2- 172.21.15.53:0/3078495135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f635e530 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff81009a230 tx=0x7ff810009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968cae2e01ce4944 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.562+0000 7ff805ffb700 1 -- 172.21.15.53:0/3078495135 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81009e170 con 0x5564f635e0f0 2023-11-06T01:44:09.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.562+0000 7ff805ffb700 1 -- 172.21.15.53:0/3078495135 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff810096040 con 0x5564f635e0f0 2023-11-06T01:44:09.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.562+0000 7ff805ffb700 1 -- 172.21.15.53:0/3078495135 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810005750 con 0x5564f635e0f0 2023-11-06T01:44:09.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.563+0000 7ff831b98b80 1 -- 172.21.15.53:0/3078495135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 msgr2=0x5564f635e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.563+0000 7ff831b98b80 1 --2- 172.21.15.53:0/3078495135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f635e530 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff81009a230 tx=0x7ff810009ca0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.563+0000 7ff831b98b80 1 -- 172.21.15.53:0/3078495135 shutdown_connections 2023-11-06T01:44:09.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.563+0000 7ff831b98b80 1 --2- 172.21.15.53:0/3078495135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f635d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.563+0000 7ff831b98b80 1 --2- 172.21.15.53:0/3078495135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f635e530 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.563+0000 7ff831b98b80 1 --2- 172.21.15.53:0/3078495135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635ea70 0x5564f635ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.563+0000 7ff831b98b80 1 -- 172.21.15.53:0/3078495135 >> 172.21.15.53:0/3078495135 conn(0x5564f62bc0d0 msgr2=0x5564f6357940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.563+0000 7ff831b98b80 1 -- 172.21.15.53:0/3078495135 shutdown_connections 2023-11-06T01:44:09.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.563+0000 7ff831b98b80 1 -- 172.21.15.53:0/3078495135 wait complete. 2023-11-06T01:44:09.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.563+0000 7ff831b98b80 1 -- start start 2023-11-06T01:44:09.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.563+0000 7ff831b98b80 1 Processor -- start 2023-11-06T01:44:09.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635d3f0 0x5564f6366700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635e0f0 0x5564f6366c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635ea70 0x5564f636dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff831b98b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f624a510 con 0x5564f635d3f0 2023-11-06T01:44:09.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff831b98b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f624a1b0 con 0x5564f635ea70 2023-11-06T01:44:09.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff831b98b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f6131920 con 0x5564f635e0f0 2023-11-06T01:44:09.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff831b98b80 10 client.0 _ll_register_callbacks cb 0x5564f6197a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:09.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff81effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635d3f0 0x5564f6366700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff81effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635d3f0 0x5564f6366700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42762/0 (socket says 172.21.15.53:42762) 2023-11-06T01:44:09.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff81effd700 1 -- 172.21.15.53:0/146630312 learned_addr learned my addr 172.21.15.53:0/146630312 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:09.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff81f7fe700 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635ea70 0x5564f636dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff81effd700 1 -- 172.21.15.53:0/146630312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635e0f0 msgr2=0x5564f6366c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff81effd700 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635e0f0 0x5564f6366c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff81effd700 1 -- 172.21.15.53:0/146630312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635ea70 msgr2=0x5564f636dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff81effd700 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635ea70 0x5564f636dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff81effd700 1 -- 172.21.15.53:0/146630312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81800e140 con 0x5564f635d3f0 2023-11-06T01:44:09.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff81effd700 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635d3f0 0x5564f6366700 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff8100a26d0 tx=0x7ff810009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5e13afa345d21c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff7feffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:44:09.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff7feffd700 1 -- 172.21.15.53:0/146630312 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810099c20 con 0x5564f635d3f0 2023-11-06T01:44:09.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff7feffd700 1 -- 172.21.15.53:0/146630312 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff81009e360 con 0x5564f635d3f0 2023-11-06T01:44:09.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff831b98b80 1 -- 172.21.15.53:0/146630312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564f62d8d10 con 0x5564f635d3f0 2023-11-06T01:44:09.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff7feffd700 1 -- 172.21.15.53:0/146630312 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff810005730 con 0x5564f635d3f0 2023-11-06T01:44:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff831b98b80 10 client.16581 fetch_fsmap learned FSMap version 44 2023-11-06T01:44:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.564+0000 7ff831b98b80 1 -- 172.21.15.53:0/146630312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7ff8201a1250 con 0x5564f635d3f0 2023-11-06T01:44:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.565+0000 7ff7feffd700 1 -- 172.21.15.53:0/146630312 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564f62d8d10 con 0x5564f635d3f0 2023-11-06T01:44:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.565+0000 7ff7feffd700 1 -- 172.21.15.53:0/146630312 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff8100a21a0 con 0x5564f635d3f0 2023-11-06T01:44:09.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.565+0000 7ff831b98b80 10 client.16581 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:44:09.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.565+0000 7ff831b98b80 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:09.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.565+0000 7ff831b98b80 20 client.16581 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:09.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.565+0000 7ff831b98b80 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5564f6372050 0x5564f6374470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.565+0000 7ff831b98b80 4 client.16581 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:44:09.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.565+0000 7ff831b98b80 1 -- 172.21.15.53:0/146630312 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564f632be10 con 0x5564f6372050 2023-11-06T01:44:09.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.565+0000 7ff81e7fc700 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5564f6372050 0x5564f6374470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.567+0000 7ff7feffd700 10 client.16581 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:09.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.567+0000 7ff81e7fc700 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5564f6372050 0x5564f6374470 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2f23881ad2a0b0 server_cookie=bc8b38c7be4f9067 in_seq=0 out_seq=0 2023-11-06T01:44:09.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff7feffd700 1 -- 172.21.15.53:0/146630312 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5564f632be10 con 0x5564f6372050 2023-11-06T01:44:09.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff7feffd700 10 client.16581 handle_command_reply: tid=0 2023-11-06T01:44:09.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 1 client.16581 shutdown 2023-11-06T01:44:09.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 1 -- 172.21.15.53:0/146630312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635d3f0 msgr2=0x5564f6366700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635d3f0 0x5564f6366700 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff8100a26d0 tx=0x7ff810009310 comp rx=0 tx=0).stop 2023-11-06T01:44:09.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 1 -- 172.21.15.53:0/146630312 shutdown_connections 2023-11-06T01:44:09.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635ea70 0x5564f636dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5564f6372050 0x5564f6374470 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635d3f0 0x5564f6366700 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff7feffd700 0 client.16581 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:09.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 1 --2- 172.21.15.53:0/146630312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635e0f0 0x5564f6366c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 1 -- 172.21.15.53:0/146630312 >> 172.21.15.53:0/146630312 conn(0x5564f62bc0d0 msgr2=0x5564f63570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 1 -- 172.21.15.53:0/146630312 shutdown_connections 2023-11-06T01:44:09.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 1 -- 172.21.15.53:0/146630312 wait complete. 2023-11-06T01:44:09.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.569+0000 7ff831b98b80 20 client.16581 trim_cache size 0 max 16384 2023-11-06T01:44:09.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff831b98b80 1 Processor -- start 2023-11-06T01:44:09.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff831b98b80 1 -- start start 2023-11-06T01:44:09.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f635ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f636bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635ea70 0x5564f63664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff831b98b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f63594d0 con 0x5564f635e0f0 2023-11-06T01:44:09.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff831b98b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f62adcc0 con 0x5564f635d3f0 2023-11-06T01:44:09.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff831b98b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f62b4a10 con 0x5564f635ea70 2023-11-06T01:44:09.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff81e7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f636bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff81e7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f636bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42772/0 (socket says 172.21.15.53:42772) 2023-11-06T01:44:09.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff81effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f635ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff81e7fc700 1 -- 172.21.15.53:0/734160981 learned_addr learned my addr 172.21.15.53:0/734160981 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:09.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff81f7fe700 1 --2- 172.21.15.53:0/734160981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635ea70 0x5564f63664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff81e7fc700 1 -- 172.21.15.53:0/734160981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635ea70 msgr2=0x5564f63664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff81e7fc700 1 --2- 172.21.15.53:0/734160981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635ea70 0x5564f63664b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff81e7fc700 1 -- 172.21.15.53:0/734160981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 msgr2=0x5564f635ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff81e7fc700 1 --2- 172.21.15.53:0/734160981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f635ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff81e7fc700 1 -- 172.21.15.53:0/734160981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564f60c4c60 con 0x5564f635e0f0 2023-11-06T01:44:09.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.589+0000 7ff81effd700 1 --2- 172.21.15.53:0/734160981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f635ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:09.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.590+0000 7ff81e7fc700 1 --2- 172.21.15.53:0/734160981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f636bc30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff814006d20 tx=0x7ff814004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ea2d83b6fc1aa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.590+0000 7ff8067fc700 1 -- 172.21.15.53:0/734160981 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400ede0 con 0x5564f635e0f0 2023-11-06T01:44:09.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.590+0000 7ff8067fc700 1 -- 172.21.15.53:0/734160981 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff81401b040 con 0x5564f635e0f0 2023-11-06T01:44:09.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.590+0000 7ff8067fc700 1 -- 172.21.15.53:0/734160981 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814015480 con 0x5564f635e0f0 2023-11-06T01:44:09.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 -- 172.21.15.53:0/734160981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 msgr2=0x5564f636bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 --2- 172.21.15.53:0/734160981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f636bc30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff814006d20 tx=0x7ff814004750 comp rx=0 tx=0).stop 2023-11-06T01:44:09.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 -- 172.21.15.53:0/734160981 shutdown_connections 2023-11-06T01:44:09.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 --2- 172.21.15.53:0/734160981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f635ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 --2- 172.21.15.53:0/734160981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635ea70 0x5564f63664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 --2- 172.21.15.53:0/734160981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635e0f0 0x5564f636bc30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 -- 172.21.15.53:0/734160981 >> 172.21.15.53:0/734160981 conn(0x5564f62bc0d0 msgr2=0x5564f63613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 -- 172.21.15.53:0/734160981 shutdown_connections 2023-11-06T01:44:09.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 -- 172.21.15.53:0/734160981 wait complete. 2023-11-06T01:44:09.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 -- start start 2023-11-06T01:44:09.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 Processor -- start 2023-11-06T01:44:09.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f63bbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635e0f0 0x5564f63bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635ea70 0x5564f63c3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f6367490 con 0x5564f635ea70 2023-11-06T01:44:09.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f6358240 con 0x5564f635d3f0 2023-11-06T01:44:09.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.591+0000 7ff831b98b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f63c3920 con 0x5564f635e0f0 2023-11-06T01:44:09.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff831b98b80 10 client.0 _ll_register_callbacks cb 0x7ff818009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:09.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635ea70 0x5564f63c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f63bbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635ea70 0x5564f63c3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42780/0 (socket says 172.21.15.53:42780) 2023-11-06T01:44:09.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81f7fe700 1 -- 172.21.15.53:0/280055110 learned_addr learned my addr 172.21.15.53:0/280055110 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:09.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f63bbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54214/0 (socket says 172.21.15.53:54214) 2023-11-06T01:44:09.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81e7fc700 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635e0f0 0x5564f63bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81f7fe700 1 -- 172.21.15.53:0/280055110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635e0f0 msgr2=0x5564f63bc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81f7fe700 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635e0f0 0x5564f63bc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81f7fe700 1 -- 172.21.15.53:0/280055110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 msgr2=0x5564f63bbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:09.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81f7fe700 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f63bbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81f7fe700 1 -- 172.21.15.53:0/280055110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8201a1250 con 0x5564f635ea70 2023-11-06T01:44:09.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff81f7fe700 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635ea70 0x5564f63c3390 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff820101830 tx=0x7ff8180049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb9ac41ced4d5e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:09.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff7feffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:44:09.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff7feffd700 1 -- 172.21.15.53:0/280055110 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818004db0 con 0x5564f635ea70 2023-11-06T01:44:09.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff7feffd700 1 -- 172.21.15.53:0/280055110 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff818016930 con 0x5564f635ea70 2023-11-06T01:44:09.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff831b98b80 1 -- 172.21.15.53:0/280055110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff810096040 con 0x5564f635ea70 2023-11-06T01:44:09.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff7feffd700 1 -- 172.21.15.53:0/280055110 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8180093a0 con 0x5564f635ea70 2023-11-06T01:44:09.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff831b98b80 10 client.16587 fetch_fsmap learned FSMap version 44 2023-11-06T01:44:09.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.592+0000 7ff831b98b80 1 -- 172.21.15.53:0/280055110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7ff814009660 con 0x5564f635ea70 2023-11-06T01:44:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.593+0000 7ff7feffd700 1 -- 172.21.15.53:0/280055110 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff810096040 con 0x5564f635ea70 2023-11-06T01:44:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.593+0000 7ff7feffd700 1 -- 172.21.15.53:0/280055110 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff818027050 con 0x5564f635ea70 2023-11-06T01:44:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.593+0000 7ff831b98b80 10 client.16587 fetch_fsmap finished waiting for FSMap version 44 2023-11-06T01:44:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.593+0000 7ff831b98b80 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.593+0000 7ff831b98b80 20 client.16587 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.593+0000 7ff831b98b80 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5564f63c5410 0x5564f63c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.593+0000 7ff831b98b80 4 client.16587 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:44:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.593+0000 7ff831b98b80 1 -- 172.21.15.53:0/280055110 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5564f62a6d40 con 0x5564f63c5410 2023-11-06T01:44:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.593+0000 7ff81effd700 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5564f63c5410 0x5564f63c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:09.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.597+0000 7ff7feffd700 10 client.16587 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:09.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.597+0000 7ff81effd700 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5564f63c5410 0x5564f63c7830 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13dbd3ef3aff27d server_cookie=84772d27c6909756 in_seq=0 out_seq=0 2023-11-06T01:44:09.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff7feffd700 1 -- 172.21.15.53:0/280055110 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5564f62a6d40 con 0x5564f63c5410 2023-11-06T01:44:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff7feffd700 10 client.16587 handle_command_reply: tid=0 2023-11-06T01:44:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 1 client.16587 shutdown 2023-11-06T01:44:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 1 -- 172.21.15.53:0/280055110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635ea70 msgr2=0x5564f63c3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635ea70 0x5564f63c3390 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff820101830 tx=0x7ff8180049c0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 1 -- 172.21.15.53:0/280055110 shutdown_connections 2023-11-06T01:44:09.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564f635d3f0 0x5564f63bbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5564f63c5410 0x5564f63c7830 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564f635ea70 0x5564f63c3390 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff7feffd700 0 client.16587 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:09.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 1 --2- 172.21.15.53:0/280055110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f635e0f0 0x5564f63bc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 1 -- 172.21.15.53:0/280055110 >> 172.21.15.53:0/280055110 conn(0x5564f62bc0d0 msgr2=0x5564f63664b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 1 -- 172.21.15.53:0/280055110 shutdown_connections 2023-11-06T01:44:09.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 1 -- 172.21.15.53:0/280055110 wait complete. 2023-11-06T01:44:09.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.602+0000 7ff831b98b80 20 client.16587 trim_cache size 0 max 16384 2023-11-06T01:44:09.614 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:44:09.614 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:44:09.614 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:44:09.614 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:44:09.615 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:44:09.615 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:44:09.615 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:44:09.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:44:09.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:44:09.615 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:44:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 -- 172.21.15.53:0/1082939980 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8080a0520 msgr2=0x7ff8080a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8080a0520 0x7ff8080a29e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff814009a10 tx=0x7ff814009190 comp rx=0 tx=0).stop 2023-11-06T01:44:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 -- 172.21.15.53:0/1082939980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff820101860 msgr2=0x7ff8201011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff820101860 0x7ff8201011d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff810004730 tx=0x7ff810004760 comp rx=0 tx=0).stop 2023-11-06T01:44:09.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 -- 172.21.15.53:0/1082939980 shutdown_connections 2023-11-06T01:44:09.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8080a0520 0x7ff8080a29e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff820106ac0 0x7ff8200ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff820101860 0x7ff8201011d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 --2- 172.21.15.53:0/1082939980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff820104190 0x7ff8200ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:09.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 -- 172.21.15.53:0/1082939980 >> 172.21.15.53:0/1082939980 conn(0x7ff8200fb1f0 msgr2=0x7ff8201024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:09.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 -- 172.21.15.53:0/1082939980 shutdown_connections 2023-11-06T01:44:09.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:09.603+0000 7ff825a08700 1 -- 172.21.15.53:0/1082939980 wait complete. 2023-11-06T01:44:09.619 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:44:09.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:09 smithi062 ceph-mon[103924]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 719 op/s 2023-11-06T01:44:09.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:09 smithi163 ceph-mon[104416]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 719 op/s 2023-11-06T01:44:09.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:09 smithi053 ceph-mon[99472]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 719 op/s 2023-11-06T01:44:11.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:11 smithi163 ceph-mon[104416]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 769 op/s 2023-11-06T01:44:11.845 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:11 smithi053 ceph-mon[99472]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 769 op/s 2023-11-06T01:44:11.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:11 smithi062 ceph-mon[103924]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 769 op/s 2023-11-06T01:44:12.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 595.0 s, 1547.6 tps, lat 64.665 ms stddev 79.106 2023-11-06T01:44:13.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:13 smithi163 ceph-mon[104416]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 10 MiB/s wr, 766 op/s 2023-11-06T01:44:13.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:13 smithi053 ceph-mon[99472]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 10 MiB/s wr, 766 op/s 2023-11-06T01:44:13.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:13 smithi062 ceph-mon[103924]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 10 MiB/s wr, 766 op/s 2023-11-06T01:44:15.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:15 smithi163 ceph-mon[104416]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 MiB/s wr, 829 op/s 2023-11-06T01:44:15.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:15 smithi053 ceph-mon[99472]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 MiB/s wr, 829 op/s 2023-11-06T01:44:15.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:15 smithi062 ceph-mon[103924]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 MiB/s wr, 829 op/s 2023-11-06T01:44:17.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 600.0 s, 1374.0 tps, lat 73.768 ms stddev 91.773 2023-11-06T01:44:17.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:17 smithi163 ceph-mon[104416]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 814 op/s 2023-11-06T01:44:17.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:17 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:44:17.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:17 smithi053 ceph-mon[99472]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 814 op/s 2023-11-06T01:44:17.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:17 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:44:17.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:17 smithi062 ceph-mon[103924]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 814 op/s 2023-11-06T01:44:17.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:17 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:44:18.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:18 smithi163 ceph-mon[104416]: scrub summary: idle+waiting paths [/] 2023-11-06T01:44:18.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:18 smithi053 ceph-mon[99472]: scrub summary: idle+waiting paths [/] 2023-11-06T01:44:18.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:18 smithi062 ceph-mon[103924]: scrub summary: idle+waiting paths [/] 2023-11-06T01:44:19.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:19 smithi163 ceph-mon[104416]: scrub summary: active paths [/] 2023-11-06T01:44:19.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:19 smithi163 ceph-mon[104416]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 812 op/s 2023-11-06T01:44:19.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:19 smithi053 ceph-mon[99472]: scrub summary: active paths [/] 2023-11-06T01:44:19.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:19 smithi053 ceph-mon[99472]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 812 op/s 2023-11-06T01:44:19.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:19 smithi062 ceph-mon[103924]: scrub summary: active paths [/] 2023-11-06T01:44:19.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:19 smithi062 ceph-mon[103924]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 812 op/s 2023-11-06T01:44:21.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:21 smithi163 ceph-mon[104416]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 795 op/s 2023-11-06T01:44:21.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:21 smithi053 ceph-mon[99472]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 795 op/s 2023-11-06T01:44:21.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:21 smithi062 ceph-mon[103924]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 795 op/s 2023-11-06T01:44:22.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 605.0 s, 1619.2 tps, lat 62.099 ms stddev 71.218 2023-11-06T01:44:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.358+0000 7f55b44e5700 1 -- 172.21.15.53:0/1670986976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ac0ffcc0 msgr2=0x7f55ac1000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.358+0000 7f55b44e5700 1 --2- 172.21.15.53:0/1670986976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ac0ffcc0 0x7f55ac1000e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009990 tx=0x7f55a8009ca0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.358+0000 7f55b44e5700 1 -- 172.21.15.53:0/1670986976 shutdown_connections 2023-11-06T01:44:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.358+0000 7f55b44e5700 1 --2- 172.21.15.53:0/1670986976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ac0ffcc0 0x7f55ac1000e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.358+0000 7f55b44e5700 1 --2- 172.21.15.53:0/1670986976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55ac101850 0x7f55ac068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.358+0000 7f55b44e5700 1 --2- 172.21.15.53:0/1670986976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55ac100eb0 0x7f55ac101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.358+0000 7f55b44e5700 1 -- 172.21.15.53:0/1670986976 >> 172.21.15.53:0/1670986976 conn(0x7f55ac0fb210 msgr2=0x7f55ac0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.358+0000 7f55b44e5700 1 -- 172.21.15.53:0/1670986976 shutdown_connections 2023-11-06T01:44:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.358+0000 7f55b44e5700 1 -- 172.21.15.53:0/1670986976 wait complete. 2023-11-06T01:44:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.359+0000 7f55b44e5700 1 Processor -- start 2023-11-06T01:44:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b44e5700 1 -- start start 2023-11-06T01:44:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b44e5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55ac0ffcc0 0x7f55ac19d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b44e5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ac100eb0 0x7f55ac19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b44e5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55ac101850 0x7f55ac19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b44e5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac19ec50 con 0x7f55ac0ffcc0 2023-11-06T01:44:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b44e5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac19edc0 con 0x7f55ac100eb0 2023-11-06T01:44:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b44e5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac19ef80 con 0x7f55ac101850 2023-11-06T01:44:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b1a80700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ac100eb0 0x7f55ac19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b2a82700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55ac101850 0x7f55ac19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b1a80700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ac100eb0 0x7f55ac19d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54228/0 (socket says 172.21.15.53:54228) 2023-11-06T01:44:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b1a80700 1 -- 172.21.15.53:0/1066211446 learned_addr learned my addr 172.21.15.53:0/1066211446 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b2a82700 1 -- 172.21.15.53:0/1066211446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ac100eb0 msgr2=0x7f55ac19d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b2a82700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ac100eb0 0x7f55ac19d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b2a82700 1 -- 172.21.15.53:0/1066211446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55ac0ffcc0 msgr2=0x7f55ac19d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b2281700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55ac0ffcc0 0x7f55ac19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b2a82700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55ac0ffcc0 0x7f55ac19d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.360+0000 7f55b2a82700 1 -- 172.21.15.53:0/1066211446 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a80095f0 con 0x7f55ac101850 2023-11-06T01:44:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.361+0000 7f55b1a80700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ac100eb0 0x7f55ac19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.361+0000 7f55b2281700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55ac0ffcc0 0x7f55ac19d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:44:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.361+0000 7f55b2a82700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55ac101850 0x7f55ac19e5f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f55a400efc0 tx=0x7f55a400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.361+0000 7f55a37fe700 1 -- 172.21.15.53:0/1066211446 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4018070 con 0x7f55ac101850 2023-11-06T01:44:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.361+0000 7f55a37fe700 1 -- 172.21.15.53:0/1066211446 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55a400f040 con 0x7f55ac101850 2023-11-06T01:44:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.361+0000 7f55b44e5700 1 -- 172.21.15.53:0/1066211446 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ac19f200 con 0x7f55ac101850 2023-11-06T01:44:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.361+0000 7f55a37fe700 1 -- 172.21.15.53:0/1066211446 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400cdf0 con 0x7f55ac101850 2023-11-06T01:44:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.361+0000 7f55b44e5700 1 -- 172.21.15.53:0/1066211446 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ac06aa10 con 0x7f55ac101850 2023-11-06T01:44:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.364+0000 7f55a37fe700 1 -- 172.21.15.53:0/1066211446 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f55a4003a00 con 0x7f55ac101850 2023-11-06T01:44:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.365+0000 7f55a37fe700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f55980a0520 0x7f55980a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.365+0000 7f55a37fe700 1 -- 172.21.15.53:0/1066211446 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f55a40cb650 con 0x7f55ac101850 2023-11-06T01:44:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.365+0000 7f55b2281700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f55980a0520 0x7f55980a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.365+0000 7f55c0675b80 1 Processor -- start 2023-11-06T01:44:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.365+0000 7f55b2281700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f55980a0520 0x7f55980a29e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f55a8000c00 tx=0x7f55a8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.365+0000 7f55c0675b80 1 -- start start 2023-11-06T01:44:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 0x55800997caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x55800997d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x55800997e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55c0675b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580098d49b0 con 0x55800997ded0 2023-11-06T01:44:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55c0675b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580098d2370 con 0x55800997cfe0 2023-11-06T01:44:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55c0675b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580098cdc60 con 0x55800997c680 2023-11-06T01:44:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55b2a82700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x55800997e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55b2281700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 0x55800997caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55b1a80700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x55800997d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55b2281700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 0x55800997caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57038/0 (socket says 172.21.15.53:57038) 2023-11-06T01:44:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55b2a82700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x55800997e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42802/0 (socket says 172.21.15.53:42802) 2023-11-06T01:44:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.366+0000 7f55b2281700 1 -- 172.21.15.53:0/726232042 learned_addr learned my addr 172.21.15.53:0/726232042 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.367+0000 7f55b2a82700 1 -- 172.21.15.53:0/726232042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 msgr2=0x55800997caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.367+0000 7f55b2a82700 1 --2- 172.21.15.53:0/726232042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 0x55800997caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.367+0000 7f55b2a82700 1 -- 172.21.15.53:0/726232042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 msgr2=0x55800997d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.367+0000 7f55b2a82700 1 --2- 172.21.15.53:0/726232042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x55800997d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.367+0000 7f55b2a82700 1 -- 172.21.15.53:0/726232042 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580096fbd80 con 0x55800997ded0 2023-11-06T01:44:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.367+0000 7f55b1a80700 1 --2- 172.21.15.53:0/726232042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x55800997d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.367+0000 7f55b2a82700 1 --2- 172.21.15.53:0/726232042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x55800997e350 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f55a400c7a0 tx=0x7f55a4098a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2be84da0910a924 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.368+0000 7f55a0ff9700 1 -- 172.21.15.53:0/726232042 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a401b320 con 0x55800997ded0 2023-11-06T01:44:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.368+0000 7f55a0ff9700 1 -- 172.21.15.53:0/726232042 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55a4099620 con 0x55800997ded0 2023-11-06T01:44:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.368+0000 7f55a0ff9700 1 -- 172.21.15.53:0/726232042 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a40993e0 con 0x55800997ded0 2023-11-06T01:44:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.368+0000 7f55c0675b80 1 -- 172.21.15.53:0/726232042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 msgr2=0x55800997e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.368+0000 7f55c0675b80 1 --2- 172.21.15.53:0/726232042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x55800997e350 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f55a400c7a0 tx=0x7f55a4098a60 comp rx=0 tx=0).stop 2023-11-06T01:44:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.368+0000 7f55c0675b80 1 -- 172.21.15.53:0/726232042 shutdown_connections 2023-11-06T01:44:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.368+0000 7f55c0675b80 1 --2- 172.21.15.53:0/726232042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x55800997e350 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.368+0000 7f55c0675b80 1 --2- 172.21.15.53:0/726232042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 0x55800997caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.368+0000 7f55c0675b80 1 --2- 172.21.15.53:0/726232042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x55800997d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.368+0000 7f55c0675b80 1 -- 172.21.15.53:0/726232042 >> 172.21.15.53:0/726232042 conn(0x5580098dc070 msgr2=0x558009979040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55c0675b80 1 -- 172.21.15.53:0/726232042 shutdown_connections 2023-11-06T01:44:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55c0675b80 1 -- 172.21.15.53:0/726232042 wait complete. 2023-11-06T01:44:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55c0675b80 1 -- start start 2023-11-06T01:44:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55c0675b80 1 Processor -- start 2023-11-06T01:44:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997c680 0x558009986580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x558009986ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997ded0 0x55800998db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55c0675b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55800986a4b0 con 0x55800997c680 2023-11-06T01:44:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55c0675b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55800986a150 con 0x55800997cfe0 2023-11-06T01:44:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55c0675b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580097518c0 con 0x55800997ded0 2023-11-06T01:44:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55c0675b80 10 client.0 _ll_register_callbacks cb 0x5580097b7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55b2281700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997c680 0x558009986580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55b2281700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997c680 0x558009986580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42804/0 (socket says 172.21.15.53:42804) 2023-11-06T01:44:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55b2281700 1 -- 172.21.15.53:0/1133256507 learned_addr learned my addr 172.21.15.53:0/1133256507 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55b1a80700 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x558009986ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.369+0000 7f55b2a82700 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997ded0 0x55800998db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.370+0000 7f55b2a82700 1 -- 172.21.15.53:0/1133256507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 msgr2=0x558009986ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.370+0000 7f55b2a82700 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x558009986ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.370+0000 7f55b2a82700 1 -- 172.21.15.53:0/1133256507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997c680 msgr2=0x558009986580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.370+0000 7f55b2a82700 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997c680 0x558009986580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.370+0000 7f55b2a82700 1 -- 172.21.15.53:0/1133256507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c009ae0 con 0x55800997ded0 2023-11-06T01:44:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.370+0000 7f55b2281700 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997c680 0x558009986580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.370+0000 7f55b2a82700 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997ded0 0x55800998db00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x558009984be0 tx=0x7f55a4099790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad33d96799586d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.370+0000 7f5595ffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:44:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.370+0000 7f5595ffb700 1 -- 172.21.15.53:0/1133256507 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a40b5070 con 0x55800997ded0 2023-11-06T01:44:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.370+0000 7f5595ffb700 1 -- 172.21.15.53:0/1133256507 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f55a4098f20 con 0x55800997ded0 2023-11-06T01:44:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.370+0000 7f55c0675b80 1 -- 172.21.15.53:0/1133256507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580098f8cb0 con 0x55800997ded0 2023-11-06T01:44:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.371+0000 7f5595ffb700 1 -- 172.21.15.53:0/1133256507 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55a401b320 con 0x55800997ded0 2023-11-06T01:44:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.371+0000 7f55c0675b80 10 client.26206 fetch_fsmap learned FSMap version 46 2023-11-06T01:44:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.371+0000 7f55c0675b80 1 -- 172.21.15.53:0/1133256507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f55ac06aa10 con 0x55800997ded0 2023-11-06T01:44:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.372+0000 7f5595ffb700 1 -- 172.21.15.53:0/1133256507 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580098f8cb0 con 0x55800997ded0 2023-11-06T01:44:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.372+0000 7f5595ffb700 1 -- 172.21.15.53:0/1133256507 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f55a4097e80 con 0x55800997ded0 2023-11-06T01:44:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.372+0000 7f55c0675b80 10 client.26206 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:44:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.372+0000 7f55c0675b80 10 client.26206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.372+0000 7f55c0675b80 20 client.26206 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.372+0000 7f55c0675b80 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558009991e30 0x558009994220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.372+0000 7f55c0675b80 4 client.26206 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:44:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.372+0000 7f55c0675b80 1 -- 172.21.15.53:0/1133256507 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55800994bdb0 con 0x558009991e30 2023-11-06T01:44:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.372+0000 7f55b2281700 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558009991e30 0x558009994220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.373+0000 7f5595ffb700 10 client.26206 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.373+0000 7f55b2281700 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558009991e30 0x558009994220 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80aa55925b468f0 server_cookie=49ae4cdefdb9a7a3 in_seq=0 out_seq=0 2023-11-06T01:44:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.373+0000 7f5595ffb700 1 -- 172.21.15.53:0/1133256507 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55800994bdb0 con 0x558009991e30 2023-11-06T01:44:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.373+0000 7f5595ffb700 10 client.26206 handle_command_reply: tid=0 2023-11-06T01:44:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.373+0000 7f55c0675b80 1 client.26206 shutdown 2023-11-06T01:44:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.374+0000 7f55c0675b80 1 -- 172.21.15.53:0/1133256507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997ded0 msgr2=0x55800998db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.374+0000 7f55c0675b80 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997ded0 0x55800998db00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x558009984be0 tx=0x7f55a4099790 comp rx=0 tx=0).stop 2023-11-06T01:44:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.375+0000 7f55c0675b80 1 -- 172.21.15.53:0/1133256507 shutdown_connections 2023-11-06T01:44:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.375+0000 7f55c0675b80 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x558009991e30 0x558009994220 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.375+0000 7f55c0675b80 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997ded0 0x55800998db00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.375+0000 7f55c0675b80 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997c680 0x558009986580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.375+0000 7f55c0675b80 1 --2- 172.21.15.53:0/1133256507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x558009986ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.375+0000 7f55c0675b80 1 -- 172.21.15.53:0/1133256507 >> 172.21.15.53:0/1133256507 conn(0x5580098dc070 msgr2=0x5580099773c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.375+0000 7f5595ffb700 0 client.26206 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.375+0000 7f55c0675b80 1 -- 172.21.15.53:0/1133256507 shutdown_connections 2023-11-06T01:44:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.375+0000 7f55c0675b80 1 -- 172.21.15.53:0/1133256507 wait complete. 2023-11-06T01:44:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.375+0000 7f55c0675b80 20 client.26206 trim_cache size 0 max 16384 2023-11-06T01:44:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.411+0000 7f55c0675b80 1 Processor -- start 2023-11-06T01:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.411+0000 7f55c0675b80 1 -- start start 2023-11-06T01:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.411+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 0x55800998bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.411+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x5580099810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.411+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x558009985140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.411+0000 7f55c0675b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55800996f7a0 con 0x55800997ded0 2023-11-06T01:44:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.411+0000 7f55c0675b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558009972500 con 0x55800997cfe0 2023-11-06T01:44:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.411+0000 7f55c0675b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580099860f0 con 0x55800997c680 2023-11-06T01:44:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.412+0000 7f55b2a82700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x558009985140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.412+0000 7f55b2a82700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x558009985140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42816/0 (socket says 172.21.15.53:42816) 2023-11-06T01:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.412+0000 7f55b2a82700 1 -- 172.21.15.53:0/3891488776 learned_addr learned my addr 172.21.15.53:0/3891488776 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.412+0000 7f55b2a82700 1 -- 172.21.15.53:0/3891488776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 msgr2=0x55800998bab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.412+0000 7f55b2a82700 1 --2- 172.21.15.53:0/3891488776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 0x55800998bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.412+0000 7f55b2a82700 1 -- 172.21.15.53:0/3891488776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 msgr2=0x5580099810a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:44:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.412+0000 7f55b2a82700 1 --2- 172.21.15.53:0/3891488776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x5580099810a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.412+0000 7f55b2a82700 1 -- 172.21.15.53:0/3891488776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580096e4c00 con 0x55800997ded0 2023-11-06T01:44:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.413+0000 7f55b2a82700 1 --2- 172.21.15.53:0/3891488776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x558009985140 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f55a40c94a0 tx=0x7f55a4098a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc76577848fb64f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.413+0000 7f55a17fa700 1 -- 172.21.15.53:0/3891488776 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a401b320 con 0x55800997ded0 2023-11-06T01:44:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.413+0000 7f55a17fa700 1 -- 172.21.15.53:0/3891488776 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55a4098f20 con 0x55800997ded0 2023-11-06T01:44:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.413+0000 7f55a17fa700 1 -- 172.21.15.53:0/3891488776 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a409dbe0 con 0x55800997ded0 2023-11-06T01:44:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.414+0000 7f55c0675b80 1 -- 172.21.15.53:0/3891488776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 msgr2=0x558009985140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.414+0000 7f55c0675b80 1 --2- 172.21.15.53:0/3891488776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x558009985140 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f55a40c94a0 tx=0x7f55a4098a60 comp rx=0 tx=0).stop 2023-11-06T01:44:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.414+0000 7f55c0675b80 1 -- 172.21.15.53:0/3891488776 shutdown_connections 2023-11-06T01:44:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.414+0000 7f55c0675b80 1 --2- 172.21.15.53:0/3891488776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x558009985140 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.414+0000 7f55c0675b80 1 --2- 172.21.15.53:0/3891488776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 0x55800998bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.414+0000 7f55c0675b80 1 --2- 172.21.15.53:0/3891488776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x5580099810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.414+0000 7f55c0675b80 1 -- 172.21.15.53:0/3891488776 >> 172.21.15.53:0/3891488776 conn(0x5580098dc070 msgr2=0x558009984360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.414+0000 7f55c0675b80 1 -- 172.21.15.53:0/3891488776 shutdown_connections 2023-11-06T01:44:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.414+0000 7f55c0675b80 1 -- 172.21.15.53:0/3891488776 wait complete. 2023-11-06T01:44:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.415+0000 7f55c0675b80 1 -- start start 2023-11-06T01:44:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.415+0000 7f55c0675b80 1 Processor -- start 2023-11-06T01:44:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.415+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 0x55800998b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.415+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x5580099f0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.415+0000 7f55c0675b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x5580099f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.415+0000 7f55c0675b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580099f3470 con 0x55800997ded0 2023-11-06T01:44:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.415+0000 7f55c0675b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580099f35e0 con 0x55800997cfe0 2023-11-06T01:44:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.415+0000 7f55c0675b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580099f37a0 con 0x55800997c680 2023-11-06T01:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.415+0000 7f55c0675b80 10 client.0 _ll_register_callbacks cb 0x7f559c0098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.416+0000 7f55b2a82700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x5580099f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.416+0000 7f55b2a82700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x5580099f2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42820/0 (socket says 172.21.15.53:42820) 2023-11-06T01:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.416+0000 7f55b2a82700 1 -- 172.21.15.53:0/2450320771 learned_addr learned my addr 172.21.15.53:0/2450320771 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.416+0000 7f55b2a82700 1 -- 172.21.15.53:0/2450320771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 msgr2=0x55800998b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.416+0000 7f55b2a82700 1 --2- 172.21.15.53:0/2450320771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 0x55800998b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.416+0000 7f55b2a82700 1 -- 172.21.15.53:0/2450320771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 msgr2=0x5580099f0860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.416+0000 7f55b2a82700 1 --2- 172.21.15.53:0/2450320771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x5580099f0860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.416+0000 7f55b2a82700 1 -- 172.21.15.53:0/2450320771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ac06aa10 con 0x55800997ded0 2023-11-06T01:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.416+0000 7f55b2a82700 1 --2- 172.21.15.53:0/2450320771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x5580099f2db0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f55a4098350 tx=0x7f55a40972c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2bea8a49b87119 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.416+0000 7f5595ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.416+0000 7f5595ffb700 1 -- 172.21.15.53:0/2450320771 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a40b5070 con 0x55800997ded0 2023-11-06T01:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.417+0000 7f5595ffb700 1 -- 172.21.15.53:0/2450320771 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f55a409cd50 con 0x55800997ded0 2023-11-06T01:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.417+0000 7f55c0675b80 1 -- 172.21.15.53:0/2450320771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55a4099620 con 0x55800997ded0 2023-11-06T01:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.417+0000 7f5595ffb700 1 -- 172.21.15.53:0/2450320771 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55a401b320 con 0x55800997ded0 2023-11-06T01:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.417+0000 7f55c0675b80 10 client.16602 fetch_fsmap learned FSMap version 46 2023-11-06T01:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.417+0000 7f55c0675b80 1 -- 172.21.15.53:0/2450320771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5580097221f0 con 0x55800997ded0 2023-11-06T01:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.419+0000 7f5595ffb700 1 -- 172.21.15.53:0/2450320771 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55a4099620 con 0x55800997ded0 2023-11-06T01:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.419+0000 7f5595ffb700 1 -- 172.21.15.53:0/2450320771 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f55a40b80e0 con 0x55800997ded0 2023-11-06T01:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.419+0000 7f55c0675b80 10 client.16602 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.419+0000 7f55c0675b80 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.419+0000 7f55c0675b80 20 client.16602 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.420+0000 7f55c0675b80 1 --2- 172.21.15.53:0/2450320771 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5580099f4bc0 0x5580099f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.420+0000 7f55c0675b80 4 client.16602 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.420+0000 7f55c0675b80 1 -- 172.21.15.53:0/2450320771 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5580098c6ce0 con 0x5580099f4bc0 2023-11-06T01:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.422+0000 7f55b2281700 1 --2- 172.21.15.53:0/2450320771 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5580099f4bc0 0x5580099f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.423+0000 7f5595ffb700 10 client.16602 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.423+0000 7f55b2281700 1 --2- 172.21.15.53:0/2450320771 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5580099f4bc0 0x5580099f6fe0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f3e2c4438bde9d server_cookie=a5b85a4d3d81b53 in_seq=0 out_seq=0 2023-11-06T01:44:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.448+0000 7f5595ffb700 1 -- 172.21.15.53:0/2450320771 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5580098c6ce0 con 0x5580099f4bc0 2023-11-06T01:44:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.448+0000 7f5595ffb700 10 client.16602 handle_command_reply: tid=0 2023-11-06T01:44:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.448+0000 7f55c0675b80 1 client.16602 shutdown 2023-11-06T01:44:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.448+0000 7f55c0675b80 1 -- 172.21.15.53:0/2450320771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 msgr2=0x5580099f2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.448+0000 7f55c0675b80 1 --2- 172.21.15.53:0/2450320771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x5580099f2db0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f55a4098350 tx=0x7f55a40972c0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.448+0000 7f55c0675b80 1 -- 172.21.15.53:0/2450320771 shutdown_connections 2023-11-06T01:44:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.448+0000 7f55c0675b80 1 --2- 172.21.15.53:0/2450320771 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5580099f4bc0 0x5580099f6fe0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.448+0000 7f55c0675b80 1 --2- 172.21.15.53:0/2450320771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800997ded0 0x5580099f2db0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.448+0000 7f55c0675b80 1 --2- 172.21.15.53:0/2450320771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55800997c680 0x55800998b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.448+0000 7f5595ffb700 0 client.16602 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.448+0000 7f55c0675b80 1 --2- 172.21.15.53:0/2450320771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55800997cfe0 0x5580099f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55c0675b80 1 -- 172.21.15.53:0/2450320771 >> 172.21.15.53:0/2450320771 conn(0x5580098dc070 msgr2=0x5580099810a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55c0675b80 1 -- 172.21.15.53:0/2450320771 shutdown_connections 2023-11-06T01:44:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55c0675b80 1 -- 172.21.15.53:0/2450320771 wait complete. 2023-11-06T01:44:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55c0675b80 20 client.16602 trim_cache size 0 max 16384 2023-11-06T01:44:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:44:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:44:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:44:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:44:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 -- 172.21.15.53:0/1066211446 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f55980a0520 msgr2=0x7f55980a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f55980a0520 0x7f55980a29e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f55a8000c00 tx=0x7f55a8005fd0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 -- 172.21.15.53:0/1066211446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55ac101850 msgr2=0x7f55ac19e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55ac101850 0x7f55ac19e5f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f55a400efc0 tx=0x7f55a400c3a0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 -- 172.21.15.53:0/1066211446 shutdown_connections 2023-11-06T01:44:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f55980a0520 0x7f55980a29e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55ac101850 0x7f55ac19e5f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55ac0ffcc0 0x7f55ac19d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 --2- 172.21.15.53:0/1066211446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55ac100eb0 0x7f55ac19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 -- 172.21.15.53:0/1066211446 >> 172.21.15.53:0/1066211446 conn(0x7f55ac0fb210 msgr2=0x7f55ac0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 -- 172.21.15.53:0/1066211446 shutdown_connections 2023-11-06T01:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:22.449+0000 7f55b44e5700 1 -- 172.21.15.53:0/1066211446 wait complete. 2023-11-06T01:44:23.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:23 smithi053 ceph-mon[99472]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 812 op/s 2023-11-06T01:44:23.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:23 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:44:23.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:23 smithi062 ceph-mon[103924]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 812 op/s 2023-11-06T01:44:23.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:23 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:44:23.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:23 smithi163 ceph-mon[104416]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 812 op/s 2023-11-06T01:44:23.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:23 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:44:25.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:25 smithi053 ceph-mon[99472]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 804 op/s 2023-11-06T01:44:25.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:25 smithi062 ceph-mon[103924]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 804 op/s 2023-11-06T01:44:25.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:25 smithi163 ceph-mon[104416]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 804 op/s 2023-11-06T01:44:27.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 610.0 s, 1742.1 tps, lat 57.535 ms stddev 66.273 2023-11-06T01:44:27.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:27 smithi053 ceph-mon[99472]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 740 op/s 2023-11-06T01:44:27.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:27 smithi062 ceph-mon[103924]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 740 op/s 2023-11-06T01:44:27.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:27 smithi163 ceph-mon[104416]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 740 op/s 2023-11-06T01:44:29.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:29 smithi053 ceph-mon[99472]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 690 op/s 2023-11-06T01:44:29.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:29 smithi062 ceph-mon[103924]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 690 op/s 2023-11-06T01:44:29.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:29 smithi163 ceph-mon[104416]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 690 op/s 2023-11-06T01:44:31.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:31 smithi062 ceph-mon[103924]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 678 op/s 2023-11-06T01:44:31.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:31 smithi163 ceph-mon[104416]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 678 op/s 2023-11-06T01:44:31.846 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:31 smithi053 ceph-mon[99472]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 678 op/s 2023-11-06T01:44:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 615.0 s, 1052.2 tps, lat 90.206 ms stddev 152.683 2023-11-06T01:44:33.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:33 smithi062 ceph-mon[103924]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 734 op/s 2023-11-06T01:44:33.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:33 smithi163 ceph-mon[104416]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 734 op/s 2023-11-06T01:44:33.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:33 smithi053 ceph-mon[99472]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 734 op/s 2023-11-06T01:44:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.637+0000 7fa02b215700 1 -- 172.21.15.53:0/2355794337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0241041b0 msgr2=0x7fa0241065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.637+0000 7fa02b215700 1 --2- 172.21.15.53:0/2355794337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0241041b0 0x7fa0241065a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa010009990 tx=0x7fa010009ca0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.637+0000 7fa02b215700 1 -- 172.21.15.53:0/2355794337 shutdown_connections 2023-11-06T01:44:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.637+0000 7fa02b215700 1 --2- 172.21.15.53:0/2355794337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa024106ae0 0x7fa024108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.637+0000 7fa02b215700 1 --2- 172.21.15.53:0/2355794337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa024101880 0x7fa024103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.637+0000 7fa02b215700 1 --2- 172.21.15.53:0/2355794337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0241041b0 0x7fa0241065a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.637+0000 7fa02b215700 1 -- 172.21.15.53:0/2355794337 >> 172.21.15.53:0/2355794337 conn(0x7fa0240fb1f0 msgr2=0x7fa0240fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.637+0000 7fa02b215700 1 -- 172.21.15.53:0/2355794337 shutdown_connections 2023-11-06T01:44:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.637+0000 7fa02b215700 1 -- 172.21.15.53:0/2355794337 wait complete. 2023-11-06T01:44:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.638+0000 7fa02b215700 1 Processor -- start 2023-11-06T01:44:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.638+0000 7fa02b215700 1 -- start start 2023-11-06T01:44:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa02b215700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa024101880 0x7fa024194910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa02b215700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0241041b0 0x7fa024194e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa02b215700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa024106ae0 0x7fa024195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa02b215700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa024196380 con 0x7fa0241041b0 2023-11-06T01:44:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa02b215700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241964f0 con 0x7fa024101880 2023-11-06T01:44:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa02b215700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241966b0 con 0x7fa024106ae0 2023-11-06T01:44:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa028fb1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa024101880 0x7fa024194910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa028fb1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa024101880 0x7fa024194910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54270/0 (socket says 172.21.15.53:54270) 2023-11-06T01:44:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa028fb1700 1 -- 172.21.15.53:0/1551473499 learned_addr learned my addr 172.21.15.53:0/1551473499 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa028fb1700 1 -- 172.21.15.53:0/1551473499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa024106ae0 msgr2=0x7fa024195d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:44:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa028fb1700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa024106ae0 0x7fa024195d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa028fb1700 1 -- 172.21.15.53:0/1551473499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0241041b0 msgr2=0x7fa024194e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa028fb1700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0241041b0 0x7fa024194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa028fb1700 1 -- 172.21.15.53:0/1551473499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa018009520 con 0x7fa024101880 2023-11-06T01:44:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.639+0000 7fa028fb1700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa024101880 0x7fa024194910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa01800eb30 tx=0x7fa01800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.640+0000 7fa021ffb700 1 -- 172.21.15.53:0/1551473499 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01800cd40 con 0x7fa024101880 2023-11-06T01:44:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.640+0000 7fa021ffb700 1 -- 172.21.15.53:0/1551473499 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01800cea0 con 0x7fa024101880 2023-11-06T01:44:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.640+0000 7fa021ffb700 1 -- 172.21.15.53:0/1551473499 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018010430 con 0x7fa024101880 2023-11-06T01:44:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.641+0000 7fa02b215700 1 -- 172.21.15.53:0/1551473499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0100095f0 con 0x7fa024101880 2023-11-06T01:44:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.641+0000 7fa02b215700 1 -- 172.21.15.53:0/1551473499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02419cf90 con 0x7fa024101880 2023-11-06T01:44:35.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:35 smithi062 ceph-mon[103924]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.39k op/s 2023-11-06T01:44:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.643+0000 7fa021ffb700 1 -- 172.21.15.53:0/1551473499 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa018003680 con 0x7fa024101880 2023-11-06T01:44:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.643+0000 7fa021ffb700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0140a05a0 0x7fa0140a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.644+0000 7fa021ffb700 1 -- 172.21.15.53:0/1551473499 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa018014070 con 0x7fa024101880 2023-11-06T01:44:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.644+0000 7fa023fff700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0140a05a0 0x7fa0140a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.644+0000 7fa023fff700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0140a05a0 0x7fa0140a2a60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa01000b680 tx=0x7fa010009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.645+0000 7fa0373a5b80 1 Processor -- start 2023-11-06T01:44:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.646+0000 7fa0373a5b80 1 -- start start 2023-11-06T01:44:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.646+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661680 0x5561d3661aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.646+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 0x5561d36628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.646+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3662ed0 0x5561d3663350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.646+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d35b99b0 con 0x5561d3662ed0 2023-11-06T01:44:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.646+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d35b7370 con 0x5561d3661fe0 2023-11-06T01:44:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.646+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d35b2c60 con 0x5561d3661680 2023-11-06T01:44:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.647+0000 7fa0297b2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3662ed0 0x5561d3663350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.647+0000 7fa0297b2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3662ed0 0x5561d3663350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42842/0 (socket says 172.21.15.53:42842) 2023-11-06T01:44:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.647+0000 7fa0297b2700 1 -- 172.21.15.53:0/11280171 learned_addr learned my addr 172.21.15.53:0/11280171 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.647+0000 7fa028fb1700 1 --2- 172.21.15.53:0/11280171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661680 0x5561d3661aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.647+0000 7fa023fff700 1 --2- 172.21.15.53:0/11280171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 0x5561d36628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.648+0000 7fa028fb1700 1 -- 172.21.15.53:0/11280171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 msgr2=0x5561d36628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.648+0000 7fa028fb1700 1 --2- 172.21.15.53:0/11280171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 0x5561d36628f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.648+0000 7fa028fb1700 1 -- 172.21.15.53:0/11280171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3662ed0 msgr2=0x5561d3663350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.648+0000 7fa028fb1700 1 --2- 172.21.15.53:0/11280171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3662ed0 0x5561d3663350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.648+0000 7fa028fb1700 1 -- 172.21.15.53:0/11280171 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561d33e0d80 con 0x5561d3661680 2023-11-06T01:44:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.648+0000 7fa023fff700 1 --2- 172.21.15.53:0/11280171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 0x5561d36628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.648+0000 7fa028fb1700 1 --2- 172.21.15.53:0/11280171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661680 0x5561d3661aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa01800eb00 tx=0x7fa01801fe20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a14f05b810e126b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.648+0000 7fa0297b2700 1 --2- 172.21.15.53:0/11280171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3662ed0 0x5561d3663350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.648+0000 7fa00effd700 1 -- 172.21.15.53:0/11280171 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01809d400 con 0x5561d3661680 2023-11-06T01:44:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.648+0000 7fa00effd700 1 -- 172.21.15.53:0/11280171 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0180a15b0 con 0x5561d3661680 2023-11-06T01:44:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.648+0000 7fa00effd700 1 -- 172.21.15.53:0/11280171 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018099830 con 0x5561d3661680 2023-11-06T01:44:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.649+0000 7fa0373a5b80 1 -- 172.21.15.53:0/11280171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661680 msgr2=0x5561d3661aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.649+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/11280171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661680 0x5561d3661aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa01800eb00 tx=0x7fa01801fe20 comp rx=0 tx=0).stop 2023-11-06T01:44:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.649+0000 7fa0373a5b80 1 -- 172.21.15.53:0/11280171 shutdown_connections 2023-11-06T01:44:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.649+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/11280171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3662ed0 0x5561d3663350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.649+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/11280171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661680 0x5561d3661aa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.649+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/11280171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 0x5561d36628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.649+0000 7fa0373a5b80 1 -- 172.21.15.53:0/11280171 >> 172.21.15.53:0/11280171 conn(0x5561d35c1070 msgr2=0x5561d365e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.649+0000 7fa0373a5b80 1 -- 172.21.15.53:0/11280171 shutdown_connections 2023-11-06T01:44:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.649+0000 7fa0373a5b80 1 -- 172.21.15.53:0/11280171 wait complete. 2023-11-06T01:44:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.650+0000 7fa0373a5b80 1 -- start start 2023-11-06T01:44:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.650+0000 7fa0373a5b80 1 Processor -- start 2023-11-06T01:44:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.650+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 0x5561d366b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.650+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 0x5561d366ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.650+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 0x5561d3672ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.650+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d354f4b0 con 0x5561d3661680 2023-11-06T01:44:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.650+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d354f150 con 0x5561d3661fe0 2023-11-06T01:44:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.650+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d34368c0 con 0x5561d3662ed0 2023-11-06T01:44:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa0373a5b80 10 client.0 _ll_register_callbacks cb 0x5561d349ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa023fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 0x5561d366ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa028fb1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 0x5561d366b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa023fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 0x5561d366ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54282/0 (socket says 172.21.15.53:54282) 2023-11-06T01:44:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa023fff700 1 -- 172.21.15.53:0/1571226655 learned_addr learned my addr 172.21.15.53:0/1571226655 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa0297b2700 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 0x5561d3672ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa023fff700 1 -- 172.21.15.53:0/1571226655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 msgr2=0x5561d3672ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa023fff700 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 0x5561d3672ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa023fff700 1 -- 172.21.15.53:0/1571226655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 msgr2=0x5561d366b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa023fff700 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 0x5561d366b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa023fff700 1 -- 172.21.15.53:0/1571226655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa024109ae0 con 0x5561d3661fe0 2023-11-06T01:44:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa023fff700 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 0x5561d366ba70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa010009990 tx=0x7fa01001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a64a8db83891fc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa003fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:44:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa003fff700 1 -- 172.21.15.53:0/1571226655 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01002b040 con 0x5561d3661fe0 2023-11-06T01:44:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa003fff700 1 -- 172.21.15.53:0/1571226655 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa01000bcb0 con 0x5561d3661fe0 2023-11-06T01:44:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.651+0000 7fa0373a5b80 1 -- 172.21.15.53:0/1571226655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561d35ddcb0 con 0x5561d3661fe0 2023-11-06T01:44:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa003fff700 1 -- 172.21.15.53:0/1571226655 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa01001d070 con 0x5561d3661fe0 2023-11-06T01:44:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa0373a5b80 10 client.26156 fetch_fsmap learned FSMap version 46 2023-11-06T01:44:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa0373a5b80 1 -- 172.21.15.53:0/1571226655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7fa01c00e140 con 0x5561d3661fe0 2023-11-06T01:44:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa003fff700 1 -- 172.21.15.53:0/1571226655 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561d35ddcb0 con 0x5561d3661fe0 2023-11-06T01:44:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa003fff700 1 -- 172.21.15.53:0/1571226655 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa010038020 con 0x5561d3661fe0 2023-11-06T01:44:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa0373a5b80 10 client.26156 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:44:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa0373a5b80 10 client.26156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa0373a5b80 20 client.26156 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5561d3677360 0x5561d3679780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa0373a5b80 4 client.26156 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:44:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa0373a5b80 1 -- 172.21.15.53:0/1571226655 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561d3630db0 con 0x5561d3677360 2023-11-06T01:44:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.652+0000 7fa028fb1700 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5561d3677360 0x5561d3679780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.653+0000 7fa003fff700 10 client.26156 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.653+0000 7fa028fb1700 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5561d3677360 0x5561d3679780 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9faf3a2ad3aa8ac server_cookie=3625a5eceacaf859 in_seq=0 out_seq=0 2023-11-06T01:44:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.653+0000 7fa003fff700 1 -- 172.21.15.53:0/1571226655 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5561d3630db0 con 0x5561d3677360 2023-11-06T01:44:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.653+0000 7fa003fff700 10 client.26156 handle_command_reply: tid=0 2023-11-06T01:44:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.653+0000 7fa0373a5b80 1 client.26156 shutdown 2023-11-06T01:44:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa0373a5b80 1 -- 172.21.15.53:0/1571226655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 msgr2=0x5561d366ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 0x5561d366ba70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa010009990 tx=0x7fa01001d000 comp rx=0 tx=0).stop 2023-11-06T01:44:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa0373a5b80 1 -- 172.21.15.53:0/1571226655 shutdown_connections 2023-11-06T01:44:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5561d3677360 0x5561d3679780 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 0x5561d3672ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 0x5561d366b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/1571226655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661fe0 0x5561d366ba70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa003fff700 0 client.26156 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa0373a5b80 1 -- 172.21.15.53:0/1571226655 >> 172.21.15.53:0/1571226655 conn(0x5561d35c1070 msgr2=0x5561d365c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa0373a5b80 1 -- 172.21.15.53:0/1571226655 shutdown_connections 2023-11-06T01:44:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa0373a5b80 1 -- 172.21.15.53:0/1571226655 wait complete. 2023-11-06T01:44:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.654+0000 7fa0373a5b80 20 client.26156 trim_cache size 0 max 16384 2023-11-06T01:44:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.674+0000 7fa0373a5b80 1 Processor -- start 2023-11-06T01:44:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0373a5b80 1 -- start start 2023-11-06T01:44:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661680 0x5561d36709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661fe0 0x5561d366a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 0x5561d365b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d3654f30 con 0x5561d3661fe0 2023-11-06T01:44:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d3630db0 con 0x5561d3661680 2023-11-06T01:44:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d36547a0 con 0x5561d3662ed0 2023-11-06T01:44:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0297b2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 0x5561d365b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0297b2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 0x5561d365b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57094/0 (socket says 172.21.15.53:57094) 2023-11-06T01:44:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0297b2700 1 -- 172.21.15.53:0/3491557914 learned_addr learned my addr 172.21.15.53:0/3491557914 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa023fff700 1 --2- 172.21.15.53:0/3491557914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661fe0 0x5561d366a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa028fb1700 1 --2- 172.21.15.53:0/3491557914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661680 0x5561d36709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0297b2700 1 -- 172.21.15.53:0/3491557914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661680 msgr2=0x5561d36709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0297b2700 1 --2- 172.21.15.53:0/3491557914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661680 0x5561d36709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0297b2700 1 -- 172.21.15.53:0/3491557914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661fe0 msgr2=0x5561d366a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.675+0000 7fa0297b2700 1 --2- 172.21.15.53:0/3491557914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661fe0 0x5561d366a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.676+0000 7fa0297b2700 1 -- 172.21.15.53:0/3491557914 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02419cf90 con 0x5561d3662ed0 2023-11-06T01:44:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.676+0000 7fa0297b2700 1 --2- 172.21.15.53:0/3491557914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 0x5561d365b660 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa01c007500 tx=0x7fa01c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d47d9203ae1cb2d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.676+0000 7fa00f7fe700 1 -- 172.21.15.53:0/3491557914 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c038610 con 0x5561d3662ed0 2023-11-06T01:44:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.676+0000 7fa00f7fe700 1 -- 172.21.15.53:0/3491557914 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01c00f040 con 0x5561d3662ed0 2023-11-06T01:44:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.676+0000 7fa00f7fe700 1 -- 172.21.15.53:0/3491557914 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c037af0 con 0x5561d3662ed0 2023-11-06T01:44:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.677+0000 7fa0373a5b80 1 -- 172.21.15.53:0/3491557914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 msgr2=0x5561d365b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.677+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/3491557914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 0x5561d365b660 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa01c007500 tx=0x7fa01c00b040 comp rx=0 tx=0).stop 2023-11-06T01:44:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.677+0000 7fa0373a5b80 1 -- 172.21.15.53:0/3491557914 shutdown_connections 2023-11-06T01:44:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.677+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/3491557914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3661680 0x5561d36709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.677+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/3491557914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3662ed0 0x5561d365b660 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.677+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/3491557914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661fe0 0x5561d366a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.677+0000 7fa0373a5b80 1 -- 172.21.15.53:0/3491557914 >> 172.21.15.53:0/3491557914 conn(0x5561d35c1070 msgr2=0x5561d36693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.677+0000 7fa0373a5b80 1 -- 172.21.15.53:0/3491557914 shutdown_connections 2023-11-06T01:44:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.677+0000 7fa0373a5b80 1 -- 172.21.15.53:0/3491557914 wait complete. 2023-11-06T01:44:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.678+0000 7fa0373a5b80 1 -- start start 2023-11-06T01:44:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.678+0000 7fa0373a5b80 1 Processor -- start 2023-11-06T01:44:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.678+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 0x5561d36c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.678+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661fe0 0x5561d36c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.678+0000 7fa0373a5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3662ed0 0x5561d36c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.678+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d36c8ae0 con 0x5561d3661680 2023-11-06T01:44:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.678+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d36c8c20 con 0x5561d3662ed0 2023-11-06T01:44:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.678+0000 7fa0373a5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d36c8db0 con 0x5561d3661fe0 2023-11-06T01:44:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.678+0000 7fa0373a5b80 10 client.0 _ll_register_callbacks cb 0x7fa010005360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.678+0000 7fa028fb1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 0x5561d36c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa028fb1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 0x5561d36c1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42858/0 (socket says 172.21.15.53:42858) 2023-11-06T01:44:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa028fb1700 1 -- 172.21.15.53:0/528145065 learned_addr learned my addr 172.21.15.53:0/528145065 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa023fff700 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661fe0 0x5561d36c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa023fff700 1 -- 172.21.15.53:0/528145065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3662ed0 msgr2=0x5561d36c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa023fff700 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3662ed0 0x5561d36c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa023fff700 1 -- 172.21.15.53:0/528145065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 msgr2=0x5561d36c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa023fff700 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 0x5561d36c1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa023fff700 1 -- 172.21.15.53:0/528145065 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c009230 con 0x5561d3661fe0 2023-11-06T01:44:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa028fb1700 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 0x5561d36c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa023fff700 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661fe0 0x5561d36c15c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa010005d90 tx=0x7fa010000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f037bba77526bd75 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa003fff700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:44:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa003fff700 1 -- 172.21.15.53:0/528145065 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01002b040 con 0x5561d3661fe0 2023-11-06T01:44:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa0373a5b80 1 -- 172.21.15.53:0/528145065 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0180a15b0 con 0x5561d3661fe0 2023-11-06T01:44:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.679+0000 7fa003fff700 1 -- 172.21.15.53:0/528145065 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa01000eb30 con 0x5561d3661fe0 2023-11-06T01:44:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.680+0000 7fa003fff700 1 -- 172.21.15.53:0/528145065 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa01000ecf0 con 0x5561d3661fe0 2023-11-06T01:44:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.680+0000 7fa0373a5b80 10 client.26221 fetch_fsmap learned FSMap version 46 2023-11-06T01:44:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.680+0000 7fa0373a5b80 1 -- 172.21.15.53:0/528145065 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5561d33c9c00 con 0x5561d3661fe0 2023-11-06T01:44:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.680+0000 7fa003fff700 1 -- 172.21.15.53:0/528145065 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0180a15b0 con 0x5561d3661fe0 2023-11-06T01:44:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.680+0000 7fa003fff700 1 -- 172.21.15.53:0/528145065 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa010038030 con 0x5561d3661fe0 2023-11-06T01:44:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.680+0000 7fa0373a5b80 10 client.26221 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:44:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.680+0000 7fa0373a5b80 10 client.26221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.680+0000 7fa0373a5b80 20 client.26221 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.680+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5561d36ca380 0x5561d36cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.680+0000 7fa0373a5b80 4 client.26221 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:44:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.680+0000 7fa0373a5b80 1 -- 172.21.15.53:0/528145065 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5561d35abce0 con 0x5561d36ca380 2023-11-06T01:44:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.681+0000 7fa028fb1700 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5561d36ca380 0x5561d36cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.682+0000 7fa003fff700 10 client.26221 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.682+0000 7fa028fb1700 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5561d36ca380 0x5561d36cc7a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3b20f9d6f7d2d3 server_cookie=b7717e1d1a5a14bd in_seq=0 out_seq=0 2023-11-06T01:44:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.686+0000 7fa003fff700 1 -- 172.21.15.53:0/528145065 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5561d35abce0 con 0x5561d36ca380 2023-11-06T01:44:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.686+0000 7fa003fff700 10 client.26221 handle_command_reply: tid=0 2023-11-06T01:44:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.686+0000 7fa0373a5b80 1 client.26221 shutdown 2023-11-06T01:44:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.686+0000 7fa0373a5b80 1 -- 172.21.15.53:0/528145065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661fe0 msgr2=0x5561d36c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.686+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661fe0 0x5561d36c15c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa010005d90 tx=0x7fa010000bc0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.687+0000 7fa0373a5b80 1 -- 172.21.15.53:0/528145065 shutdown_connections 2023-11-06T01:44:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.687+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561d3662ed0 0x5561d36c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.687+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5561d36ca380 0x5561d36cc7a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.687+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d3661680 0x5561d36c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.687+0000 7fa0373a5b80 1 --2- 172.21.15.53:0/528145065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561d3661fe0 0x5561d36c15c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.687+0000 7fa0373a5b80 1 -- 172.21.15.53:0/528145065 >> 172.21.15.53:0/528145065 conn(0x5561d35c1070 msgr2=0x5561d366a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.687+0000 7fa003fff700 0 client.26221 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.687+0000 7fa0373a5b80 1 -- 172.21.15.53:0/528145065 shutdown_connections 2023-11-06T01:44:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.687+0000 7fa0373a5b80 1 -- 172.21.15.53:0/528145065 wait complete. 2023-11-06T01:44:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa0373a5b80 20 client.26221 trim_cache size 0 max 16384 2023-11-06T01:44:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:44:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:44:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:44:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:44:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 -- 172.21.15.53:0/1551473499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0140a05a0 msgr2=0x7fa0140a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0140a05a0 0x7fa0140a2a60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa01000b680 tx=0x7fa010009510 comp rx=0 tx=0).stop 2023-11-06T01:44:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 -- 172.21.15.53:0/1551473499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa024101880 msgr2=0x7fa024194910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa024101880 0x7fa024194910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa01800eb30 tx=0x7fa01800eef0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 -- 172.21.15.53:0/1551473499 shutdown_connections 2023-11-06T01:44:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0140a05a0 0x7fa0140a2a60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa024101880 0x7fa024194910 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa024106ae0 0x7fa024195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 --2- 172.21.15.53:0/1551473499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0241041b0 0x7fa024194e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 -- 172.21.15.53:0/1551473499 >> 172.21.15.53:0/1551473499 conn(0x7fa0240fb1f0 msgr2=0x7fa0240fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 -- 172.21.15.53:0/1551473499 shutdown_connections 2023-11-06T01:44:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:35.689+0000 7fa02b215700 1 -- 172.21.15.53:0/1551473499 wait complete. 2023-11-06T01:44:35.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:35 smithi163 ceph-mon[104416]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.39k op/s 2023-11-06T01:44:35.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:35 smithi053 ceph-mon[99472]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.39k op/s 2023-11-06T01:44:37.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 620.0 s, 1732.2 tps, lat 59.877 ms stddev 91.370 2023-11-06T01:44:37.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:37 smithi062 ceph-mon[103924]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 33 MiB/s wr, 1.88k op/s 2023-11-06T01:44:37.740 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:37 smithi163 ceph-mon[104416]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 33 MiB/s wr, 1.88k op/s 2023-11-06T01:44:37.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:37 smithi053 ceph-mon[99472]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 33 MiB/s wr, 1.88k op/s 2023-11-06T01:44:39.620 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:44:39.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:39 smithi163 ceph-mon[104416]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-11-06T01:44:39.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.831+0000 7f579c532700 1 -- 172.21.15.53:0/1233159670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5794101f60 msgr2=0x7f57941043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:39.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.831+0000 7f579c532700 1 --2- 172.21.15.53:0/1233159670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5794101f60 0x7f57941043f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f578c00b0a0 tx=0x7f578c00b3b0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.831+0000 7f579c532700 1 -- 172.21.15.53:0/1233159670 shutdown_connections 2023-11-06T01:44:39.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.831+0000 7f579c532700 1 --2- 172.21.15.53:0/1233159670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5794104930 0x7f5794104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.831+0000 7f579c532700 1 --2- 172.21.15.53:0/1233159670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57940ff5f0 0x7f5794101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.831+0000 7f579c532700 1 --2- 172.21.15.53:0/1233159670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5794101f60 0x7f57941043f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.831+0000 7f579c532700 1 -- 172.21.15.53:0/1233159670 >> 172.21.15.53:0/1233159670 conn(0x7f57940fb1f0 msgr2=0x7f57940fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:39.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.831+0000 7f579c532700 1 -- 172.21.15.53:0/1233159670 shutdown_connections 2023-11-06T01:44:39.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.831+0000 7f579c532700 1 -- 172.21.15.53:0/1233159670 wait complete. 2023-11-06T01:44:39.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.832+0000 7f579c532700 1 Processor -- start 2023-11-06T01:44:39.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.832+0000 7f579c532700 1 -- start start 2023-11-06T01:44:39.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.832+0000 7f579c532700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57940ff5f0 0x7f579419d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.832+0000 7f579c532700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5794101f60 0x7f579419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579c532700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5794104930 0x7f57941a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579c532700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57941a4ce0 con 0x7f5794104930 2023-11-06T01:44:39.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579c532700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57941a4e20 con 0x7f57940ff5f0 2023-11-06T01:44:39.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579c532700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57941a4fb0 con 0x7f5794101f60 2023-11-06T01:44:39.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579a2ce700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57940ff5f0 0x7f579419d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579aacf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5794104930 0x7f57941a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579a2ce700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57940ff5f0 0x7f579419d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54308/0 (socket says 172.21.15.53:54308) 2023-11-06T01:44:39.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579a2ce700 1 -- 172.21.15.53:0/2174682676 learned_addr learned my addr 172.21.15.53:0/2174682676 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:39.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579aacf700 1 -- 172.21.15.53:0/2174682676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5794101f60 msgr2=0x7f579419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:39.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579aacf700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5794101f60 0x7f579419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579aacf700 1 -- 172.21.15.53:0/2174682676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57940ff5f0 msgr2=0x7f579419d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:39.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579aacf700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57940ff5f0 0x7f579419d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579aacf700 1 -- 172.21.15.53:0/2174682676 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c009d00 con 0x7f5794104930 2023-11-06T01:44:39.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.833+0000 7f579aacf700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5794104930 0x7f57941a4710 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f578400b3c0 tx=0x7f578400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:39.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.834+0000 7f579a2ce700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57940ff5f0 0x7f579419d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:44:39.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.834+0000 7f578b7fe700 1 -- 172.21.15.53:0/2174682676 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784011830 con 0x7f5794104930 2023-11-06T01:44:39.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.834+0000 7f578b7fe700 1 -- 172.21.15.53:0/2174682676 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5784011e50 con 0x7f5794104930 2023-11-06T01:44:39.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.834+0000 7f579c532700 1 -- 172.21.15.53:0/2174682676 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57941a5290 con 0x7f5794104930 2023-11-06T01:44:39.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.834+0000 7f578b7fe700 1 -- 172.21.15.53:0/2174682676 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784010e80 con 0x7f5794104930 2023-11-06T01:44:39.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.834+0000 7f579c532700 1 -- 172.21.15.53:0/2174682676 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57941a5870 con 0x7f5794104930 2023-11-06T01:44:39.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.837+0000 7f578b7fe700 1 -- 172.21.15.53:0/2174682676 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5784010380 con 0x7f5794104930 2023-11-06T01:44:39.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.837+0000 7f578b7fe700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57800a05a0 0x7f57800a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.838+0000 7f578b7fe700 1 -- 172.21.15.53:0/2174682676 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f57840cb3b0 con 0x7f5794104930 2023-11-06T01:44:39.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.838+0000 7f579a2ce700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57800a05a0 0x7f57800a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.838+0000 7f579a2ce700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57800a05a0 0x7f57800a2a60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5790009d70 tx=0x7f5790009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:39.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f57a86c2b80 1 Processor -- start 2023-11-06T01:44:39.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f57a86c2b80 1 -- start start 2023-11-06T01:44:39.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb10f0 0x56340deb1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 0x56340deafe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb03f0 0x56340deb0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340de07a10 con 0x56340deb1a70 2023-11-06T01:44:39.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340de053d0 con 0x56340deb03f0 2023-11-06T01:44:39.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340de00cc0 con 0x56340deb10f0 2023-11-06T01:44:39.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f5799acd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 0x56340deafe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f5799acd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 0x56340deafe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42880/0 (socket says 172.21.15.53:42880) 2023-11-06T01:44:39.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f579aacf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb03f0 0x56340deb0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f5799acd700 1 -- 172.21.15.53:0/3807350603 learned_addr learned my addr 172.21.15.53:0/3807350603 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:39.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.840+0000 7f579a2ce700 1 --2- 172.21.15.53:0/3807350603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb10f0 0x56340deb1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.841+0000 7f5799acd700 1 -- 172.21.15.53:0/3807350603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb10f0 msgr2=0x56340deb1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.841+0000 7f5799acd700 1 --2- 172.21.15.53:0/3807350603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb10f0 0x56340deb1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.841+0000 7f5799acd700 1 -- 172.21.15.53:0/3807350603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb03f0 msgr2=0x56340deb0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.841+0000 7f5799acd700 1 --2- 172.21.15.53:0/3807350603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb03f0 0x56340deb0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.841+0000 7f5799acd700 1 -- 172.21.15.53:0/3807350603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56340dc2ede0 con 0x56340deb1a70 2023-11-06T01:44:39.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.841+0000 7f579aacf700 1 --2- 172.21.15.53:0/3807350603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb03f0 0x56340deb0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:39.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.841+0000 7f5799acd700 1 --2- 172.21.15.53:0/3807350603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 0x56340deafe60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f578c0062a0 tx=0x7f578c0097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37c0b45dba2d988 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:39.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.841+0000 7f5788ff9700 1 -- 172.21.15.53:0/3807350603 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c00e040 con 0x56340deb1a70 2023-11-06T01:44:39.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.841+0000 7f5788ff9700 1 -- 172.21.15.53:0/3807350603 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f578c00bcb0 con 0x56340deb1a70 2023-11-06T01:44:39.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.841+0000 7f5788ff9700 1 -- 172.21.15.53:0/3807350603 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0053d0 con 0x56340deb1a70 2023-11-06T01:44:39.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.842+0000 7f57a86c2b80 1 -- 172.21.15.53:0/3807350603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 msgr2=0x56340deafe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:39.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.842+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/3807350603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 0x56340deafe60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f578c0062a0 tx=0x7f578c0097e0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.842+0000 7f57a86c2b80 1 -- 172.21.15.53:0/3807350603 shutdown_connections 2023-11-06T01:44:39.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.842+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/3807350603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb03f0 0x56340deb0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.842+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/3807350603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb10f0 0x56340deb1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.842+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/3807350603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 0x56340deafe60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.842+0000 7f57a86c2b80 1 -- 172.21.15.53:0/3807350603 >> 172.21.15.53:0/3807350603 conn(0x56340de0f0d0 msgr2=0x56340deaa940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:39.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.842+0000 7f57a86c2b80 1 -- 172.21.15.53:0/3807350603 shutdown_connections 2023-11-06T01:44:39.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.842+0000 7f57a86c2b80 1 -- 172.21.15.53:0/3807350603 wait complete. 2023-11-06T01:44:39.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f57a86c2b80 1 -- start start 2023-11-06T01:44:39.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f57a86c2b80 1 Processor -- start 2023-11-06T01:44:39.850 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:39 smithi053 ceph-mon[99472]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-11-06T01:44:39.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 0x56340deb9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 0x56340deb9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb1a70 0x56340dec0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340dd9d510 con 0x56340deb10f0 2023-11-06T01:44:39.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340dd9d1b0 con 0x56340deb1a70 2023-11-06T01:44:39.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340dc84920 con 0x56340deb03f0 2023-11-06T01:44:39.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f57a86c2b80 10 client.0 _ll_register_callbacks cb 0x56340dceaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:39.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f5799acd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 0x56340deb9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f5799acd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 0x56340deb9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42886/0 (socket says 172.21.15.53:42886) 2023-11-06T01:44:39.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f5799acd700 1 -- 172.21.15.53:0/1049685055 learned_addr learned my addr 172.21.15.53:0/1049685055 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:39.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.843+0000 7f579a2ce700 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 0x56340deb9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f5799acd700 1 -- 172.21.15.53:0/1049685055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 msgr2=0x56340deb9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:39.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f5799acd700 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 0x56340deb9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f5799acd700 1 -- 172.21.15.53:0/1049685055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb1a70 msgr2=0x56340dec0eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:44:39.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f5799acd700 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb1a70 0x56340dec0eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f5799acd700 1 -- 172.21.15.53:0/1049685055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57941a5870 con 0x56340deb10f0 2023-11-06T01:44:39.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f579a2ce700 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 0x56340deb9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:39.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f5799acd700 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 0x56340deb9e70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f578c00b780 tx=0x7f578c0052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a7dcb6e8782c3c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:39.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f577dffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:44:39.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f577dffb700 1 -- 172.21.15.53:0/1049685055 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c00e040 con 0x56340deb10f0 2023-11-06T01:44:39.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f577dffb700 1 -- 172.21.15.53:0/1049685055 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f578c0074e0 con 0x56340deb10f0 2023-11-06T01:44:39.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f57a86c2b80 1 -- 172.21.15.53:0/1049685055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56340de2bd10 con 0x56340deb10f0 2023-11-06T01:44:39.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f577dffb700 1 -- 172.21.15.53:0/1049685055 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f578c00be20 con 0x56340deb10f0 2023-11-06T01:44:39.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f57a86c2b80 10 client.16629 fetch_fsmap learned FSMap version 46 2023-11-06T01:44:39.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.844+0000 7f57a86c2b80 1 -- 172.21.15.53:0/1049685055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f57941058e0 con 0x56340deb10f0 2023-11-06T01:44:39.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.845+0000 7f577dffb700 1 -- 172.21.15.53:0/1049685055 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56340de2bd10 con 0x56340deb10f0 2023-11-06T01:44:39.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.845+0000 7f577dffb700 1 -- 172.21.15.53:0/1049685055 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f578c019040 con 0x56340deb10f0 2023-11-06T01:44:39.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.845+0000 7f57a86c2b80 10 client.16629 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:44:39.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.845+0000 7f57a86c2b80 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:39.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.845+0000 7f57a86c2b80 20 client.16629 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:39.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.845+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56340dec5300 0x56340dec7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.845+0000 7f57a86c2b80 4 client.16629 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:44:39.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.845+0000 7f57a86c2b80 1 -- 172.21.15.53:0/1049685055 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56340de7ee10 con 0x56340dec5300 2023-11-06T01:44:39.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.845+0000 7f579a2ce700 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56340dec5300 0x56340dec7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.845+0000 7f577dffb700 10 client.16629 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:39.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.845+0000 7f579a2ce700 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56340dec5300 0x56340dec7720 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454f64492945a5cb server_cookie=1425708bfb6a82c9 in_seq=0 out_seq=0 2023-11-06T01:44:39.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f577dffb700 1 -- 172.21.15.53:0/1049685055 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56340de7ee10 con 0x56340dec5300 2023-11-06T01:44:39.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f577dffb700 10 client.16629 handle_command_reply: tid=0 2023-11-06T01:44:39.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f57a86c2b80 1 client.16629 shutdown 2023-11-06T01:44:39.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f57a86c2b80 1 -- 172.21.15.53:0/1049685055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 msgr2=0x56340deb9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:39.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 0x56340deb9e70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f578c00b780 tx=0x7f578c0052e0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f57a86c2b80 1 -- 172.21.15.53:0/1049685055 shutdown_connections 2023-11-06T01:44:39.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb1a70 0x56340dec0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56340dec5300 0x56340dec7720 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 0x56340deb9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f577dffb700 0 client.16629 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:39.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/1049685055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 0x56340deb9e70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f57a86c2b80 1 -- 172.21.15.53:0/1049685055 >> 172.21.15.53:0/1049685055 conn(0x56340de0f0d0 msgr2=0x56340deaa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:39.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f57a86c2b80 1 -- 172.21.15.53:0/1049685055 shutdown_connections 2023-11-06T01:44:39.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.846+0000 7f57a86c2b80 1 -- 172.21.15.53:0/1049685055 wait complete. 2023-11-06T01:44:39.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.847+0000 7f57a86c2b80 20 client.16629 trim_cache size 0 max 16384 2023-11-06T01:44:39.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.868+0000 7f57a86c2b80 1 Processor -- start 2023-11-06T01:44:39.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.868+0000 7f57a86c2b80 1 -- start start 2023-11-06T01:44:39.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.868+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb03f0 0x56340deafe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.868+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 0x56340debedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.868+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb1a70 0x56340deb98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340deb84a0 con 0x56340deb10f0 2023-11-06T01:44:39.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340dea2b50 con 0x56340deb03f0 2023-11-06T01:44:39.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340deac170 con 0x56340deb1a70 2023-11-06T01:44:39.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f5799acd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 0x56340debedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f5799acd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 0x56340debedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42894/0 (socket says 172.21.15.53:42894) 2023-11-06T01:44:39.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f579aacf700 1 --2- 172.21.15.53:0/877374993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb1a70 0x56340deb98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f579a2ce700 1 --2- 172.21.15.53:0/877374993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb03f0 0x56340deafe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f5799acd700 1 -- 172.21.15.53:0/877374993 learned_addr learned my addr 172.21.15.53:0/877374993 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:39.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f579aacf700 1 -- 172.21.15.53:0/877374993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb03f0 msgr2=0x56340deafe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:39.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f579aacf700 1 --2- 172.21.15.53:0/877374993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb03f0 0x56340deafe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f579aacf700 1 -- 172.21.15.53:0/877374993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 msgr2=0x56340debedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:39.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f579aacf700 1 --2- 172.21.15.53:0/877374993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 0x56340debedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f579aacf700 1 -- 172.21.15.53:0/877374993 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56340dc17c60 con 0x56340deb1a70 2023-11-06T01:44:39.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f579aacf700 1 --2- 172.21.15.53:0/877374993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb1a70 0x56340deb98f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f57840c8ee0 tx=0x7f5784098730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa54219f69b3b49 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:39.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.869+0000 7f57897fa700 1 -- 172.21.15.53:0/877374993 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57840b0070 con 0x56340deb1a70 2023-11-06T01:44:39.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.870+0000 7f57897fa700 1 -- 172.21.15.53:0/877374993 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57840992f0 con 0x56340deb1a70 2023-11-06T01:44:39.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.870+0000 7f57897fa700 1 -- 172.21.15.53:0/877374993 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784098e50 con 0x56340deb1a70 2023-11-06T01:44:39.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.871+0000 7f57a86c2b80 1 -- 172.21.15.53:0/877374993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb1a70 msgr2=0x56340deb98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:39.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.871+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/877374993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb1a70 0x56340deb98f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f57840c8ee0 tx=0x7f5784098730 comp rx=0 tx=0).stop 2023-11-06T01:44:39.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.871+0000 7f57a86c2b80 1 -- 172.21.15.53:0/877374993 shutdown_connections 2023-11-06T01:44:39.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.871+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/877374993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb03f0 0x56340deafe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.871+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/877374993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb1a70 0x56340deb98f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.871+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/877374993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb10f0 0x56340debedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.871+0000 7f57a86c2b80 1 -- 172.21.15.53:0/877374993 >> 172.21.15.53:0/877374993 conn(0x56340de0f0d0 msgr2=0x56340deb42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:39.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.871+0000 7f57a86c2b80 1 -- 172.21.15.53:0/877374993 shutdown_connections 2023-11-06T01:44:39.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.871+0000 7f57a86c2b80 1 -- 172.21.15.53:0/877374993 wait complete. 2023-11-06T01:44:39.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f57a86c2b80 1 -- start start 2023-11-06T01:44:39.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f57a86c2b80 1 Processor -- start 2023-11-06T01:44:39.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 0x56340debcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb10f0 0x56340debd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f57a86c2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 0x56340debe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340debe860 con 0x56340deb1a70 2023-11-06T01:44:39.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340debe9d0 con 0x56340deb10f0 2023-11-06T01:44:39.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f57a86c2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340debeb90 con 0x56340deb03f0 2023-11-06T01:44:39.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f57a86c2b80 10 client.0 _ll_register_callbacks cb 0x7f5784004790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:39.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f579a2ce700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 0x56340debcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f5799acd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb10f0 0x56340debd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f579aacf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 0x56340debe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f579a2ce700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 0x56340debcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57138/0 (socket says 172.21.15.53:57138) 2023-11-06T01:44:39.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.872+0000 7f579a2ce700 1 -- 172.21.15.53:0/963706561 learned_addr learned my addr 172.21.15.53:0/963706561 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:39.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f579a2ce700 1 -- 172.21.15.53:0/963706561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb10f0 msgr2=0x56340debd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:39.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f579a2ce700 1 --2- 172.21.15.53:0/963706561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb10f0 0x56340debd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f579a2ce700 1 -- 172.21.15.53:0/963706561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 msgr2=0x56340debe320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:39.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f579a2ce700 1 --2- 172.21.15.53:0/963706561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 0x56340debe320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f579a2ce700 1 -- 172.21.15.53:0/963706561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57941a5290 con 0x56340deb03f0 2023-11-06T01:44:39.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f579a2ce700 1 --2- 172.21.15.53:0/963706561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 0x56340debcdd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x56340deb0e90 tx=0x7f57900047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f34f43c4e49b31a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:39.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f577dffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:44:39.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f577dffb700 1 -- 172.21.15.53:0/963706561 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790025040 con 0x56340deb03f0 2023-11-06T01:44:39.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f577dffb700 1 -- 172.21.15.53:0/963706561 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5790018070 con 0x56340deb03f0 2023-11-06T01:44:39.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f57a86c2b80 1 -- 172.21.15.53:0/963706561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f578c00bcb0 con 0x56340deb03f0 2023-11-06T01:44:39.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f577dffb700 1 -- 172.21.15.53:0/963706561 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f579002a980 con 0x56340deb03f0 2023-11-06T01:44:39.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f57a86c2b80 10 client.26236 fetch_fsmap learned FSMap version 46 2023-11-06T01:44:39.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f57a86c2b80 1 -- 172.21.15.53:0/963706561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56340dc55250 con 0x56340deb03f0 2023-11-06T01:44:39.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f577dffb700 1 -- 172.21.15.53:0/963706561 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f578c00bcb0 con 0x56340deb03f0 2023-11-06T01:44:39.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f577dffb700 1 -- 172.21.15.53:0/963706561 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f579003c050 con 0x56340deb03f0 2023-11-06T01:44:39.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f57a86c2b80 10 client.26236 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:44:39.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f57a86c2b80 10 client.26236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:39.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.873+0000 7f57a86c2b80 20 client.26236 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:39.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/963706561 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56340df18060 0x56340df1a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:39.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f57a86c2b80 4 client.26236 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:44:39.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f57a86c2b80 1 -- 172.21.15.53:0/963706561 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56340ddf9d40 con 0x56340df18060 2023-11-06T01:44:39.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f5799acd700 1 --2- 172.21.15.53:0/963706561 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56340df18060 0x56340df1a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:39.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f577dffb700 10 client.26236 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:39.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f5799acd700 1 --2- 172.21.15.53:0/963706561 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56340df18060 0x56340df1a480 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b1c088b0e7bf36a server_cookie=6f034a39c5f0dce2 in_seq=0 out_seq=0 2023-11-06T01:44:39.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f577dffb700 1 -- 172.21.15.53:0/963706561 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56340ddf9d40 con 0x56340df18060 2023-11-06T01:44:39.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f577dffb700 10 client.26236 handle_command_reply: tid=0 2023-11-06T01:44:39.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f57a86c2b80 1 client.26236 shutdown 2023-11-06T01:44:39.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f57a86c2b80 1 -- 172.21.15.53:0/963706561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 msgr2=0x56340debcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:39.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/963706561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 0x56340debcdd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x56340deb0e90 tx=0x7f57900047c0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f57a86c2b80 1 -- 172.21.15.53:0/963706561 shutdown_connections 2023-11-06T01:44:39.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/963706561 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56340df18060 0x56340df1a480 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.874+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/963706561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56340deb1a70 0x56340debe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f577dffb700 0 client.26236 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:39.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/963706561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56340deb03f0 0x56340debcdd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f57a86c2b80 1 --2- 172.21.15.53:0/963706561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56340deb10f0 0x56340debd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f57a86c2b80 1 -- 172.21.15.53:0/963706561 >> 172.21.15.53:0/963706561 conn(0x56340de0f0d0 msgr2=0x56340deb98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:39.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f57a86c2b80 1 -- 172.21.15.53:0/963706561 shutdown_connections 2023-11-06T01:44:39.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f57a86c2b80 1 -- 172.21.15.53:0/963706561 wait complete. 2023-11-06T01:44:39.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f57a86c2b80 20 client.26236 trim_cache size 0 max 16384 2023-11-06T01:44:39.888 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:44:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:44:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:44:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:44:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:44:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:44:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:44:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:44:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:44:39.890 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:44:39.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 -- 172.21.15.53:0/2174682676 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57800a05a0 msgr2=0x7f57800a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:39.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57800a05a0 0x7f57800a2a60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5790009d70 tx=0x7f5790009260 comp rx=0 tx=0).stop 2023-11-06T01:44:39.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 -- 172.21.15.53:0/2174682676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5794104930 msgr2=0x7f57941a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:39.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5794104930 0x7f57941a4710 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f578400b3c0 tx=0x7f578400b780 comp rx=0 tx=0).stop 2023-11-06T01:44:39.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 -- 172.21.15.53:0/2174682676 shutdown_connections 2023-11-06T01:44:39.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f57800a05a0 0x7f57800a2a60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57940ff5f0 0x7f579419d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5794104930 0x7f57941a4710 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 --2- 172.21.15.53:0/2174682676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5794101f60 0x7f579419d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:39.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 -- 172.21.15.53:0/2174682676 >> 172.21.15.53:0/2174682676 conn(0x7f57940fb1f0 msgr2=0x7f5794100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:39.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 -- 172.21.15.53:0/2174682676 shutdown_connections 2023-11-06T01:44:39.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:44:39.875+0000 7f579c532700 1 -- 172.21.15.53:0/2174682676 wait complete. 2023-11-06T01:44:39.894 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:44:39.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:39 smithi062 ceph-mon[103924]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-11-06T01:44:41.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:41 smithi163 ceph-mon[104416]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.95k op/s 2023-11-06T01:44:41.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:41 smithi053 ceph-mon[99472]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.95k op/s 2023-11-06T01:44:41.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:41 smithi062 ceph-mon[103924]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.95k op/s 2023-11-06T01:44:42.184 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 625.0 s, 1777.6 tps, lat 56.695 ms stddev 62.330 2023-11-06T01:44:43.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:43 smithi163 ceph-mon[104416]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-11-06T01:44:43.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:43 smithi053 ceph-mon[99472]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-11-06T01:44:43.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:43 smithi062 ceph-mon[103924]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-11-06T01:44:45.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:45 smithi163 ceph-mon[104416]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.90k op/s 2023-11-06T01:44:45.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:45 smithi053 ceph-mon[99472]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.90k op/s 2023-11-06T01:44:45.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:45 smithi062 ceph-mon[103924]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.90k op/s 2023-11-06T01:44:47.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 630.0 s, 1783.4 tps, lat 56.334 ms stddev 74.970 2023-11-06T01:44:47.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:47 smithi163 ceph-mon[104416]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 MiB/s wr, 1.22k op/s 2023-11-06T01:44:47.851 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:47 smithi053 ceph-mon[99472]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 MiB/s wr, 1.22k op/s 2023-11-06T01:44:47.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:47 smithi062 ceph-mon[103924]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 MiB/s wr, 1.22k op/s 2023-11-06T01:44:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.873+0000 7fda97aa2700 1 -- 172.21.15.53:0/3004670904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda90106ac0 msgr2=0x7fda90108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.873+0000 7fda97aa2700 1 --2- 172.21.15.53:0/3004670904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda90106ac0 0x7fda90108f30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fda8800b0a0 tx=0x7fda8800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.873+0000 7fda97aa2700 1 -- 172.21.15.53:0/3004670904 shutdown_connections 2023-11-06T01:44:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.873+0000 7fda97aa2700 1 --2- 172.21.15.53:0/3004670904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda90106ac0 0x7fda90108f30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.873+0000 7fda97aa2700 1 --2- 172.21.15.53:0/3004670904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda90101860 0x7fda90103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.873+0000 7fda97aa2700 1 --2- 172.21.15.53:0/3004670904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda90104190 0x7fda90106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.873+0000 7fda97aa2700 1 -- 172.21.15.53:0/3004670904 >> 172.21.15.53:0/3004670904 conn(0x7fda900fb250 msgr2=0x7fda900fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.873+0000 7fda97aa2700 1 -- 172.21.15.53:0/3004670904 shutdown_connections 2023-11-06T01:44:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.873+0000 7fda97aa2700 1 -- 172.21.15.53:0/3004670904 wait complete. 2023-11-06T01:44:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda97aa2700 1 Processor -- start 2023-11-06T01:44:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda97aa2700 1 -- start start 2023-11-06T01:44:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda97aa2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda90101860 0x7fda900ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda97aa2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda90104190 0x7fda900ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda97aa2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda90106ac0 0x7fda90100290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda97aa2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda901a0bc0 con 0x7fda90106ac0 2023-11-06T01:44:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda97aa2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda901a0d00 con 0x7fda90101860 2023-11-06T01:44:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda97aa2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda901a0e40 con 0x7fda90104190 2023-11-06T01:44:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda9603f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda90106ac0 0x7fda90100290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda9603f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda90106ac0 0x7fda90100290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42916/0 (socket says 172.21.15.53:42916) 2023-11-06T01:44:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda9603f700 1 -- 172.21.15.53:0/4237693529 learned_addr learned my addr 172.21.15.53:0/4237693529 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda9583e700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda90101860 0x7fda900ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda9603f700 1 -- 172.21.15.53:0/4237693529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda90104190 msgr2=0x7fda900ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda9603f700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda90104190 0x7fda900ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda9603f700 1 -- 172.21.15.53:0/4237693529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda90101860 msgr2=0x7fda900ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda9603f700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda90101860 0x7fda900ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.874+0000 7fda9603f700 1 -- 172.21.15.53:0/4237693529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda88009d00 con 0x7fda90106ac0 2023-11-06T01:44:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.875+0000 7fda9603f700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda90106ac0 0x7fda90100290 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fda8800b780 tx=0x7fda8800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.875+0000 7fda86ffd700 1 -- 172.21.15.53:0/4237693529 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8800e040 con 0x7fda90106ac0 2023-11-06T01:44:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.875+0000 7fda86ffd700 1 -- 172.21.15.53:0/4237693529 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda880094a0 con 0x7fda90106ac0 2023-11-06T01:44:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.875+0000 7fda97aa2700 1 -- 172.21.15.53:0/4237693529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda901a1070 con 0x7fda90106ac0 2023-11-06T01:44:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.875+0000 7fda86ffd700 1 -- 172.21.15.53:0/4237693529 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88005330 con 0x7fda90106ac0 2023-11-06T01:44:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.875+0000 7fda97aa2700 1 -- 172.21.15.53:0/4237693529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda901a14f0 con 0x7fda90106ac0 2023-11-06T01:44:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.878+0000 7fda86ffd700 1 -- 172.21.15.53:0/4237693529 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fda8801da30 con 0x7fda90106ac0 2023-11-06T01:44:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.879+0000 7fdaa3c32b80 1 Processor -- start 2023-11-06T01:44:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.879+0000 7fda86ffd700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda7c0a0520 0x7fda7c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.879+0000 7fdaa3c32b80 1 -- start start 2023-11-06T01:44:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.879+0000 7fda86ffd700 1 -- 172.21.15.53:0/4237693529 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fda880cd050 con 0x7fda90106ac0 2023-11-06T01:44:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c232b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 0x55fc99c24100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c246e0 0x55fc99c24b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc99b729b0 con 0x55fc99c237f0 2023-11-06T01:44:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc99b70370 con 0x55fc99c22e90 2023-11-06T01:44:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc99b6bc60 con 0x55fc99c246e0 2023-11-06T01:44:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fda9503d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 0x55fc99c24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fda9503d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 0x55fc99c24100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42922/0 (socket says 172.21.15.53:42922) 2023-11-06T01:44:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fda9503d700 1 -- 172.21.15.53:0/952924546 learned_addr learned my addr 172.21.15.53:0/952924546 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fda9503d700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda7c0a0520 0x7fda7c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fda9503d700 1 -- 172.21.15.53:0/952924546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c246e0 msgr2=0x55fc99c24b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fda9503d700 1 --2- 172.21.15.53:0/952924546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c246e0 0x55fc99c24b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fda9503d700 1 -- 172.21.15.53:0/952924546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 msgr2=0x55fc99c232b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fda9503d700 1 --2- 172.21.15.53:0/952924546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fda9503d700 1 -- 172.21.15.53:0/952924546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc99999d80 con 0x55fc99c237f0 2023-11-06T01:44:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.880+0000 7fda9503d700 1 --2- 172.21.15.53:0/952924546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 0x55fc99c24100 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fda80012810 tx=0x7fda80012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7da55aa7f2e060 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.881+0000 7fda7bfff700 1 -- 172.21.15.53:0/952924546 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80003b80 con 0x55fc99c237f0 2023-11-06T01:44:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.881+0000 7fda7bfff700 1 -- 172.21.15.53:0/952924546 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda80018070 con 0x55fc99c237f0 2023-11-06T01:44:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.881+0000 7fda7bfff700 1 -- 172.21.15.53:0/952924546 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8001d480 con 0x55fc99c237f0 2023-11-06T01:44:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.882+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/952924546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 msgr2=0x55fc99c24100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.882+0000 7fda9503d700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda7c0a0520 0x7fda7c0a29e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda8001f000 tx=0x7fda8001b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.882+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/952924546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 0x55fc99c24100 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fda80012810 tx=0x7fda80012b20 comp rx=0 tx=0).stop 2023-11-06T01:44:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.882+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/952924546 shutdown_connections 2023-11-06T01:44:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.882+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/952924546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c232b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.882+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/952924546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c246e0 0x55fc99c24b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.882+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/952924546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 0x55fc99c24100 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.882+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/952924546 >> 172.21.15.53:0/952924546 conn(0x55fc99b7a070 msgr2=0x55fc99c17040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.882+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/952924546 shutdown_connections 2023-11-06T01:44:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.882+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/952924546 wait complete. 2023-11-06T01:44:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.882+0000 7fdaa3c32b80 1 -- start start 2023-11-06T01:44:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fdaa3c32b80 1 Processor -- start 2023-11-06T01:44:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c22e90 0x55fc99c3dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c237f0 0x55fc99c3e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c246e0 0x55fc99c45290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc99b084b0 con 0x55fc99c246e0 2023-11-06T01:44:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc99b08150 con 0x55fc99c237f0 2023-11-06T01:44:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc999ef8c0 con 0x55fc99c22e90 2023-11-06T01:44:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fdaa3c32b80 10 client.0 _ll_register_callbacks cb 0x55fc99a55a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fda9503d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c237f0 0x55fc99c3e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fda9503d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c237f0 0x55fc99c3e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54364/0 (socket says 172.21.15.53:54364) 2023-11-06T01:44:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fda9503d700 1 -- 172.21.15.53:0/684292470 learned_addr learned my addr 172.21.15.53:0/684292470 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fda9583e700 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c22e90 0x55fc99c3dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.883+0000 7fda9603f700 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c246e0 0x55fc99c45290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.884+0000 7fda9583e700 1 -- 172.21.15.53:0/684292470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c237f0 msgr2=0x55fc99c3e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.884+0000 7fda9583e700 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c237f0 0x55fc99c3e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.884+0000 7fda9583e700 1 -- 172.21.15.53:0/684292470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c246e0 msgr2=0x55fc99c45290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.884+0000 7fda9583e700 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c246e0 0x55fc99c45290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.884+0000 7fda9583e700 1 -- 172.21.15.53:0/684292470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda80012450 con 0x55fc99c22e90 2023-11-06T01:44:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.884+0000 7fda9503d700 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c237f0 0x55fc99c3e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.884+0000 7fda9583e700 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c22e90 0x55fc99c3dd10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00b6c0 tx=0x7fda8c0106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44dd60deb091290d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.884+0000 7fda797fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:44:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.884+0000 7fda797fa700 1 -- 172.21.15.53:0/684292470 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c004030 con 0x55fc99c22e90 2023-11-06T01:44:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.884+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/684292470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc99b96cb0 con 0x55fc99c22e90 2023-11-06T01:44:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fda797fa700 1 -- 172.21.15.53:0/684292470 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fda8c017070 con 0x55fc99c22e90 2023-11-06T01:44:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fda797fa700 1 -- 172.21.15.53:0/684292470 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda8c004190 con 0x55fc99c22e90 2023-11-06T01:44:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fdaa3c32b80 10 client.26242 fetch_fsmap learned FSMap version 46 2023-11-06T01:44:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/684292470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55fc99982c00 con 0x55fc99c22e90 2023-11-06T01:44:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fda797fa700 1 -- 172.21.15.53:0/684292470 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc99b96cb0 con 0x55fc99c22e90 2023-11-06T01:44:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fda797fa700 1 -- 172.21.15.53:0/684292470 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda8c005280 con 0x55fc99c22e90 2023-11-06T01:44:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fdaa3c32b80 10 client.26242 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:44:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fdaa3c32b80 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fdaa3c32b80 20 client.26242 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fc99c49b20 0x55fc99c4bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fdaa3c32b80 4 client.26242 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:44:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.887+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/684292470 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc99be9db0 con 0x55fc99c49b20 2023-11-06T01:44:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.888+0000 7fda9503d700 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fc99c49b20 0x55fc99c4bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.888+0000 7fda797fa700 10 client.26242 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.888+0000 7fda9503d700 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fc99c49b20 0x55fc99c4bf40 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee60b9ca67113bb server_cookie=a4cbd78df23caf47 in_seq=0 out_seq=0 2023-11-06T01:44:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fda797fa700 1 -- 172.21.15.53:0/684292470 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fc99be9db0 con 0x55fc99c49b20 2023-11-06T01:44:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fda797fa700 10 client.26242 handle_command_reply: tid=0 2023-11-06T01:44:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 1 client.26242 shutdown 2023-11-06T01:44:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/684292470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c22e90 msgr2=0x55fc99c3dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c22e90 0x55fc99c3dd10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00b6c0 tx=0x7fda8c0106b0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/684292470 shutdown_connections 2023-11-06T01:44:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fc99c49b20 0x55fc99c4bf40 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c246e0 0x55fc99c45290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c22e90 0x55fc99c3dd10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/684292470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c237f0 0x55fc99c3e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fda797fa700 0 client.26242 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/684292470 >> 172.21.15.53:0/684292470 conn(0x55fc99b7a070 msgr2=0x55fc99c153c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/684292470 shutdown_connections 2023-11-06T01:44:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/684292470 wait complete. 2023-11-06T01:44:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.889+0000 7fdaa3c32b80 20 client.26242 trim_cache size 0 max 16384 2023-11-06T01:44:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.908+0000 7fdaa3c32b80 1 Processor -- start 2023-11-06T01:44:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.908+0000 7fdaa3c32b80 1 -- start start 2023-11-06T01:44:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.908+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c23270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.908+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 0x55fc99c43210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.908+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c246e0 0x55fc99c3d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.908+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc99c176f0 con 0x55fc99c237f0 2023-11-06T01:44:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.908+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc99c49660 con 0x55fc99c22e90 2023-11-06T01:44:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.908+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc99c16890 con 0x55fc99c246e0 2023-11-06T01:44:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.908+0000 7fda9503d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 0x55fc99c43210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.908+0000 7fda9583e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c23270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.909+0000 7fda9603f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c246e0 0x55fc99c3d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.909+0000 7fda9503d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 0x55fc99c43210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42934/0 (socket says 172.21.15.53:42934) 2023-11-06T01:44:48.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.909+0000 7fda9503d700 1 -- 172.21.15.53:0/1485007703 learned_addr learned my addr 172.21.15.53:0/1485007703 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.909+0000 7fda9603f700 1 -- 172.21.15.53:0/1485007703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 msgr2=0x55fc99c23270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.909+0000 7fda9603f700 1 --2- 172.21.15.53:0/1485007703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c23270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.909+0000 7fda9603f700 1 -- 172.21.15.53:0/1485007703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 msgr2=0x55fc99c43210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.909+0000 7fda9603f700 1 --2- 172.21.15.53:0/1485007703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 0x55fc99c43210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.909+0000 7fda9603f700 1 -- 172.21.15.53:0/1485007703 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc999c01f0 con 0x55fc99c246e0 2023-11-06T01:44:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.909+0000 7fda9603f700 1 --2- 172.21.15.53:0/1485007703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c246e0 0x55fc99c3d3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55fc99c23ec0 tx=0x7fda88009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c098bd4155024dd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.910+0000 7fda84daa700 1 -- 172.21.15.53:0/1485007703 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8809a920 con 0x55fc99c246e0 2023-11-06T01:44:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.910+0000 7fda84daa700 1 -- 172.21.15.53:0/1485007703 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda8809e750 con 0x55fc99c246e0 2023-11-06T01:44:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.910+0000 7fda84daa700 1 -- 172.21.15.53:0/1485007703 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8809a070 con 0x55fc99c246e0 2023-11-06T01:44:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.911+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/1485007703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c246e0 msgr2=0x55fc99c3d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.911+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/1485007703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c246e0 0x55fc99c3d3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55fc99c23ec0 tx=0x7fda88009b80 comp rx=0 tx=0).stop 2023-11-06T01:44:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.912+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/1485007703 shutdown_connections 2023-11-06T01:44:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.912+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/1485007703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c23270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.912+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/1485007703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c246e0 0x55fc99c3d3c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.912+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/1485007703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c237f0 0x55fc99c43210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.912+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/1485007703 >> 172.21.15.53:0/1485007703 conn(0x55fc99b7a070 msgr2=0x55fc99c3c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.912+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/1485007703 shutdown_connections 2023-11-06T01:44:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.912+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/1485007703 wait complete. 2023-11-06T01:44:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.912+0000 7fdaa3c32b80 1 -- start start 2023-11-06T01:44:48.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fdaa3c32b80 1 Processor -- start 2023-11-06T01:44:48.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c426b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.917 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c237f0 0x55fc99c42bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fdaa3c32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c246e0 0x55fc99c99a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc99c99fd0 con 0x55fc99c246e0 2023-11-06T01:44:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc99c9a110 con 0x55fc99c22e90 2023-11-06T01:44:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fdaa3c32b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc99c9a2d0 con 0x55fc99c237f0 2023-11-06T01:44:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fdaa3c32b80 10 client.0 _ll_register_callbacks cb 0x7fda8c00b970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:44:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fda9583e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fda9603f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c246e0 0x55fc99c99a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fda9583e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c426b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54378/0 (socket says 172.21.15.53:54378) 2023-11-06T01:44:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fda9503d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c237f0 0x55fc99c42bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.913+0000 7fda9583e700 1 -- 172.21.15.53:0/2955389696 learned_addr learned my addr 172.21.15.53:0/2955389696 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:44:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fda9583e700 1 -- 172.21.15.53:0/2955389696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c237f0 msgr2=0x55fc99c42bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fda9583e700 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c237f0 0x55fc99c42bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fda9583e700 1 -- 172.21.15.53:0/2955389696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c246e0 msgr2=0x55fc99c99a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:44:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fda9583e700 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c246e0 0x55fc99c99a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fda9583e700 1 -- 172.21.15.53:0/2955389696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda901a14f0 con 0x55fc99c22e90 2023-11-06T01:44:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fda9503d700 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c237f0 0x55fc99c42bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:44:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fda9583e700 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c426b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0059d0 tx=0x7fda8c010fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99aa5064928d2f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:44:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fda797fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:44:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fda797fa700 1 -- 172.21.15.53:0/2955389696 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c003a90 con 0x55fc99c22e90 2023-11-06T01:44:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/2955389696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda80018070 con 0x55fc99c22e90 2023-11-06T01:44:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fda797fa700 1 -- 172.21.15.53:0/2955389696 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fda8c017070 con 0x55fc99c22e90 2023-11-06T01:44:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fda797fa700 1 -- 172.21.15.53:0/2955389696 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda8c005c40 con 0x55fc99c22e90 2023-11-06T01:44:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fdaa3c32b80 10 client.26186 fetch_fsmap learned FSMap version 46 2023-11-06T01:44:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.914+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/2955389696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55fc99998250 con 0x55fc99c22e90 2023-11-06T01:44:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.915+0000 7fda797fa700 1 -- 172.21.15.53:0/2955389696 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda80018070 con 0x55fc99c22e90 2023-11-06T01:44:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.915+0000 7fda797fa700 1 -- 172.21.15.53:0/2955389696 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda8c011ac0 con 0x55fc99c22e90 2023-11-06T01:44:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.915+0000 7fdaa3c32b80 10 client.26186 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:44:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.915+0000 7fdaa3c32b80 10 client.26186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:44:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.915+0000 7fdaa3c32b80 20 client.26186 populate_metadata read hostname 'smithi053' 2023-11-06T01:44:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.915+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fc99c9bc10 0x55fc99c9e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:44:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.915+0000 7fdaa3c32b80 4 client.26186 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:44:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.915+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/2955389696 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc99b64ce0 con 0x55fc99c9bc10 2023-11-06T01:44:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.915+0000 7fda9503d700 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fc99c9bc10 0x55fc99c9e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:44:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.916+0000 7fda9503d700 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fc99c9bc10 0x55fc99c9e030 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d4b8a473fbf34f server_cookie=895b22f1573bc519 in_seq=0 out_seq=0 2023-11-06T01:44:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.916+0000 7fda797fa700 10 client.26186 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.931+0000 7fda797fa700 1 -- 172.21.15.53:0/2955389696 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc99b64ce0 con 0x55fc99c9bc10 2023-11-06T01:44:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.931+0000 7fda797fa700 10 client.26186 handle_command_reply: tid=0 2023-11-06T01:44:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.931+0000 7fdaa3c32b80 1 client.26186 shutdown 2023-11-06T01:44:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/2955389696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 msgr2=0x55fc99c426b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c426b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0059d0 tx=0x7fda8c010fa0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/2955389696 shutdown_connections 2023-11-06T01:44:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc99c22e90 0x55fc99c426b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fc99c9bc10 0x55fc99c9e030 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc99c246e0 0x55fc99c99a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fdaa3c32b80 1 --2- 172.21.15.53:0/2955389696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc99c237f0 0x55fc99c42bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/2955389696 >> 172.21.15.53:0/2955389696 conn(0x55fc99b7a070 msgr2=0x55fc99c16c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fda797fa700 0 client.26186 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:44:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/2955389696 shutdown_connections 2023-11-06T01:44:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fdaa3c32b80 1 -- 172.21.15.53:0/2955389696 wait complete. 2023-11-06T01:44:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fdaa3c32b80 20 client.26186 trim_cache size 0 max 16384 2023-11-06T01:44:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:44:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:44:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:44:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:44:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fda97aa2700 1 -- 172.21.15.53:0/4237693529 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda7c0a0520 msgr2=0x7fda7c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.932+0000 7fda97aa2700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda7c0a0520 0x7fda7c0a29e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda8001f000 tx=0x7fda8001b000 comp rx=0 tx=0).stop 2023-11-06T01:44:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.933+0000 7fda97aa2700 1 -- 172.21.15.53:0/4237693529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda90106ac0 msgr2=0x7fda90100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:44:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.933+0000 7fda97aa2700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda90106ac0 0x7fda90100290 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fda8800b780 tx=0x7fda8800bd60 comp rx=0 tx=0).stop 2023-11-06T01:44:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.933+0000 7fda97aa2700 1 -- 172.21.15.53:0/4237693529 shutdown_connections 2023-11-06T01:44:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.933+0000 7fda97aa2700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fda7c0a0520 0x7fda7c0a29e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.933+0000 7fda97aa2700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda90101860 0x7fda900ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.933+0000 7fda97aa2700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda90106ac0 0x7fda90100290 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.933+0000 7fda97aa2700 1 --2- 172.21.15.53:0/4237693529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda90104190 0x7fda900ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:44:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.933+0000 7fda97aa2700 1 -- 172.21.15.53:0/4237693529 >> 172.21.15.53:0/4237693529 conn(0x7fda900fb250 msgr2=0x7fda901024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:44:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.933+0000 7fda97aa2700 1 -- 172.21.15.53:0/4237693529 shutdown_connections 2023-11-06T01:44:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:44:48.933+0000 7fda97aa2700 1 -- 172.21.15.53:0/4237693529 wait complete. 2023-11-06T01:44:49.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:49 smithi053 ceph-mon[99472]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 703 op/s 2023-11-06T01:44:49.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:49 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:44:49.894 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:49 smithi062 ceph-mon[103924]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 703 op/s 2023-11-06T01:44:49.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:49 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:44:49.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:49 smithi163 ceph-mon[104416]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 703 op/s 2023-11-06T01:44:49.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:49 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:44:51.852 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:51 smithi053 ceph-mon[99472]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 650 op/s 2023-11-06T01:44:51.920 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:51 smithi062 ceph-mon[103924]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 650 op/s 2023-11-06T01:44:51.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:51 smithi163 ceph-mon[104416]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 650 op/s 2023-11-06T01:44:52.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 635.0 s, 1372.6 tps, lat 71.927 ms stddev 79.724 2023-11-06T01:44:53.499 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:53 smithi163 ceph-mon[104416]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 623 op/s 2023-11-06T01:44:53.499 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:53 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.499 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:53 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.499 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:53 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.500 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:53 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.500 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:53 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.500 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:53 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:53 smithi053 ceph-mon[99472]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 623 op/s 2023-11-06T01:44:53.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:53 smithi062 ceph-mon[103924]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 623 op/s 2023-11-06T01:44:53.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:53 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:53 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:53 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:53 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:53 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:53.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:53 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:54.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:54 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:44:54.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:54 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:44:54.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:54 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:54.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:54 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:44:54.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:54 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:44:54.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:54 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:54.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:54 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:44:54.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:54 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:44:54.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:54 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:44:55.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:55 smithi053 ceph-mon[99472]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 678 op/s 2023-11-06T01:44:55.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:55 smithi062 ceph-mon[103924]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 678 op/s 2023-11-06T01:44:55.761 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:55 smithi163 ceph-mon[104416]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 678 op/s 2023-11-06T01:44:57.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 640.0 s, 1522.6 tps, lat 65.402 ms stddev 80.451 2023-11-06T01:44:57.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:57 smithi062 ceph-mon[103924]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 631 op/s 2023-11-06T01:44:57.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:57 smithi163 ceph-mon[104416]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 631 op/s 2023-11-06T01:44:57.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:57 smithi053 ceph-mon[99472]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 631 op/s 2023-11-06T01:44:59.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:44:59 smithi062 ceph-mon[103924]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 MiB/s wr, 605 op/s 2023-11-06T01:44:59.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:44:59 smithi163 ceph-mon[104416]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 MiB/s wr, 605 op/s 2023-11-06T01:44:59.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:44:59 smithi053 ceph-mon[99472]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 MiB/s wr, 605 op/s 2023-11-06T01:45:01.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:01 smithi163 ceph-mon[104416]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 665 op/s 2023-11-06T01:45:01.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:01 smithi053 ceph-mon[99472]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 665 op/s 2023-11-06T01:45:01.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:01 smithi062 ceph-mon[103924]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 665 op/s 2023-11-06T01:45:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.088+0000 7f79000a9700 1 -- 172.21.15.53:0/891087048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78f8100200 msgr2=0x7f78f8100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.088+0000 7f79000a9700 1 --2- 172.21.15.53:0/891087048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78f8100200 0x7f78f8100620 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009990 tx=0x7f78f4009ca0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.089+0000 7f79000a9700 1 -- 172.21.15.53:0/891087048 shutdown_connections 2023-11-06T01:45:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.089+0000 7f79000a9700 1 --2- 172.21.15.53:0/891087048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f81018d0 0x7f78f8108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.089+0000 7f79000a9700 1 --2- 172.21.15.53:0/891087048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78f8100200 0x7f78f8100620 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.089+0000 7f79000a9700 1 --2- 172.21.15.53:0/891087048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f8100f10 0x7f78f8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.089+0000 7f79000a9700 1 -- 172.21.15.53:0/891087048 >> 172.21.15.53:0/891087048 conn(0x7f78f80fa9d0 msgr2=0x7f78f80fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.091+0000 7f79000a9700 1 -- 172.21.15.53:0/891087048 shutdown_connections 2023-11-06T01:45:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.091+0000 7f79000a9700 1 -- 172.21.15.53:0/891087048 wait complete. 2023-11-06T01:45:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.092+0000 7f79000a9700 1 Processor -- start 2023-11-06T01:45:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.092+0000 7f79000a9700 1 -- start start 2023-11-06T01:45:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.092+0000 7f79000a9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78f8100200 0x7f78f8110280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.092+0000 7f79000a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f8100f10 0x7f78f81107c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.092+0000 7f79000a9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f81018d0 0x7f78f810e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.092+0000 7f79000a9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f810eee0 con 0x7f78f8100200 2023-11-06T01:45:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.092+0000 7f79000a9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f810f050 con 0x7f78f8100f10 2023-11-06T01:45:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.092+0000 7f79000a9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f810f210 con 0x7f78f81018d0 2023-11-06T01:45:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fde45700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78f8100200 0x7f78f8110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fe646700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f81018d0 0x7f78f810e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fd644700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f8100f10 0x7f78f81107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fe646700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f81018d0 0x7f78f810e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57196/0 (socket says 172.21.15.53:57196) 2023-11-06T01:45:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fde45700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78f8100200 0x7f78f8110280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42954/0 (socket says 172.21.15.53:42954) 2023-11-06T01:45:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fe646700 1 -- 172.21.15.53:0/3690716473 learned_addr learned my addr 172.21.15.53:0/3690716473 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fe646700 1 -- 172.21.15.53:0/3690716473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f8100f10 msgr2=0x7f78f81107c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fe646700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f8100f10 0x7f78f81107c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fe646700 1 -- 172.21.15.53:0/3690716473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78f8100200 msgr2=0x7f78f8110280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fe646700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78f8100200 0x7f78f8110280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fe646700 1 -- 172.21.15.53:0/3690716473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f40095f0 con 0x7f78f81018d0 2023-11-06T01:45:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fd644700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f8100f10 0x7f78f81107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78fe646700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f81018d0 0x7f78f810e9a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f78f000ef30 tx=0x7f78f000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78eeffd700 1 -- 172.21.15.53:0/3690716473 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0018070 con 0x7f78f81018d0 2023-11-06T01:45:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78eeffd700 1 -- 172.21.15.53:0/3690716473 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78f000f040 con 0x7f78f81018d0 2023-11-06T01:45:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f79000a9700 1 -- 172.21.15.53:0/3690716473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f810f490 con 0x7f78f81018d0 2023-11-06T01:45:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.093+0000 7f78eeffd700 1 -- 172.21.15.53:0/3690716473 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f000cc20 con 0x7f78f81018d0 2023-11-06T01:45:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.094+0000 7f79000a9700 1 -- 172.21.15.53:0/3690716473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f81ae280 con 0x7f78f81018d0 2023-11-06T01:45:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.096+0000 7f78eeffd700 1 -- 172.21.15.53:0/3690716473 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f78f0003a00 con 0x7f78f81018d0 2023-11-06T01:45:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.097+0000 7f78eeffd700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78e40a05f0 0x7f78e40a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.097+0000 7f78eeffd700 1 -- 172.21.15.53:0/3690716473 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f78f00cb4f0 con 0x7f78f81018d0 2023-11-06T01:45:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.097+0000 7f78fde45700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78e40a05f0 0x7f78e40a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.098+0000 7f78fde45700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78e40a05f0 0x7f78e40a2ab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f78f81112c0 tx=0x7f78f4005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f790c239b80 1 Processor -- start 2023-11-06T01:45:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f790c239b80 1 -- start start 2023-11-06T01:45:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f790c239b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 0x56073b171aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f790c239b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171fe0 0x56073b1728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f790c239b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b172ed0 0x56073b173350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f790c239b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073b0c99b0 con 0x56073b171680 2023-11-06T01:45:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f790c239b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073b0c7370 con 0x56073b171fe0 2023-11-06T01:45:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f790c239b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073b0c2c60 con 0x56073b172ed0 2023-11-06T01:45:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f78fde45700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 0x56073b171aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f78fd644700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171fe0 0x56073b1728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f78fde45700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 0x56073b171aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42962/0 (socket says 172.21.15.53:42962) 2023-11-06T01:45:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f78fd644700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171fe0 0x56073b1728f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54400/0 (socket says 172.21.15.53:54400) 2023-11-06T01:45:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.099+0000 7f78fde45700 1 -- 172.21.15.53:0/3785017766 learned_addr learned my addr 172.21.15.53:0/3785017766 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.100+0000 7f78fe646700 1 --2- 172.21.15.53:0/3785017766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b172ed0 0x56073b173350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.100+0000 7f78fd644700 1 -- 172.21.15.53:0/3785017766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b172ed0 msgr2=0x56073b173350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.100+0000 7f78fd644700 1 --2- 172.21.15.53:0/3785017766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b172ed0 0x56073b173350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.100+0000 7f78fd644700 1 -- 172.21.15.53:0/3785017766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 msgr2=0x56073b171aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.100+0000 7f78fd644700 1 --2- 172.21.15.53:0/3785017766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 0x56073b171aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.100+0000 7f78fd644700 1 -- 172.21.15.53:0/3785017766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56073aef0d80 con 0x56073b171fe0 2023-11-06T01:45:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.100+0000 7f78fd644700 1 --2- 172.21.15.53:0/3785017766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171fe0 0x56073b1728f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f78f80fff20 tx=0x7f78e80117a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9049363d2c29879 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.100+0000 7f78e3fff700 1 -- 172.21.15.53:0/3785017766 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e80042f0 con 0x56073b171fe0 2023-11-06T01:45:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.100+0000 7f78e3fff700 1 -- 172.21.15.53:0/3785017766 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78e8004450 con 0x56073b171fe0 2023-11-06T01:45:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.100+0000 7f78e3fff700 1 -- 172.21.15.53:0/3785017766 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e80056e0 con 0x56073b171fe0 2023-11-06T01:45:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.101+0000 7f790c239b80 1 -- 172.21.15.53:0/3785017766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171fe0 msgr2=0x56073b1728f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.101+0000 7f790c239b80 1 --2- 172.21.15.53:0/3785017766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171fe0 0x56073b1728f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f78f80fff20 tx=0x7f78e80117a0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.101+0000 7f790c239b80 1 -- 172.21.15.53:0/3785017766 shutdown_connections 2023-11-06T01:45:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.101+0000 7f790c239b80 1 --2- 172.21.15.53:0/3785017766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b172ed0 0x56073b173350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.101+0000 7f790c239b80 1 --2- 172.21.15.53:0/3785017766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 0x56073b171aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.101+0000 7f790c239b80 1 --2- 172.21.15.53:0/3785017766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171fe0 0x56073b1728f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.101+0000 7f790c239b80 1 -- 172.21.15.53:0/3785017766 >> 172.21.15.53:0/3785017766 conn(0x56073b0d1070 msgr2=0x56073b16e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.101+0000 7f790c239b80 1 -- 172.21.15.53:0/3785017766 shutdown_connections 2023-11-06T01:45:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.101+0000 7f790c239b80 1 -- 172.21.15.53:0/3785017766 wait complete. 2023-11-06T01:45:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.102+0000 7f790c239b80 1 -- start start 2023-11-06T01:45:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.102+0000 7f790c239b80 1 Processor -- start 2023-11-06T01:45:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.103+0000 7f790c239b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171680 0x56073b17b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.103+0000 7f790c239b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171fe0 0x56073b17b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.103+0000 7f790c239b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b172ed0 0x56073b182950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.103+0000 7f790c239b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073b05f4b0 con 0x56073b171fe0 2023-11-06T01:45:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.103+0000 7f790c239b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073b05f150 con 0x56073b171680 2023-11-06T01:45:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.103+0000 7f790c239b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073af468c0 con 0x56073b172ed0 2023-11-06T01:45:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.103+0000 7f790c239b80 10 client.0 _ll_register_callbacks cb 0x56073afaca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.103+0000 7f78fd644700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171fe0 0x56073b17b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.103+0000 7f78fd644700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171fe0 0x56073b17b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42968/0 (socket says 172.21.15.53:42968) 2023-11-06T01:45:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.103+0000 7f78fd644700 1 -- 172.21.15.53:0/602790797 learned_addr learned my addr 172.21.15.53:0/602790797 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.103+0000 7f78fde45700 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171680 0x56073b17b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.104+0000 7f78fd644700 1 -- 172.21.15.53:0/602790797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b172ed0 msgr2=0x56073b182950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:45:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.104+0000 7f78fd644700 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b172ed0 0x56073b182950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.104+0000 7f78fd644700 1 -- 172.21.15.53:0/602790797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171680 msgr2=0x56073b17b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.104+0000 7f78fd644700 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171680 0x56073b17b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.104+0000 7f78fd644700 1 -- 172.21.15.53:0/602790797 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e8009c90 con 0x56073b171fe0 2023-11-06T01:45:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.104+0000 7f78fde45700 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171680 0x56073b17b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.104+0000 7f78fd644700 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171fe0 0x56073b17b910 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f78e8004420 tx=0x7f78e80056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332520d7c88abe33 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.104+0000 7f78e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:45:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.104+0000 7f78e17fa700 1 -- 172.21.15.53:0/602790797 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8004b60 con 0x56073b171fe0 2023-11-06T01:45:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.104+0000 7f78e17fa700 1 -- 172.21.15.53:0/602790797 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f78e8004cc0 con 0x56073b171fe0 2023-11-06T01:45:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.104+0000 7f790c239b80 1 -- 172.21.15.53:0/602790797 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56073b0edcb0 con 0x56073b171fe0 2023-11-06T01:45:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.105+0000 7f78e17fa700 1 -- 172.21.15.53:0/602790797 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78e8003680 con 0x56073b171fe0 2023-11-06T01:45:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.105+0000 7f790c239b80 10 client.16665 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.105+0000 7f790c239b80 1 -- 172.21.15.53:0/602790797 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56073aed9c00 con 0x56073b171fe0 2023-11-06T01:45:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.105+0000 7f78e17fa700 1 -- 172.21.15.53:0/602790797 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78e8033020 con 0x56073b171fe0 2023-11-06T01:45:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.106+0000 7f78e17fa700 1 -- 172.21.15.53:0/602790797 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f78e80037e0 con 0x56073b171fe0 2023-11-06T01:45:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.106+0000 7f790c239b80 10 client.16665 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.106+0000 7f790c239b80 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.106+0000 7f790c239b80 20 client.16665 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.106+0000 7f790c239b80 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56073b1870d0 0x56073b1894f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.106+0000 7f790c239b80 4 client.16665 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:45:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.106+0000 7f790c239b80 1 -- 172.21.15.53:0/602790797 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56073b140db0 con 0x56073b1870d0 2023-11-06T01:45:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.106+0000 7f78fde45700 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56073b1870d0 0x56073b1894f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.106+0000 7f78e17fa700 10 client.16665 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.106+0000 7f78fde45700 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56073b1870d0 0x56073b1894f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=524efa15b4cea5bd server_cookie=3e4ddb3e59b31ec6 in_seq=0 out_seq=0 2023-11-06T01:45:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f78e17fa700 1 -- 172.21.15.53:0/602790797 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56073b140db0 con 0x56073b1870d0 2023-11-06T01:45:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f78e17fa700 10 client.16665 handle_command_reply: tid=0 2023-11-06T01:45:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f790c239b80 1 client.16665 shutdown 2023-11-06T01:45:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f790c239b80 1 -- 172.21.15.53:0/602790797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171fe0 msgr2=0x56073b17b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f790c239b80 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171fe0 0x56073b17b910 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f78e8004420 tx=0x7f78e80056c0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f790c239b80 1 -- 172.21.15.53:0/602790797 shutdown_connections 2023-11-06T01:45:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f790c239b80 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171680 0x56073b17b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f790c239b80 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56073b1870d0 0x56073b1894f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f790c239b80 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b172ed0 0x56073b182950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f790c239b80 1 --2- 172.21.15.53:0/602790797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171fe0 0x56073b17b910 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f78e17fa700 0 client.16665 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f790c239b80 1 -- 172.21.15.53:0/602790797 >> 172.21.15.53:0/602790797 conn(0x56073b0d1070 msgr2=0x56073b16c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f790c239b80 1 -- 172.21.15.53:0/602790797 shutdown_connections 2023-11-06T01:45:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.107+0000 7f790c239b80 1 -- 172.21.15.53:0/602790797 wait complete. 2023-11-06T01:45:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.108+0000 7f790c239b80 20 client.16665 trim_cache size 0 max 16384 2023-11-06T01:45:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.134+0000 7f790c239b80 1 Processor -- start 2023-11-06T01:45:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.134+0000 7f790c239b80 1 -- start start 2023-11-06T01:45:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.134+0000 7f790c239b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 0x56073b18de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.134+0000 7f790c239b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b171fe0 0x56073b182910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.134+0000 7f790c239b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b172ed0 0x56073b1732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.134+0000 7f790c239b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073b167500 con 0x56073b171680 2023-11-06T01:45:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.134+0000 7f790c239b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073b186ef0 con 0x56073b172ed0 2023-11-06T01:45:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.134+0000 7f790c239b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073b16d880 con 0x56073b171fe0 2023-11-06T01:45:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.136+0000 7f78fe646700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b172ed0 0x56073b1732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.142+0000 7f78fe646700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b172ed0 0x56073b1732b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54414/0 (socket says 172.21.15.53:54414) 2023-11-06T01:45:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.142+0000 7f78fe646700 1 -- 172.21.15.53:0/1872982042 learned_addr learned my addr 172.21.15.53:0/1872982042 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.144+0000 7f78fde45700 1 --2- 172.21.15.53:0/1872982042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 0x56073b18de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.144+0000 7f78fde45700 1 -- 172.21.15.53:0/1872982042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b171fe0 msgr2=0x56073b182910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.144+0000 7f78fde45700 1 --2- 172.21.15.53:0/1872982042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b171fe0 0x56073b182910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.144+0000 7f78fde45700 1 -- 172.21.15.53:0/1872982042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b172ed0 msgr2=0x56073b1732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.144+0000 7f78fde45700 1 --2- 172.21.15.53:0/1872982042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b172ed0 0x56073b1732b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.144+0000 7f78fde45700 1 -- 172.21.15.53:0/1872982042 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56073af171f0 con 0x56073b171680 2023-11-06T01:45:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.144+0000 7f78fde45700 1 --2- 172.21.15.53:0/1872982042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 0x56073b18de50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f78f40098e0 tx=0x7f78f4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e267b8708567ca9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.145+0000 7f78ecff9700 1 -- 172.21.15.53:0/1872982042 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4024bd0 con 0x56073b171680 2023-11-06T01:45:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.145+0000 7f78ecff9700 1 -- 172.21.15.53:0/1872982042 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78f401e040 con 0x56073b171680 2023-11-06T01:45:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.145+0000 7f78ecff9700 1 -- 172.21.15.53:0/1872982042 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4036450 con 0x56073b171680 2023-11-06T01:45:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.146+0000 7f790c239b80 1 -- 172.21.15.53:0/1872982042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 msgr2=0x56073b18de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.146+0000 7f790c239b80 1 --2- 172.21.15.53:0/1872982042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 0x56073b18de50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f78f40098e0 tx=0x7f78f4020000 comp rx=0 tx=0).stop 2023-11-06T01:45:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.148+0000 7f790c239b80 1 -- 172.21.15.53:0/1872982042 shutdown_connections 2023-11-06T01:45:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.148+0000 7f790c239b80 1 --2- 172.21.15.53:0/1872982042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b172ed0 0x56073b1732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.148+0000 7f790c239b80 1 --2- 172.21.15.53:0/1872982042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b171680 0x56073b18de50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.148+0000 7f790c239b80 1 --2- 172.21.15.53:0/1872982042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b171fe0 0x56073b182910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.148+0000 7f790c239b80 1 -- 172.21.15.53:0/1872982042 >> 172.21.15.53:0/1872982042 conn(0x56073b0d1070 msgr2=0x56073b16dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.152+0000 7f790c239b80 1 -- 172.21.15.53:0/1872982042 shutdown_connections 2023-11-06T01:45:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.152+0000 7f790c239b80 1 -- 172.21.15.53:0/1872982042 wait complete. 2023-11-06T01:45:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.153+0000 7f790c239b80 1 -- start start 2023-11-06T01:45:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.153+0000 7f790c239b80 1 Processor -- start 2023-11-06T01:45:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f790c239b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b171680 0x56073b1806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f790c239b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171fe0 0x56073b180c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f790c239b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b172ed0 0x56073b181c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f790c239b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073b182160 con 0x56073b172ed0 2023-11-06T01:45:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f790c239b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073b1822d0 con 0x56073b171fe0 2023-11-06T01:45:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f790c239b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073b182490 con 0x56073b171680 2023-11-06T01:45:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f790c239b80 10 client.0 _ll_register_callbacks cb 0x7f78f4004620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f78fe646700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b172ed0 0x56073b181c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f78fde45700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b171680 0x56073b1806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f78fe646700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b172ed0 0x56073b181c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42980/0 (socket says 172.21.15.53:42980) 2023-11-06T01:45:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f78fe646700 1 -- 172.21.15.53:0/2145900348 learned_addr learned my addr 172.21.15.53:0/2145900348 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f78fe646700 1 -- 172.21.15.53:0/2145900348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b171680 msgr2=0x56073b1806d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f78fe646700 1 --2- 172.21.15.53:0/2145900348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b171680 0x56073b1806d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f78fe646700 1 -- 172.21.15.53:0/2145900348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171fe0 msgr2=0x56073b180c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f78fe646700 1 --2- 172.21.15.53:0/2145900348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171fe0 0x56073b180c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.154+0000 7f78fe646700 1 -- 172.21.15.53:0/2145900348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f8108a90 con 0x56073b172ed0 2023-11-06T01:45:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.155+0000 7f78fe646700 1 --2- 172.21.15.53:0/2145900348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b172ed0 0x56073b181c20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f78f00957d0 tx=0x7f78f001b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a9a74a27a5d4f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.155+0000 7f78e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:45:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.155+0000 7f78e17fa700 1 -- 172.21.15.53:0/2145900348 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f009cb90 con 0x56073b172ed0 2023-11-06T01:45:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.155+0000 7f790c239b80 1 -- 172.21.15.53:0/2145900348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78e8004450 con 0x56073b172ed0 2023-11-06T01:45:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.155+0000 7f78e17fa700 1 -- 172.21.15.53:0/2145900348 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f78f00a0d40 con 0x56073b172ed0 2023-11-06T01:45:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.155+0000 7f78e17fa700 1 -- 172.21.15.53:0/2145900348 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78f0097100 con 0x56073b172ed0 2023-11-06T01:45:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.155+0000 7f790c239b80 10 client.16671 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.155+0000 7f790c239b80 1 -- 172.21.15.53:0/2145900348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56073aeef250 con 0x56073b172ed0 2023-11-06T01:45:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.155+0000 7f78e17fa700 1 -- 172.21.15.53:0/2145900348 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78f0077e90 con 0x56073b172ed0 2023-11-06T01:45:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.156+0000 7f78e17fa700 1 -- 172.21.15.53:0/2145900348 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f78f00cb160 con 0x56073b172ed0 2023-11-06T01:45:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.156+0000 7f790c239b80 10 client.16671 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.156+0000 7f790c239b80 10 client.16671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.156+0000 7f790c239b80 20 client.16671 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.156+0000 7f790c239b80 1 --2- 172.21.15.53:0/2145900348 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56073b1d9c50 0x56073b1dc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.156+0000 7f790c239b80 4 client.16671 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:45:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.156+0000 7f790c239b80 1 -- 172.21.15.53:0/2145900348 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56073b0bbce0 con 0x56073b1d9c50 2023-11-06T01:45:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.156+0000 7f78fde45700 1 --2- 172.21.15.53:0/2145900348 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56073b1d9c50 0x56073b1dc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.156+0000 7f78e17fa700 10 client.16671 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.156+0000 7f78fde45700 1 --2- 172.21.15.53:0/2145900348 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56073b1d9c50 0x56073b1dc070 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f775191073d259e server_cookie=58c5ceb261f478c1 in_seq=0 out_seq=0 2023-11-06T01:45:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f78e17fa700 1 -- 172.21.15.53:0/2145900348 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56073b0bbce0 con 0x56073b1d9c50 2023-11-06T01:45:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f78e17fa700 10 client.16671 handle_command_reply: tid=0 2023-11-06T01:45:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 1 client.16671 shutdown 2023-11-06T01:45:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 1 -- 172.21.15.53:0/2145900348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b172ed0 msgr2=0x56073b181c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 1 --2- 172.21.15.53:0/2145900348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b172ed0 0x56073b181c20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f78f00957d0 tx=0x7f78f001b3d0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 1 -- 172.21.15.53:0/2145900348 shutdown_connections 2023-11-06T01:45:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 1 --2- 172.21.15.53:0/2145900348 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56073b1d9c50 0x56073b1dc070 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 1 --2- 172.21.15.53:0/2145900348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56073b172ed0 0x56073b181c20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 1 --2- 172.21.15.53:0/2145900348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56073b171680 0x56073b1806d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 1 --2- 172.21.15.53:0/2145900348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56073b171fe0 0x56073b180c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 1 -- 172.21.15.53:0/2145900348 >> 172.21.15.53:0/2145900348 conn(0x56073b0d1070 msgr2=0x56073b17b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f78e17fa700 0 client.16671 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 1 -- 172.21.15.53:0/2145900348 shutdown_connections 2023-11-06T01:45:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 1 -- 172.21.15.53:0/2145900348 wait complete. 2023-11-06T01:45:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.173+0000 7f790c239b80 20 client.16671 trim_cache size 0 max 16384 2023-11-06T01:45:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:45:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:45:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:45:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:45:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 -- 172.21.15.53:0/3690716473 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78e40a05f0 msgr2=0x7f78e40a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78e40a05f0 0x7f78e40a2ab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f78f81112c0 tx=0x7f78f4005ad0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 -- 172.21.15.53:0/3690716473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f81018d0 msgr2=0x7f78f810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f81018d0 0x7f78f810e9a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f78f000ef30 tx=0x7f78f000c3a0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 -- 172.21.15.53:0/3690716473 shutdown_connections 2023-11-06T01:45:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f78e40a05f0 0x7f78e40a2ab0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f81018d0 0x7f78f810e9a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78f8100200 0x7f78f8110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 --2- 172.21.15.53:0/3690716473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f8100f10 0x7f78f81107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 -- 172.21.15.53:0/3690716473 >> 172.21.15.53:0/3690716473 conn(0x7f78f80fa9d0 msgr2=0x7f78f8104400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 -- 172.21.15.53:0/3690716473 shutdown_connections 2023-11-06T01:45:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:02.174+0000 7f79000a9700 1 -- 172.21.15.53:0/3690716473 wait complete. 2023-11-06T01:45:02.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 645.0 s, 1891.0 tps, lat 53.216 ms stddev 64.884 2023-11-06T01:45:03.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:03 smithi163 ceph-mon[104416]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.7 MiB/s wr, 598 op/s 2023-11-06T01:45:03.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:03 smithi053 ceph-mon[99472]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.7 MiB/s wr, 598 op/s 2023-11-06T01:45:03.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:03 smithi062 ceph-mon[103924]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.7 MiB/s wr, 598 op/s 2023-11-06T01:45:05.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:05 smithi163 ceph-mon[104416]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.8 MiB/s wr, 718 op/s 2023-11-06T01:45:05.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:05 smithi053 ceph-mon[99472]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.8 MiB/s wr, 718 op/s 2023-11-06T01:45:05.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:05 smithi062 ceph-mon[103924]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.8 MiB/s wr, 718 op/s 2023-11-06T01:45:07.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 650.0 s, 1538.9 tps, lat 64.898 ms stddev 68.666 2023-11-06T01:45:07.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:07 smithi163 ceph-mon[104416]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.6 MiB/s wr, 678 op/s 2023-11-06T01:45:07.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:07 smithi053 ceph-mon[99472]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.6 MiB/s wr, 678 op/s 2023-11-06T01:45:07.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:07 smithi062 ceph-mon[103924]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.6 MiB/s wr, 678 op/s 2023-11-06T01:45:09.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:09 smithi163 ceph-mon[104416]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 724 op/s 2023-11-06T01:45:09.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:09 smithi053 ceph-mon[99472]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 724 op/s 2023-11-06T01:45:09.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:09 smithi062 ceph-mon[103924]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 724 op/s 2023-11-06T01:45:09.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:45:10.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.082+0000 7ff8a6736700 1 -- 172.21.15.53:0/301587322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8a0101810 msgr2=0x7ff8a01080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:10.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.082+0000 7ff8a6736700 1 --2- 172.21.15.53:0/301587322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8a0101810 0x7ff8a01080c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff898009f90 tx=0x7ff89800b330 comp rx=0 tx=0).stop 2023-11-06T01:45:10.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.082+0000 7ff8a6736700 1 -- 172.21.15.53:0/301587322 shutdown_connections 2023-11-06T01:45:10.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.082+0000 7ff8a6736700 1 --2- 172.21.15.53:0/301587322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8a0101810 0x7ff8a01080c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.082+0000 7ff8a6736700 1 --2- 172.21.15.53:0/301587322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a00ffc80 0x7ff8a01000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.082+0000 7ff8a6736700 1 --2- 172.21.15.53:0/301587322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a0100e70 0x7ff8a01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.082+0000 7ff8a6736700 1 -- 172.21.15.53:0/301587322 >> 172.21.15.53:0/301587322 conn(0x7ff8a00fb1f0 msgr2=0x7ff8a00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:10.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.082+0000 7ff8a6736700 1 -- 172.21.15.53:0/301587322 shutdown_connections 2023-11-06T01:45:10.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.082+0000 7ff8a6736700 1 -- 172.21.15.53:0/301587322 wait complete. 2023-11-06T01:45:10.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff8a6736700 1 Processor -- start 2023-11-06T01:45:10.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff8a6736700 1 -- start start 2023-11-06T01:45:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff8a6736700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a00ffc80 0x7ff8a0198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff8a6736700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a0100e70 0x7ff8a0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff8a6736700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8a0101810 0x7ff8a01a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff8a6736700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a01a0880 con 0x7ff8a00ffc80 2023-11-06T01:45:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff8a6736700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a01a09f0 con 0x7ff8a0100e70 2023-11-06T01:45:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff8a6736700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a01a0bb0 con 0x7ff8a0101810 2023-11-06T01:45:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a00ffc80 0x7ff8a0198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a00ffc80 0x7ff8a0198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42992/0 (socket says 172.21.15.53:42992) 2023-11-06T01:45:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff8a4cd3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8a0101810 0x7ff8a01a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff89ffff700 1 -- 172.21.15.53:0/115307023 learned_addr learned my addr 172.21.15.53:0/115307023 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff89f7fe700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a0100e70 0x7ff8a0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff89ffff700 1 -- 172.21.15.53:0/115307023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8a0101810 msgr2=0x7ff8a01a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff89ffff700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8a0101810 0x7ff8a01a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff89ffff700 1 -- 172.21.15.53:0/115307023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a0100e70 msgr2=0x7ff8a0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff89ffff700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a0100e70 0x7ff8a0199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.083+0000 7ff89ffff700 1 -- 172.21.15.53:0/115307023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894009520 con 0x7ff8a00ffc80 2023-11-06T01:45:10.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.084+0000 7ff89ffff700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a00ffc80 0x7ff8a0198d40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff89000e9f0 tx=0x7ff89000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:10.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.084+0000 7ff89d7fa700 1 -- 172.21.15.53:0/115307023 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890004220 con 0x7ff8a00ffc80 2023-11-06T01:45:10.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.084+0000 7ff8a6736700 1 -- 172.21.15.53:0/115307023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff898009c40 con 0x7ff8a00ffc80 2023-11-06T01:45:10.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.084+0000 7ff89d7fa700 1 -- 172.21.15.53:0/115307023 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff890004380 con 0x7ff8a00ffc80 2023-11-06T01:45:10.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.084+0000 7ff89d7fa700 1 -- 172.21.15.53:0/115307023 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890003be0 con 0x7ff8a00ffc80 2023-11-06T01:45:10.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.084+0000 7ff8a6736700 1 -- 172.21.15.53:0/115307023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a01a11b0 con 0x7ff8a00ffc80 2023-11-06T01:45:10.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.087+0000 7ff89d7fa700 1 -- 172.21.15.53:0/115307023 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff890003d40 con 0x7ff8a00ffc80 2023-11-06T01:45:10.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.088+0000 7ff89d7fa700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8880a05a0 0x7ff8880a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.088+0000 7ff89f7fe700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8880a05a0 0x7ff8880a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.088+0000 7ff89d7fa700 1 -- 172.21.15.53:0/115307023 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff890014070 con 0x7ff8a00ffc80 2023-11-06T01:45:10.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff89f7fe700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8880a05a0 0x7ff8880a2a60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff894009df0 tx=0x7ff894009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:10.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8b28c6b80 1 Processor -- start 2023-11-06T01:45:10.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8b28c6b80 1 -- start start 2023-11-06T01:45:10.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f330f0 0x5633e0f33530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f33a70 0x5633e0f31e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f32870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0e89a10 con 0x5633e0f323f0 2023-11-06T01:45:10.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0e873d0 con 0x5633e0f33a70 2023-11-06T01:45:10.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0e82cc0 con 0x5633e0f330f0 2023-11-06T01:45:10.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8a4cd3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f32870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f33a70 0x5633e0f31e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8a4cd3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f32870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43004/0 (socket says 172.21.15.53:43004) 2023-11-06T01:45:10.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f330f0 0x5633e0f33530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8a4cd3700 1 -- 172.21.15.53:0/420179467 learned_addr learned my addr 172.21.15.53:0/420179467 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:10.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8a4cd3700 1 -- 172.21.15.53:0/420179467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f330f0 msgr2=0x5633e0f33530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:10.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.089+0000 7ff8a4cd3700 1 --2- 172.21.15.53:0/420179467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f330f0 0x5633e0f33530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.090+0000 7ff8a4cd3700 1 -- 172.21.15.53:0/420179467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f33a70 msgr2=0x5633e0f31e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:10.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.090+0000 7ff8a4cd3700 1 --2- 172.21.15.53:0/420179467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f33a70 0x5633e0f31e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.090+0000 7ff8a4cd3700 1 -- 172.21.15.53:0/420179467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e0cb0de0 con 0x5633e0f323f0 2023-11-06T01:45:10.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.090+0000 7ff89f7fe700 1 --2- 172.21.15.53:0/420179467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f33a70 0x5633e0f31e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:10.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.090+0000 7ff8a4cd3700 1 --2- 172.21.15.53:0/420179467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f32870 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff898009040 tx=0x7ff898009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d813073832471fc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:10.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.090+0000 7ff8867fc700 1 -- 172.21.15.53:0/420179467 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89801f070 con 0x5633e0f323f0 2023-11-06T01:45:10.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.090+0000 7ff8867fc700 1 -- 172.21.15.53:0/420179467 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8980092e0 con 0x5633e0f323f0 2023-11-06T01:45:10.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.090+0000 7ff8867fc700 1 -- 172.21.15.53:0/420179467 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898007790 con 0x5633e0f323f0 2023-11-06T01:45:10.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.091+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/420179467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 msgr2=0x5633e0f32870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:10.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.091+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/420179467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f32870 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff898009040 tx=0x7ff898009ac0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.091+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/420179467 shutdown_connections 2023-11-06T01:45:10.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.091+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/420179467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f32870 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.091+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/420179467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f330f0 0x5633e0f33530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.091+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/420179467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f33a70 0x5633e0f31e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.091+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/420179467 >> 172.21.15.53:0/420179467 conn(0x5633e0e910d0 msgr2=0x5633e0f2c940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:10.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.091+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/420179467 shutdown_connections 2023-11-06T01:45:10.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.091+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/420179467 wait complete. 2023-11-06T01:45:10.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.091+0000 7ff8b28c6b80 1 -- start start 2023-11-06T01:45:10.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff8b28c6b80 1 Processor -- start 2023-11-06T01:45:10.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 0x5633e0f3b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f330f0 0x5633e0f3be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f33a70 0x5633e0f42e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0e1f510 con 0x5633e0f33a70 2023-11-06T01:45:10.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0e1f1b0 con 0x5633e0f330f0 2023-11-06T01:45:10.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0d06920 con 0x5633e0f323f0 2023-11-06T01:45:10.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff8b28c6b80 10 client.0 _ll_register_callbacks cb 0x5633e0d6ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:10.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 0x5633e0f3b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 0x5633e0f3b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57246/0 (socket says 172.21.15.53:57246) 2023-11-06T01:45:10.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff89ffff700 1 -- 172.21.15.53:0/2117889420 learned_addr learned my addr 172.21.15.53:0/2117889420 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:10.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.092+0000 7ff8a4cd3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f33a70 0x5633e0f42e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.093+0000 7ff89ffff700 1 -- 172.21.15.53:0/2117889420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f330f0 msgr2=0x5633e0f3be50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:45:10.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.093+0000 7ff89ffff700 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f330f0 0x5633e0f3be50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.093+0000 7ff89ffff700 1 -- 172.21.15.53:0/2117889420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f33a70 msgr2=0x5633e0f42e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:10.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.093+0000 7ff89ffff700 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f33a70 0x5633e0f42e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.093+0000 7ff89ffff700 1 -- 172.21.15.53:0/2117889420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff894017070 con 0x5633e0f323f0 2023-11-06T01:45:10.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.093+0000 7ff8a4cd3700 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f33a70 0x5633e0f42e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:10.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.093+0000 7ff89ffff700 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 0x5633e0f3b910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff8900c9540 tx=0x7ff890098c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a26c290e4c5d3fc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:10.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.093+0000 7ff87f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:45:10.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.093+0000 7ff87f7fe700 1 -- 172.21.15.53:0/2117889420 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890099700 con 0x5633e0f323f0 2023-11-06T01:45:10.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.093+0000 7ff87f7fe700 1 -- 172.21.15.53:0/2117889420 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff8900b0070 con 0x5633e0f323f0 2023-11-06T01:45:10.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.093+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/2117889420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633e0eadd10 con 0x5633e0f323f0 2023-11-06T01:45:10.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff87f7fe700 1 -- 172.21.15.53:0/2117889420 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8900992a0 con 0x5633e0f323f0 2023-11-06T01:45:10.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff8b28c6b80 10 client.26272 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:10.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/2117889420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7ff8a0108a10 con 0x5633e0f323f0 2023-11-06T01:45:10.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff87f7fe700 1 -- 172.21.15.53:0/2117889420 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633e0eadd10 con 0x5633e0f323f0 2023-11-06T01:45:10.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff87f7fe700 1 -- 172.21.15.53:0/2117889420 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff890064f90 con 0x5633e0f323f0 2023-11-06T01:45:10.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff8b28c6b80 10 client.26272 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:10.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff8b28c6b80 10 client.26272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:10.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff8b28c6b80 20 client.26272 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:10.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5633e0f47660 0x5633e0f49a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff8b28c6b80 4 client.26272 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:45:10.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/2117889420 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633e0f00e10 con 0x5633e0f47660 2023-11-06T01:45:10.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.094+0000 7ff89f7fe700 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5633e0f47660 0x5633e0f49a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.095+0000 7ff89f7fe700 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5633e0f47660 0x5633e0f49a80 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f65329ddc29eab7 server_cookie=1ca8bab520513804 in_seq=0 out_seq=0 2023-11-06T01:45:10.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.095+0000 7ff87f7fe700 10 client.26272 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:10.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.095+0000 7ff87f7fe700 1 -- 172.21.15.53:0/2117889420 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5633e0f00e10 con 0x5633e0f47660 2023-11-06T01:45:10.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.095+0000 7ff87f7fe700 10 client.26272 handle_command_reply: tid=0 2023-11-06T01:45:10.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.095+0000 7ff8b28c6b80 1 client.26272 shutdown 2023-11-06T01:45:10.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.095+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/2117889420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 msgr2=0x5633e0f3b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:10.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.095+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 0x5633e0f3b910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff8900c9540 tx=0x7ff890098c80 comp rx=0 tx=0).stop 2023-11-06T01:45:10.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.096+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/2117889420 shutdown_connections 2023-11-06T01:45:10.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.096+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5633e0f47660 0x5633e0f49a80 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.096+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f33a70 0x5633e0f42e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.096+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 0x5633e0f3b910 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.096+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/2117889420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f330f0 0x5633e0f3be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.096+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/2117889420 >> 172.21.15.53:0/2117889420 conn(0x5633e0e910d0 msgr2=0x5633e0f2c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:10.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.096+0000 7ff87f7fe700 0 client.26272 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:10.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.097+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/2117889420 shutdown_connections 2023-11-06T01:45:10.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.097+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/2117889420 wait complete. 2023-11-06T01:45:10.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.097+0000 7ff8b28c6b80 20 client.26272 trim_cache size 0 max 16384 2023-11-06T01:45:10.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.115+0000 7ff8b28c6b80 1 Processor -- start 2023-11-06T01:45:10.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.115+0000 7ff8b28c6b80 1 -- start start 2023-11-06T01:45:10.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.115+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0e910d0 0x5633e0f31e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.115+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 0x5633e0f40e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.115+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f330f0 0x5633e0f3b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.115+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0f3adc0 con 0x5633e0f330f0 2023-11-06T01:45:10.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.115+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0f2ddd0 con 0x5633e0e910d0 2023-11-06T01:45:10.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.115+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0f3ca90 con 0x5633e0f323f0 2023-11-06T01:45:10.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0e910d0 0x5633e0f31e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff8a4cd3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f330f0 0x5633e0f3b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0e910d0 0x5633e0f31e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54452/0 (socket says 172.21.15.53:54452) 2023-11-06T01:45:10.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff89ffff700 1 -- 172.21.15.53:0/35430965 learned_addr learned my addr 172.21.15.53:0/35430965 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:10.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 0x5633e0f40e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff8a4cd3700 1 -- 172.21.15.53:0/35430965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 msgr2=0x5633e0f40e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:10.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff8a4cd3700 1 --2- 172.21.15.53:0/35430965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 0x5633e0f40e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff8a4cd3700 1 -- 172.21.15.53:0/35430965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0e910d0 msgr2=0x5633e0f31e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:10.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff8a4cd3700 1 --2- 172.21.15.53:0/35430965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0e910d0 0x5633e0f31e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff8a4cd3700 1 -- 172.21.15.53:0/35430965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e0c99c60 con 0x5633e0f330f0 2023-11-06T01:45:10.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff8a4cd3700 1 --2- 172.21.15.53:0/35430965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f330f0 0x5633e0f3b6c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff89800b6c0 tx=0x7ff89800b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eccace963a63bb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:10.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.116+0000 7ff886ffd700 1 -- 172.21.15.53:0/35430965 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89801f040 con 0x5633e0f330f0 2023-11-06T01:45:10.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.117+0000 7ff886ffd700 1 -- 172.21.15.53:0/35430965 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff898007420 con 0x5633e0f330f0 2023-11-06T01:45:10.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.117+0000 7ff886ffd700 1 -- 172.21.15.53:0/35430965 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8980116c0 con 0x5633e0f330f0 2023-11-06T01:45:10.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.117+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/35430965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f330f0 msgr2=0x5633e0f3b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:10.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.117+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/35430965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f330f0 0x5633e0f3b6c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff89800b6c0 tx=0x7ff89800b770 comp rx=0 tx=0).stop 2023-11-06T01:45:10.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.117+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/35430965 shutdown_connections 2023-11-06T01:45:10.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.117+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/35430965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0e910d0 0x5633e0f31e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.117+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/35430965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f330f0 0x5633e0f3b6c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.117+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/35430965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0f323f0 0x5633e0f40e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.117+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/35430965 >> 172.21.15.53:0/35430965 conn(0x5633e0f3a9b0 msgr2=0x5633e0f36290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:10.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.117+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/35430965 shutdown_connections 2023-11-06T01:45:10.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.117+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/35430965 wait complete. 2023-11-06T01:45:10.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff8b28c6b80 1 -- start start 2023-11-06T01:45:10.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff8b28c6b80 1 Processor -- start 2023-11-06T01:45:10.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0e910d0 0x5633e0f40ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f3f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff8b28c6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f330f0 0x5633e0f3f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0f3fb40 con 0x5633e0f323f0 2023-11-06T01:45:10.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0f3fcb0 con 0x5633e0f330f0 2023-11-06T01:45:10.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff8b28c6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e0f3fe70 con 0x5633e0e910d0 2023-11-06T01:45:10.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff8b28c6b80 10 client.0 _ll_register_callbacks cb 0x7ff890098de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:10.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f3f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f3f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43022/0 (socket says 172.21.15.53:43022) 2023-11-06T01:45:10.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0e910d0 0x5633e0f40ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff89f7fe700 1 -- 172.21.15.53:0/4180032548 learned_addr learned my addr 172.21.15.53:0/4180032548 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:10.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff8a4cd3700 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f330f0 0x5633e0f3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff89f7fe700 1 -- 172.21.15.53:0/4180032548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0e910d0 msgr2=0x5633e0f40ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:10.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff89f7fe700 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0e910d0 0x5633e0f40ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff89f7fe700 1 -- 172.21.15.53:0/4180032548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f330f0 msgr2=0x5633e0f3f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:10.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff89f7fe700 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f330f0 0x5633e0f3f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.118+0000 7ff89f7fe700 1 -- 172.21.15.53:0/4180032548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633e0cb0de0 con 0x5633e0f323f0 2023-11-06T01:45:10.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff89f7fe700 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f3f0c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff894013010 tx=0x7ff8940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa7d4aad659c715 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:10.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff87f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:45:10.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff87f7fe700 1 -- 172.21.15.53:0/4180032548 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894022040 con 0x5633e0f323f0 2023-11-06T01:45:10.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff87f7fe700 1 -- 172.21.15.53:0/4180032548 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff89401d820 con 0x5633e0f323f0 2023-11-06T01:45:10.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/4180032548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8980092e0 con 0x5633e0f323f0 2023-11-06T01:45:10.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff87f7fe700 1 -- 172.21.15.53:0/4180032548 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff89401d9e0 con 0x5633e0f323f0 2023-11-06T01:45:10.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff8b28c6b80 10 client.16689 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:10.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/4180032548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5633e0cd7250 con 0x5633e0f323f0 2023-11-06T01:45:10.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff87f7fe700 1 -- 172.21.15.53:0/4180032548 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8980092e0 con 0x5633e0f323f0 2023-11-06T01:45:10.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff87f7fe700 1 -- 172.21.15.53:0/4180032548 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff894039020 con 0x5633e0f323f0 2023-11-06T01:45:10.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff8b28c6b80 10 client.16689 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:10.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff8b28c6b80 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:10.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff8b28c6b80 20 client.16689 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:10.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5633e0f33a70 0x5633e0fa50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:10.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff8b28c6b80 4 client.16689 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:45:10.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.119+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/4180032548 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633e0e7bd40 con 0x5633e0f33a70 2023-11-06T01:45:10.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff89ffff700 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5633e0f33a70 0x5633e0fa50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:10.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff87f7fe700 10 client.16689 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:10.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff89ffff700 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5633e0f33a70 0x5633e0fa50f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bf8fe68e02d6c1 server_cookie=f63a3835779b6eab in_seq=0 out_seq=0 2023-11-06T01:45:10.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff87f7fe700 1 -- 172.21.15.53:0/4180032548 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5633e0e7bd40 con 0x5633e0f33a70 2023-11-06T01:45:10.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff87f7fe700 10 client.16689 handle_command_reply: tid=0 2023-11-06T01:45:10.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff8b28c6b80 1 client.16689 shutdown 2023-11-06T01:45:10.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/4180032548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 msgr2=0x5633e0f3f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:10.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f3f0c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff894013010 tx=0x7ff8940047c0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/4180032548 shutdown_connections 2023-11-06T01:45:10.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633e0f330f0 0x5633e0f3f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5633e0f33a70 0x5633e0fa50f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5633e0e910d0 0x5633e0f40ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff87f7fe700 0 client.16689 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:10.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff8b28c6b80 1 --2- 172.21.15.53:0/4180032548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633e0f323f0 0x5633e0f3f0c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/4180032548 >> 172.21.15.53:0/4180032548 conn(0x5633e0f3a9b0 msgr2=0x5633e0f3b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:10.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/4180032548 shutdown_connections 2023-11-06T01:45:10.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.120+0000 7ff8b28c6b80 1 -- 172.21.15.53:0/4180032548 wait complete. 2023-11-06T01:45:10.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.121+0000 7ff8b28c6b80 20 client.16689 trim_cache size 0 max 16384 2023-11-06T01:45:10.136 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:45:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:45:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:45:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:45:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:45:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:45:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:45:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:45:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:45:10.138 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:45:10.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.121+0000 7ff8a6736700 1 -- 172.21.15.53:0/115307023 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8880a05a0 msgr2=0x7ff8880a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:10.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.121+0000 7ff8a6736700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8880a05a0 0x7ff8880a2a60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff894009df0 tx=0x7ff894009260 comp rx=0 tx=0).stop 2023-11-06T01:45:10.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.122+0000 7ff8a6736700 1 -- 172.21.15.53:0/115307023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a00ffc80 msgr2=0x7ff8a0198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:10.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.122+0000 7ff8a6736700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a00ffc80 0x7ff8a0198d40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff89000e9f0 tx=0x7ff89000ed00 comp rx=0 tx=0).stop 2023-11-06T01:45:10.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.122+0000 7ff8a6736700 1 -- 172.21.15.53:0/115307023 shutdown_connections 2023-11-06T01:45:10.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.122+0000 7ff8a6736700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff8880a05a0 0x7ff8880a2a60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.122+0000 7ff8a6736700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8a0101810 0x7ff8a01a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.122+0000 7ff8a6736700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a00ffc80 0x7ff8a0198d40 secure :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff89000e9f0 tx=0x7ff89000ed00 comp rx=0 tx=0).stop 2023-11-06T01:45:10.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.122+0000 7ff8a6736700 1 --2- 172.21.15.53:0/115307023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a0100e70 0x7ff8a0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:10.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.122+0000 7ff8a6736700 1 -- 172.21.15.53:0/115307023 >> 172.21.15.53:0/115307023 conn(0x7ff8a00fb1f0 msgr2=0x7ff8a00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:10.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.122+0000 7ff8a6736700 1 -- 172.21.15.53:0/115307023 shutdown_connections 2023-11-06T01:45:10.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:10.122+0000 7ff8a6736700 1 -- 172.21.15.53:0/115307023 wait complete. 2023-11-06T01:45:10.142 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:45:11.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:11 smithi163 ceph-mon[104416]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 11 MiB/s wr, 804 op/s 2023-11-06T01:45:11.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:11 smithi053 ceph-mon[99472]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 11 MiB/s wr, 804 op/s 2023-11-06T01:45:11.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:11 smithi062 ceph-mon[103924]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 11 MiB/s wr, 804 op/s 2023-11-06T01:45:12.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 655.0 s, 1664.3 tps, lat 60.582 ms stddev 67.046 2023-11-06T01:45:13.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:13 smithi163 ceph-mon[104416]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.7 MiB/s wr, 707 op/s 2023-11-06T01:45:13.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:13 smithi053 ceph-mon[99472]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.7 MiB/s wr, 707 op/s 2023-11-06T01:45:13.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:13 smithi062 ceph-mon[103924]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.7 MiB/s wr, 707 op/s 2023-11-06T01:45:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.336+0000 7fc314ae7700 1 -- 172.21.15.53:0/2687999149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc310101820 msgr2=0x7fc3101080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:15.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.336+0000 7fc314ae7700 1 --2- 172.21.15.53:0/2687999149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc310101820 0x7fc3101080d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc308009fc0 tx=0x7fc30800b330 comp rx=0 tx=0).stop 2023-11-06T01:45:15.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.336+0000 7fc314ae7700 1 -- 172.21.15.53:0/2687999149 shutdown_connections 2023-11-06T01:45:15.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.336+0000 7fc314ae7700 1 --2- 172.21.15.53:0/2687999149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc310101820 0x7fc3101080d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.336+0000 7fc314ae7700 1 --2- 172.21.15.53:0/2687999149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3100ffc90 0x7fc3101000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.336+0000 7fc314ae7700 1 --2- 172.21.15.53:0/2687999149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc310100e80 0x7fc3101012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.336+0000 7fc314ae7700 1 -- 172.21.15.53:0/2687999149 >> 172.21.15.53:0/2687999149 conn(0x7fc3100fb1e0 msgr2=0x7fc3100fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.337+0000 7fc314ae7700 1 -- 172.21.15.53:0/2687999149 shutdown_connections 2023-11-06T01:45:15.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.337+0000 7fc314ae7700 1 -- 172.21.15.53:0/2687999149 wait complete. 2023-11-06T01:45:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc314ae7700 1 Processor -- start 2023-11-06T01:45:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc314ae7700 1 -- start start 2023-11-06T01:45:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc314ae7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3100ffc90 0x7fc3101a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc314ae7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc310100e80 0x7fc3101a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc314ae7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc310101820 0x7fc3101a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc314ae7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3101a72d0 con 0x7fc310100e80 2023-11-06T01:45:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc314ae7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3101a7440 con 0x7fc3100ffc90 2023-11-06T01:45:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc314ae7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3101a7600 con 0x7fc310101820 2023-11-06T01:45:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc30e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3100ffc90 0x7fc3101a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc30dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc310100e80 0x7fc3101a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc30e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3100ffc90 0x7fc3101a58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54472/0 (socket says 172.21.15.53:54472) 2023-11-06T01:45:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc30e59c700 1 -- 172.21.15.53:0/1660658053 learned_addr learned my addr 172.21.15.53:0/1660658053 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc30ed9d700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc310101820 0x7fc3101a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc30dd9b700 1 -- 172.21.15.53:0/1660658053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc310101820 msgr2=0x7fc3101a6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc310101820 0x7fc3101a6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc30dd9b700 1 -- 172.21.15.53:0/1660658053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3100ffc90 msgr2=0x7fc3101a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3100ffc90 0x7fc3101a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc30dd9b700 1 -- 172.21.15.53:0/1660658053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc308009c40 con 0x7fc310100e80 2023-11-06T01:45:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.338+0000 7fc30e59c700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3100ffc90 0x7fc3101a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.339+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc310100e80 0x7fc3101a5df0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc30400e9c0 tx=0x7fc30400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.339+0000 7fc2ff7fe700 1 -- 172.21.15.53:0/1660658053 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30400c9c0 con 0x7fc310100e80 2023-11-06T01:45:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.339+0000 7fc2ff7fe700 1 -- 172.21.15.53:0/1660658053 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc304010430 con 0x7fc310100e80 2023-11-06T01:45:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.339+0000 7fc2ff7fe700 1 -- 172.21.15.53:0/1660658053 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304018680 con 0x7fc310100e80 2023-11-06T01:45:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.339+0000 7fc314ae7700 1 -- 172.21.15.53:0/1660658053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3101a78e0 con 0x7fc310100e80 2023-11-06T01:45:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.340+0000 7fc314ae7700 1 -- 172.21.15.53:0/1660658053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3101ae0a0 con 0x7fc310100e80 2023-11-06T01:45:15.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.342+0000 7fc2ff7fe700 1 -- 172.21.15.53:0/1660658053 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fc304010860 con 0x7fc310100e80 2023-11-06T01:45:15.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.342+0000 7fc2ff7fe700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2f80a0660 0x7fc2f80a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.343+0000 7fc2ff7fe700 1 -- 172.21.15.53:0/1660658053 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc304014070 con 0x7fc310100e80 2023-11-06T01:45:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.343+0000 7fc30e59c700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2f80a0660 0x7fc2f80a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.343+0000 7fc320c77b80 1 Processor -- start 2023-11-06T01:45:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.343+0000 7fc30e59c700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2f80a0660 0x7fc2f80a2b20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc3000105c0 tx=0x7fc30000b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.343+0000 7fc320c77b80 1 -- start start 2023-11-06T01:45:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.343+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a8baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.343+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a8c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.343+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a8d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.343+0000 7fc320c77b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd39e39b0 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.343+0000 7fc320c77b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd39e1370 con 0x55fbd3a8b680 2023-11-06T01:45:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.343+0000 7fc320c77b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd39dcc60 con 0x55fbd3a8ced0 2023-11-06T01:45:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a8c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a8baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a8baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54480/0 (socket says 172.21.15.53:54480) 2023-11-06T01:45:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30e59c700 1 -- 172.21.15.53:0/2003744209 learned_addr learned my addr 172.21.15.53:0/2003744209 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30ed9d700 1 --2- 172.21.15.53:0/2003744209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a8d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30dd9b700 1 -- 172.21.15.53:0/2003744209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8ced0 msgr2=0x55fbd3a8d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/2003744209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a8d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30dd9b700 1 -- 172.21.15.53:0/2003744209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 msgr2=0x55fbd3a8baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/2003744209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a8baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30dd9b700 1 -- 172.21.15.53:0/2003744209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbd380ad80 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30e59c700 1 --2- 172.21.15.53:0/2003744209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a8baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.344+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/2003744209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a8c8f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc304005fd0 tx=0x7fc304098d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae48d541a65304b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.345+0000 7fc2fcff9700 1 -- 172.21.15.53:0/2003744209 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30409b030 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.346+0000 7fc2fcff9700 1 -- 172.21.15.53:0/2003744209 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc304099120 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.346+0000 7fc2fcff9700 1 -- 172.21.15.53:0/2003744209 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3040ace40 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.347+0000 7fc320c77b80 1 -- 172.21.15.53:0/2003744209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 msgr2=0x55fbd3a8c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.347+0000 7fc320c77b80 1 --2- 172.21.15.53:0/2003744209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a8c8f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc304005fd0 tx=0x7fc304098d80 comp rx=0 tx=0).stop 2023-11-06T01:45:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.347+0000 7fc320c77b80 1 -- 172.21.15.53:0/2003744209 shutdown_connections 2023-11-06T01:45:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.347+0000 7fc320c77b80 1 --2- 172.21.15.53:0/2003744209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a8baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.347+0000 7fc320c77b80 1 --2- 172.21.15.53:0/2003744209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a8d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.347+0000 7fc320c77b80 1 --2- 172.21.15.53:0/2003744209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a8c8f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.347+0000 7fc320c77b80 1 -- 172.21.15.53:0/2003744209 >> 172.21.15.53:0/2003744209 conn(0x55fbd39eb070 msgr2=0x55fbd3a88040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.347+0000 7fc320c77b80 1 -- 172.21.15.53:0/2003744209 shutdown_connections 2023-11-06T01:45:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.347+0000 7fc320c77b80 1 -- 172.21.15.53:0/2003744209 wait complete. 2023-11-06T01:45:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.347+0000 7fc320c77b80 1 -- start start 2023-11-06T01:45:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.347+0000 7fc320c77b80 1 Processor -- start 2023-11-06T01:45:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a95540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a95a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a9cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc320c77b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd39794b0 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc320c77b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd3979150 con 0x55fbd3a8b680 2023-11-06T01:45:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc320c77b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd38608c0 con 0x55fbd3a8ced0 2023-11-06T01:45:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc320c77b80 10 client.0 _ll_register_callbacks cb 0x55fbd38c6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a95a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a95540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a9cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a95540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54486/0 (socket says 172.21.15.53:54486) 2023-11-06T01:45:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30e59c700 1 -- 172.21.15.53:0/175715234 learned_addr learned my addr 172.21.15.53:0/175715234 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30e59c700 1 -- 172.21.15.53:0/175715234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8ced0 msgr2=0x55fbd3a9cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30e59c700 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a9cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30e59c700 1 -- 172.21.15.53:0/175715234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 msgr2=0x55fbd3a95a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30e59c700 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a95a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30e59c700 1 -- 172.21.15.53:0/175715234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3000114e0 con 0x55fbd3a8b680 2023-11-06T01:45:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30ed9d700 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a9cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a95a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc30e59c700 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a95540 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc300011c60 tx=0x7fc3000047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9e278553f0da724 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.348+0000 7fc2f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:45:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.349+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/175715234 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300004020 con 0x55fbd3a8b680 2023-11-06T01:45:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.349+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/175715234 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc300017070 con 0x55fbd3a8b680 2023-11-06T01:45:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.349+0000 7fc320c77b80 1 -- 172.21.15.53:0/175715234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbd3a07cb0 con 0x55fbd3a8b680 2023-11-06T01:45:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.349+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/175715234 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc300004180 con 0x55fbd3a8b680 2023-11-06T01:45:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.349+0000 7fc320c77b80 10 client.26228 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.349+0000 7fc320c77b80 1 -- 172.21.15.53:0/175715234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7fc3101ae0a0 con 0x55fbd3a8b680 2023-11-06T01:45:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.349+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/175715234 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fbd3a07cb0 con 0x55fbd3a8b680 2023-11-06T01:45:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.350+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/175715234 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc300031050 con 0x55fbd3a8b680 2023-11-06T01:45:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.350+0000 7fc320c77b80 10 client.26228 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.350+0000 7fc320c77b80 10 client.26228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.350+0000 7fc320c77b80 20 client.26228 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.350+0000 7fc320c77b80 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fbd3aa13a0 0x55fbd3aa37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.350+0000 7fc320c77b80 4 client.26228 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:45:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.350+0000 7fc320c77b80 1 -- 172.21.15.53:0/175715234 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbd3a5adb0 con 0x55fbd3aa13a0 2023-11-06T01:45:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.350+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fbd3aa13a0 0x55fbd3aa37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.350+0000 7fc2f5ffb700 10 client.26228 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.350+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fbd3aa13a0 0x55fbd3aa37c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a4be702d1286107 server_cookie=615b4be9dc3e1a1a in_seq=0 out_seq=0 2023-11-06T01:45:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/175715234 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fbd3a5adb0 con 0x55fbd3aa13a0 2023-11-06T01:45:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc2f5ffb700 10 client.26228 handle_command_reply: tid=0 2023-11-06T01:45:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 1 client.26228 shutdown 2023-11-06T01:45:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 1 -- 172.21.15.53:0/175715234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 msgr2=0x55fbd3a95540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a95540 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc300011c60 tx=0x7fc3000047c0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 1 -- 172.21.15.53:0/175715234 shutdown_connections 2023-11-06T01:45:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8b680 0x55fbd3a95540 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fbd3aa13a0 0x55fbd3aa37c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a9cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 1 --2- 172.21.15.53:0/175715234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a95a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc2f5ffb700 0 client.26228 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 1 -- 172.21.15.53:0/175715234 >> 172.21.15.53:0/175715234 conn(0x55fbd39eb070 msgr2=0x55fbd3a863c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 1 -- 172.21.15.53:0/175715234 shutdown_connections 2023-11-06T01:45:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 1 -- 172.21.15.53:0/175715234 wait complete. 2023-11-06T01:45:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.351+0000 7fc320c77b80 20 client.26228 trim_cache size 0 max 16384 2023-11-06T01:45:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.369+0000 7fc320c77b80 1 Processor -- start 2023-11-06T01:45:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.369+0000 7fc320c77b80 1 -- start start 2023-11-06T01:45:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8b680 0x55fbd3a94bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc320c77b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd3a81530 con 0x55fbd3a8b680 2023-11-06T01:45:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc320c77b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd3a887e0 con 0x55fbd3a8ced0 2023-11-06T01:45:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc320c77b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd3a7eab0 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc30dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc30dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57296/0 (socket says 172.21.15.53:57296) 2023-11-06T01:45:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc30dd9b700 1 -- 172.21.15.53:0/907679382 learned_addr learned my addr 172.21.15.53:0/907679382 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc30e59c700 1 --2- 172.21.15.53:0/907679382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8b680 0x55fbd3a94bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc30ed9d700 1 --2- 172.21.15.53:0/907679382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc30dd9b700 1 -- 172.21.15.53:0/907679382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8ced0 msgr2=0x55fbd3a94100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/907679382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a94100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc30dd9b700 1 -- 172.21.15.53:0/907679382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8b680 msgr2=0x55fbd3a94bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/907679382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8b680 0x55fbd3a94bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.370+0000 7fc30dd9b700 1 -- 172.21.15.53:0/907679382 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbd37f3c00 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.371+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/907679382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ca80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc304095620 tx=0x7fc304098b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6665e5963df1d0a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.371+0000 7fc2fd7fa700 1 -- 172.21.15.53:0/907679382 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30409b040 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.371+0000 7fc2fd7fa700 1 -- 172.21.15.53:0/907679382 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3040105a0 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.371+0000 7fc2fd7fa700 1 -- 172.21.15.53:0/907679382 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3040a0d00 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 -- 172.21.15.53:0/907679382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8bfe0 msgr2=0x55fbd3a9ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 --2- 172.21.15.53:0/907679382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ca80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc304095620 tx=0x7fc304098b80 comp rx=0 tx=0).stop 2023-11-06T01:45:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 -- 172.21.15.53:0/907679382 shutdown_connections 2023-11-06T01:45:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 --2- 172.21.15.53:0/907679382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a94100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 --2- 172.21.15.53:0/907679382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8b680 0x55fbd3a94bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 --2- 172.21.15.53:0/907679382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ca80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 -- 172.21.15.53:0/907679382 >> 172.21.15.53:0/907679382 conn(0x55fbd39eb070 msgr2=0x55fbd3a87c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 -- 172.21.15.53:0/907679382 shutdown_connections 2023-11-06T01:45:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 -- 172.21.15.53:0/907679382 wait complete. 2023-11-06T01:45:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 -- start start 2023-11-06T01:45:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 Processor -- start 2023-11-06T01:45:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8b680 0x55fbd3a9a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a9be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd3a9c3d0 con 0x55fbd3a8ced0 2023-11-06T01:45:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd3a9c540 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbd3a9c700 con 0x55fbd3a8b680 2023-11-06T01:45:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.372+0000 7fc320c77b80 10 client.0 _ll_register_callbacks cb 0x7fc300011840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a9be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8b680 0x55fbd3a9a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8b680 0x55fbd3a9a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57302/0 (socket says 172.21.15.53:57302) 2023-11-06T01:45:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54502/0 (socket says 172.21.15.53:54502) 2023-11-06T01:45:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30e59c700 1 -- 172.21.15.53:0/1281320575 learned_addr learned my addr 172.21.15.53:0/1281320575 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a9be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43066/0 (socket says 172.21.15.53:43066) 2023-11-06T01:45:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30dd9b700 1 -- 172.21.15.53:0/1281320575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8b680 msgr2=0x55fbd3a9a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8b680 0x55fbd3a9a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30dd9b700 1 -- 172.21.15.53:0/1281320575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8ced0 msgr2=0x55fbd3a9be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a9be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30dd9b700 1 -- 172.21.15.53:0/1281320575 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fbd380ad80 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30ed9d700 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a9be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc30dd9b700 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ae80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc304098bf0 tx=0x7fc304098fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd24624a4ad45ed5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc2f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:45:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/1281320575 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30409b040 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc320c77b80 1 -- 172.21.15.53:0/1281320575 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc304099120 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.373+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/1281320575 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc30409c070 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/1281320575 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3040a1190 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc320c77b80 10 client.26234 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc320c77b80 1 -- 172.21.15.53:0/1281320575 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55fbd38311f0 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/1281320575 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc304099120 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/1281320575 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc3040950a0 con 0x55fbd3a8bfe0 2023-11-06T01:45:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc320c77b80 10 client.26234 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc320c77b80 10 client.26234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc320c77b80 20 client.26234 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc320c77b80 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fbd3af3980 0x55fbd3af5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc320c77b80 4 client.26234 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:45:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc320c77b80 1 -- 172.21.15.53:0/1281320575 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fbd39d5ce0 con 0x55fbd3af3980 2023-11-06T01:45:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.374+0000 7fc30e59c700 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fbd3af3980 0x55fbd3af5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.376+0000 7fc2f5ffb700 10 client.26234 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.376+0000 7fc30e59c700 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fbd3af3980 0x55fbd3af5da0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c47cae749a1378 server_cookie=acaa17f67370cf0e in_seq=0 out_seq=0 2023-11-06T01:45:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc2f5ffb700 1 -- 172.21.15.53:0/1281320575 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fbd39d5ce0 con 0x55fbd3af3980 2023-11-06T01:45:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc2f5ffb700 10 client.26234 handle_command_reply: tid=0 2023-11-06T01:45:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 1 client.26234 shutdown 2023-11-06T01:45:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 1 -- 172.21.15.53:0/1281320575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8bfe0 msgr2=0x55fbd3a9ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ae80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc304098bf0 tx=0x7fc304098fe0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 1 -- 172.21.15.53:0/1281320575 shutdown_connections 2023-11-06T01:45:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55fbd3af3980 0x55fbd3af5da0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbd3a8ced0 0x55fbd3a9be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc2f5ffb700 0 client.26234 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fbd3a8b680 0x55fbd3a9a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 1 --2- 172.21.15.53:0/1281320575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbd3a8bfe0 0x55fbd3a9ae80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 1 -- 172.21.15.53:0/1281320575 >> 172.21.15.53:0/1281320575 conn(0x55fbd39eb070 msgr2=0x55fbd3a94100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 1 -- 172.21.15.53:0/1281320575 shutdown_connections 2023-11-06T01:45:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 1 -- 172.21.15.53:0/1281320575 wait complete. 2023-11-06T01:45:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.384+0000 7fc320c77b80 20 client.26234 trim_cache size 0 max 16384 2023-11-06T01:45:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:45:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:45:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:45:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:45:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 -- 172.21.15.53:0/1660658053 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2f80a0660 msgr2=0x7fc2f80a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2f80a0660 0x7fc2f80a2b20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc3000105c0 tx=0x7fc30000b4a0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 -- 172.21.15.53:0/1660658053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc310100e80 msgr2=0x7fc3101a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc310100e80 0x7fc3101a5df0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc30400e9c0 tx=0x7fc30400ecd0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 -- 172.21.15.53:0/1660658053 shutdown_connections 2023-11-06T01:45:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc2f80a0660 0x7fc2f80a2b20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3100ffc90 0x7fc3101a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc310101820 0x7fc3101a6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 --2- 172.21.15.53:0/1660658053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc310100e80 0x7fc3101a5df0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 -- 172.21.15.53:0/1660658053 >> 172.21.15.53:0/1660658053 conn(0x7fc3100fb1e0 msgr2=0x7fc3101042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 -- 172.21.15.53:0/1660658053 shutdown_connections 2023-11-06T01:45:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:15.385+0000 7fc314ae7700 1 -- 172.21.15.53:0/1660658053 wait complete. 2023-11-06T01:45:15.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:15 smithi163 ceph-mon[104416]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 MiB/s wr, 797 op/s 2023-11-06T01:45:15.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:15 smithi053 ceph-mon[99472]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 MiB/s wr, 797 op/s 2023-11-06T01:45:16.079 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:15 smithi062 ceph-mon[103924]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 MiB/s wr, 797 op/s 2023-11-06T01:45:17.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 660.0 s, 1428.6 tps, lat 66.396 ms stddev 68.606 2023-11-06T01:45:17.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:17 smithi053 ceph-mon[99472]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 698 op/s 2023-11-06T01:45:17.898 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:17 smithi062 ceph-mon[103924]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 698 op/s 2023-11-06T01:45:17.988 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:17 smithi163 ceph-mon[104416]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 698 op/s 2023-11-06T01:45:19.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:19 smithi053 ceph-mon[99472]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 696 op/s 2023-11-06T01:45:19.911 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:19 smithi062 ceph-mon[103924]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 696 op/s 2023-11-06T01:45:19.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:19 smithi163 ceph-mon[104416]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 696 op/s 2023-11-06T01:45:20.848 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:20 smithi053 ceph-mon[99472]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 718 op/s 2023-11-06T01:45:20.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:20 smithi062 ceph-mon[103924]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 718 op/s 2023-11-06T01:45:20.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:20 smithi163 ceph-mon[104416]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 718 op/s 2023-11-06T01:45:22.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 665.0 s, 1017.8 tps, lat 102.286 ms stddev 126.245 2023-11-06T01:45:22.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:22 smithi062 ceph-mon[103924]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 613 op/s 2023-11-06T01:45:22.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:22 smithi163 ceph-mon[104416]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 613 op/s 2023-11-06T01:45:22.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:22 smithi053 ceph-mon[99472]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 613 op/s 2023-11-06T01:45:24.241 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:23 smithi163 ceph-mon[104416]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 728 op/s 2023-11-06T01:45:24.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:23 smithi053 ceph-mon[99472]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 728 op/s 2023-11-06T01:45:24.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:23 smithi062 ceph-mon[103924]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 728 op/s 2023-11-06T01:45:26.237 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:25 smithi163 ceph-mon[104416]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 734 op/s 2023-11-06T01:45:26.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:25 smithi053 ceph-mon[99472]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 734 op/s 2023-11-06T01:45:26.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:25 smithi062 ceph-mon[103924]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 734 op/s 2023-11-06T01:45:27.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 670.0 s, 1398.8 tps, lat 71.112 ms stddev 79.299 2023-11-06T01:45:28.237 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:27 smithi163 ceph-mon[104416]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 746 op/s 2023-11-06T01:45:28.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:27 smithi053 ceph-mon[99472]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 746 op/s 2023-11-06T01:45:28.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:27 smithi062 ceph-mon[103924]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 746 op/s 2023-11-06T01:45:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.564+0000 7f29b5978700 1 -- 172.21.15.53:0/1901158952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b010d0d0 msgr2=0x7f29b010d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.564+0000 7f29b5978700 1 --2- 172.21.15.53:0/1901158952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b010d0d0 0x7f29b010d550 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2998009990 tx=0x7f2998009ca0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.573+0000 7f29b5978700 1 -- 172.21.15.53:0/1901158952 shutdown_connections 2023-11-06T01:45:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.573+0000 7f29b5978700 1 --2- 172.21.15.53:0/1901158952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b0107d90 0x7f29b010a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.573+0000 7f29b5978700 1 --2- 172.21.15.53:0/1901158952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b010d0d0 0x7f29b010d550 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.573+0000 7f29b5978700 1 --2- 172.21.15.53:0/1901158952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29b010a700 0x7f29b010cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.573+0000 7f29b5978700 1 -- 172.21.15.53:0/1901158952 >> 172.21.15.53:0/1901158952 conn(0x7f29b006dbc0 msgr2=0x7f29b0070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.578+0000 7f29b5978700 1 -- 172.21.15.53:0/1901158952 shutdown_connections 2023-11-06T01:45:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.579+0000 7f29b5978700 1 -- 172.21.15.53:0/1901158952 wait complete. 2023-11-06T01:45:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.579+0000 7f29b5978700 1 Processor -- start 2023-11-06T01:45:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.581+0000 7f29b5978700 1 -- start start 2023-11-06T01:45:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.581+0000 7f29b5978700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b0107d90 0x7f29b0114450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.581+0000 7f29b5978700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29b010a700 0x7f29b0114990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.581+0000 7f29b5978700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b010d0d0 0x7f29b0115810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.581+0000 7f29b5978700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b0115e70 con 0x7f29b0107d90 2023-11-06T01:45:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.581+0000 7f29b5978700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b0115fe0 con 0x7f29b010d0d0 2023-11-06T01:45:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.581+0000 7f29b5978700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b01161a0 con 0x7f29b010a700 2023-11-06T01:45:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.582+0000 7f29af7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b010d0d0 0x7f29b0115810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.582+0000 7f29af7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b010d0d0 0x7f29b0115810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54510/0 (socket says 172.21.15.53:54510) 2023-11-06T01:45:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.582+0000 7f29af7fe700 1 -- 172.21.15.53:0/3585189184 learned_addr learned my addr 172.21.15.53:0/3585189184 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.582+0000 7f29af7fe700 1 -- 172.21.15.53:0/3585189184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29b010a700 msgr2=0x7f29b0114990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.582+0000 7f29af7fe700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29b010a700 0x7f29b0114990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.582+0000 7f29af7fe700 1 -- 172.21.15.53:0/3585189184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b0107d90 msgr2=0x7f29b0114450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:45:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.582+0000 7f29af7fe700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b0107d90 0x7f29b0114450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.582+0000 7f29af7fe700 1 -- 172.21.15.53:0/3585189184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29980095f0 con 0x7f29b010d0d0 2023-11-06T01:45:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.582+0000 7f29af7fe700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b010d0d0 0x7f29b0115810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2998005e50 tx=0x7f299800be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.584+0000 7f29b4976700 1 -- 172.21.15.53:0/3585189184 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299801d070 con 0x7f29b010d0d0 2023-11-06T01:45:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.585+0000 7f29b4976700 1 -- 172.21.15.53:0/3585189184 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f299800f460 con 0x7f29b010d0d0 2023-11-06T01:45:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.585+0000 7f29b5978700 1 -- 172.21.15.53:0/3585189184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b0116420 con 0x7f29b010d0d0 2023-11-06T01:45:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.585+0000 7f29b4976700 1 -- 172.21.15.53:0/3585189184 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998017650 con 0x7f29b010d0d0 2023-11-06T01:45:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.585+0000 7f29b5978700 1 -- 172.21.15.53:0/3585189184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b0116cb0 con 0x7f29b010d0d0 2023-11-06T01:45:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.587+0000 7f29b4976700 1 -- 172.21.15.53:0/3585189184 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f2998005390 con 0x7f29b010d0d0 2023-11-06T01:45:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.588+0000 7f29b4976700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f299c0a05f0 0x7f299c0a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.588+0000 7f29b4976700 1 -- 172.21.15.53:0/3585189184 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f29980ccbf0 con 0x7f29b010d0d0 2023-11-06T01:45:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.588+0000 7f29aeffd700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f299c0a05f0 0x7f299c0a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29c1b08b80 1 Processor -- start 2023-11-06T01:45:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29c1b08b80 1 -- start start 2023-11-06T01:45:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29aeffd700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f299c0a05f0 0x7f299c0a2ab0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f29b01141c0 tx=0x7f29a0006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b482aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b4838f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b484350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b3da9b0 con 0x55914b483ed0 2023-11-06T01:45:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b3d8370 con 0x55914b482fe0 2023-11-06T01:45:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b3d3c60 con 0x55914b482680 2023-11-06T01:45:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29af7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b484350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29af7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b484350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43080/0 (socket says 172.21.15.53:43080) 2023-11-06T01:45:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29af7fe700 1 -- 172.21.15.53:0/3513734296 learned_addr learned my addr 172.21.15.53:0/3513734296 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.589+0000 7f29ae7fc700 1 --2- 172.21.15.53:0/3513734296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b4838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.590+0000 7f29af7fe700 1 -- 172.21.15.53:0/3513734296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 msgr2=0x55914b482aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.590+0000 7f29af7fe700 1 --2- 172.21.15.53:0/3513734296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b482aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.590+0000 7f29af7fe700 1 -- 172.21.15.53:0/3513734296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 msgr2=0x55914b4838f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.590+0000 7f29af7fe700 1 --2- 172.21.15.53:0/3513734296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b4838f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.590+0000 7f29af7fe700 1 -- 172.21.15.53:0/3513734296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55914b201d80 con 0x55914b483ed0 2023-11-06T01:45:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.590+0000 7f29af7fe700 1 --2- 172.21.15.53:0/3513734296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b484350 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f29980969c0 tx=0x7f299809a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390d0ba3aabb073a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.591+0000 7f29a5ffb700 1 -- 172.21.15.53:0/3513734296 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299809e120 con 0x55914b483ed0 2023-11-06T01:45:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.591+0000 7f29a5ffb700 1 -- 172.21.15.53:0/3513734296 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f299809a4c0 con 0x55914b483ed0 2023-11-06T01:45:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.591+0000 7f29a5ffb700 1 -- 172.21.15.53:0/3513734296 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299809a980 con 0x55914b483ed0 2023-11-06T01:45:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.592+0000 7f29c1b08b80 1 -- 172.21.15.53:0/3513734296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 msgr2=0x55914b484350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.592+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/3513734296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b484350 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f29980969c0 tx=0x7f299809a000 comp rx=0 tx=0).stop 2023-11-06T01:45:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.592+0000 7f29c1b08b80 1 -- 172.21.15.53:0/3513734296 shutdown_connections 2023-11-06T01:45:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.592+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/3513734296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b484350 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.592+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/3513734296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b482aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.592+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/3513734296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b4838f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.592+0000 7f29c1b08b80 1 -- 172.21.15.53:0/3513734296 >> 172.21.15.53:0/3513734296 conn(0x55914b3e2070 msgr2=0x55914b47f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.592+0000 7f29c1b08b80 1 -- 172.21.15.53:0/3513734296 shutdown_connections 2023-11-06T01:45:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.592+0000 7f29c1b08b80 1 -- 172.21.15.53:0/3513734296 wait complete. 2023-11-06T01:45:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29c1b08b80 1 -- start start 2023-11-06T01:45:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29c1b08b80 1 Processor -- start 2023-11-06T01:45:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b48c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482fe0 0x55914b48c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b483ed0 0x55914b493730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b3704b0 con 0x55914b482fe0 2023-11-06T01:45:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b370150 con 0x55914b483ed0 2023-11-06T01:45:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b2578c0 con 0x55914b482680 2023-11-06T01:45:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29c1b08b80 10 client.0 _ll_register_callbacks cb 0x55914b2bda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29aeffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b48c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29aeffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b48c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57322/0 (socket says 172.21.15.53:57322) 2023-11-06T01:45:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.593+0000 7f29aeffd700 1 -- 172.21.15.53:0/651087665 learned_addr learned my addr 172.21.15.53:0/651087665 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f29af7fe700 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b483ed0 0x55914b493730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f29ae7fc700 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482fe0 0x55914b48c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f29aeffd700 1 -- 172.21.15.53:0/651087665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b483ed0 msgr2=0x55914b493730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f29aeffd700 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b483ed0 0x55914b493730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f29aeffd700 1 -- 172.21.15.53:0/651087665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482fe0 msgr2=0x55914b48c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f29aeffd700 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482fe0 0x55914b48c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f29aeffd700 1 -- 172.21.15.53:0/651087665 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b0116cb0 con 0x55914b482680 2023-11-06T01:45:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f29ae7fc700 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482fe0 0x55914b48c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:45:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f29aeffd700 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b48c1b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29a00123e0 tx=0x7f29a00126f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bd5198037a795da server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f298effd700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:45:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f298effd700 1 -- 172.21.15.53:0/651087665 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a001a070 con 0x55914b482680 2023-11-06T01:45:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f298effd700 1 -- 172.21.15.53:0/651087665 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f29a0012ba0 con 0x55914b482680 2023-11-06T01:45:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f29c1b08b80 1 -- 172.21.15.53:0/651087665 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55914b3fecb0 con 0x55914b482680 2023-11-06T01:45:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f298effd700 1 -- 172.21.15.53:0/651087665 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a00152e0 con 0x55914b482680 2023-11-06T01:45:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.594+0000 7f29c1b08b80 10 client.26299 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.595+0000 7f29c1b08b80 1 -- 172.21.15.53:0/651087665 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55914b1eac00 con 0x55914b482680 2023-11-06T01:45:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.595+0000 7f298effd700 1 -- 172.21.15.53:0/651087665 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29a002d080 con 0x55914b482680 2023-11-06T01:45:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.595+0000 7f298effd700 1 -- 172.21.15.53:0/651087665 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f29a0027030 con 0x55914b482680 2023-11-06T01:45:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.595+0000 7f29c1b08b80 10 client.26299 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.595+0000 7f29c1b08b80 10 client.26299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.595+0000 7f29c1b08b80 20 client.26299 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.595+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55914b497ff0 0x55914b49a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.595+0000 7f29c1b08b80 4 client.26299 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:45:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.595+0000 7f29c1b08b80 1 -- 172.21.15.53:0/651087665 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55914b451db0 con 0x55914b497ff0 2023-11-06T01:45:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.598+0000 7f29ae7fc700 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55914b497ff0 0x55914b49a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.598+0000 7f298effd700 10 client.26299 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.598+0000 7f29ae7fc700 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55914b497ff0 0x55914b49a410 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8211f808cb4964b server_cookie=bad377a5375d132b in_seq=0 out_seq=0 2023-11-06T01:45:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.599+0000 7f298effd700 1 -- 172.21.15.53:0/651087665 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55914b451db0 con 0x55914b497ff0 2023-11-06T01:45:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.599+0000 7f298effd700 10 client.26299 handle_command_reply: tid=0 2023-11-06T01:45:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.599+0000 7f29c1b08b80 1 client.26299 shutdown 2023-11-06T01:45:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f29c1b08b80 1 -- 172.21.15.53:0/651087665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 msgr2=0x55914b48c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b48c1b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29a00123e0 tx=0x7f29a00126f0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f29c1b08b80 1 -- 172.21.15.53:0/651087665 shutdown_connections 2023-11-06T01:45:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b483ed0 0x55914b493730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55914b497ff0 0x55914b49a410 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b48c1b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/651087665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482fe0 0x55914b48c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f29c1b08b80 1 -- 172.21.15.53:0/651087665 >> 172.21.15.53:0/651087665 conn(0x55914b3e2070 msgr2=0x55914b47d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f298effd700 0 client.26299 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f29c1b08b80 1 -- 172.21.15.53:0/651087665 shutdown_connections 2023-11-06T01:45:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f29c1b08b80 1 -- 172.21.15.53:0/651087665 wait complete. 2023-11-06T01:45:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.600+0000 7f29c1b08b80 20 client.26299 trim_cache size 0 max 16384 2023-11-06T01:45:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.620+0000 7f29c1b08b80 1 Processor -- start 2023-11-06T01:45:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.620+0000 7f29c1b08b80 1 -- start start 2023-11-06T01:45:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.620+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b4916e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.620+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b48a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.620+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b49a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.620+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b475f30 con 0x55914b483ed0 2023-11-06T01:45:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.620+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b478530 con 0x55914b482fe0 2023-11-06T01:45:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.620+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b48d3b0 con 0x55914b482680 2023-11-06T01:45:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.621+0000 7f29af7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b49a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.621+0000 7f29af7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b49a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43094/0 (socket says 172.21.15.53:43094) 2023-11-06T01:45:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.621+0000 7f29af7fe700 1 -- 172.21.15.53:0/2386950091 learned_addr learned my addr 172.21.15.53:0/2386950091 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.621+0000 7f29ae7fc700 1 --2- 172.21.15.53:0/2386950091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b48a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.621+0000 7f29af7fe700 1 -- 172.21.15.53:0/2386950091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 msgr2=0x55914b4916e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.621+0000 7f29af7fe700 1 --2- 172.21.15.53:0/2386950091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b4916e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.621+0000 7f29af7fe700 1 -- 172.21.15.53:0/2386950091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 msgr2=0x55914b48a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.621+0000 7f29af7fe700 1 --2- 172.21.15.53:0/2386950091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b48a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.621+0000 7f29af7fe700 1 -- 172.21.15.53:0/2386950091 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55914b2281f0 con 0x55914b483ed0 2023-11-06T01:45:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.621+0000 7f29ae7fc700 1 --2- 172.21.15.53:0/2386950091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b48a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.622+0000 7f29af7fe700 1 --2- 172.21.15.53:0/2386950091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b49a000 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f299809a450 tx=0x7f2998009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6397a097619443 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.622+0000 7f29a67fc700 1 -- 172.21.15.53:0/2386950091 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980a24a0 con 0x55914b483ed0 2023-11-06T01:45:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.622+0000 7f29a67fc700 1 -- 172.21.15.53:0/2386950091 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2998098860 con 0x55914b483ed0 2023-11-06T01:45:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.622+0000 7f29a67fc700 1 -- 172.21.15.53:0/2386950091 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299809a980 con 0x55914b483ed0 2023-11-06T01:45:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2386950091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 msgr2=0x55914b49a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/2386950091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b49a000 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f299809a450 tx=0x7f2998009ca0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2386950091 shutdown_connections 2023-11-06T01:45:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/2386950091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b483ed0 0x55914b49a000 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/2386950091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b482680 0x55914b4916e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/2386950091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b48a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2386950091 >> 172.21.15.53:0/2386950091 conn(0x55914b3e2070 msgr2=0x55914b47ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2386950091 shutdown_connections 2023-11-06T01:45:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2386950091 wait complete. 2023-11-06T01:45:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 -- start start 2023-11-06T01:45:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 Processor -- start 2023-11-06T01:45:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.623+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482680 0x55914b4e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b4e5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29c1b08b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b483ed0 0x55914b4e8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b4e8c20 con 0x55914b482680 2023-11-06T01:45:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b4e8d90 con 0x55914b482fe0 2023-11-06T01:45:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29c1b08b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914b4e8f50 con 0x55914b483ed0 2023-11-06T01:45:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29c1b08b80 10 client.0 _ll_register_callbacks cb 0x7f29a00159d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29aeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482680 0x55914b4e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29ae7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b4e5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29aeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482680 0x55914b4e3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43098/0 (socket says 172.21.15.53:43098) 2023-11-06T01:45:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29aeffd700 1 -- 172.21.15.53:0/2709824454 learned_addr learned my addr 172.21.15.53:0/2709824454 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29af7fe700 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b483ed0 0x55914b4e8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29ae7fc700 1 -- 172.21.15.53:0/2709824454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b483ed0 msgr2=0x55914b4e8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29ae7fc700 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b483ed0 0x55914b4e8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29ae7fc700 1 -- 172.21.15.53:0/2709824454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482680 msgr2=0x55914b4e3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29ae7fc700 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482680 0x55914b4e3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29ae7fc700 1 -- 172.21.15.53:0/2709824454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a8009cd0 con 0x55914b482fe0 2023-11-06T01:45:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.624+0000 7f29aeffd700 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482680 0x55914b4e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.625+0000 7f29ae7fc700 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b4e5fe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f29a8004bc0 tx=0x7f29a801af40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d987501d9f1ff24b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.625+0000 7f298effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:45:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.625+0000 7f298effd700 1 -- 172.21.15.53:0/2709824454 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8018400 con 0x55914b482fe0 2023-11-06T01:45:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.625+0000 7f298effd700 1 -- 172.21.15.53:0/2709824454 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f29a8018560 con 0x55914b482fe0 2023-11-06T01:45:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.625+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2709824454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f299809a4c0 con 0x55914b482fe0 2023-11-06T01:45:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.625+0000 7f298effd700 1 -- 172.21.15.53:0/2709824454 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a8018720 con 0x55914b482fe0 2023-11-06T01:45:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.625+0000 7f29c1b08b80 10 client.26249 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.625+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2709824454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55914b201d80 con 0x55914b482fe0 2023-11-06T01:45:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.625+0000 7f298effd700 1 -- 172.21.15.53:0/2709824454 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f299809a4c0 con 0x55914b482fe0 2023-11-06T01:45:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.626+0000 7f298effd700 1 -- 172.21.15.53:0/2709824454 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f29a802eac0 con 0x55914b482fe0 2023-11-06T01:45:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.626+0000 7f29c1b08b80 10 client.26249 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.626+0000 7f29c1b08b80 10 client.26249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.626+0000 7f29c1b08b80 20 client.26249 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.626+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55914b4ea960 0x55914b4ecd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.626+0000 7f29c1b08b80 4 client.26249 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:45:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.626+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2709824454 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55914b3ccce0 con 0x55914b4ea960 2023-11-06T01:45:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.626+0000 7f29aeffd700 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55914b4ea960 0x55914b4ecd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.626+0000 7f298effd700 10 client.26249 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.626+0000 7f29aeffd700 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55914b4ea960 0x55914b4ecd80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace072cfd3a45ea5 server_cookie=18fb810aebbc4975 in_seq=0 out_seq=0 2023-11-06T01:45:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f298effd700 1 -- 172.21.15.53:0/2709824454 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55914b3ccce0 con 0x55914b4ea960 2023-11-06T01:45:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f298effd700 10 client.26249 handle_command_reply: tid=0 2023-11-06T01:45:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 1 client.26249 shutdown 2023-11-06T01:45:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2709824454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 msgr2=0x55914b4e5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b4e5fe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f29a8004bc0 tx=0x7f29a801af40 comp rx=0 tx=0).stop 2023-11-06T01:45:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2709824454 shutdown_connections 2023-11-06T01:45:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55914b4ea960 0x55914b4ecd80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55914b483ed0 0x55914b4e8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55914b482680 0x55914b4e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 1 --2- 172.21.15.53:0/2709824454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55914b482fe0 0x55914b4e5fe0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f298effd700 0 client.26249 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2709824454 >> 172.21.15.53:0/2709824454 conn(0x55914b3e2070 msgr2=0x55914b48a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2709824454 shutdown_connections 2023-11-06T01:45:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 1 -- 172.21.15.53:0/2709824454 wait complete. 2023-11-06T01:45:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.634+0000 7f29c1b08b80 20 client.26249 trim_cache size 0 max 16384 2023-11-06T01:45:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:45:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:45:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:45:28.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:45:28.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 -- 172.21.15.53:0/3585189184 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f299c0a05f0 msgr2=0x7f299c0a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:28.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f299c0a05f0 0x7f299c0a2ab0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f29b01141c0 tx=0x7f29a0006c60 comp rx=0 tx=0).stop 2023-11-06T01:45:28.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 -- 172.21.15.53:0/3585189184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b010d0d0 msgr2=0x7f29b0115810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:28.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b010d0d0 0x7f29b0115810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2998005e50 tx=0x7f299800be70 comp rx=0 tx=0).stop 2023-11-06T01:45:28.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 -- 172.21.15.53:0/3585189184 shutdown_connections 2023-11-06T01:45:28.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f299c0a05f0 0x7f299c0a2ab0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b010d0d0 0x7f29b0115810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b0107d90 0x7f29b0114450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 --2- 172.21.15.53:0/3585189184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29b010a700 0x7f29b0114990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:28.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 -- 172.21.15.53:0/3585189184 >> 172.21.15.53:0/3585189184 conn(0x7f29b006dbc0 msgr2=0x7f29b01095c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 -- 172.21.15.53:0/3585189184 shutdown_connections 2023-11-06T01:45:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:28.635+0000 7f29b5978700 1 -- 172.21.15.53:0/3585189184 wait complete. 2023-11-06T01:45:30.237 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:29 smithi163 ceph-mon[104416]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 780 op/s 2023-11-06T01:45:30.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:29 smithi053 ceph-mon[99472]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 780 op/s 2023-11-06T01:45:30.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:29 smithi062 ceph-mon[103924]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 780 op/s 2023-11-06T01:45:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 675.0 s, 1745.5 tps, lat 57.688 ms stddev 66.888 2023-11-06T01:45:32.752 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:32 smithi163 ceph-mon[104416]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 689 op/s 2023-11-06T01:45:32.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:32 smithi053 ceph-mon[99472]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 689 op/s 2023-11-06T01:45:32.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:32 smithi062 ceph-mon[103924]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 689 op/s 2023-11-06T01:45:34.237 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:33 smithi163 ceph-mon[104416]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 800 op/s 2023-11-06T01:45:34.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:33 smithi053 ceph-mon[99472]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 800 op/s 2023-11-06T01:45:34.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:33 smithi062 ceph-mon[103924]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 800 op/s 2023-11-06T01:45:36.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:35 smithi053 ceph-mon[99472]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 737 op/s 2023-11-06T01:45:36.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:36 smithi062 ceph-mon[103924]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 737 op/s 2023-11-06T01:45:36.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:36 smithi163 ceph-mon[104416]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 737 op/s 2023-11-06T01:45:37.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 680.0 s, 1552.6 tps, lat 64.667 ms stddev 77.740 2023-11-06T01:45:38.243 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:37 smithi163 ceph-mon[104416]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 723 op/s 2023-11-06T01:45:38.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:37 smithi053 ceph-mon[99472]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 723 op/s 2023-11-06T01:45:38.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:37 smithi062 ceph-mon[103924]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 723 op/s 2023-11-06T01:45:40.143 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:45:40.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:40 smithi053 ceph-mon[99472]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 780 op/s 2023-11-06T01:45:40.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:40 smithi062 ceph-mon[103924]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 780 op/s 2023-11-06T01:45:40.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.443+0000 7f124adbd700 1 -- 172.21.15.53:0/4183058950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f124410e150 msgr2=0x7f12441105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:40.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.443+0000 7f124adbd700 1 --2- 172.21.15.53:0/4183058950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f124410e150 0x7f12441105e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1238009990 tx=0x7f1238009ca0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.444+0000 7f124adbd700 1 -- 172.21.15.53:0/4183058950 shutdown_connections 2023-11-06T01:45:40.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.444+0000 7f124adbd700 1 --2- 172.21.15.53:0/4183058950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f124410e150 0x7f12441105e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.444+0000 7f124adbd700 1 --2- 172.21.15.53:0/4183058950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244108ea0 0x7f124410b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.444+0000 7f124adbd700 1 --2- 172.21.15.53:0/4183058950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f124410b7d0 0x7f124410dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.444+0000 7f124adbd700 1 -- 172.21.15.53:0/4183058950 >> 172.21.15.53:0/4183058950 conn(0x7f124406db40 msgr2=0x7f124406ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:40.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.447+0000 7f124adbd700 1 -- 172.21.15.53:0/4183058950 shutdown_connections 2023-11-06T01:45:40.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.447+0000 7f124adbd700 1 -- 172.21.15.53:0/4183058950 wait complete. 2023-11-06T01:45:40.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.448+0000 7f124adbd700 1 Processor -- start 2023-11-06T01:45:40.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.449+0000 7f124adbd700 1 -- start start 2023-11-06T01:45:40.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.449+0000 7f124adbd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244108ea0 0x7f124419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.449+0000 7f124adbd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f124410b7d0 0x7f124419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.449+0000 7f124adbd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f124410e150 0x7f12441a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.449+0000 7f124adbd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12441a4d60 con 0x7f124410e150 2023-11-06T01:45:40.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.449+0000 7f124adbd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12441a4ed0 con 0x7f124410b7d0 2023-11-06T01:45:40.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.449+0000 7f124adbd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12441a5090 con 0x7f1244108ea0 2023-11-06T01:45:40.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.449+0000 7f124935a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f124410e150 0x7f12441a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.449+0000 7f124935a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f124410e150 0x7f12441a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43106/0 (socket says 172.21.15.53:43106) 2023-11-06T01:45:40.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.450+0000 7f124935a700 1 -- 172.21.15.53:0/3405656170 learned_addr learned my addr 172.21.15.53:0/3405656170 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:40.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.450+0000 7f1248b59700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244108ea0 0x7f124419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.450+0000 7f124935a700 1 -- 172.21.15.53:0/3405656170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244108ea0 msgr2=0x7f124419d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:40.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.450+0000 7f1243fff700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f124410b7d0 0x7f124419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.450+0000 7f124935a700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244108ea0 0x7f124419d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.450+0000 7f124935a700 1 -- 172.21.15.53:0/3405656170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f124410b7d0 msgr2=0x7f124419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:40.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.450+0000 7f124935a700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f124410b7d0 0x7f124419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.450+0000 7f124935a700 1 -- 172.21.15.53:0/3405656170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12380095f0 con 0x7f124410e150 2023-11-06T01:45:40.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.450+0000 7f1243fff700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f124410b7d0 0x7f124419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:45:40.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.450+0000 7f124935a700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f124410e150 0x7f12441a4790 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1238003cd0 tx=0x7f1238003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:40.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.451+0000 7f1241ffb700 1 -- 172.21.15.53:0/3405656170 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123801d070 con 0x7f124410e150 2023-11-06T01:45:40.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.451+0000 7f1241ffb700 1 -- 172.21.15.53:0/3405656170 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f123800f460 con 0x7f124410e150 2023-11-06T01:45:40.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.451+0000 7f1241ffb700 1 -- 172.21.15.53:0/3405656170 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238017650 con 0x7f124410e150 2023-11-06T01:45:40.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.451+0000 7f124adbd700 1 -- 172.21.15.53:0/3405656170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12441a52b0 con 0x7f124410e150 2023-11-06T01:45:40.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.452+0000 7f124adbd700 1 -- 172.21.15.53:0/3405656170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12441a5790 con 0x7f124410e150 2023-11-06T01:45:40.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.457+0000 7f1241ffb700 1 -- 172.21.15.53:0/3405656170 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f12380052d0 con 0x7f124410e150 2023-11-06T01:45:40.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.457+0000 7f1256f4db80 1 Processor -- start 2023-11-06T01:45:40.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.463+0000 7f1241ffb700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f122c0a0620 0x7f122c0a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.463+0000 7f1241ffb700 1 -- 172.21.15.53:0/3405656170 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f12380ccbe0 con 0x7f124410e150 2023-11-06T01:45:40.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.463+0000 7f1256f4db80 1 -- start start 2023-11-06T01:45:40.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.464+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540ab0f0 0x55ad540ab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.464+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.464+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aa3f0 0x55ad540aa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.464+0000 7f1256f4db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad54001a10 con 0x55ad540aa3f0 2023-11-06T01:45:40.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.464+0000 7f1256f4db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad53fff3d0 con 0x55ad540ab0f0 2023-11-06T01:45:40.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.464+0000 7f1256f4db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad53ffacc0 con 0x55ad540aba70 2023-11-06T01:45:40.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.465+0000 7f1243fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.465+0000 7f1243fff700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f122c0a0620 0x7f122c0a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.465+0000 7f1243fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540a9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57354/0 (socket says 172.21.15.53:57354) 2023-11-06T01:45:40.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.465+0000 7f1243fff700 1 -- 172.21.15.53:0/441628558 learned_addr learned my addr 172.21.15.53:0/441628558 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:40.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.466+0000 7f124935a700 1 --2- 172.21.15.53:0/441628558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aa3f0 0x55ad540aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.466+0000 7f1243fff700 1 -- 172.21.15.53:0/441628558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540ab0f0 msgr2=0x55ad540ab530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:45:40.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.466+0000 7f1243fff700 1 --2- 172.21.15.53:0/441628558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540ab0f0 0x55ad540ab530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.466+0000 7f1243fff700 1 -- 172.21.15.53:0/441628558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aa3f0 msgr2=0x55ad540aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:40.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.466+0000 7f1243fff700 1 --2- 172.21.15.53:0/441628558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aa3f0 0x55ad540aa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.466+0000 7f1243fff700 1 -- 172.21.15.53:0/441628558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad53e28de0 con 0x55ad540aba70 2023-11-06T01:45:40.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.467+0000 7f1243fff700 1 --2- 172.21.15.53:0/441628558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540a9e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f12340138f0 tx=0x7f1234013c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc8a7237587b3fb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:40.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.467+0000 7f1243fff700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f122c0a0620 0x7f122c0a2ae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1234015300 tx=0x7f1234009000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:40.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.467+0000 7f122affd700 1 -- 172.21.15.53:0/441628558 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12340132f0 con 0x55ad540aba70 2023-11-06T01:45:40.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.468+0000 7f122affd700 1 -- 172.21.15.53:0/441628558 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12340044e0 con 0x55ad540aba70 2023-11-06T01:45:40.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.468+0000 7f122affd700 1 -- 172.21.15.53:0/441628558 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234003db0 con 0x55ad540aba70 2023-11-06T01:45:40.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.470+0000 7f1256f4db80 1 -- 172.21.15.53:0/441628558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 msgr2=0x55ad540a9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:40.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.470+0000 7f1256f4db80 1 --2- 172.21.15.53:0/441628558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540a9e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f12340138f0 tx=0x7f1234013c00 comp rx=0 tx=0).stop 2023-11-06T01:45:40.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.471+0000 7f1256f4db80 1 -- 172.21.15.53:0/441628558 shutdown_connections 2023-11-06T01:45:40.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.471+0000 7f1256f4db80 1 --2- 172.21.15.53:0/441628558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540ab0f0 0x55ad540ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.471+0000 7f1256f4db80 1 --2- 172.21.15.53:0/441628558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aa3f0 0x55ad540aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.471+0000 7f1256f4db80 1 --2- 172.21.15.53:0/441628558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540a9e60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.471+0000 7f1256f4db80 1 -- 172.21.15.53:0/441628558 >> 172.21.15.53:0/441628558 conn(0x55ad540090d0 msgr2=0x55ad540a4940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:40.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.471+0000 7f1256f4db80 1 -- 172.21.15.53:0/441628558 shutdown_connections 2023-11-06T01:45:40.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.471+0000 7f1256f4db80 1 -- 172.21.15.53:0/441628558 wait complete. 2023-11-06T01:45:40.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.471+0000 7f1256f4db80 1 -- start start 2023-11-06T01:45:40.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.471+0000 7f1256f4db80 1 Processor -- start 2023-11-06T01:45:40.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 0x55ad540b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540ab0f0 0x55ad540b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aba70 0x55ad540bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1256f4db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad53f97510 con 0x55ad540aba70 2023-11-06T01:45:40.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1256f4db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad53f971b0 con 0x55ad540aa3f0 2023-11-06T01:45:40.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1256f4db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad53e7e920 con 0x55ad540ab0f0 2023-11-06T01:45:40.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1256f4db80 10 client.0 _ll_register_callbacks cb 0x55ad53ee4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:40.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1248b59700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 0x55ad540b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1248b59700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 0x55ad540b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54556/0 (socket says 172.21.15.53:54556) 2023-11-06T01:45:40.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1248b59700 1 -- 172.21.15.53:0/1594389515 learned_addr learned my addr 172.21.15.53:0/1594389515 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:40.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f124935a700 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aba70 0x55ad540bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1243fff700 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540ab0f0 0x55ad540b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1248b59700 1 -- 172.21.15.53:0/1594389515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540ab0f0 msgr2=0x55ad540b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:40.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1248b59700 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540ab0f0 0x55ad540b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1248b59700 1 -- 172.21.15.53:0/1594389515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aba70 msgr2=0x55ad540bafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:40.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1248b59700 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aba70 0x55ad540bafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f1248b59700 1 -- 172.21.15.53:0/1594389515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12340134a0 con 0x55ad540aa3f0 2023-11-06T01:45:40.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.472+0000 7f124935a700 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aba70 0x55ad540bafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:45:40.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.473+0000 7f1248b59700 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 0x55ad540b3a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f123c00c9c0 tx=0x7f123c00c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5a113236d5d7a42 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:40.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.473+0000 7f1223fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:45:40.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.473+0000 7f1223fff700 1 -- 172.21.15.53:0/1594389515 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c011070 con 0x55ad540aa3f0 2023-11-06T01:45:40.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.473+0000 7f1223fff700 1 -- 172.21.15.53:0/1594389515 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f123c015410 con 0x55ad540aa3f0 2023-11-06T01:45:40.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.473+0000 7f1256f4db80 1 -- 172.21.15.53:0/1594389515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad54025d10 con 0x55ad540aa3f0 2023-11-06T01:45:40.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.473+0000 7f1223fff700 1 -- 172.21.15.53:0/1594389515 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f123c009970 con 0x55ad540aa3f0 2023-11-06T01:45:40.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.473+0000 7f1256f4db80 10 client.26255 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:40.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.473+0000 7f1256f4db80 1 -- 172.21.15.53:0/1594389515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ad53e11c60 con 0x55ad540aa3f0 2023-11-06T01:45:40.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.473+0000 7f1223fff700 1 -- 172.21.15.53:0/1594389515 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f123c024080 con 0x55ad540aa3f0 2023-11-06T01:45:40.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.474+0000 7f1223fff700 1 -- 172.21.15.53:0/1594389515 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f123c01e030 con 0x55ad540aa3f0 2023-11-06T01:45:40.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.474+0000 7f1256f4db80 10 client.26255 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:40.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.474+0000 7f1256f4db80 10 client.26255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:40.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.474+0000 7f1256f4db80 20 client.26255 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:40.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.474+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad540bf910 0x55ad540c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.474+0000 7f1256f4db80 4 client.26255 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:45:40.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.474+0000 7f1256f4db80 1 -- 172.21.15.53:0/1594389515 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad54078e10 con 0x55ad540bf910 2023-11-06T01:45:40.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.474+0000 7f1243fff700 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad540bf910 0x55ad540c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.476+0000 7f1243fff700 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad540bf910 0x55ad540c1d30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=422b00c721d558fe server_cookie=89fd16d265d8a96f in_seq=0 out_seq=0 2023-11-06T01:45:40.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.479+0000 7f1223fff700 10 client.26255 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:40.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.479+0000 7f1223fff700 1 -- 172.21.15.53:0/1594389515 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ad54078e10 con 0x55ad540bf910 2023-11-06T01:45:40.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.479+0000 7f1223fff700 10 client.26255 handle_command_reply: tid=0 2023-11-06T01:45:40.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.479+0000 7f1256f4db80 1 client.26255 shutdown 2023-11-06T01:45:40.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.479+0000 7f1256f4db80 1 -- 172.21.15.53:0/1594389515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 msgr2=0x55ad540b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:40.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.479+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 0x55ad540b3a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f123c00c9c0 tx=0x7f123c00c9f0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.480+0000 7f1256f4db80 1 -- 172.21.15.53:0/1594389515 shutdown_connections 2023-11-06T01:45:40.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.480+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 0x55ad540b3a40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.480+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad540bf910 0x55ad540c1d30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.480+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aba70 0x55ad540bafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.480+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1594389515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540ab0f0 0x55ad540b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.480+0000 7f1223fff700 0 client.26255 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:40.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.480+0000 7f1256f4db80 1 -- 172.21.15.53:0/1594389515 >> 172.21.15.53:0/1594389515 conn(0x55ad540090d0 msgr2=0x55ad540a40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:40.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.480+0000 7f1256f4db80 1 -- 172.21.15.53:0/1594389515 shutdown_connections 2023-11-06T01:45:40.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.480+0000 7f1256f4db80 1 -- 172.21.15.53:0/1594389515 wait complete. 2023-11-06T01:45:40.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.480+0000 7f1256f4db80 20 client.26255 trim_cache size 0 max 16384 2023-11-06T01:45:40.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:40 smithi163 ceph-mon[104416]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 780 op/s 2023-11-06T01:45:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.505+0000 7f1256f4db80 1 Processor -- start 2023-11-06T01:45:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.505+0000 7f1256f4db80 1 -- start start 2023-11-06T01:45:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.505+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 0x55ad540a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.505+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540ab0f0 0x55ad540b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.505+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.505+0000 7f1256f4db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad540b3610 con 0x55ad540ab0f0 2023-11-06T01:45:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.505+0000 7f1256f4db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad540b24a0 con 0x55ad540aa3f0 2023-11-06T01:45:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.505+0000 7f1256f4db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad540a5db0 con 0x55ad540aba70 2023-11-06T01:45:40.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.505+0000 7f124935a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.506+0000 7f124935a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57366/0 (socket says 172.21.15.53:57366) 2023-11-06T01:45:40.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.506+0000 7f124935a700 1 -- 172.21.15.53:0/1352709706 learned_addr learned my addr 172.21.15.53:0/1352709706 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:40.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.506+0000 7f1248b59700 1 --2- 172.21.15.53:0/1352709706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 0x55ad540a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.506+0000 7f124935a700 1 -- 172.21.15.53:0/1352709706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 msgr2=0x55ad540a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:40.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.506+0000 7f124935a700 1 --2- 172.21.15.53:0/1352709706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 0x55ad540a9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.506+0000 7f124935a700 1 -- 172.21.15.53:0/1352709706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540ab0f0 msgr2=0x55ad540b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:40.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.506+0000 7f124935a700 1 --2- 172.21.15.53:0/1352709706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540ab0f0 0x55ad540b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.506+0000 7f124935a700 1 -- 172.21.15.53:0/1352709706 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad53e4f250 con 0x55ad540aba70 2023-11-06T01:45:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.506+0000 7f124935a700 1 --2- 172.21.15.53:0/1352709706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540b37f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1238096ec0 tx=0x7f123809a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7662cfe2c92baea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.506+0000 7f122b7fe700 1 -- 172.21.15.53:0/1352709706 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123809ab50 con 0x55ad540aba70 2023-11-06T01:45:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.507+0000 7f122b7fe700 1 -- 172.21.15.53:0/1352709706 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f123809a450 con 0x55ad540aba70 2023-11-06T01:45:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.507+0000 7f122b7fe700 1 -- 172.21.15.53:0/1352709706 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238085070 con 0x55ad540aba70 2023-11-06T01:45:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.508+0000 7f1256f4db80 1 -- 172.21.15.53:0/1352709706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 msgr2=0x55ad540b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.508+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1352709706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540b37f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1238096ec0 tx=0x7f123809a2a0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.508+0000 7f1256f4db80 1 -- 172.21.15.53:0/1352709706 shutdown_connections 2023-11-06T01:45:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.508+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1352709706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540aa3f0 0x55ad540a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.508+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1352709706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540b37f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.508+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1352709706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540ab0f0 0x55ad540b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.508+0000 7f1256f4db80 1 -- 172.21.15.53:0/1352709706 >> 172.21.15.53:0/1352709706 conn(0x55ad540090d0 msgr2=0x55ad540ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.508+0000 7f1256f4db80 1 -- 172.21.15.53:0/1352709706 shutdown_connections 2023-11-06T01:45:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.508+0000 7f1256f4db80 1 -- 172.21.15.53:0/1352709706 wait complete. 2023-11-06T01:45:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.508+0000 7f1256f4db80 1 -- start start 2023-11-06T01:45:40.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.508+0000 7f1256f4db80 1 Processor -- start 2023-11-06T01:45:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aa3f0 0x55ad540b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540ab0f0 0x55ad540b7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1256f4db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1256f4db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad540b8280 con 0x55ad540aa3f0 2023-11-06T01:45:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1256f4db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad540b83f0 con 0x55ad540ab0f0 2023-11-06T01:45:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1256f4db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad541114a0 con 0x55ad540aba70 2023-11-06T01:45:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1256f4db80 10 client.0 _ll_register_callbacks cb 0x7f123c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1243fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540ab0f0 0x55ad540b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1248b59700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aa3f0 0x55ad540b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1243fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540ab0f0 0x55ad540b7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54572/0 (socket says 172.21.15.53:54572) 2023-11-06T01:45:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1243fff700 1 -- 172.21.15.53:0/1513646850 learned_addr learned my addr 172.21.15.53:0/1513646850 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f124935a700 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1243fff700 1 -- 172.21.15.53:0/1513646850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 msgr2=0x55ad540b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1243fff700 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1243fff700 1 -- 172.21.15.53:0/1513646850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aa3f0 msgr2=0x55ad540b8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1243fff700 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aa3f0 0x55ad540b8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.509+0000 7f1243fff700 1 -- 172.21.15.53:0/1513646850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12441a5790 con 0x55ad540ab0f0 2023-11-06T01:45:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.510+0000 7f1243fff700 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540ab0f0 0x55ad540b7540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55ad540abe50 tx=0x7f1234013860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f76ba4fd9a92d0d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.510+0000 7f1223fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:45:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.510+0000 7f1223fff700 1 -- 172.21.15.53:0/1513646850 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12340322f0 con 0x55ad540ab0f0 2023-11-06T01:45:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.510+0000 7f1223fff700 1 -- 172.21.15.53:0/1513646850 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1234006be0 con 0x55ad540ab0f0 2023-11-06T01:45:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.510+0000 7f1256f4db80 1 -- 172.21.15.53:0/1513646850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12340044e0 con 0x55ad540ab0f0 2023-11-06T01:45:40.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.510+0000 7f1223fff700 1 -- 172.21.15.53:0/1513646850 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1234032450 con 0x55ad540ab0f0 2023-11-06T01:45:40.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.510+0000 7f1256f4db80 10 client.26261 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:40.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.510+0000 7f1256f4db80 1 -- 172.21.15.53:0/1513646850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ad53e272b0 con 0x55ad540ab0f0 2023-11-06T01:45:40.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.510+0000 7f1223fff700 1 -- 172.21.15.53:0/1513646850 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12340044e0 con 0x55ad540ab0f0 2023-11-06T01:45:40.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.511+0000 7f1223fff700 1 -- 172.21.15.53:0/1513646850 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f123401e030 con 0x55ad540ab0f0 2023-11-06T01:45:40.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.511+0000 7f1256f4db80 10 client.26261 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:40.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.511+0000 7f1256f4db80 10 client.26261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:40.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.511+0000 7f1256f4db80 20 client.26261 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:40.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.511+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad54112e40 0x55ad54115260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:40.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.511+0000 7f1256f4db80 4 client.26261 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:45:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.511+0000 7f1256f4db80 1 -- 172.21.15.53:0/1513646850 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ad53ff3d40 con 0x55ad54112e40 2023-11-06T01:45:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.511+0000 7f1248b59700 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad54112e40 0x55ad54115260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1223fff700 10 client.26261 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1248b59700 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad54112e40 0x55ad54115260 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c0f9038737c18 server_cookie=3e1df6b249e76a1d in_seq=0 out_seq=0 2023-11-06T01:45:40.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1223fff700 1 -- 172.21.15.53:0/1513646850 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ad53ff3d40 con 0x55ad54112e40 2023-11-06T01:45:40.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1223fff700 10 client.26261 handle_command_reply: tid=0 2023-11-06T01:45:40.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1256f4db80 1 client.26261 shutdown 2023-11-06T01:45:40.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1256f4db80 1 -- 172.21.15.53:0/1513646850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540ab0f0 msgr2=0x55ad540b7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:40.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540ab0f0 0x55ad540b7540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55ad540abe50 tx=0x7f1234013860 comp rx=0 tx=0).stop 2023-11-06T01:45:40.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1256f4db80 1 -- 172.21.15.53:0/1513646850 shutdown_connections 2023-11-06T01:45:40.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad54112e40 0x55ad54115260 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad540aba70 0x55ad540b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1223fff700 0 client.26261 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:40.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.513+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad540aa3f0 0x55ad540b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.514+0000 7f1256f4db80 1 --2- 172.21.15.53:0/1513646850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad540ab0f0 0x55ad540b7540 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.514+0000 7f1256f4db80 1 -- 172.21.15.53:0/1513646850 >> 172.21.15.53:0/1513646850 conn(0x55ad540090d0 msgr2=0x55ad540b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:40.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.514+0000 7f1256f4db80 1 -- 172.21.15.53:0/1513646850 shutdown_connections 2023-11-06T01:45:40.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.514+0000 7f1256f4db80 1 -- 172.21.15.53:0/1513646850 wait complete. 2023-11-06T01:45:40.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.514+0000 7f1256f4db80 20 client.26261 trim_cache size 0 max 16384 2023-11-06T01:45:40.529 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:45:40.529 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:45:40.529 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:45:40.529 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:45:40.530 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:45:40.530 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:45:40.530 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:45:40.530 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:45:40.530 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:45:40.531 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:45:40.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 -- 172.21.15.53:0/3405656170 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f122c0a0620 msgr2=0x7f122c0a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:40.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f122c0a0620 0x7f122c0a2ae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1234015300 tx=0x7f1234009000 comp rx=0 tx=0).stop 2023-11-06T01:45:40.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 -- 172.21.15.53:0/3405656170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f124410e150 msgr2=0x7f12441a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:40.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f124410e150 0x7f12441a4790 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1238003cd0 tx=0x7f1238003db0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 -- 172.21.15.53:0/3405656170 shutdown_connections 2023-11-06T01:45:40.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f122c0a0620 0x7f122c0a2ae0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f124410e150 0x7f12441a4790 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244108ea0 0x7f124419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 --2- 172.21.15.53:0/3405656170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f124410b7d0 0x7f124419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:40.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 -- 172.21.15.53:0/3405656170 >> 172.21.15.53:0/3405656170 conn(0x7f124406db40 msgr2=0x7f124410e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:40.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 -- 172.21.15.53:0/3405656170 shutdown_connections 2023-11-06T01:45:40.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:45:40.515+0000 7f124adbd700 1 -- 172.21.15.53:0/3405656170 wait complete. 2023-11-06T01:45:40.535 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:45:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.804+0000 7f490e65e700 1 -- 172.21.15.53:0/2176235595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4908101fc0 msgr2=0x7f4908102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.804+0000 7f490e65e700 1 --2- 172.21.15.53:0/2176235595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4908101fc0 0x7f4908102420 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0099c0 tx=0x7f48fc009cd0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.804+0000 7f490e65e700 1 -- 172.21.15.53:0/2176235595 shutdown_connections 2023-11-06T01:45:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.804+0000 7f490e65e700 1 --2- 172.21.15.53:0/2176235595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4908100dd0 0x7f49081011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.804+0000 7f490e65e700 1 --2- 172.21.15.53:0/2176235595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4908102960 0x7f4908109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.804+0000 7f490e65e700 1 --2- 172.21.15.53:0/2176235595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4908101fc0 0x7f4908102420 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.804+0000 7f490e65e700 1 -- 172.21.15.53:0/2176235595 >> 172.21.15.53:0/2176235595 conn(0x7f49080fc340 msgr2=0x7f49080fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.804+0000 7f490e65e700 1 -- 172.21.15.53:0/2176235595 shutdown_connections 2023-11-06T01:45:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.804+0000 7f490e65e700 1 -- 172.21.15.53:0/2176235595 wait complete. 2023-11-06T01:45:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.805+0000 7f490e65e700 1 Processor -- start 2023-11-06T01:45:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.805+0000 7f490e65e700 1 -- start start 2023-11-06T01:45:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.805+0000 7f490e65e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4908100dd0 0x7f4908196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.805+0000 7f490e65e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4908101fc0 0x7f4908197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.805+0000 7f490e65e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4908102960 0x7f4908197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.805+0000 7f490e65e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49081985f0 con 0x7f4908101fc0 2023-11-06T01:45:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.805+0000 7f490e65e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4908198760 con 0x7f4908102960 2023-11-06T01:45:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.805+0000 7f490e65e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4908198920 con 0x7f4908100dd0 2023-11-06T01:45:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f49077fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4908101fc0 0x7f4908197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f4907fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4908100dd0 0x7f4908196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f4907fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4908100dd0 0x7f4908196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57386/0 (socket says 172.21.15.53:57386) 2023-11-06T01:45:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f4907fff700 1 -- 172.21.15.53:0/2955413433 learned_addr learned my addr 172.21.15.53:0/2955413433 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f49077fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4908101fc0 0x7f4908197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43148/0 (socket says 172.21.15.53:43148) 2023-11-06T01:45:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f490cbfb700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4908102960 0x7f4908197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f4907fff700 1 -- 172.21.15.53:0/2955413433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4908102960 msgr2=0x7f4908197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f4907fff700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4908102960 0x7f4908197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f4907fff700 1 -- 172.21.15.53:0/2955413433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4908101fc0 msgr2=0x7f4908197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f4907fff700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4908101fc0 0x7f4908197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f4907fff700 1 -- 172.21.15.53:0/2955413433 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc0095f0 con 0x7f4908100dd0 2023-11-06T01:45:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f490cbfb700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4908102960 0x7f4908197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f4907fff700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4908100dd0 0x7f4908196bd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f48f800e970 tx=0x7f48f800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f49057fa700 1 -- 172.21.15.53:0/2955413433 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8009790 con 0x7f4908100dd0 2023-11-06T01:45:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f490e65e700 1 -- 172.21.15.53:0/2955413433 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4908198c00 con 0x7f4908100dd0 2023-11-06T01:45:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f49057fa700 1 -- 172.21.15.53:0/2955413433 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f800c7e0 con 0x7f4908100dd0 2023-11-06T01:45:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f49057fa700 1 -- 172.21.15.53:0/2955413433 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8003c70 con 0x7f4908100dd0 2023-11-06T01:45:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.806+0000 7f490e65e700 1 -- 172.21.15.53:0/2955413433 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490819f3c0 con 0x7f4908100dd0 2023-11-06T01:45:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.809+0000 7f49057fa700 1 -- 172.21.15.53:0/2955413433 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f48f800cc50 con 0x7f4908100dd0 2023-11-06T01:45:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.809+0000 7f49057fa700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f48f00a0520 0x7f48f00a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.810+0000 7f49057fa700 1 -- 172.21.15.53:0/2955413433 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f48f8014070 con 0x7f4908100dd0 2023-11-06T01:45:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.811+0000 7f491a7eeb80 1 Processor -- start 2023-11-06T01:45:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.812+0000 7f49077fe700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f48f00a0520 0x7f48f00a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.812+0000 7f491a7eeb80 1 -- start start 2023-11-06T01:45:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d726aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726fe0 0x561f7d7278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d728350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d67e9b0 con 0x561f7d726fe0 2023-11-06T01:45:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d67c370 con 0x561f7d726680 2023-11-06T01:45:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d677c60 con 0x561f7d727ed0 2023-11-06T01:45:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f49077fe700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f48f00a0520 0x7f48f00a29e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00b580 tx=0x7f48fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f4907fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d726aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f4907fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d726aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54592/0 (socket says 172.21.15.53:54592) 2023-11-06T01:45:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f4907fff700 1 -- 172.21.15.53:0/3173954987 learned_addr learned my addr 172.21.15.53:0/3173954987 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f490cbfb700 1 --2- 172.21.15.53:0/3173954987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d728350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f49077fe700 1 --2- 172.21.15.53:0/3173954987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726fe0 0x561f7d7278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f4907fff700 1 -- 172.21.15.53:0/3173954987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 msgr2=0x561f7d728350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f4907fff700 1 --2- 172.21.15.53:0/3173954987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d728350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f4907fff700 1 -- 172.21.15.53:0/3173954987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726fe0 msgr2=0x561f7d7278f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f4907fff700 1 --2- 172.21.15.53:0/3173954987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726fe0 0x561f7d7278f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f4907fff700 1 -- 172.21.15.53:0/3173954987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7d4a5d80 con 0x561f7d726680 2023-11-06T01:45:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.813+0000 7f4907fff700 1 --2- 172.21.15.53:0/3173954987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d726aa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f48f80a0ee0 tx=0x7f48f8003710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ac564531f67d2e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.814+0000 7f48ee7fc700 1 -- 172.21.15.53:0/3173954987 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f80840c0 con 0x561f7d726680 2023-11-06T01:45:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.814+0000 7f48ee7fc700 1 -- 172.21.15.53:0/3173954987 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f80993a0 con 0x561f7d726680 2023-11-06T01:45:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.814+0000 7f48ee7fc700 1 -- 172.21.15.53:0/3173954987 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8099860 con 0x561f7d726680 2023-11-06T01:45:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.814+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3173954987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 msgr2=0x561f7d726aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.814+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3173954987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d726aa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f48f80a0ee0 tx=0x7f48f8003710 comp rx=0 tx=0).stop 2023-11-06T01:45:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.814+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3173954987 shutdown_connections 2023-11-06T01:45:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.814+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3173954987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d726aa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.814+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3173954987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d728350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.814+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3173954987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726fe0 0x561f7d7278f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.814+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3173954987 >> 172.21.15.53:0/3173954987 conn(0x561f7d686070 msgr2=0x561f7d723040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.814+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3173954987 shutdown_connections 2023-11-06T01:45:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3173954987 wait complete. 2023-11-06T01:45:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f491a7eeb80 1 -- start start 2023-11-06T01:45:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f491a7eeb80 1 Processor -- start 2023-11-06T01:45:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726680 0x561f7d730520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726fe0 0x561f7d730a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d737aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d6144b0 con 0x561f7d726680 2023-11-06T01:45:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d614150 con 0x561f7d726fe0 2023-11-06T01:45:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d4fb8c0 con 0x561f7d727ed0 2023-11-06T01:45:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f491a7eeb80 10 client.0 _ll_register_callbacks cb 0x561f7d561a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f4907fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726680 0x561f7d730520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f49077fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726fe0 0x561f7d730a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f4907fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726680 0x561f7d730520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43160/0 (socket says 172.21.15.53:43160) 2023-11-06T01:45:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f4907fff700 1 -- 172.21.15.53:0/3258599500 learned_addr learned my addr 172.21.15.53:0/3258599500 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.815+0000 7f490cbfb700 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d737aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f4907fff700 1 -- 172.21.15.53:0/3258599500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 msgr2=0x561f7d737aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f4907fff700 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d737aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f4907fff700 1 -- 172.21.15.53:0/3258599500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726fe0 msgr2=0x561f7d730a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f4907fff700 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726fe0 0x561f7d730a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f4907fff700 1 -- 172.21.15.53:0/3258599500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490819f3c0 con 0x561f7d726680 2023-11-06T01:45:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f490cbfb700 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d737aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:45:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f4907fff700 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726680 0x561f7d730520 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f48f80c99b0 tx=0x7f48f8005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef607b63911b433 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f48e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:45:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f48e77fe700 1 -- 172.21.15.53:0/3258599500 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8084050 con 0x561f7d726680 2023-11-06T01:45:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f48e77fe700 1 -- 172.21.15.53:0/3258599500 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f48f8097740 con 0x561f7d726680 2023-11-06T01:45:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3258599500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f7d6a2cb0 con 0x561f7d726680 2023-11-06T01:45:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f48e77fe700 1 -- 172.21.15.53:0/3258599500 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f8095c00 con 0x561f7d726680 2023-11-06T01:45:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f491a7eeb80 10 client.16746 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.816+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3258599500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x561f7d48ec00 con 0x561f7d726680 2023-11-06T01:45:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.817+0000 7f48e77fe700 1 -- 172.21.15.53:0/3258599500 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f7d6a2cb0 con 0x561f7d726680 2023-11-06T01:45:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.817+0000 7f48e77fe700 1 -- 172.21.15.53:0/3258599500 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f48f80a0ff0 con 0x561f7d726680 2023-11-06T01:45:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.817+0000 7f491a7eeb80 10 client.16746 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.817+0000 7f491a7eeb80 10 client.16746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.817+0000 7f491a7eeb80 20 client.16746 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.817+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561f7d73bf90 0x561f7d73e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.817+0000 7f491a7eeb80 4 client.16746 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:45:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.817+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3258599500 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f7d6f5db0 con 0x561f7d73bf90 2023-11-06T01:45:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.817+0000 7f49077fe700 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561f7d73bf90 0x561f7d73e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.818+0000 7f48e77fe700 10 client.16746 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.818+0000 7f49077fe700 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561f7d73bf90 0x561f7d73e3b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a2eb58fd87f43e server_cookie=7e0523390e8464a2 in_seq=0 out_seq=0 2023-11-06T01:45:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f48e77fe700 1 -- 172.21.15.53:0/3258599500 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561f7d6f5db0 con 0x561f7d73bf90 2023-11-06T01:45:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f48e77fe700 10 client.16746 handle_command_reply: tid=0 2023-11-06T01:45:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 1 client.16746 shutdown 2023-11-06T01:45:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3258599500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726680 msgr2=0x561f7d730520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726680 0x561f7d730520 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f48f80c99b0 tx=0x7f48f8005bb0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3258599500 shutdown_connections 2023-11-06T01:45:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561f7d73bf90 0x561f7d73e3b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d737aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f48e77fe700 0 client.16746 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726680 0x561f7d730520 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3258599500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726fe0 0x561f7d730a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3258599500 >> 172.21.15.53:0/3258599500 conn(0x561f7d686070 msgr2=0x561f7d7213c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3258599500 shutdown_connections 2023-11-06T01:45:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3258599500 wait complete. 2023-11-06T01:45:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.819+0000 7f491a7eeb80 20 client.16746 trim_cache size 0 max 16384 2023-11-06T01:45:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f491a7eeb80 1 Processor -- start 2023-11-06T01:45:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f491a7eeb80 1 -- start start 2023-11-06T01:45:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d735a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d726fe0 0x561f7d72fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d727ed0 0x561f7d722c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d72ef90 con 0x561f7d727ed0 2023-11-06T01:45:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d727d50 con 0x561f7d726680 2023-11-06T01:45:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d7197a0 con 0x561f7d726fe0 2023-11-06T01:45:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f490cbfb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d727ed0 0x561f7d722c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f49077fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d726fe0 0x561f7d72fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f4907fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d735a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f49077fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d726fe0 0x561f7d72fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57410/0 (socket says 172.21.15.53:57410) 2023-11-06T01:45:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f490cbfb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d727ed0 0x561f7d722c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43172/0 (socket says 172.21.15.53:43172) 2023-11-06T01:45:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f4907fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d735a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54606/0 (socket says 172.21.15.53:54606) 2023-11-06T01:45:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f49077fe700 1 -- 172.21.15.53:0/3796738953 learned_addr learned my addr 172.21.15.53:0/3796738953 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f49077fe700 1 -- 172.21.15.53:0/3796738953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 msgr2=0x561f7d735a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f49077fe700 1 --2- 172.21.15.53:0/3796738953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d735a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f49077fe700 1 -- 172.21.15.53:0/3796738953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d727ed0 msgr2=0x561f7d722c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.839+0000 7f49077fe700 1 --2- 172.21.15.53:0/3796738953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d727ed0 0x561f7d722c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.840+0000 7f49077fe700 1 -- 172.21.15.53:0/3796738953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7d4cc1f0 con 0x561f7d726fe0 2023-11-06T01:45:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.840+0000 7f49077fe700 1 --2- 172.21.15.53:0/3796738953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d726fe0 0x561f7d72fbd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005040 tx=0x7f48fc021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=873dbfb2b0f4175d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.840+0000 7f48eeffd700 1 -- 172.21.15.53:0/3796738953 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc02b040 con 0x561f7d726fe0 2023-11-06T01:45:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.840+0000 7f48eeffd700 1 -- 172.21.15.53:0/3796738953 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48fc02c040 con 0x561f7d726fe0 2023-11-06T01:45:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.840+0000 7f48eeffd700 1 -- 172.21.15.53:0/3796738953 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc003a30 con 0x561f7d726fe0 2023-11-06T01:45:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.841+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3796738953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d726fe0 msgr2=0x561f7d72fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.841+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3796738953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d726fe0 0x561f7d72fbd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005040 tx=0x7f48fc021000 comp rx=0 tx=0).stop 2023-11-06T01:45:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.841+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3796738953 shutdown_connections 2023-11-06T01:45:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.841+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3796738953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d735a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.841+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3796738953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d727ed0 0x561f7d722c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.841+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/3796738953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d726fe0 0x561f7d72fbd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.841+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3796738953 >> 172.21.15.53:0/3796738953 conn(0x561f7d686070 msgr2=0x561f7d72f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.841+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3796738953 shutdown_connections 2023-11-06T01:45:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.841+0000 7f491a7eeb80 1 -- 172.21.15.53:0/3796738953 wait complete. 2023-11-06T01:45:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.841+0000 7f491a7eeb80 1 -- start start 2023-11-06T01:45:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.841+0000 7f491a7eeb80 1 Processor -- start 2023-11-06T01:45:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.842+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d787c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.842+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726fe0 0x561f7d78a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.842+0000 7f491a7eeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d78c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.842+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d78cf10 con 0x561f7d726fe0 2023-11-06T01:45:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.842+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d78d080 con 0x561f7d726680 2023-11-06T01:45:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.842+0000 7f491a7eeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7d78d240 con 0x561f7d727ed0 2023-11-06T01:45:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.842+0000 7f491a7eeb80 10 client.0 _ll_register_callbacks cb 0x7f48fc00b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.843+0000 7f490cbfb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d78c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.843+0000 7f49077fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726fe0 0x561f7d78a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.843+0000 7f49077fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726fe0 0x561f7d78a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43176/0 (socket says 172.21.15.53:43176) 2023-11-06T01:45:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.843+0000 7f49077fe700 1 -- 172.21.15.53:0/1728153649 learned_addr learned my addr 172.21.15.53:0/1728153649 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.844+0000 7f490cbfb700 1 -- 172.21.15.53:0/1728153649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 msgr2=0x561f7d787c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:45:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.844+0000 7f490cbfb700 1 --2- 172.21.15.53:0/1728153649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d787c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.844+0000 7f490cbfb700 1 -- 172.21.15.53:0/1728153649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726fe0 msgr2=0x561f7d78a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.844+0000 7f490cbfb700 1 --2- 172.21.15.53:0/1728153649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726fe0 0x561f7d78a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.844+0000 7f490cbfb700 1 -- 172.21.15.53:0/1728153649 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4908109b60 con 0x561f7d727ed0 2023-11-06T01:45:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.844+0000 7f490cbfb700 1 --2- 172.21.15.53:0/1728153649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d78c7d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x561f7d71c210 tx=0x7f4900004460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68bdd2d269c25c08 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.844+0000 7f48e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:45:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.844+0000 7f48e77fe700 1 -- 172.21.15.53:0/1728153649 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490001d070 con 0x561f7d727ed0 2023-11-06T01:45:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.845+0000 7f48e77fe700 1 -- 172.21.15.53:0/1728153649 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4900004bb0 con 0x561f7d727ed0 2023-11-06T01:45:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.845+0000 7f491a7eeb80 1 -- 172.21.15.53:0/1728153649 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48f80993a0 con 0x561f7d727ed0 2023-11-06T01:45:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.845+0000 7f48e77fe700 1 -- 172.21.15.53:0/1728153649 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4900004d70 con 0x561f7d727ed0 2023-11-06T01:45:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.845+0000 7f491a7eeb80 10 client.26335 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.845+0000 7f491a7eeb80 1 -- 172.21.15.53:0/1728153649 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x561f7d4a4250 con 0x561f7d727ed0 2023-11-06T01:45:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.845+0000 7f48e77fe700 1 -- 172.21.15.53:0/1728153649 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f80993a0 con 0x561f7d727ed0 2023-11-06T01:45:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.845+0000 7f48e77fe700 1 -- 172.21.15.53:0/1728153649 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4900022770 con 0x561f7d727ed0 2023-11-06T01:45:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.846+0000 7f491a7eeb80 10 client.26335 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.846+0000 7f491a7eeb80 10 client.26335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.846+0000 7f491a7eeb80 20 client.26335 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.846+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/1728153649 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561f7d793040 0x561f7d795460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.846+0000 7f491a7eeb80 4 client.26335 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:45:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.846+0000 7f491a7eeb80 1 -- 172.21.15.53:0/1728153649 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f7d670ce0 con 0x561f7d793040 2023-11-06T01:45:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.846+0000 7f4907fff700 1 --2- 172.21.15.53:0/1728153649 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561f7d793040 0x561f7d795460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.846+0000 7f48e77fe700 10 client.26335 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.846+0000 7f4907fff700 1 --2- 172.21.15.53:0/1728153649 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561f7d793040 0x561f7d795460 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b93b810afe55cdc server_cookie=5523ddcbf6dedee6 in_seq=0 out_seq=0 2023-11-06T01:45:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.899+0000 7f48e77fe700 1 -- 172.21.15.53:0/1728153649 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f7d670ce0 con 0x561f7d793040 2023-11-06T01:45:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.899+0000 7f48e77fe700 10 client.26335 handle_command_reply: tid=0 2023-11-06T01:45:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.899+0000 7f491a7eeb80 1 client.26335 shutdown 2023-11-06T01:45:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.899+0000 7f491a7eeb80 1 -- 172.21.15.53:0/1728153649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 msgr2=0x561f7d78c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.899+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/1728153649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d78c7d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x561f7d71c210 tx=0x7f4900004460 comp rx=0 tx=0).stop 2023-11-06T01:45:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.900+0000 7f491a7eeb80 1 -- 172.21.15.53:0/1728153649 shutdown_connections 2023-11-06T01:45:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.900+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/1728153649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f7d726680 0x561f7d787c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.900+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/1728153649 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x561f7d793040 0x561f7d795460 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.900+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/1728153649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561f7d727ed0 0x561f7d78c7d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.900+0000 7f491a7eeb80 1 --2- 172.21.15.53:0/1728153649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f7d726fe0 0x561f7d78a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.900+0000 7f491a7eeb80 1 -- 172.21.15.53:0/1728153649 >> 172.21.15.53:0/1728153649 conn(0x561f7d686070 msgr2=0x561f7d72f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.900+0000 7f48e77fe700 0 client.26335 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.902+0000 7f491a7eeb80 1 -- 172.21.15.53:0/1728153649 shutdown_connections 2023-11-06T01:45:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f491a7eeb80 1 -- 172.21.15.53:0/1728153649 wait complete. 2023-11-06T01:45:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f491a7eeb80 20 client.26335 trim_cache size 0 max 16384 2023-11-06T01:45:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:45:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:45:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:45:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:45:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f490e65e700 1 -- 172.21.15.53:0/2955413433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f48f00a0520 msgr2=0x7f48f00a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f490e65e700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f48f00a0520 0x7f48f00a29e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00b580 tx=0x7f48fc005fb0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f490e65e700 1 -- 172.21.15.53:0/2955413433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4908100dd0 msgr2=0x7f4908196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f490e65e700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4908100dd0 0x7f4908196bd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f48f800e970 tx=0x7f48f800ec80 comp rx=0 tx=0).stop 2023-11-06T01:45:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f490e65e700 1 -- 172.21.15.53:0/2955413433 shutdown_connections 2023-11-06T01:45:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f490e65e700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f48f00a0520 0x7f48f00a29e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f490e65e700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4908102960 0x7f4908197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f490e65e700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4908100dd0 0x7f4908196bd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f490e65e700 1 --2- 172.21.15.53:0/2955413433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4908101fc0 0x7f4908197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.905+0000 7f490e65e700 1 -- 172.21.15.53:0/2955413433 >> 172.21.15.53:0/2955413433 conn(0x7f49080fc340 msgr2=0x7f4908105520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.906+0000 7f490e65e700 1 -- 172.21.15.53:0/2955413433 shutdown_connections 2023-11-06T01:45:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:41.906+0000 7f490e65e700 1 -- 172.21.15.53:0/2955413433 wait complete. 2023-11-06T01:45:42.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 685.0 s, 1140.4 tps, lat 88.160 ms stddev 164.840 2023-11-06T01:45:42.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:42 smithi062 ceph-mon[103924]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 1.03k op/s 2023-11-06T01:45:42.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:42 smithi163 ceph-mon[104416]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 1.03k op/s 2023-11-06T01:45:42.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:42 smithi053 ceph-mon[99472]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 1.03k op/s 2023-11-06T01:45:44.238 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:43 smithi163 ceph-mon[104416]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 1.71k op/s 2023-11-06T01:45:44.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:43 smithi053 ceph-mon[99472]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 1.71k op/s 2023-11-06T01:45:44.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:43 smithi062 ceph-mon[103924]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 1.71k op/s 2023-11-06T01:45:46.237 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:45 smithi163 ceph-mon[104416]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2023-11-06T01:45:46.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:45 smithi053 ceph-mon[99472]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2023-11-06T01:45:46.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:45 smithi062 ceph-mon[103924]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2023-11-06T01:45:47.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 690.0 s, 1613.4 tps, lat 61.306 ms stddev 64.984 2023-11-06T01:45:48.237 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:47 smithi163 ceph-mon[104416]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.88k op/s 2023-11-06T01:45:48.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:47 smithi053 ceph-mon[99472]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.88k op/s 2023-11-06T01:45:48.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:47 smithi062 ceph-mon[103924]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.88k op/s 2023-11-06T01:45:50.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:49 smithi053 ceph-mon[99472]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.87k op/s 2023-11-06T01:45:50.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:49 smithi062 ceph-mon[103924]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.87k op/s 2023-11-06T01:45:50.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:50 smithi163 ceph-mon[104416]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.87k op/s 2023-11-06T01:45:52.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 695.0 s, 1648.9 tps, lat 60.689 ms stddev 66.645 2023-11-06T01:45:52.742 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:52 smithi163 ceph-mon[104416]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.78k op/s 2023-11-06T01:45:52.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:52 smithi053 ceph-mon[99472]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.78k op/s 2023-11-06T01:45:52.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:52 smithi062 ceph-mon[103924]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.78k op/s 2023-11-06T01:45:53.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:53 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:45:54.092 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:53 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:45:54.148 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:53 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:45:54.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:54 smithi163 ceph-mon[104416]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 MiB/s wr, 1.52k op/s 2023-11-06T01:45:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.082+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1082273039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e8107d90 msgr2=0x7fc0e810a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.082+0000 7fc0ef3e0700 1 --2- 172.21.15.53:0/1082273039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e8107d90 0x7fc0e810a1c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400b0a0 tx=0x7fc0e400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.085+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1082273039 shutdown_connections 2023-11-06T01:45:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.085+0000 7fc0ef3e0700 1 --2- 172.21.15.53:0/1082273039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e8107d90 0x7fc0e810a1c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.085+0000 7fc0ef3e0700 1 --2- 172.21.15.53:0/1082273039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e810d0d0 0x7fc0e810d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.085+0000 7fc0ef3e0700 1 --2- 172.21.15.53:0/1082273039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e810a700 0x7fc0e810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.085+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1082273039 >> 172.21.15.53:0/1082273039 conn(0x7fc0e806dbc0 msgr2=0x7fc0e8070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.085+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1082273039 shutdown_connections 2023-11-06T01:45:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.085+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1082273039 wait complete. 2023-11-06T01:45:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.086+0000 7fc0ef3e0700 1 Processor -- start 2023-11-06T01:45:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.086+0000 7fc0ef3e0700 1 -- start start 2023-11-06T01:45:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.087+0000 7fc0ef3e0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e8107d90 0x7fc0e81a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.087+0000 7fc0ef3e0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e810a700 0x7fc0e81a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.087+0000 7fc0ef3e0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e810d0d0 0x7fc0e81a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.087+0000 7fc0ef3e0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e81a74f0 con 0x7fc0e810d0d0 2023-11-06T01:45:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.087+0000 7fc0ef3e0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e81a7660 con 0x7fc0e810a700 2023-11-06T01:45:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.087+0000 7fc0ef3e0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e81a7820 con 0x7fc0e8107d90 2023-11-06T01:45:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.087+0000 7fc0ec97b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e810a700 0x7fc0e81a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.087+0000 7fc0ec97b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e810a700 0x7fc0e81a6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54624/0 (socket says 172.21.15.53:54624) 2023-11-06T01:45:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.087+0000 7fc0ec97b700 1 -- 172.21.15.53:0/1247661028 learned_addr learned my addr 172.21.15.53:0/1247661028 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.087+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e8107d90 0x7fc0e81a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0ec97b700 1 -- 172.21.15.53:0/1247661028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e8107d90 msgr2=0x7fc0e81a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0ec97b700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e8107d90 0x7fc0e81a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0ec97b700 1 -- 172.21.15.53:0/1247661028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e810d0d0 msgr2=0x7fc0e81a6e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:45:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0ec97b700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e810d0d0 0x7fc0e81a6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0ec97b700 1 -- 172.21.15.53:0/1247661028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4009d00 con 0x7fc0e810a700 2023-11-06T01:45:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e8107d90 0x7fc0e81a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:45:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0ec97b700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e810a700 0x7fc0e81a6010 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e9a0 tx=0x7fc0dc00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0da7fc700 1 -- 172.21.15.53:0/1247661028 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc00cbd0 con 0x7fc0e810a700 2023-11-06T01:45:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0da7fc700 1 -- 172.21.15.53:0/1247661028 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0dc00cd30 con 0x7fc0e810a700 2023-11-06T01:45:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0da7fc700 1 -- 172.21.15.53:0/1247661028 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc018680 con 0x7fc0e810a700 2023-11-06T01:45:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1247661028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e81a7b00 con 0x7fc0e810a700 2023-11-06T01:45:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.088+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1247661028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e810f6a0 con 0x7fc0e810a700 2023-11-06T01:45:55.093 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:54 smithi053 ceph-mon[99472]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 MiB/s wr, 1.52k op/s 2023-11-06T01:45:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.095+0000 7fc0fb570b80 1 Processor -- start 2023-11-06T01:45:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.095+0000 7fc0fb570b80 1 -- start start 2023-11-06T01:45:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.095+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6f680 0x55baf7b6faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.095+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6ffe0 0x55baf7b708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.095+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 0x55baf7b71350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf7ac79b0 con 0x55baf7b6f680 2023-11-06T01:45:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf7ac5370 con 0x55baf7b70ed0 2023-11-06T01:45:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf7ac0c60 con 0x55baf7b6ffe0 2023-11-06T01:45:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0ed97d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6f680 0x55baf7b6faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0ed97d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6f680 0x55baf7b6faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43190/0 (socket says 172.21.15.53:43190) 2023-11-06T01:45:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0ed97d700 1 -- 172.21.15.53:0/871565315 learned_addr learned my addr 172.21.15.53:0/871565315 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/871565315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6ffe0 0x55baf7b708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0da7fc700 1 -- 172.21.15.53:0/1247661028 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fc0dc0187e0 con 0x7fc0e810a700 2023-11-06T01:45:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0ed97d700 1 -- 172.21.15.53:0/871565315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6ffe0 msgr2=0x55baf7b708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0ed97d700 1 --2- 172.21.15.53:0/871565315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6ffe0 0x55baf7b708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0ed97d700 1 -- 172.21.15.53:0/871565315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 msgr2=0x55baf7b71350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:45:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0ed97d700 1 --2- 172.21.15.53:0/871565315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 0x55baf7b71350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.096+0000 7fc0ed97d700 1 -- 172.21.15.53:0/871565315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baf78eed80 con 0x55baf7b6f680 2023-11-06T01:45:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.097+0000 7fc0ed97d700 1 --2- 172.21.15.53:0/871565315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6f680 0x55baf7b6faa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009c60 tx=0x7fc0e0009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb375474ce39ca95 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.097+0000 7fc0d37fe700 1 -- 172.21.15.53:0/871565315 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e00097b0 con 0x55baf7b6f680 2023-11-06T01:45:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.097+0000 7fc0d37fe700 1 -- 172.21.15.53:0/871565315 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0e0004830 con 0x55baf7b6f680 2023-11-06T01:45:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.097+0000 7fc0d37fe700 1 -- 172.21.15.53:0/871565315 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e00055c0 con 0x55baf7b6f680 2023-11-06T01:45:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.097+0000 7fc0da7fc700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0d40a05f0 0x7fc0d40a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.097+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0d40a05f0 0x7fc0d40a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.097+0000 7fc0da7fc700 1 -- 172.21.15.53:0/1247661028 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc0dc014070 con 0x7fc0e810a700 2023-11-06T01:45:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.098+0000 7fc0fb570b80 1 -- 172.21.15.53:0/871565315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6f680 msgr2=0x55baf7b6faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.098+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/871565315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6f680 0x55baf7b6faa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009c60 tx=0x7fc0e0009f70 comp rx=0 tx=0).stop 2023-11-06T01:45:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.098+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0d40a05f0 0x7fc0d40a2ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4000f80 tx=0x7fc0e400b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.098+0000 7fc0fb570b80 1 -- 172.21.15.53:0/871565315 shutdown_connections 2023-11-06T01:45:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.098+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/871565315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 0x55baf7b71350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.098+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/871565315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6f680 0x55baf7b6faa0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.098+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/871565315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6ffe0 0x55baf7b708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.098+0000 7fc0fb570b80 1 -- 172.21.15.53:0/871565315 >> 172.21.15.53:0/871565315 conn(0x55baf7acf070 msgr2=0x55baf7b6c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.099+0000 7fc0fb570b80 1 -- 172.21.15.53:0/871565315 shutdown_connections 2023-11-06T01:45:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.099+0000 7fc0fb570b80 1 -- 172.21.15.53:0/871565315 wait complete. 2023-11-06T01:45:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0fb570b80 1 -- start start 2023-11-06T01:45:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0fb570b80 1 Processor -- start 2023-11-06T01:45:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 0x55baf7b79550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6ffe0 0x55baf7b79a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 0x55baf7b80ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf7a5d4b0 con 0x55baf7b6ffe0 2023-11-06T01:45:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf7a5d150 con 0x55baf7b70ed0 2023-11-06T01:45:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf79448c0 con 0x55baf7b6f680 2023-11-06T01:45:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0fb570b80 10 client.0 _ll_register_callbacks cb 0x55baf79aaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0ec97b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6ffe0 0x55baf7b79a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0ed17c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 0x55baf7b79550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0ec97b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6ffe0 0x55baf7b79a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43198/0 (socket says 172.21.15.53:43198) 2023-11-06T01:45:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0ec97b700 1 -- 172.21.15.53:0/3981741188 learned_addr learned my addr 172.21.15.53:0/3981741188 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.100+0000 7fc0ed97d700 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 0x55baf7b80ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.101+0000 7fc0ed17c700 1 -- 172.21.15.53:0/3981741188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 msgr2=0x55baf7b80ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.101+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 0x55baf7b80ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.101+0000 7fc0ed17c700 1 -- 172.21.15.53:0/3981741188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6ffe0 msgr2=0x55baf7b79a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.101+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6ffe0 0x55baf7b79a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.101+0000 7fc0ed17c700 1 -- 172.21.15.53:0/3981741188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e0009910 con 0x55baf7b6f680 2023-11-06T01:45:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.101+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 0x55baf7b79550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009fd0 tx=0x7fc0e400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb9c610dc10b55b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.101+0000 7fc0d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:45:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.101+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3981741188 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400e040 con 0x55baf7b6f680 2023-11-06T01:45:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.101+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3981741188 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc0e4023040 con 0x55baf7b6f680 2023-11-06T01:45:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.101+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3981741188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55baf7aebcb0 con 0x55baf7b6f680 2023-11-06T01:45:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3981741188 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0e401c820 con 0x55baf7b6f680 2023-11-06T01:45:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0fb570b80 10 client.26344 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3981741188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55baf78d7c00 con 0x55baf7b6f680 2023-11-06T01:45:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3981741188 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55baf7aebcb0 con 0x55baf7b6f680 2023-11-06T01:45:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3981741188 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc0e4005710 con 0x55baf7b6f680 2023-11-06T01:45:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0fb570b80 10 client.26344 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0fb570b80 10 client.26344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0fb570b80 20 client.26344 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55baf7b85190 0x55baf7b875b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0fb570b80 4 client.26344 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:45:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3981741188 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55baf7b3edb0 con 0x55baf7b85190 2023-11-06T01:45:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.102+0000 7fc0ec97b700 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55baf7b85190 0x55baf7b875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.103+0000 7fc0ec97b700 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55baf7b85190 0x55baf7b875b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80fa41c157ff14a1 server_cookie=93047e33cbe37961 in_seq=0 out_seq=0 2023-11-06T01:45:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.103+0000 7fc0d0ff9700 10 client.26344 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3981741188 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55baf7b3edb0 con 0x55baf7b85190 2023-11-06T01:45:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0d0ff9700 10 client.26344 handle_command_reply: tid=0 2023-11-06T01:45:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0fb570b80 1 client.26344 shutdown 2023-11-06T01:45:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3981741188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 msgr2=0x55baf7b79550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 0x55baf7b79550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009fd0 tx=0x7fc0e400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3981741188 shutdown_connections 2023-11-06T01:45:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 0x55baf7b80ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55baf7b85190 0x55baf7b875b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 0x55baf7b79550 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0d0ff9700 0 client.26344 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3981741188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6ffe0 0x55baf7b79a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3981741188 >> 172.21.15.53:0/3981741188 conn(0x55baf7acf070 msgr2=0x55baf7b6a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3981741188 shutdown_connections 2023-11-06T01:45:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.104+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3981741188 wait complete. 2023-11-06T01:45:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.105+0000 7fc0fb570b80 20 client.26344 trim_cache size 0 max 16384 2023-11-06T01:45:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.127+0000 7fc0fb570b80 1 Processor -- start 2023-11-06T01:45:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.127+0000 7fc0fb570b80 1 -- start start 2023-11-06T01:45:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.127+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b6f680 0x55baf7b6fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.127+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6ffe0 0x55baf7b7ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.127+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b70ed0 0x55baf7b77320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.127+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf7b7a740 con 0x55baf7b70ed0 2023-11-06T01:45:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.127+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf7b627a0 con 0x55baf7b6f680 2023-11-06T01:45:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.127+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf7b78a00 con 0x55baf7b6ffe0 2023-11-06T01:45:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.128+0000 7fc0ed97d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b70ed0 0x55baf7b77320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.128+0000 7fc0ed97d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b70ed0 0x55baf7b77320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43208/0 (socket says 172.21.15.53:43208) 2023-11-06T01:45:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.128+0000 7fc0ed97d700 1 -- 172.21.15.53:0/501551431 learned_addr learned my addr 172.21.15.53:0/501551431 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.128+0000 7fc0ec97b700 1 --2- 172.21.15.53:0/501551431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6ffe0 0x55baf7b7ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.129+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/501551431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b6f680 0x55baf7b6fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.129+0000 7fc0ed97d700 1 -- 172.21.15.53:0/501551431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6ffe0 msgr2=0x55baf7b7ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.129+0000 7fc0ed97d700 1 --2- 172.21.15.53:0/501551431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6ffe0 0x55baf7b7ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.129+0000 7fc0ed97d700 1 -- 172.21.15.53:0/501551431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b6f680 msgr2=0x55baf7b6fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.129+0000 7fc0ed97d700 1 --2- 172.21.15.53:0/501551431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b6f680 0x55baf7b6fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.129+0000 7fc0ed97d700 1 -- 172.21.15.53:0/501551431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baf79151f0 con 0x55baf7b70ed0 2023-11-06T01:45:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.129+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/501551431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b6f680 0x55baf7b6fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:45:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.129+0000 7fc0ed97d700 1 --2- 172.21.15.53:0/501551431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b70ed0 0x55baf7b77320 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009c30 tx=0x7fc0e00094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b34caf6e13be4f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.130+0000 7fc0d3fff700 1 -- 172.21.15.53:0/501551431 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0004ba0 con 0x55baf7b70ed0 2023-11-06T01:45:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.130+0000 7fc0d3fff700 1 -- 172.21.15.53:0/501551431 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0e00056c0 con 0x55baf7b70ed0 2023-11-06T01:45:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.130+0000 7fc0d3fff700 1 -- 172.21.15.53:0/501551431 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0023480 con 0x55baf7b70ed0 2023-11-06T01:45:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.131+0000 7fc0fb570b80 1 -- 172.21.15.53:0/501551431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b70ed0 msgr2=0x55baf7b77320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.131+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/501551431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b70ed0 0x55baf7b77320 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009c30 tx=0x7fc0e00094b0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.133+0000 7fc0fb570b80 1 -- 172.21.15.53:0/501551431 shutdown_connections 2023-11-06T01:45:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.133+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/501551431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b6f680 0x55baf7b6fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.133+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/501551431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b70ed0 0x55baf7b77320 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.133+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/501551431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6ffe0 0x55baf7b7ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.133+0000 7fc0fb570b80 1 -- 172.21.15.53:0/501551431 >> 172.21.15.53:0/501551431 conn(0x55baf7acf070 msgr2=0x55baf7b78170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.133+0000 7fc0fb570b80 1 -- 172.21.15.53:0/501551431 shutdown_connections 2023-11-06T01:45:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.133+0000 7fc0fb570b80 1 -- 172.21.15.53:0/501551431 wait complete. 2023-11-06T01:45:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.133+0000 7fc0fb570b80 1 -- start start 2023-11-06T01:45:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0fb570b80 1 Processor -- start 2023-11-06T01:45:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 0x55baf7bd0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6ffe0 0x55baf7bd31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0fb570b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 0x55baf7bd5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf7bd5e40 con 0x55baf7b6ffe0 2023-11-06T01:45:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf7bd5fb0 con 0x55baf7b70ed0 2023-11-06T01:45:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0fb570b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baf7bd6170 con 0x55baf7b6f680 2023-11-06T01:45:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0fb570b80 10 client.0 _ll_register_callbacks cb 0x7fc0e40078f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:45:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0ec97b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6ffe0 0x55baf7bd31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0ed97d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 0x55baf7bd5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0ed17c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 0x55baf7bd0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0ed17c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 0x55baf7bd0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57452/0 (socket says 172.21.15.53:57452) 2023-11-06T01:45:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.134+0000 7fc0ed17c700 1 -- 172.21.15.53:0/3262210354 learned_addr learned my addr 172.21.15.53:0/3262210354 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:45:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.135+0000 7fc0ed17c700 1 -- 172.21.15.53:0/3262210354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 msgr2=0x55baf7bd5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.135+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/3262210354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 0x55baf7bd5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.135+0000 7fc0ed17c700 1 -- 172.21.15.53:0/3262210354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6ffe0 msgr2=0x55baf7bd31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:45:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.135+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/3262210354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6ffe0 0x55baf7bd31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.135+0000 7fc0ed17c700 1 -- 172.21.15.53:0/3262210354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55baf78eed80 con 0x55baf7b6f680 2023-11-06T01:45:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.135+0000 7fc0ed17c700 1 --2- 172.21.15.53:0/3262210354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 0x55baf7bd0c70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40079c0 tx=0x7fc0e401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c8aec89f758190 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:45:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.135+0000 7fc0d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:45:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.135+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3262210354 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400e040 con 0x55baf7b6f680 2023-11-06T01:45:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.135+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3262210354 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc0e40307f0 con 0x55baf7b6f680 2023-11-06T01:45:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.135+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3262210354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0e0004830 con 0x55baf7b6f680 2023-11-06T01:45:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.136+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3262210354 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0e40309b0 con 0x55baf7b6f680 2023-11-06T01:45:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.136+0000 7fc0fb570b80 10 client.26350 fetch_fsmap learned FSMap version 46 2023-11-06T01:45:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.136+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3262210354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55baf78ed250 con 0x55baf7b6f680 2023-11-06T01:45:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.136+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3262210354 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0e404a020 con 0x55baf7b6f680 2023-11-06T01:45:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.136+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3262210354 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc0e4046450 con 0x55baf7b6f680 2023-11-06T01:45:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.136+0000 7fc0fb570b80 10 client.26350 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:45:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.136+0000 7fc0fb570b80 10 client.26350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:45:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.136+0000 7fc0fb570b80 20 client.26350 populate_metadata read hostname 'smithi053' 2023-11-06T01:45:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.136+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3262210354 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55baf7bd76f0 0x55baf7bd9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:45:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.137+0000 7fc0fb570b80 4 client.26350 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:45:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.137+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3262210354 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55baf7ab9ce0 con 0x55baf7bd76f0 2023-11-06T01:45:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.137+0000 7fc0ec97b700 1 --2- 172.21.15.53:0/3262210354 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55baf7bd76f0 0x55baf7bd9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:45:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.138+0000 7fc0ec97b700 1 --2- 172.21.15.53:0/3262210354 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55baf7bd76f0 0x55baf7bd9b10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a07d4a9a01671054 server_cookie=d374c2345f246a11 in_seq=0 out_seq=0 2023-11-06T01:45:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.138+0000 7fc0d0ff9700 10 client.26350 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:55.149 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:54 smithi062 ceph-mon[103924]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 MiB/s wr, 1.52k op/s 2023-11-06T01:45:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0d0ff9700 1 -- 172.21.15.53:0/3262210354 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55baf7ab9ce0 con 0x55baf7bd76f0 2023-11-06T01:45:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0d0ff9700 10 client.26350 handle_command_reply: tid=0 2023-11-06T01:45:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0fb570b80 1 client.26350 shutdown 2023-11-06T01:45:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3262210354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 msgr2=0x55baf7bd0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3262210354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 0x55baf7bd0c70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40079c0 tx=0x7fc0e401c000 comp rx=0 tx=0).stop 2023-11-06T01:45:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3262210354 shutdown_connections 2023-11-06T01:45:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3262210354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baf7b70ed0 0x55baf7bd5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3262210354 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55baf7bd76f0 0x55baf7bd9b10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3262210354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55baf7b6f680 0x55baf7bd0c70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0fb570b80 1 --2- 172.21.15.53:0/3262210354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55baf7b6ffe0 0x55baf7bd31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3262210354 >> 172.21.15.53:0/3262210354 conn(0x55baf7acf070 msgr2=0x55baf7b81300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0d0ff9700 0 client.26350 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:45:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3262210354 shutdown_connections 2023-11-06T01:45:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.194+0000 7fc0fb570b80 1 -- 172.21.15.53:0/3262210354 wait complete. 2023-11-06T01:45:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0fb570b80 20 client.26350 trim_cache size 0 max 16384 2023-11-06T01:45:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:45:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:45:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:45:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:45:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1247661028 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0d40a05f0 msgr2=0x7fc0d40a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0d40a05f0 0x7fc0d40a2ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4000f80 tx=0x7fc0e400b740 comp rx=0 tx=0).stop 2023-11-06T01:45:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1247661028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e810a700 msgr2=0x7fc0e81a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:45:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e810a700 0x7fc0e81a6010 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e9a0 tx=0x7fc0dc00ed60 comp rx=0 tx=0).stop 2023-11-06T01:45:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1247661028 shutdown_connections 2023-11-06T01:45:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fc0d40a05f0 0x7fc0d40a2ab0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e810d0d0 0x7fc0e81a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e8107d90 0x7fc0e81a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 --2- 172.21.15.53:0/1247661028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e810a700 0x7fc0e81a6010 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:45:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1247661028 >> 172.21.15.53:0/1247661028 conn(0x7fc0e806dbc0 msgr2=0x7fc0e810bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:45:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1247661028 shutdown_connections 2023-11-06T01:45:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:45:55.199+0000 7fc0ef3e0700 1 -- 172.21.15.53:0/1247661028 wait complete. 2023-11-06T01:45:56.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:56 smithi053 ceph-mon[99472]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 877 op/s 2023-11-06T01:45:56.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:56 smithi062 ceph-mon[103924]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 877 op/s 2023-11-06T01:45:56.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:56 smithi163 ceph-mon[104416]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 877 op/s 2023-11-06T01:45:57.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 700.0 s, 1504.3 tps, lat 66.684 ms stddev 80.997 2023-11-06T01:45:57.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:45:57.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:45:57.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:45:57.397 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:45:57.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:45:57.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:45:57.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:45:57.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:45:57.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:45:58.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:45:58 smithi053 ceph-mon[99472]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 650 op/s 2023-11-06T01:45:58.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:45:58 smithi062 ceph-mon[103924]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 650 op/s 2023-11-06T01:45:58.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:45:58 smithi163 ceph-mon[104416]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 650 op/s 2023-11-06T01:46:00.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:00 smithi053 ceph-mon[99472]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 636 op/s 2023-11-06T01:46:00.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:00 smithi062 ceph-mon[103924]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 636 op/s 2023-11-06T01:46:00.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:00 smithi163 ceph-mon[104416]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 636 op/s 2023-11-06T01:46:02.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 705.0 s, 1519.2 tps, lat 65.152 ms stddev 70.711 2023-11-06T01:46:02.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:02 smithi062 ceph-mon[103924]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 605 op/s 2023-11-06T01:46:02.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:02 smithi163 ceph-mon[104416]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 605 op/s 2023-11-06T01:46:02.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:02 smithi053 ceph-mon[99472]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 605 op/s 2023-11-06T01:46:04.351 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:04 smithi053 ceph-mon[99472]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 729 op/s 2023-11-06T01:46:04.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:04 smithi062 ceph-mon[103924]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 729 op/s 2023-11-06T01:46:04.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:04 smithi163 ceph-mon[104416]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 729 op/s 2023-11-06T01:46:07.184 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 710.0 s, 1620.9 tps, lat 62.170 ms stddev 72.172 2023-11-06T01:46:07.353 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:07 smithi053 ceph-mon[99472]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 MiB/s wr, 692 op/s 2023-11-06T01:46:07.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:07 smithi062 ceph-mon[103924]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 MiB/s wr, 692 op/s 2023-11-06T01:46:07.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:07 smithi163 ceph-mon[104416]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 MiB/s wr, 692 op/s 2023-11-06T01:46:08.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:08 smithi053 ceph-mon[99472]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 719 op/s 2023-11-06T01:46:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.345+0000 7fe266890700 1 -- 172.21.15.53:0/4060636825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe260101830 msgr2=0x7fe2601080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.345+0000 7fe266890700 1 --2- 172.21.15.53:0/4060636825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe260101830 0x7fe2601080e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe25800b0d0 tx=0x7fe25800b3e0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.345+0000 7fe266890700 1 -- 172.21.15.53:0/4060636825 shutdown_connections 2023-11-06T01:46:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.345+0000 7fe266890700 1 --2- 172.21.15.53:0/4060636825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe260101830 0x7fe2601080e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.345+0000 7fe266890700 1 --2- 172.21.15.53:0/4060636825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2600ffca0 0x7fe2601000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.345+0000 7fe266890700 1 --2- 172.21.15.53:0/4060636825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe260100e90 0x7fe2601012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.345+0000 7fe266890700 1 -- 172.21.15.53:0/4060636825 >> 172.21.15.53:0/4060636825 conn(0x7fe2600fb230 msgr2=0x7fe2600fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.345+0000 7fe266890700 1 -- 172.21.15.53:0/4060636825 shutdown_connections 2023-11-06T01:46:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.345+0000 7fe266890700 1 -- 172.21.15.53:0/4060636825 wait complete. 2023-11-06T01:46:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.346+0000 7fe266890700 1 Processor -- start 2023-11-06T01:46:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.346+0000 7fe266890700 1 -- start start 2023-11-06T01:46:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.346+0000 7fe266890700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2600ffca0 0x7fe260199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.346+0000 7fe266890700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe260100e90 0x7fe26019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe266890700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe260101830 0x7fe26019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe266890700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26019e900 con 0x7fe260101830 2023-11-06T01:46:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe266890700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26019ea40 con 0x7fe260100e90 2023-11-06T01:46:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe266890700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26019ec00 con 0x7fe2600ffca0 2023-11-06T01:46:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe25ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2600ffca0 0x7fe260199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe25ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2600ffca0 0x7fe260199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57468/0 (socket says 172.21.15.53:57468) 2023-11-06T01:46:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe25ffff700 1 -- 172.21.15.53:0/963619863 learned_addr learned my addr 172.21.15.53:0/963619863 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe264e2d700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe260101830 0x7fe26019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe264e2d700 1 -- 172.21.15.53:0/963619863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2600ffca0 msgr2=0x7fe260199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe264e2d700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2600ffca0 0x7fe260199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe264e2d700 1 -- 172.21.15.53:0/963619863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe260100e90 msgr2=0x7fe26019bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe264e2d700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe260100e90 0x7fe26019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.347+0000 7fe264e2d700 1 -- 172.21.15.53:0/963619863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe258009d00 con 0x7fe260101830 2023-11-06T01:46:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.348+0000 7fe264e2d700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe260101830 0x7fe26019e330 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe25800b770 tx=0x7fe258008f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.389+0000 7fe25d7fa700 1 -- 172.21.15.53:0/963619863 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25800e040 con 0x7fe260101830 2023-11-06T01:46:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.389+0000 7fe25d7fa700 1 -- 172.21.15.53:0/963619863 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe258020070 con 0x7fe260101830 2023-11-06T01:46:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.389+0000 7fe25d7fa700 1 -- 172.21.15.53:0/963619863 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258027b90 con 0x7fe260101830 2023-11-06T01:46:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.389+0000 7fe266890700 1 -- 172.21.15.53:0/963619863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe260195670 con 0x7fe260101830 2023-11-06T01:46:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.389+0000 7fe266890700 1 -- 172.21.15.53:0/963619863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe260195b30 con 0x7fe260101830 2023-11-06T01:46:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.394+0000 7fe272a20b80 1 Processor -- start 2023-11-06T01:46:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.394+0000 7fe272a20b80 1 -- start start 2023-11-06T01:46:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.394+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c36ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.394+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c37000 0x557c19c37480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.394+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c379c0 0x557c19c37e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.394+0000 7fe272a20b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19b859b0 con 0x557c19c366a0 2023-11-06T01:46:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.394+0000 7fe272a20b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19b83370 con 0x557c19c379c0 2023-11-06T01:46:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.394+0000 7fe272a20b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19b7ec60 con 0x557c19c37000 2023-11-06T01:46:08.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:08 smithi062 ceph-mon[103924]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 719 op/s 2023-11-06T01:46:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.403+0000 7fe25d7fa700 1 -- 172.21.15.53:0/963619863 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe258027cf0 con 0x7fe260101830 2023-11-06T01:46:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.404+0000 7fe25d7fa700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2480a0670 0x7fe2480a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.404+0000 7fe25f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c36ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.404+0000 7fe25f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c36ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43236/0 (socket says 172.21.15.53:43236) 2023-11-06T01:46:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.404+0000 7fe25f7fe700 1 -- 172.21.15.53:0/2231711965 learned_addr learned my addr 172.21.15.53:0/2231711965 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.404+0000 7fe25d7fa700 1 -- 172.21.15.53:0/963619863 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe258022070 con 0x7fe260101830 2023-11-06T01:46:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.405+0000 7fe264e2d700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2480a0670 0x7fe2480a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.405+0000 7fe25f7fe700 1 -- 172.21.15.53:0/2231711965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c37000 msgr2=0x557c19c37480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.405+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/2231711965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c37000 0x557c19c37480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.405+0000 7fe25f7fe700 1 -- 172.21.15.53:0/2231711965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c379c0 msgr2=0x557c19c37e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.405+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/2231711965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c379c0 0x557c19c37e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.405+0000 7fe25f7fe700 1 -- 172.21.15.53:0/2231711965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c199acd80 con 0x557c19c366a0 2023-11-06T01:46:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.405+0000 7fe264e2d700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2480a0670 0x7fe2480a2b30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe25800b0d0 tx=0x7fe2580275f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.405+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/2231711965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c36ac0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe254009c90 tx=0x7fe2540103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9cdbe77de05ea7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.415+0000 7fe245ffb700 1 -- 172.21.15.53:0/2231711965 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254010c60 con 0x557c19c366a0 2023-11-06T01:46:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.415+0000 7fe245ffb700 1 -- 172.21.15.53:0/2231711965 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2540044f0 con 0x557c19c366a0 2023-11-06T01:46:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.418+0000 7fe245ffb700 1 -- 172.21.15.53:0/2231711965 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254011440 con 0x557c19c366a0 2023-11-06T01:46:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.419+0000 7fe272a20b80 1 -- 172.21.15.53:0/2231711965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 msgr2=0x557c19c36ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.419+0000 7fe272a20b80 1 --2- 172.21.15.53:0/2231711965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c36ac0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe254009c90 tx=0x7fe2540103d0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.423+0000 7fe272a20b80 1 -- 172.21.15.53:0/2231711965 shutdown_connections 2023-11-06T01:46:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.423+0000 7fe272a20b80 1 --2- 172.21.15.53:0/2231711965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c379c0 0x557c19c37e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.423+0000 7fe272a20b80 1 --2- 172.21.15.53:0/2231711965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c36ac0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.423+0000 7fe272a20b80 1 --2- 172.21.15.53:0/2231711965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c37000 0x557c19c37480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.423+0000 7fe272a20b80 1 -- 172.21.15.53:0/2231711965 >> 172.21.15.53:0/2231711965 conn(0x557c19b8d070 msgr2=0x557c19c2a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.423+0000 7fe272a20b80 1 -- 172.21.15.53:0/2231711965 shutdown_connections 2023-11-06T01:46:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.423+0000 7fe272a20b80 1 -- 172.21.15.53:0/2231711965 wait complete. 2023-11-06T01:46:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.424+0000 7fe272a20b80 1 -- start start 2023-11-06T01:46:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.424+0000 7fe272a20b80 1 Processor -- start 2023-11-06T01:46:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.425+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c2f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.425+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19c2f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.425+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c379c0 0x557c19c347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.425+0000 7fe272a20b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19b1b4b0 con 0x557c19c366a0 2023-11-06T01:46:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.425+0000 7fe272a20b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19b1b150 con 0x557c19c37000 2023-11-06T01:46:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.425+0000 7fe272a20b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19a028c0 con 0x557c19c379c0 2023-11-06T01:46:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.425+0000 7fe272a20b80 10 client.0 _ll_register_callbacks cb 0x557c19a68a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.425+0000 7fe25f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c2f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.425+0000 7fe25f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c2f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43242/0 (socket says 172.21.15.53:43242) 2023-11-06T01:46:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.425+0000 7fe25f7fe700 1 -- 172.21.15.53:0/3826517804 learned_addr learned my addr 172.21.15.53:0/3826517804 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.426+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c379c0 0x557c19c347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.426+0000 7fe25f7fe700 1 -- 172.21.15.53:0/3826517804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c379c0 msgr2=0x557c19c347c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.426+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c379c0 0x557c19c347c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.426+0000 7fe25f7fe700 1 -- 172.21.15.53:0/3826517804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 msgr2=0x557c19c2f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.426+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19c2f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.426+0000 7fe25f7fe700 1 -- 172.21.15.53:0/3826517804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe254009940 con 0x557c19c366a0 2023-11-06T01:46:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.426+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c2f250 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe254009680 tx=0x7fe254004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40802b3d3b2dff64 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.426+0000 7fe23f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:46:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.426+0000 7fe23f7fe700 1 -- 172.21.15.53:0/3826517804 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25401d070 con 0x557c19c366a0 2023-11-06T01:46:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe23f7fe700 1 -- 172.21.15.53:0/3826517804 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe254003fc0 con 0x557c19c366a0 2023-11-06T01:46:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe272a20b80 1 -- 172.21.15.53:0/3826517804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c19ba9cb0 con 0x557c19c366a0 2023-11-06T01:46:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe23f7fe700 1 -- 172.21.15.53:0/3826517804 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe254004180 con 0x557c19c366a0 2023-11-06T01:46:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe272a20b80 10 client.16779 fetch_fsmap learned FSMap version 46 2023-11-06T01:46:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe272a20b80 1 -- 172.21.15.53:0/3826517804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x557c19995c00 con 0x557c19c366a0 2023-11-06T01:46:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe23f7fe700 1 -- 172.21.15.53:0/3826517804 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c19ba9cb0 con 0x557c19c366a0 2023-11-06T01:46:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe23f7fe700 1 -- 172.21.15.53:0/3826517804 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe254030020 con 0x557c19c366a0 2023-11-06T01:46:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe272a20b80 10 client.16779 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:46:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe272a20b80 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe272a20b80 20 client.16779 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe272a20b80 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557c19c52180 0x557c19c545a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe272a20b80 4 client.16779 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:46:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.427+0000 7fe272a20b80 1 -- 172.21.15.53:0/3826517804 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c19bfcdb0 con 0x557c19c52180 2023-11-06T01:46:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.428+0000 7fe25ffff700 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557c19c52180 0x557c19c545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.429+0000 7fe25ffff700 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557c19c52180 0x557c19c545a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e974e1104b10e02 server_cookie=7758c4df6c6909c5 in_seq=0 out_seq=0 2023-11-06T01:46:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.429+0000 7fe23f7fe700 10 client.16779 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.430+0000 7fe23f7fe700 1 -- 172.21.15.53:0/3826517804 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557c19bfcdb0 con 0x557c19c52180 2023-11-06T01:46:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.430+0000 7fe23f7fe700 10 client.16779 handle_command_reply: tid=0 2023-11-06T01:46:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.430+0000 7fe272a20b80 1 client.16779 shutdown 2023-11-06T01:46:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.432+0000 7fe272a20b80 1 -- 172.21.15.53:0/3826517804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 msgr2=0x557c19c2f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.432+0000 7fe272a20b80 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c2f250 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe254009680 tx=0x7fe254004680 comp rx=0 tx=0).stop 2023-11-06T01:46:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.432+0000 7fe272a20b80 1 -- 172.21.15.53:0/3826517804 shutdown_connections 2023-11-06T01:46:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.432+0000 7fe272a20b80 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557c19c52180 0x557c19c545a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.432+0000 7fe272a20b80 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c379c0 0x557c19c347c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.432+0000 7fe272a20b80 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c2f250 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.432+0000 7fe272a20b80 1 --2- 172.21.15.53:0/3826517804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19c2f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.432+0000 7fe272a20b80 1 -- 172.21.15.53:0/3826517804 >> 172.21.15.53:0/3826517804 conn(0x557c19b8d070 msgr2=0x557c19c2cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.435+0000 7fe23f7fe700 0 client.16779 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.435+0000 7fe272a20b80 1 -- 172.21.15.53:0/3826517804 shutdown_connections 2023-11-06T01:46:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.435+0000 7fe272a20b80 1 -- 172.21.15.53:0/3826517804 wait complete. 2023-11-06T01:46:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.435+0000 7fe272a20b80 20 client.16779 trim_cache size 0 max 16384 2023-11-06T01:46:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.454+0000 7fe272a20b80 1 Processor -- start 2023-11-06T01:46:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.454+0000 7fe272a20b80 1 -- start start 2023-11-06T01:46:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c366a0 0x557c19c2e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19c34750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c379c0 0x557c19c29e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe272a20b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19c2df20 con 0x557c19c379c0 2023-11-06T01:46:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe272a20b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19c20940 con 0x557c19c37000 2023-11-06T01:46:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe272a20b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19c23090 con 0x557c19c366a0 2023-11-06T01:46:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c379c0 0x557c19c29e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19c34750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19c34750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54688/0 (socket says 172.21.15.53:54688) 2023-11-06T01:46:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c379c0 0x557c19c29e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43252/0 (socket says 172.21.15.53:43252) 2023-11-06T01:46:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25ffff700 1 -- 172.21.15.53:0/3752209489 learned_addr learned my addr 172.21.15.53:0/3752209489 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/3752209489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c366a0 0x557c19c2e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25f7fe700 1 -- 172.21.15.53:0/3752209489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c366a0 msgr2=0x557c19c2e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/3752209489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c366a0 0x557c19c2e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25f7fe700 1 -- 172.21.15.53:0/3752209489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 msgr2=0x557c19c34750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/3752209489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19c34750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25f7fe700 1 -- 172.21.15.53:0/3752209489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c199d31f0 con 0x557c19c379c0 2023-11-06T01:46:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25ffff700 1 --2- 172.21.15.53:0/3752209489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19c34750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:46:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.455+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/3752209489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c379c0 0x557c19c29e10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe254001eb0 tx=0x7fe2540103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c198aae11596e5c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.456+0000 7fe2467fc700 1 -- 172.21.15.53:0/3752209489 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25401d040 con 0x557c19c379c0 2023-11-06T01:46:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.456+0000 7fe2467fc700 1 -- 172.21.15.53:0/3752209489 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe254026380 con 0x557c19c379c0 2023-11-06T01:46:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.456+0000 7fe2467fc700 1 -- 172.21.15.53:0/3752209489 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2540267c0 con 0x557c19c379c0 2023-11-06T01:46:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.456+0000 7fe272a20b80 1 -- 172.21.15.53:0/3752209489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c379c0 msgr2=0x557c19c29e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.456+0000 7fe272a20b80 1 --2- 172.21.15.53:0/3752209489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c379c0 0x557c19c29e10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe254001eb0 tx=0x7fe2540103d0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.456+0000 7fe272a20b80 1 -- 172.21.15.53:0/3752209489 shutdown_connections 2023-11-06T01:46:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.456+0000 7fe272a20b80 1 --2- 172.21.15.53:0/3752209489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c379c0 0x557c19c29e10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.457+0000 7fe272a20b80 1 --2- 172.21.15.53:0/3752209489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c366a0 0x557c19c2e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.457+0000 7fe272a20b80 1 --2- 172.21.15.53:0/3752209489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19c34750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.457+0000 7fe272a20b80 1 -- 172.21.15.53:0/3752209489 >> 172.21.15.53:0/3752209489 conn(0x557c19b8d070 msgr2=0x557c19c37da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.457+0000 7fe272a20b80 1 -- 172.21.15.53:0/3752209489 shutdown_connections 2023-11-06T01:46:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.457+0000 7fe272a20b80 1 -- 172.21.15.53:0/3752209489 wait complete. 2023-11-06T01:46:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.457+0000 7fe272a20b80 1 -- start start 2023-11-06T01:46:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.457+0000 7fe272a20b80 1 Processor -- start 2023-11-06T01:46:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c9fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19ca21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe272a20b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c379c0 0x557c19ca4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe272a20b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19c28510 con 0x557c19c366a0 2023-11-06T01:46:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe272a20b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19ca4c80 con 0x557c19c37000 2023-11-06T01:46:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe272a20b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c19ca4e40 con 0x557c19c379c0 2023-11-06T01:46:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe272a20b80 10 client.0 _ll_register_callbacks cb 0x7fe254009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c9fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c9fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43254/0 (socket says 172.21.15.53:43254) 2023-11-06T01:46:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25f7fe700 1 -- 172.21.15.53:0/1218584104 learned_addr learned my addr 172.21.15.53:0/1218584104 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c379c0 0x557c19ca4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25ffff700 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19ca21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25f7fe700 1 -- 172.21.15.53:0/1218584104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c379c0 msgr2=0x557c19ca4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c379c0 0x557c19ca4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25f7fe700 1 -- 172.21.15.53:0/1218584104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 msgr2=0x557c19ca21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19ca21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25f7fe700 1 -- 172.21.15.53:0/1218584104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c19995c00 con 0x557c19c366a0 2023-11-06T01:46:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25ffff700 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19ca21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:46:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe25f7fe700 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c9fc70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe254003a00 tx=0x7fe254005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a600fb487ce173ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe23f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:46:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe23f7fe700 1 -- 172.21.15.53:0/1218584104 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25401d070 con 0x557c19c366a0 2023-11-06T01:46:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe23f7fe700 1 -- 172.21.15.53:0/1218584104 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe254000e40 con 0x557c19c366a0 2023-11-06T01:46:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.458+0000 7fe272a20b80 1 -- 172.21.15.53:0/1218584104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2540044f0 con 0x557c19c366a0 2023-11-06T01:46:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.459+0000 7fe23f7fe700 1 -- 172.21.15.53:0/1218584104 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe254004830 con 0x557c19c366a0 2023-11-06T01:46:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.459+0000 7fe272a20b80 10 client.16785 fetch_fsmap learned FSMap version 46 2023-11-06T01:46:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.459+0000 7fe272a20b80 1 -- 172.21.15.53:0/1218584104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x557c199ab250 con 0x557c19c366a0 2023-11-06T01:46:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.459+0000 7fe23f7fe700 1 -- 172.21.15.53:0/1218584104 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2540044f0 con 0x557c19c366a0 2023-11-06T01:46:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.459+0000 7fe23f7fe700 1 -- 172.21.15.53:0/1218584104 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe254030070 con 0x557c19c366a0 2023-11-06T01:46:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.459+0000 7fe272a20b80 10 client.16785 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:46:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.459+0000 7fe272a20b80 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.460+0000 7fe272a20b80 20 client.16785 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.460+0000 7fe272a20b80 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557c19ca64b0 0x557c19ca88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.460+0000 7fe272a20b80 4 client.16785 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:46:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.460+0000 7fe272a20b80 1 -- 172.21.15.53:0/1218584104 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c19b77ce0 con 0x557c19ca64b0 2023-11-06T01:46:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.460+0000 7fe25ffff700 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557c19ca64b0 0x557c19ca88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.460+0000 7fe23f7fe700 10 client.16785 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.460+0000 7fe25ffff700 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557c19ca64b0 0x557c19ca88d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=364dea59117203ba server_cookie=86a90c6d6e3bf5d3 in_seq=0 out_seq=0 2023-11-06T01:46:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe23f7fe700 1 -- 172.21.15.53:0/1218584104 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c19b77ce0 con 0x557c19ca64b0 2023-11-06T01:46:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe23f7fe700 10 client.16785 handle_command_reply: tid=0 2023-11-06T01:46:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 1 client.16785 shutdown 2023-11-06T01:46:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 1 -- 172.21.15.53:0/1218584104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 msgr2=0x557c19c9fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c9fc70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe254003a00 tx=0x7fe254005fa0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 1 -- 172.21.15.53:0/1218584104 shutdown_connections 2023-11-06T01:46:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557c19ca64b0 0x557c19ca88d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c19c379c0 0x557c19ca4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe23f7fe700 0 client.16785 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c19c366a0 0x557c19c9fc70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 1 --2- 172.21.15.53:0/1218584104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c19c37000 0x557c19ca21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 1 -- 172.21.15.53:0/1218584104 >> 172.21.15.53:0/1218584104 conn(0x557c19b8d070 msgr2=0x557c19c29e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 1 -- 172.21.15.53:0/1218584104 shutdown_connections 2023-11-06T01:46:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 1 -- 172.21.15.53:0/1218584104 wait complete. 2023-11-06T01:46:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.470+0000 7fe272a20b80 20 client.16785 trim_cache size 0 max 16384 2023-11-06T01:46:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:46:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:46:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:46:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:46:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 -- 172.21.15.53:0/963619863 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2480a0670 msgr2=0x7fe2480a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2480a0670 0x7fe2480a2b30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe25800b0d0 tx=0x7fe2580275f0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 -- 172.21.15.53:0/963619863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe260101830 msgr2=0x7fe26019e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe260101830 0x7fe26019e330 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe25800b770 tx=0x7fe258008f70 comp rx=0 tx=0).stop 2023-11-06T01:46:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 -- 172.21.15.53:0/963619863 shutdown_connections 2023-11-06T01:46:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe2480a0670 0x7fe2480a2b30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe260101830 0x7fe26019e330 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2600ffca0 0x7fe260199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 --2- 172.21.15.53:0/963619863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe260100e90 0x7fe26019bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 -- 172.21.15.53:0/963619863 >> 172.21.15.53:0/963619863 conn(0x7fe2600fb230 msgr2=0x7fe260104350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 -- 172.21.15.53:0/963619863 shutdown_connections 2023-11-06T01:46:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:08.471+0000 7fe246ffd700 1 -- 172.21.15.53:0/963619863 wait complete. 2023-11-06T01:46:08.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:08 smithi163 ceph-mon[104416]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 719 op/s 2023-11-06T01:46:10.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:10 smithi053 ceph-mon[99472]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 746 op/s 2023-11-06T01:46:10.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:10 smithi062 ceph-mon[103924]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 746 op/s 2023-11-06T01:46:10.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:10 smithi163 ceph-mon[104416]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 746 op/s 2023-11-06T01:46:10.536 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:46:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.740+0000 7ff4c0b93700 1 -- 172.21.15.53:0/3590094515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4bc0784d0 msgr2=0x7ff4bc07a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.740+0000 7ff4c0b93700 1 --2- 172.21.15.53:0/3590094515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4bc0784d0 0x7ff4bc07a960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400b0a0 tx=0x7ff4b400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.743+0000 7ff4c0b93700 1 -- 172.21.15.53:0/3590094515 shutdown_connections 2023-11-06T01:46:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.744+0000 7ff4c0b93700 1 --2- 172.21.15.53:0/3590094515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4bc0784d0 0x7ff4bc07a960 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.744+0000 7ff4c0b93700 1 --2- 172.21.15.53:0/3590094515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4bc072c20 0x7ff4bc073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.744+0000 7ff4c0b93700 1 --2- 172.21.15.53:0/3590094515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4bc075b00 0x7ff4bc077f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.744+0000 7ff4c0b93700 1 -- 172.21.15.53:0/3590094515 >> 172.21.15.53:0/3590094515 conn(0x7ff4bc06db40 msgr2=0x7ff4bc06ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.744+0000 7ff4c0b93700 1 -- 172.21.15.53:0/3590094515 shutdown_connections 2023-11-06T01:46:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.744+0000 7ff4c0b93700 1 -- 172.21.15.53:0/3590094515 wait complete. 2023-11-06T01:46:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.745+0000 7ff4c0b93700 1 Processor -- start 2023-11-06T01:46:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.745+0000 7ff4c0b93700 1 -- start start 2023-11-06T01:46:10.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.745+0000 7ff4c0b93700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4bc072c20 0x7ff4bc1b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.745+0000 7ff4c0b93700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4bc075b00 0x7ff4bc1b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.745+0000 7ff4c0b93700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4bc0784d0 0x7ff4bc1b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.745+0000 7ff4c0b93700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc1b5aa0 con 0x7ff4bc0784d0 2023-11-06T01:46:10.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.745+0000 7ff4c0b93700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc1b5be0 con 0x7ff4bc072c20 2023-11-06T01:46:10.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.746+0000 7ff4c0b93700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc1b5d70 con 0x7ff4bc075b00 2023-11-06T01:46:10.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.746+0000 7ff4ba59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4bc072c20 0x7ff4bc1b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.746+0000 7ff4ba59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4bc072c20 0x7ff4bc1b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54708/0 (socket says 172.21.15.53:54708) 2023-11-06T01:46:10.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.746+0000 7ff4ba59c700 1 -- 172.21.15.53:0/2589835017 learned_addr learned my addr 172.21.15.53:0/2589835017 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:10.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.746+0000 7ff4ba59c700 1 -- 172.21.15.53:0/2589835017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4bc075b00 msgr2=0x7ff4bc1b3010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:46:10.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.746+0000 7ff4ba59c700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4bc075b00 0x7ff4bc1b3010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.746+0000 7ff4ba59c700 1 -- 172.21.15.53:0/2589835017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4bc0784d0 msgr2=0x7ff4bc1b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:10.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.746+0000 7ff4ba59c700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4bc0784d0 0x7ff4bc1b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.746+0000 7ff4ba59c700 1 -- 172.21.15.53:0/2589835017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b4009d00 con 0x7ff4bc072c20 2023-11-06T01:46:10.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.746+0000 7ff4ba59c700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4bc072c20 0x7ff4bc1b0ac0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00bf20 tx=0x7ff4ac009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:10.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.749+0000 7ff4ab7fe700 1 -- 172.21.15.53:0/2589835017 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac011e80 con 0x7ff4bc072c20 2023-11-06T01:46:10.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.749+0000 7ff4ab7fe700 1 -- 172.21.15.53:0/2589835017 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ac011530 con 0x7ff4bc072c20 2023-11-06T01:46:10.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.749+0000 7ff4c0b93700 1 -- 172.21.15.53:0/2589835017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4bc1b5ff0 con 0x7ff4bc072c20 2023-11-06T01:46:10.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.749+0000 7ff4ab7fe700 1 -- 172.21.15.53:0/2589835017 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00f860 con 0x7ff4bc072c20 2023-11-06T01:46:10.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.749+0000 7ff4c0b93700 1 -- 172.21.15.53:0/2589835017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4bc1b64a0 con 0x7ff4bc072c20 2023-11-06T01:46:10.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.753+0000 7ff4ab7fe700 1 -- 172.21.15.53:0/2589835017 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff4ac023400 con 0x7ff4bc072c20 2023-11-06T01:46:10.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ccd23b80 1 Processor -- start 2023-11-06T01:46:10.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ccd23b80 1 -- start start 2023-11-06T01:46:10.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf10f0 0x55b3adcf1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 0x55b3adcefe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf03f0 0x55b3adcf0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adc47a10 con 0x55b3adcf1a70 2023-11-06T01:46:10.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adc453d0 con 0x55b3adcf03f0 2023-11-06T01:46:10.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adc40cc0 con 0x55b3adcf10f0 2023-11-06T01:46:10.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ab7fe700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff4a40a0650 0x7ff4a40a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ba59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 0x55b3adcefe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4bad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf10f0 0x55b3adcf1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ba59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 0x55b3adcefe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43274/0 (socket says 172.21.15.53:43274) 2023-11-06T01:46:10.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4bad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf10f0 0x55b3adcf1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57512/0 (socket says 172.21.15.53:57512) 2023-11-06T01:46:10.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.754+0000 7ff4ba59c700 1 -- 172.21.15.53:0/608014441 learned_addr learned my addr 172.21.15.53:0/608014441 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:10.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4b9d9b700 1 --2- 172.21.15.53:0/608014441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf03f0 0x55b3adcf0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4ab7fe700 1 -- 172.21.15.53:0/2589835017 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff4ac0d3dc0 con 0x7ff4bc072c20 2023-11-06T01:46:10.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4bad9d700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff4a40a0650 0x7ff4a40a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4bad9d700 1 -- 172.21.15.53:0/608014441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf03f0 msgr2=0x55b3adcf0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:10.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4bad9d700 1 --2- 172.21.15.53:0/608014441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf03f0 0x55b3adcf0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4bad9d700 1 -- 172.21.15.53:0/608014441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 msgr2=0x55b3adcefe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:10.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4bad9d700 1 --2- 172.21.15.53:0/608014441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 0x55b3adcefe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4bad9d700 1 -- 172.21.15.53:0/608014441 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3ada6ede0 con 0x55b3adcf10f0 2023-11-06T01:46:10.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4bad9d700 1 --2- 172.21.15.53:0/608014441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf10f0 0x55b3adcf1530 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400b070 tx=0x7ff4b40091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4357eaec74a2398a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:10.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4bad9d700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff4a40a0650 0x7ff4a40a2b10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009fd0 tx=0x7ff4b4025000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:10.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4a8ff9700 1 -- 172.21.15.53:0/608014441 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400e050 con 0x55b3adcf10f0 2023-11-06T01:46:10.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.755+0000 7ff4a8ff9700 1 -- 172.21.15.53:0/608014441 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4b40079c0 con 0x55b3adcf10f0 2023-11-06T01:46:10.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.756+0000 7ff4a8ff9700 1 -- 172.21.15.53:0/608014441 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4005860 con 0x55b3adcf10f0 2023-11-06T01:46:10.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.756+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/608014441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf10f0 msgr2=0x55b3adcf1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:10.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.756+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/608014441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf10f0 0x55b3adcf1530 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400b070 tx=0x7ff4b40091d0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.756+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/608014441 shutdown_connections 2023-11-06T01:46:10.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.756+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/608014441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf03f0 0x55b3adcf0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.756+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/608014441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf10f0 0x55b3adcf1530 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.756+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/608014441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 0x55b3adcefe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.756+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/608014441 >> 172.21.15.53:0/608014441 conn(0x55b3adc4f0d0 msgr2=0x55b3adcea940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:10.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.756+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/608014441 shutdown_connections 2023-11-06T01:46:10.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.756+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/608014441 wait complete. 2023-11-06T01:46:10.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.757+0000 7ff4ccd23b80 1 -- start start 2023-11-06T01:46:10.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.757+0000 7ff4ccd23b80 1 Processor -- start 2023-11-06T01:46:10.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.757+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf03f0 0x55b3adcf9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.757+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf10f0 0x55b3adcf9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.757+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 0x55b3add00e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.757+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adbdd510 con 0x55b3adcf1a70 2023-11-06T01:46:10.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.757+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adbdd1b0 con 0x55b3adcf10f0 2023-11-06T01:46:10.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.757+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adac4920 con 0x55b3adcf03f0 2023-11-06T01:46:10.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.757+0000 7ff4ccd23b80 10 client.0 _ll_register_callbacks cb 0x55b3adb2aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:10.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.758+0000 7ff4bad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 0x55b3add00e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.758+0000 7ff4bad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 0x55b3add00e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43282/0 (socket says 172.21.15.53:43282) 2023-11-06T01:46:10.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.758+0000 7ff4bad9d700 1 -- 172.21.15.53:0/3893412365 learned_addr learned my addr 172.21.15.53:0/3893412365 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:10.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.758+0000 7ff4ba59c700 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf03f0 0x55b3adcf9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.758+0000 7ff4ba59c700 1 -- 172.21.15.53:0/3893412365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf10f0 msgr2=0x55b3adcf9e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:46:10.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.758+0000 7ff4ba59c700 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf10f0 0x55b3adcf9e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.758+0000 7ff4ba59c700 1 -- 172.21.15.53:0/3893412365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 msgr2=0x55b3add00e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:10.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.758+0000 7ff4ba59c700 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 0x55b3add00e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.758+0000 7ff4ba59c700 1 -- 172.21.15.53:0/3893412365 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4bc07b390 con 0x55b3adcf03f0 2023-11-06T01:46:10.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.759+0000 7ff4ba59c700 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf03f0 0x55b3adcf9910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac09c140 tx=0x7ff4ac0a1230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d566add13aa4a37 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:10.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.759+0000 7ff4a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:46:10.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.759+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/3893412365 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0a1600 con 0x55b3adcf03f0 2023-11-06T01:46:10.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.759+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/3893412365 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff4ac0a5430 con 0x55b3adcf03f0 2023-11-06T01:46:10.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.759+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/3893412365 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3adc6bd10 con 0x55b3adcf03f0 2023-11-06T01:46:10.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.760+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/3893412365 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4ac0a9670 con 0x55b3adcf03f0 2023-11-06T01:46:10.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.760+0000 7ff4ccd23b80 10 client.26371 fetch_fsmap learned FSMap version 46 2023-11-06T01:46:10.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.760+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/3893412365 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55b3ada57c60 con 0x55b3adcf03f0 2023-11-06T01:46:10.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.760+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/3893412365 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3adc6bd10 con 0x55b3adcf03f0 2023-11-06T01:46:10.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.761+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/3893412365 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4ac0a0b80 con 0x55b3adcf03f0 2023-11-06T01:46:10.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.761+0000 7ff4ccd23b80 10 client.26371 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:46:10.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.761+0000 7ff4ccd23b80 10 client.26371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:10.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.761+0000 7ff4ccd23b80 20 client.26371 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:10.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.761+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3add166c0 0x55b3add18ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.761+0000 7ff4ccd23b80 4 client.26371 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:46:10.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.761+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/3893412365 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3adcbee10 con 0x55b3add166c0 2023-11-06T01:46:10.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.761+0000 7ff4b9d9b700 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3add166c0 0x55b3add18ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.761+0000 7ff4a1ffb700 10 client.26371 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:10.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.761+0000 7ff4b9d9b700 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3add166c0 0x55b3add18ae0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768a5fa412f2533e server_cookie=a42e7ca0bf38dd7e in_seq=0 out_seq=0 2023-11-06T01:46:10.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.764+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/3893412365 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b3adcbee10 con 0x55b3add166c0 2023-11-06T01:46:10.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.764+0000 7ff4a1ffb700 10 client.26371 handle_command_reply: tid=0 2023-11-06T01:46:10.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.764+0000 7ff4ccd23b80 1 client.26371 shutdown 2023-11-06T01:46:10.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.788+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/3893412365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf03f0 msgr2=0x55b3adcf9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:10.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.788+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf03f0 0x55b3adcf9910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac09c140 tx=0x7ff4ac0a1230 comp rx=0 tx=0).stop 2023-11-06T01:46:10.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.788+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/3893412365 shutdown_connections 2023-11-06T01:46:10.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.788+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3add166c0 0x55b3add18ae0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.788+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf1a70 0x55b3add00e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.788+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf03f0 0x55b3adcf9910 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.788+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/3893412365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf10f0 0x55b3adcf9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.788+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/3893412365 >> 172.21.15.53:0/3893412365 conn(0x55b3adc4f0d0 msgr2=0x55b3adcea0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:10.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.788+0000 7ff4a1ffb700 0 client.26371 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:10.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.788+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/3893412365 shutdown_connections 2023-11-06T01:46:10.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.788+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/3893412365 wait complete. 2023-11-06T01:46:10.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.791+0000 7ff4ccd23b80 20 client.26371 trim_cache size 0 max 16384 2023-11-06T01:46:10.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.837+0000 7ff4ccd23b80 1 Processor -- start 2023-11-06T01:46:10.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.837+0000 7ff4ccd23b80 1 -- start start 2023-11-06T01:46:10.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.838+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf03f0 0x55b3adcefe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.838+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.838+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf1a70 0x55b3adcf4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.838+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adcf8dc0 con 0x55b3adcf10f0 2023-11-06T01:46:10.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.838+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adce53d0 con 0x55b3adcf1a70 2023-11-06T01:46:10.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.838+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adce2be0 con 0x55b3adcf03f0 2023-11-06T01:46:10.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.843+0000 7ff4b9d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.843+0000 7ff4b9d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43290/0 (socket says 172.21.15.53:43290) 2023-11-06T01:46:10.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.843+0000 7ff4b9d9b700 1 -- 172.21.15.53:0/2538954383 learned_addr learned my addr 172.21.15.53:0/2538954383 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:10.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.843+0000 7ff4b9d9b700 1 -- 172.21.15.53:0/2538954383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf03f0 msgr2=0x55b3adcefe60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:46:10.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.843+0000 7ff4b9d9b700 1 --2- 172.21.15.53:0/2538954383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf03f0 0x55b3adcefe60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.843+0000 7ff4b9d9b700 1 -- 172.21.15.53:0/2538954383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf1a70 msgr2=0x55b3adcf4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:10.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.843+0000 7ff4b9d9b700 1 --2- 172.21.15.53:0/2538954383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf1a70 0x55b3adcf4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.843+0000 7ff4b9d9b700 1 -- 172.21.15.53:0/2538954383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3ada95250 con 0x55b3adcf10f0 2023-11-06T01:46:10.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.844+0000 7ff4b9d9b700 1 --2- 172.21.15.53:0/2538954383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00e50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000ad10 tx=0x7ff4b000a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6a2114239ddef6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:10.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.849+0000 7ff4a97fa700 1 -- 172.21.15.53:0/2538954383 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0019440 con 0x55b3adcf10f0 2023-11-06T01:46:10.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.849+0000 7ff4a97fa700 1 -- 172.21.15.53:0/2538954383 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4b001d070 con 0x55b3adcf10f0 2023-11-06T01:46:10.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.849+0000 7ff4a97fa700 1 -- 172.21.15.53:0/2538954383 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0010430 con 0x55b3adcf10f0 2023-11-06T01:46:10.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.852+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/2538954383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 msgr2=0x55b3add00e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:10.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.852+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/2538954383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00e50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000ad10 tx=0x7ff4b000a300 comp rx=0 tx=0).stop 2023-11-06T01:46:10.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.853+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/2538954383 shutdown_connections 2023-11-06T01:46:10.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.853+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/2538954383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf1a70 0x55b3adcf4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.853+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/2538954383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf03f0 0x55b3adcefe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.853+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/2538954383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00e50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.853+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/2538954383 >> 172.21.15.53:0/2538954383 conn(0x55b3adc4f0d0 msgr2=0x55b3adcf7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:10.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.853+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/2538954383 shutdown_connections 2023-11-06T01:46:10.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.853+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/2538954383 wait complete. 2023-11-06T01:46:10.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.853+0000 7ff4ccd23b80 1 -- start start 2023-11-06T01:46:10.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.853+0000 7ff4ccd23b80 1 Processor -- start 2023-11-06T01:46:10.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.854+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf03f0 0x55b3add00630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.854+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.854+0000 7ff4ccd23b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf1a70 0x55b3adcfd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.854+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adcfd610 con 0x55b3adcf10f0 2023-11-06T01:46:10.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.854+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adcfd780 con 0x55b3adcf03f0 2023-11-06T01:46:10.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.854+0000 7ff4ccd23b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3adcfd940 con 0x55b3adcf1a70 2023-11-06T01:46:10.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.854+0000 7ff4ccd23b80 10 client.0 _ll_register_callbacks cb 0x7ff4ac0a13d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:10.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.854+0000 7ff4b9d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.854+0000 7ff4bad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf1a70 0x55b3adcfd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.854+0000 7ff4b9d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43296/0 (socket says 172.21.15.53:43296) 2023-11-06T01:46:10.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.854+0000 7ff4b9d9b700 1 -- 172.21.15.53:0/153705854 learned_addr learned my addr 172.21.15.53:0/153705854 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:10.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.855+0000 7ff4b9d9b700 1 -- 172.21.15.53:0/153705854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf1a70 msgr2=0x55b3adcfd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:10.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.855+0000 7ff4b9d9b700 1 --2- 172.21.15.53:0/153705854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf1a70 0x55b3adcfd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.855+0000 7ff4b9d9b700 1 -- 172.21.15.53:0/153705854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf03f0 msgr2=0x55b3add00630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:10.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.855+0000 7ff4b9d9b700 1 --2- 172.21.15.53:0/153705854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf03f0 0x55b3add00630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.855+0000 7ff4b9d9b700 1 -- 172.21.15.53:0/153705854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b000a8a0 con 0x55b3adcf10f0 2023-11-06T01:46:10.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.856+0000 7ff4b9d9b700 1 --2- 172.21.15.53:0/153705854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00b70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0019530 tx=0x7ff4b00052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49952f849c71dbe0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:10.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.856+0000 7ff4a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:46:10.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.856+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/153705854 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00049c0 con 0x55b3adcf10f0 2023-11-06T01:46:10.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.856+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/153705854 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff4b0015040 con 0x55b3adcf10f0 2023-11-06T01:46:10.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.856+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/153705854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4b40079c0 con 0x55b3adcf10f0 2023-11-06T01:46:10.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.857+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/153705854 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4b0004b20 con 0x55b3adcf10f0 2023-11-06T01:46:10.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.857+0000 7ff4ccd23b80 10 client.16800 fetch_fsmap learned FSMap version 46 2023-11-06T01:46:10.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.857+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/153705854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55b3ada6d2b0 con 0x55b3adcf10f0 2023-11-06T01:46:10.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.857+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/153705854 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4b40079c0 con 0x55b3adcf10f0 2023-11-06T01:46:10.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.857+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/153705854 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4b0010430 con 0x55b3adcf10f0 2023-11-06T01:46:10.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.857+0000 7ff4ccd23b80 10 client.16800 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T01:46:10.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.857+0000 7ff4ccd23b80 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:10.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.857+0000 7ff4ccd23b80 20 client.16800 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:10.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.857+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/153705854 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3add69610 0x55b3add6ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:10.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.857+0000 7ff4ccd23b80 4 client.16800 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:46:10.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.857+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/153705854 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b3adc39d40 con 0x55b3add69610 2023-11-06T01:46:10.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.860+0000 7ff4ba59c700 1 --2- 172.21.15.53:0/153705854 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3add69610 0x55b3add6ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:10.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.861+0000 7ff4ba59c700 1 --2- 172.21.15.53:0/153705854 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3add69610 0x55b3add6ba30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c77db740ff372a0 server_cookie=eccc231068f7594d in_seq=0 out_seq=0 2023-11-06T01:46:10.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.861+0000 7ff4a1ffb700 10 client.16800 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:10.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.861+0000 7ff4a1ffb700 1 -- 172.21.15.53:0/153705854 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b3adc39d40 con 0x55b3add69610 2023-11-06T01:46:10.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.861+0000 7ff4a1ffb700 10 client.16800 handle_command_reply: tid=0 2023-11-06T01:46:10.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.861+0000 7ff4ccd23b80 1 client.16800 shutdown 2023-11-06T01:46:10.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/153705854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 msgr2=0x55b3add00b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:10.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/153705854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00b70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0019530 tx=0x7ff4b00052f0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/153705854 shutdown_connections 2023-11-06T01:46:10.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/153705854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b3adcf03f0 0x55b3add00630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/153705854 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55b3add69610 0x55b3add6ba30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/153705854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b3adcf1a70 0x55b3adcfd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4a1ffb700 0 client.16800 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:10.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4ccd23b80 1 --2- 172.21.15.53:0/153705854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3adcf10f0 0x55b3add00b70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/153705854 >> 172.21.15.53:0/153705854 conn(0x55b3adc4f0d0 msgr2=0x55b3adcf7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:10.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/153705854 shutdown_connections 2023-11-06T01:46:10.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4ccd23b80 1 -- 172.21.15.53:0/153705854 wait complete. 2023-11-06T01:46:10.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.862+0000 7ff4ccd23b80 20 client.16800 trim_cache size 0 max 16384 2023-11-06T01:46:10.868 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:46:10.868 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:46:10.868 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:46:10.869 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:46:10.869 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:46:10.869 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:46:10.869 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:46:10.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:46:10.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:46:10.870 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:46:10.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.863+0000 7ff4c0b93700 1 -- 172.21.15.53:0/2589835017 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff4a40a0650 msgr2=0x7ff4a40a2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:10.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.863+0000 7ff4c0b93700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff4a40a0650 0x7ff4a40a2b10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009fd0 tx=0x7ff4b4025000 comp rx=0 tx=0).stop 2023-11-06T01:46:10.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.863+0000 7ff4c0b93700 1 -- 172.21.15.53:0/2589835017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4bc072c20 msgr2=0x7ff4bc1b0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:10.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.863+0000 7ff4c0b93700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4bc072c20 0x7ff4bc1b0ac0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00bf20 tx=0x7ff4ac009520 comp rx=0 tx=0).stop 2023-11-06T01:46:10.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.863+0000 7ff4c0b93700 1 -- 172.21.15.53:0/2589835017 shutdown_connections 2023-11-06T01:46:10.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.863+0000 7ff4c0b93700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff4a40a0650 0x7ff4a40a2b10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.863+0000 7ff4c0b93700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4bc072c20 0x7ff4bc1b0ac0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.863+0000 7ff4c0b93700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4bc0784d0 0x7ff4bc1b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.863+0000 7ff4c0b93700 1 --2- 172.21.15.53:0/2589835017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4bc075b00 0x7ff4bc1b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:10.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.863+0000 7ff4c0b93700 1 -- 172.21.15.53:0/2589835017 >> 172.21.15.53:0/2589835017 conn(0x7ff4bc06db40 msgr2=0x7ff4bc076810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:10.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.866+0000 7ff4c0b93700 1 -- 172.21.15.53:0/2589835017 shutdown_connections 2023-11-06T01:46:10.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:10.866+0000 7ff4c0b93700 1 -- 172.21.15.53:0/2589835017 wait complete. 2023-11-06T01:46:10.887 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:46:12.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 715.0 s, 1556.8 tps, lat 64.438 ms stddev 73.859 2023-11-06T01:46:12.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:12 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:46:12.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:12 smithi163 ceph-mon[104416]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 11 MiB/s wr, 725 op/s 2023-11-06T01:46:12.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:12 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:46:12.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:12 smithi053 ceph-mon[99472]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 11 MiB/s wr, 725 op/s 2023-11-06T01:46:12.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:12 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:46:12.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:12 smithi062 ceph-mon[103924]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 11 MiB/s wr, 725 op/s 2023-11-06T01:46:14.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:14 smithi053 ceph-mon[99472]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 806 op/s 2023-11-06T01:46:14.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:14 smithi062 ceph-mon[103924]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 806 op/s 2023-11-06T01:46:14.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:14 smithi163 ceph-mon[104416]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 806 op/s 2023-11-06T01:46:17.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 720.0 s, 1522.4 tps, lat 65.855 ms stddev 74.404 2023-11-06T01:46:17.237 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:16 smithi163 ceph-mon[104416]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 685 op/s 2023-11-06T01:46:17.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:16 smithi053 ceph-mon[99472]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 685 op/s 2023-11-06T01:46:17.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:16 smithi062 ceph-mon[103924]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 685 op/s 2023-11-06T01:46:19.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:18 smithi053 ceph-mon[99472]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 720 op/s 2023-11-06T01:46:19.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:18 smithi062 ceph-mon[103924]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 720 op/s 2023-11-06T01:46:19.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:18 smithi163 ceph-mon[104416]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 720 op/s 2023-11-06T01:46:21.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:20 smithi053 ceph-mon[99472]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 698 op/s 2023-11-06T01:46:21.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:20 smithi062 ceph-mon[103924]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 698 op/s 2023-11-06T01:46:21.495 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:20 smithi163 ceph-mon[104416]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 698 op/s 2023-11-06T01:46:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.638+0000 7f43f64c9700 1 -- 172.21.15.53:0/4021160741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43f0101830 msgr2=0x7f43f01080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.638+0000 7f43f64c9700 1 --2- 172.21.15.53:0/4021160741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43f0101830 0x7f43f01080e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f43e800b0d0 tx=0x7f43e800b3e0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.639+0000 7f43f64c9700 1 -- 172.21.15.53:0/4021160741 shutdown_connections 2023-11-06T01:46:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.639+0000 7f43f64c9700 1 --2- 172.21.15.53:0/4021160741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43f0101830 0x7f43f01080e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.639+0000 7f43f64c9700 1 --2- 172.21.15.53:0/4021160741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f00ffca0 0x7f43f01000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.639+0000 7f43f64c9700 1 --2- 172.21.15.53:0/4021160741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f0100e90 0x7f43f01012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.639+0000 7f43f64c9700 1 -- 172.21.15.53:0/4021160741 >> 172.21.15.53:0/4021160741 conn(0x7f43f00fb230 msgr2=0x7f43f00fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.639+0000 7f43f64c9700 1 -- 172.21.15.53:0/4021160741 shutdown_connections 2023-11-06T01:46:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.640+0000 7f43f64c9700 1 -- 172.21.15.53:0/4021160741 wait complete. 2023-11-06T01:46:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.640+0000 7f43f64c9700 1 Processor -- start 2023-11-06T01:46:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.640+0000 7f43f64c9700 1 -- start start 2023-11-06T01:46:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43f64c9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43f00ffca0 0x7f43f0199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43f64c9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f0100e90 0x7f43f019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43f64c9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f0101830 0x7f43f019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43f64c9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f019e900 con 0x7f43f00ffca0 2023-11-06T01:46:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43f64c9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f019ea70 con 0x7f43f0101830 2023-11-06T01:46:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43f64c9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f019ec30 con 0x7f43f0100e90 2023-11-06T01:46:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43effff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43f00ffca0 0x7f43f0199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43effff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43f00ffca0 0x7f43f0199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43308/0 (socket says 172.21.15.53:43308) 2023-11-06T01:46:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43effff700 1 -- 172.21.15.53:0/631350163 learned_addr learned my addr 172.21.15.53:0/631350163 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43f4a66700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f0101830 0x7f43f019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43ef7fe700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f0100e90 0x7f43f019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43effff700 1 -- 172.21.15.53:0/631350163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f0100e90 msgr2=0x7f43f019bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43effff700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f0100e90 0x7f43f019bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43effff700 1 -- 172.21.15.53:0/631350163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f0101830 msgr2=0x7f43f019e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43effff700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f0101830 0x7f43f019e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.641+0000 7f43effff700 1 -- 172.21.15.53:0/631350163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e8009d00 con 0x7f43f00ffca0 2023-11-06T01:46:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.642+0000 7f43effff700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43f00ffca0 0x7f43f0199860 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f43e000e970 tx=0x7f43e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.642+0000 7f43ed7fa700 1 -- 172.21.15.53:0/631350163 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0009790 con 0x7f43f00ffca0 2023-11-06T01:46:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.642+0000 7f43ed7fa700 1 -- 172.21.15.53:0/631350163 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e000c7e0 con 0x7f43f00ffca0 2023-11-06T01:46:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.642+0000 7f43f64c9700 1 -- 172.21.15.53:0/631350163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f019ef10 con 0x7f43f00ffca0 2023-11-06T01:46:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.642+0000 7f43ed7fa700 1 -- 172.21.15.53:0/631350163 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0003c70 con 0x7f43f00ffca0 2023-11-06T01:46:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.642+0000 7f43f64c9700 1 -- 172.21.15.53:0/631350163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f019f330 con 0x7f43f00ffca0 2023-11-06T01:46:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f4402659b80 1 Processor -- start 2023-11-06T01:46:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f4402659b80 1 -- start start 2023-11-06T01:46:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f4402659b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x55721950da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f4402659b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x55721950e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f4402659b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721950f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f4402659b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572194659b0 con 0x55721950df90 2023-11-06T01:46:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f4402659b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557219463370 con 0x55721950d630 2023-11-06T01:46:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f4402659b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721945ec60 con 0x55721950ee80 2023-11-06T01:46:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f43f4a66700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x55721950da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f43ef7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721950f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f43f4a66700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x55721950da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54752/0 (socket says 172.21.15.53:54752) 2023-11-06T01:46:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f43f4a66700 1 -- 172.21.15.53:0/3851250087 learned_addr learned my addr 172.21.15.53:0/3851250087 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f43ef7fe700 1 -- 172.21.15.53:0/3851250087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 msgr2=0x55721950da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f43ef7fe700 1 --2- 172.21.15.53:0/3851250087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x55721950da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f43ef7fe700 1 -- 172.21.15.53:0/3851250087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 msgr2=0x55721950e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f43ef7fe700 1 --2- 172.21.15.53:0/3851250087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x55721950e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.646+0000 7f43ef7fe700 1 -- 172.21.15.53:0/3851250087 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55721928cd80 con 0x55721950ee80 2023-11-06T01:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.647+0000 7f43f4a66700 1 --2- 172.21.15.53:0/3851250087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x55721950da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.647+0000 7f43ef7fe700 1 --2- 172.21.15.53:0/3851250087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721950f300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009c80 tx=0x7f43e4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb998a66ac3838ee server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.647+0000 7f43d67fc700 1 -- 172.21.15.53:0/3851250087 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4011cb0 con 0x55721950ee80 2023-11-06T01:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.647+0000 7f43d67fc700 1 -- 172.21.15.53:0/3851250087 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e4011e10 con 0x55721950ee80 2023-11-06T01:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.647+0000 7f43d67fc700 1 -- 172.21.15.53:0/3851250087 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40193e0 con 0x55721950ee80 2023-11-06T01:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f43ed7fa700 1 -- 172.21.15.53:0/631350163 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f43e000cc50 con 0x7f43f00ffca0 2023-11-06T01:46:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f4402659b80 1 -- 172.21.15.53:0/3851250087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 msgr2=0x55721950f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f4402659b80 1 --2- 172.21.15.53:0/3851250087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721950f300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009c80 tx=0x7f43e4009f90 comp rx=0 tx=0).stop 2023-11-06T01:46:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f4402659b80 1 -- 172.21.15.53:0/3851250087 shutdown_connections 2023-11-06T01:46:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f4402659b80 1 --2- 172.21.15.53:0/3851250087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x55721950da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f4402659b80 1 --2- 172.21.15.53:0/3851250087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721950f300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f4402659b80 1 --2- 172.21.15.53:0/3851250087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x55721950e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f4402659b80 1 -- 172.21.15.53:0/3851250087 >> 172.21.15.53:0/3851250087 conn(0x55721946d070 msgr2=0x55721950a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f4402659b80 1 -- 172.21.15.53:0/3851250087 shutdown_connections 2023-11-06T01:46:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f4402659b80 1 -- 172.21.15.53:0/3851250087 wait complete. 2023-11-06T01:46:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f4402659b80 1 -- start start 2023-11-06T01:46:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.648+0000 7f4402659b80 1 Processor -- start 2023-11-06T01:46:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43ed7fa700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43d80a0430 0x7f43d80a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f4402659b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950d630 0x55721951fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43f4a66700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43d80a0430 0x7f43d80a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f4402659b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950df90 0x557219520190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f4402659b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x5572195271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f4402659b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572193fb4b0 con 0x55721950d630 2023-11-06T01:46:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f4402659b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572193fb150 con 0x55721950df90 2023-11-06T01:46:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f4402659b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572192e28c0 con 0x55721950ee80 2023-11-06T01:46:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43ed7fa700 1 -- 172.21.15.53:0/631350163 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f43e0014070 con 0x7f43f00ffca0 2023-11-06T01:46:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f4402659b80 10 client.0 _ll_register_callbacks cb 0x557219348a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43effff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950d630 0x55721951fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43effff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950d630 0x55721951fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43322/0 (socket says 172.21.15.53:43322) 2023-11-06T01:46:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43effff700 1 -- 172.21.15.53:0/369857817 learned_addr learned my addr 172.21.15.53:0/369857817 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43ef7fe700 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950df90 0x557219520190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43f4a66700 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x5572195271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43effff700 1 -- 172.21.15.53:0/369857817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 msgr2=0x5572195271d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43effff700 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x5572195271d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43effff700 1 -- 172.21.15.53:0/369857817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950df90 msgr2=0x557219520190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43effff700 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950df90 0x557219520190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43effff700 1 -- 172.21.15.53:0/369857817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f0108a30 con 0x55721950d630 2023-11-06T01:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43ef7fe700 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950df90 0x557219520190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43f4a66700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43d80a0430 0x7f43d80a28f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009cd0 tx=0x7f43e8007710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43effff700 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950d630 0x55721951fc50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f43e009b1c0 tx=0x7f43e009ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb5df8047f4e0f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.649+0000 7f43cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.650+0000 7f43cf7fe700 1 -- 172.21.15.53:0/369857817 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e009b4c0 con 0x55721950d630 2023-11-06T01:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.650+0000 7f43cf7fe700 1 -- 172.21.15.53:0/369857817 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f43e009f2f0 con 0x55721950d630 2023-11-06T01:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.650+0000 7f4402659b80 1 -- 172.21.15.53:0/369857817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557219489cb0 con 0x55721950d630 2023-11-06T01:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.650+0000 7f43cf7fe700 1 -- 172.21.15.53:0/369857817 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43e00a3530 con 0x55721950d630 2023-11-06T01:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.650+0000 7f4402659b80 10 client.16809 fetch_fsmap learned FSMap version 47 2023-11-06T01:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.650+0000 7f4402659b80 1 -- 172.21.15.53:0/369857817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f43e40098e0 con 0x55721950d630 2023-11-06T01:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.652+0000 7f43cf7fe700 1 -- 172.21.15.53:0/369857817 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e0065020 con 0x55721950d630 2023-11-06T01:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.652+0000 7f43cf7fe700 1 -- 172.21.15.53:0/369857817 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f43e00a30a0 con 0x55721950d630 2023-11-06T01:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.652+0000 7f4402659b80 10 client.16809 fetch_fsmap finished waiting for FSMap version 47 2023-11-06T01:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.652+0000 7f4402659b80 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.652+0000 7f4402659b80 20 client.16809 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.653+0000 7f4402659b80 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55721952b9b0 0x55721952ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.653+0000 7f4402659b80 4 client.16809 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:46:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.653+0000 7f4402659b80 1 -- 172.21.15.53:0/369857817 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572194dcdb0 con 0x55721952b9b0 2023-11-06T01:46:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.653+0000 7f43ef7fe700 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55721952b9b0 0x55721952ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.653+0000 7f43ef7fe700 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55721952b9b0 0x55721952ddd0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51f25089af42fee6 server_cookie=ba1cf76c9eff189b in_seq=0 out_seq=0 2023-11-06T01:46:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.653+0000 7f43cf7fe700 10 client.16809 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f43cf7fe700 1 -- 172.21.15.53:0/369857817 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5572194dcdb0 con 0x55721952b9b0 2023-11-06T01:46:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f43cf7fe700 10 client.16809 handle_command_reply: tid=0 2023-11-06T01:46:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 1 client.16809 shutdown 2023-11-06T01:46:21.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 1 -- 172.21.15.53:0/369857817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950d630 msgr2=0x55721951fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:21.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950d630 0x55721951fc50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f43e009b1c0 tx=0x7f43e009ae20 comp rx=0 tx=0).stop 2023-11-06T01:46:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 1 -- 172.21.15.53:0/369857817 shutdown_connections 2023-11-06T01:46:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55721952b9b0 0x55721952ddd0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x5572195271d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f43cf7fe700 0 client.16809 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950d630 0x55721951fc50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 1 --2- 172.21.15.53:0/369857817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950df90 0x557219520190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 1 -- 172.21.15.53:0/369857817 >> 172.21.15.53:0/369857817 conn(0x55721946d070 msgr2=0x5572195083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:21.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 1 -- 172.21.15.53:0/369857817 shutdown_connections 2023-11-06T01:46:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 1 -- 172.21.15.53:0/369857817 wait complete. 2023-11-06T01:46:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.654+0000 7f4402659b80 20 client.16809 trim_cache size 0 max 16384 2023-11-06T01:46:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.695+0000 7f4402659b80 1 Processor -- start 2023-11-06T01:46:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.695+0000 7f4402659b80 1 -- start start 2023-11-06T01:46:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.695+0000 7f4402659b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x557219525150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.695+0000 7f4402659b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x557219520110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.695+0000 7f4402659b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721950e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.695+0000 7f4402659b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557219505710 con 0x55721950df90 2023-11-06T01:46:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.695+0000 7f4402659b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572195007a0 con 0x55721950d630 2023-11-06T01:46:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.695+0000 7f4402659b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55721951f100 con 0x55721950ee80 2023-11-06T01:46:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.695+0000 7f43f4a66700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721950e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.696+0000 7f43f4a66700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721950e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57570/0 (socket says 172.21.15.53:57570) 2023-11-06T01:46:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.696+0000 7f43f4a66700 1 -- 172.21.15.53:0/1008084653 learned_addr learned my addr 172.21.15.53:0/1008084653 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.697+0000 7f43f4a66700 1 -- 172.21.15.53:0/1008084653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 msgr2=0x557219525150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:46:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.697+0000 7f43f4a66700 1 --2- 172.21.15.53:0/1008084653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x557219525150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.698+0000 7f43f4a66700 1 -- 172.21.15.53:0/1008084653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 msgr2=0x557219520110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.698+0000 7f43f4a66700 1 --2- 172.21.15.53:0/1008084653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x557219520110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.698+0000 7f43f4a66700 1 -- 172.21.15.53:0/1008084653 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557219275c00 con 0x55721950ee80 2023-11-06T01:46:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.698+0000 7f43f4a66700 1 --2- 172.21.15.53:0/1008084653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721950e850 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43e800be10 tx=0x7f43e8028000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f54f57c182db3032 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.703+0000 7f43d6ffd700 1 -- 172.21.15.53:0/1008084653 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e800e040 con 0x55721950ee80 2023-11-06T01:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.703+0000 7f43d6ffd700 1 -- 172.21.15.53:0/1008084653 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e8025070 con 0x55721950ee80 2023-11-06T01:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.703+0000 7f43d6ffd700 1 -- 172.21.15.53:0/1008084653 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e802b870 con 0x55721950ee80 2023-11-06T01:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.704+0000 7f4402659b80 1 -- 172.21.15.53:0/1008084653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 msgr2=0x55721950e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.704+0000 7f4402659b80 1 --2- 172.21.15.53:0/1008084653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721950e850 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43e800be10 tx=0x7f43e8028000 comp rx=0 tx=0).stop 2023-11-06T01:46:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.704+0000 7f4402659b80 1 -- 172.21.15.53:0/1008084653 shutdown_connections 2023-11-06T01:46:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.704+0000 7f4402659b80 1 --2- 172.21.15.53:0/1008084653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x557219525150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.704+0000 7f4402659b80 1 --2- 172.21.15.53:0/1008084653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721950e850 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.704+0000 7f4402659b80 1 --2- 172.21.15.53:0/1008084653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x557219520110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.704+0000 7f4402659b80 1 -- 172.21.15.53:0/1008084653 >> 172.21.15.53:0/1008084653 conn(0x55721946d070 msgr2=0x55721951fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.705+0000 7f4402659b80 1 -- 172.21.15.53:0/1008084653 shutdown_connections 2023-11-06T01:46:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.707+0000 7f4402659b80 1 -- 172.21.15.53:0/1008084653 wait complete. 2023-11-06T01:46:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.707+0000 7f4402659b80 1 -- start start 2023-11-06T01:46:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.707+0000 7f4402659b80 1 Processor -- start 2023-11-06T01:46:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.708+0000 7f4402659b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x55721957a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.708+0000 7f4402659b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x5572195250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.708+0000 7f4402659b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721957ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.708+0000 7f4402659b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557219579480 con 0x55721950df90 2023-11-06T01:46:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.708+0000 7f4402659b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572195795f0 con 0x55721950d630 2023-11-06T01:46:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.708+0000 7f4402659b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572195797b0 con 0x55721950ee80 2023-11-06T01:46:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.708+0000 7f4402659b80 10 client.0 _ll_register_callbacks cb 0x7f43e009b360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.709+0000 7f43ef7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x5572195250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.709+0000 7f43ef7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x5572195250e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43334/0 (socket says 172.21.15.53:43334) 2023-11-06T01:46:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.709+0000 7f43ef7fe700 1 -- 172.21.15.53:0/2640299740 learned_addr learned my addr 172.21.15.53:0/2640299740 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.709+0000 7f43ef7fe700 1 -- 172.21.15.53:0/2640299740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 msgr2=0x55721957ad60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:46:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.709+0000 7f43ef7fe700 1 --2- 172.21.15.53:0/2640299740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721957ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.709+0000 7f43ef7fe700 1 -- 172.21.15.53:0/2640299740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 msgr2=0x55721957a7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:46:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.709+0000 7f43ef7fe700 1 --2- 172.21.15.53:0/2640299740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x55721957a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.709+0000 7f43ef7fe700 1 -- 172.21.15.53:0/2640299740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e800bb50 con 0x55721950df90 2023-11-06T01:46:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.710+0000 7f43ef7fe700 1 --2- 172.21.15.53:0/2640299740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x5572195250e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f43e4011720 tx=0x7f43e4005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4abc1e99de4571a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.710+0000 7f43cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:46:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.711+0000 7f43cf7fe700 1 -- 172.21.15.53:0/2640299740 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4003b40 con 0x55721950df90 2023-11-06T01:46:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.711+0000 7f43cf7fe700 1 -- 172.21.15.53:0/2640299740 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f43e4003680 con 0x55721950df90 2023-11-06T01:46:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.711+0000 7f4402659b80 1 -- 172.21.15.53:0/2640299740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43e4011e10 con 0x55721950df90 2023-11-06T01:46:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.711+0000 7f43cf7fe700 1 -- 172.21.15.53:0/2640299740 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43e4005420 con 0x55721950df90 2023-11-06T01:46:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.711+0000 7f4402659b80 10 client.16812 fetch_fsmap learned FSMap version 47 2023-11-06T01:46:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.711+0000 7f4402659b80 1 -- 172.21.15.53:0/2640299740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5572192b31f0 con 0x55721950df90 2023-11-06T01:46:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.712+0000 7f43cf7fe700 1 -- 172.21.15.53:0/2640299740 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e4011e10 con 0x55721950df90 2023-11-06T01:46:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.712+0000 7f43cf7fe700 1 -- 172.21.15.53:0/2640299740 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 47) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f43e401a440 con 0x55721950df90 2023-11-06T01:46:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.712+0000 7f4402659b80 10 client.16812 fetch_fsmap finished waiting for FSMap version 47 2023-11-06T01:46:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.712+0000 7f4402659b80 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.712+0000 7f4402659b80 20 client.16812 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.712+0000 7f4402659b80 1 --2- 172.21.15.53:0/2640299740 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55721952b9b0 0x55721957dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.712+0000 7f4402659b80 4 client.16812 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:46:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.712+0000 7f4402659b80 1 -- 172.21.15.53:0/2640299740 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557219457ce0 con 0x55721952b9b0 2023-11-06T01:46:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.712+0000 7f43effff700 1 --2- 172.21.15.53:0/2640299740 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55721952b9b0 0x55721957dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.713+0000 7f43cf7fe700 10 client.16812 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.713+0000 7f43effff700 1 --2- 172.21.15.53:0/2640299740 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55721952b9b0 0x55721957dc50 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d642cd081e56ea2 server_cookie=735751a7060b235 in_seq=0 out_seq=0 2023-11-06T01:46:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.726+0000 7f43cf7fe700 1 -- 172.21.15.53:0/2640299740 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557219457ce0 con 0x55721952b9b0 2023-11-06T01:46:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.726+0000 7f43cf7fe700 10 client.16812 handle_command_reply: tid=0 2023-11-06T01:46:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.726+0000 7f4402659b80 1 client.16812 shutdown 2023-11-06T01:46:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.728+0000 7f4402659b80 1 -- 172.21.15.53:0/2640299740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 msgr2=0x5572195250e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.728+0000 7f4402659b80 1 --2- 172.21.15.53:0/2640299740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x5572195250e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f43e4011720 tx=0x7f43e4005b40 comp rx=0 tx=0).stop 2023-11-06T01:46:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.728+0000 7f4402659b80 1 -- 172.21.15.53:0/2640299740 shutdown_connections 2023-11-06T01:46:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.728+0000 7f4402659b80 1 --2- 172.21.15.53:0/2640299740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55721950d630 0x55721957a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.728+0000 7f4402659b80 1 --2- 172.21.15.53:0/2640299740 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55721952b9b0 0x55721957dc50 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.728+0000 7f4402659b80 1 --2- 172.21.15.53:0/2640299740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55721950ee80 0x55721957ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.728+0000 7f4402659b80 1 --2- 172.21.15.53:0/2640299740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55721950df90 0x5572195250e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.728+0000 7f43cf7fe700 0 client.16812 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.728+0000 7f4402659b80 1 -- 172.21.15.53:0/2640299740 >> 172.21.15.53:0/2640299740 conn(0x55721946d070 msgr2=0x5572195101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.728+0000 7f4402659b80 1 -- 172.21.15.53:0/2640299740 shutdown_connections 2023-11-06T01:46:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.728+0000 7f4402659b80 1 -- 172.21.15.53:0/2640299740 wait complete. 2023-11-06T01:46:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f4402659b80 20 client.16812 trim_cache size 0 max 16384 2023-11-06T01:46:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:46:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:46:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:46:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:46:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 -- 172.21.15.53:0/631350163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43d80a0430 msgr2=0x7f43d80a28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43d80a0430 0x7f43d80a28f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009cd0 tx=0x7f43e8007710 comp rx=0 tx=0).stop 2023-11-06T01:46:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 -- 172.21.15.53:0/631350163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43f00ffca0 msgr2=0x7f43f0199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43f00ffca0 0x7f43f0199860 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f43e000e970 tx=0x7f43e000ec80 comp rx=0 tx=0).stop 2023-11-06T01:46:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 -- 172.21.15.53:0/631350163 shutdown_connections 2023-11-06T01:46:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43d80a0430 0x7f43d80a28f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43f0101830 0x7f43f019e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43f00ffca0 0x7f43f0199860 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 --2- 172.21.15.53:0/631350163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43f0100e90 0x7f43f019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 -- 172.21.15.53:0/631350163 >> 172.21.15.53:0/631350163 conn(0x7f43f00fb230 msgr2=0x7f43f00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 -- 172.21.15.53:0/631350163 shutdown_connections 2023-11-06T01:46:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:21.729+0000 7f43f64c9700 1 -- 172.21.15.53:0/631350163 wait complete. 2023-11-06T01:46:22.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 725.0 s, 1515.4 tps, lat 65.985 ms stddev 72.467 2023-11-06T01:46:23.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:22 smithi053 ceph-mon[99472]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 613 op/s 2023-11-06T01:46:23.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:22 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:46:23.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:22 smithi062 ceph-mon[103924]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 613 op/s 2023-11-06T01:46:23.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:22 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:46:23.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:22 smithi163 ceph-mon[104416]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 613 op/s 2023-11-06T01:46:23.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:22 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:46:25.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:25 smithi053 ceph-mon[99472]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 712 op/s 2023-11-06T01:46:25.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:25 smithi062 ceph-mon[103924]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 712 op/s 2023-11-06T01:46:25.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:25 smithi163 ceph-mon[104416]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 712 op/s 2023-11-06T01:46:26.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:26 smithi053 ceph-mon[99472]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 672 op/s 2023-11-06T01:46:26.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:26 smithi062 ceph-mon[103924]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 672 op/s 2023-11-06T01:46:26.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:26 smithi163 ceph-mon[104416]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 672 op/s 2023-11-06T01:46:27.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 730.0 s, 1059.8 tps, lat 93.424 ms stddev 114.104 2023-11-06T01:46:29.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:29 smithi053 ceph-mon[99472]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 706 op/s 2023-11-06T01:46:29.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:29 smithi062 ceph-mon[103924]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 706 op/s 2023-11-06T01:46:29.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:29 smithi163 ceph-mon[104416]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 706 op/s 2023-11-06T01:46:30.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:30 smithi053 ceph-mon[99472]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 705 op/s 2023-11-06T01:46:30.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:30 smithi062 ceph-mon[103924]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 705 op/s 2023-11-06T01:46:30.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:30 smithi163 ceph-mon[104416]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 705 op/s 2023-11-06T01:46:32.184 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 735.0 s, 1437.1 tps, lat 69.421 ms stddev 77.972 2023-11-06T01:46:32.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:32 smithi163 ceph-mon[104416]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 701 op/s 2023-11-06T01:46:32.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:32 smithi053 ceph-mon[99472]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 701 op/s 2023-11-06T01:46:32.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:32 smithi062 ceph-mon[103924]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 701 op/s 2023-11-06T01:46:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.899+0000 7f24d82ce700 1 -- 172.21.15.53:0/3304216313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24d0100e80 msgr2=0x7f24d01012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.899+0000 7f24d82ce700 1 --2- 172.21.15.53:0/3304216313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24d0100e80 0x7f24d01012e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f24c00099c0 tx=0x7f24c0009cd0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.899+0000 7f24d82ce700 1 -- 172.21.15.53:0/3304216313 shutdown_connections 2023-11-06T01:46:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.899+0000 7f24d82ce700 1 --2- 172.21.15.53:0/3304216313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d0101820 0x7f24d01080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.899+0000 7f24d82ce700 1 --2- 172.21.15.53:0/3304216313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24d00ffc90 0x7f24d01000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.899+0000 7f24d82ce700 1 --2- 172.21.15.53:0/3304216313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24d0100e80 0x7f24d01012e0 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f24c00099c0 tx=0x7f24c0009cd0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.899+0000 7f24d82ce700 1 -- 172.21.15.53:0/3304216313 >> 172.21.15.53:0/3304216313 conn(0x7f24d00fb1e0 msgr2=0x7f24d00fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.901+0000 7f24d82ce700 1 -- 172.21.15.53:0/3304216313 shutdown_connections 2023-11-06T01:46:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.901+0000 7f24d82ce700 1 -- 172.21.15.53:0/3304216313 wait complete. 2023-11-06T01:46:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d82ce700 1 Processor -- start 2023-11-06T01:46:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d82ce700 1 -- start start 2023-11-06T01:46:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d82ce700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24d00ffc90 0x7f24d0198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d82ce700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d0101820 0x7f24d0199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d82ce700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24d019a3a0 0x7f24d019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d82ce700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d019adf0 con 0x7f24d0101820 2023-11-06T01:46:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d82ce700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d019af60 con 0x7f24d00ffc90 2023-11-06T01:46:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d82ce700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d019b120 con 0x7f24d019a3a0 2023-11-06T01:46:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d5869700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d0101820 0x7f24d0199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d5869700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d0101820 0x7f24d0199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43348/0 (socket says 172.21.15.53:43348) 2023-11-06T01:46:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d5869700 1 -- 172.21.15.53:0/1755006506 learned_addr learned my addr 172.21.15.53:0/1755006506 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d606a700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24d00ffc90 0x7f24d0198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.902+0000 7f24d686b700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24d019a3a0 0x7f24d019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.903+0000 7f24d5869700 1 -- 172.21.15.53:0/1755006506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24d019a3a0 msgr2=0x7f24d019a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.903+0000 7f24d5869700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24d019a3a0 0x7f24d019a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.903+0000 7f24d5869700 1 -- 172.21.15.53:0/1755006506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24d00ffc90 msgr2=0x7f24d0198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.903+0000 7f24d5869700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24d00ffc90 0x7f24d0198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.903+0000 7f24d5869700 1 -- 172.21.15.53:0/1755006506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c8009c40 con 0x7f24d0101820 2023-11-06T01:46:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.903+0000 7f24d5869700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d0101820 0x7f24d0199520 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009fd0 tx=0x7f24c000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.903+0000 7f24c77fe700 1 -- 172.21.15.53:0/1755006506 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c001d070 con 0x7f24d0101820 2023-11-06T01:46:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.903+0000 7f24c77fe700 1 -- 172.21.15.53:0/1755006506 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24c00054e0 con 0x7f24d0101820 2023-11-06T01:46:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.903+0000 7f24c77fe700 1 -- 172.21.15.53:0/1755006506 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0004a90 con 0x7f24d0101820 2023-11-06T01:46:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.903+0000 7f24d82ce700 1 -- 172.21.15.53:0/1755006506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c00095f0 con 0x7f24d0101820 2023-11-06T01:46:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.904+0000 7f24d82ce700 1 -- 172.21.15.53:0/1755006506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d01a1890 con 0x7f24d0101820 2023-11-06T01:46:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.906+0000 7f24c77fe700 1 -- 172.21.15.53:0/1755006506 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f24c0004c50 con 0x7f24d0101820 2023-11-06T01:46:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.907+0000 7f24c77fe700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f24bc0a0620 0x7f24bc0a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.908+0000 7f24c77fe700 1 -- 172.21.15.53:0/1755006506 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f24c00cc500 con 0x7f24d0101820 2023-11-06T01:46:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.909+0000 7f24e445eb80 1 Processor -- start 2023-11-06T01:46:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.909+0000 7f24e445eb80 1 -- start start 2023-11-06T01:46:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.909+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3cad7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.909+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3cad88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.909+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3cad9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.909+0000 7f24e445eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3ca2f9b0 con 0x55df3cad7680 2023-11-06T01:46:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.909+0000 7f24e445eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3ca2d370 con 0x55df3cad8ed0 2023-11-06T01:46:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.909+0000 7f24e445eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3ca28c60 con 0x55df3cad7fe0 2023-11-06T01:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d606a700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f24bc0a0620 0x7f24bc0a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d686b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3cad9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d686b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3cad9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54794/0 (socket says 172.21.15.53:54794) 2023-11-06T01:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d686b700 1 -- 172.21.15.53:0/1874531428 learned_addr learned my addr 172.21.15.53:0/1874531428 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d5869700 1 --2- 172.21.15.53:0/1874531428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3cad88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d686b700 1 -- 172.21.15.53:0/1874531428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 msgr2=0x55df3cad88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d686b700 1 --2- 172.21.15.53:0/1874531428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3cad88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d686b700 1 -- 172.21.15.53:0/1874531428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 msgr2=0x55df3cad7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d686b700 1 --2- 172.21.15.53:0/1874531428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3cad7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d686b700 1 -- 172.21.15.53:0/1874531428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df3c856d80 con 0x55df3cad8ed0 2023-11-06T01:46:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d606a700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f24bc0a0620 0x7f24bc0a2ae0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f24d0100ce0 tx=0x7f24cc00b800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.910+0000 7f24d686b700 1 --2- 172.21.15.53:0/1874531428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3cad9350 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e730 tx=0x7f24c8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04c15a94517760f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.911+0000 7f24c4daa700 1 -- 172.21.15.53:0/1874531428 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800e040 con 0x55df3cad8ed0 2023-11-06T01:46:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.911+0000 7f24c4daa700 1 -- 172.21.15.53:0/1874531428 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24c800f040 con 0x55df3cad8ed0 2023-11-06T01:46:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.911+0000 7f24c4daa700 1 -- 172.21.15.53:0/1874531428 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8007760 con 0x55df3cad8ed0 2023-11-06T01:46:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.912+0000 7f24e445eb80 1 -- 172.21.15.53:0/1874531428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 msgr2=0x55df3cad9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.912+0000 7f24e445eb80 1 --2- 172.21.15.53:0/1874531428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3cad9350 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e730 tx=0x7f24c8009920 comp rx=0 tx=0).stop 2023-11-06T01:46:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.912+0000 7f24e445eb80 1 -- 172.21.15.53:0/1874531428 shutdown_connections 2023-11-06T01:46:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.912+0000 7f24e445eb80 1 --2- 172.21.15.53:0/1874531428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3cad9350 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.912+0000 7f24e445eb80 1 --2- 172.21.15.53:0/1874531428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3cad7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.912+0000 7f24e445eb80 1 --2- 172.21.15.53:0/1874531428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3cad88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.912+0000 7f24e445eb80 1 -- 172.21.15.53:0/1874531428 >> 172.21.15.53:0/1874531428 conn(0x55df3ca37070 msgr2=0x55df3cad4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.914+0000 7f24e445eb80 1 -- 172.21.15.53:0/1874531428 shutdown_connections 2023-11-06T01:46:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.914+0000 7f24e445eb80 1 -- 172.21.15.53:0/1874531428 wait complete. 2023-11-06T01:46:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.915+0000 7f24e445eb80 1 -- start start 2023-11-06T01:46:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.915+0000 7f24e445eb80 1 Processor -- start 2023-11-06T01:46:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.915+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3cae9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.915+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3caea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.915+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caf1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.915+0000 7f24e445eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3c9c54b0 con 0x55df3cad7680 2023-11-06T01:46:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.915+0000 7f24e445eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3c9c5150 con 0x55df3cad8ed0 2023-11-06T01:46:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.915+0000 7f24e445eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3c8ac8c0 con 0x55df3cad7fe0 2023-11-06T01:46:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.915+0000 7f24e445eb80 10 client.0 _ll_register_callbacks cb 0x55df3c912a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.915+0000 7f24d686b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caf1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24d686b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caf1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54802/0 (socket says 172.21.15.53:54802) 2023-11-06T01:46:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24d686b700 1 -- 172.21.15.53:0/182587214 learned_addr learned my addr 172.21.15.53:0/182587214 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24d606a700 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3cae9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24d5869700 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3caea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24d686b700 1 -- 172.21.15.53:0/182587214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 msgr2=0x55df3caea1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24d686b700 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3caea1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24d686b700 1 -- 172.21.15.53:0/182587214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 msgr2=0x55df3cae9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24d686b700 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3cae9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24d686b700 1 -- 172.21.15.53:0/182587214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c800e410 con 0x55df3cad8ed0 2023-11-06T01:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24d606a700 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3cae9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24d686b700 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caf1210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f24c8004470 tx=0x7f24c80048c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1b3102c7f10504 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.916+0000 7f24b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.917+0000 7f24b9ffb700 1 -- 172.21.15.53:0/182587214 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800e040 con 0x55df3cad8ed0 2023-11-06T01:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.917+0000 7f24e445eb80 1 -- 172.21.15.53:0/182587214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df3ca53cb0 con 0x55df3cad8ed0 2023-11-06T01:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.917+0000 7f24b9ffb700 1 -- 172.21.15.53:0/182587214 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f24c8007500 con 0x55df3cad8ed0 2023-11-06T01:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.917+0000 7f24b9ffb700 1 -- 172.21.15.53:0/182587214 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24c80076c0 con 0x55df3cad8ed0 2023-11-06T01:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.917+0000 7f24e445eb80 10 client.26330 fetch_fsmap learned FSMap version 48 2023-11-06T01:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.917+0000 7f24e445eb80 1 -- 172.21.15.53:0/182587214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55df3c83fc00 con 0x55df3cad8ed0 2023-11-06T01:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.918+0000 7f24b9ffb700 1 -- 172.21.15.53:0/182587214 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df3ca53cb0 con 0x55df3cad8ed0 2023-11-06T01:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.918+0000 7f24b9ffb700 1 -- 172.21.15.53:0/182587214 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 48) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f24c8028910 con 0x55df3cad8ed0 2023-11-06T01:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.918+0000 7f24e445eb80 10 client.26330 fetch_fsmap finished waiting for FSMap version 48 2023-11-06T01:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.918+0000 7f24e445eb80 10 client.26330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.918+0000 7f24e445eb80 20 client.26330 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.918+0000 7f24e445eb80 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55df3caf59a0 0x55df3caf7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.918+0000 7f24e445eb80 4 client.26330 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.918+0000 7f24e445eb80 1 -- 172.21.15.53:0/182587214 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df3caa6db0 con 0x55df3caf59a0 2023-11-06T01:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.918+0000 7f24d606a700 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55df3caf59a0 0x55df3caf7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.919+0000 7f24b9ffb700 10 client.26330 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.919+0000 7f24d606a700 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55df3caf59a0 0x55df3caf7dc0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1d2a13ccf933ee server_cookie=b9ddf7b34fed403b in_seq=0 out_seq=0 2023-11-06T01:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.920+0000 7f24b9ffb700 1 -- 172.21.15.53:0/182587214 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55df3caa6db0 con 0x55df3caf59a0 2023-11-06T01:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.920+0000 7f24b9ffb700 10 client.26330 handle_command_reply: tid=0 2023-11-06T01:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.920+0000 7f24e445eb80 1 client.26330 shutdown 2023-11-06T01:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.921+0000 7f24e445eb80 1 -- 172.21.15.53:0/182587214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 msgr2=0x55df3caf1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.921+0000 7f24e445eb80 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caf1210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f24c8004470 tx=0x7f24c80048c0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.922+0000 7f24e445eb80 1 -- 172.21.15.53:0/182587214 shutdown_connections 2023-11-06T01:46:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.922+0000 7f24e445eb80 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caf1210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.922+0000 7f24e445eb80 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55df3caf59a0 0x55df3caf7dc0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.922+0000 7f24e445eb80 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3cae9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.922+0000 7f24e445eb80 1 --2- 172.21.15.53:0/182587214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3caea1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.922+0000 7f24b9ffb700 0 client.26330 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.922+0000 7f24e445eb80 1 -- 172.21.15.53:0/182587214 >> 172.21.15.53:0/182587214 conn(0x55df3ca37070 msgr2=0x55df3cad23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.922+0000 7f24e445eb80 1 -- 172.21.15.53:0/182587214 shutdown_connections 2023-11-06T01:46:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.922+0000 7f24e445eb80 1 -- 172.21.15.53:0/182587214 wait complete. 2023-11-06T01:46:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.922+0000 7f24e445eb80 20 client.26330 trim_cache size 0 max 16384 2023-11-06T01:46:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.946+0000 7f24e445eb80 1 Processor -- start 2023-11-06T01:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.946+0000 7f24e445eb80 1 -- start start 2023-11-06T01:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.946+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3caef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.946+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3cad92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.946+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3cad3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.947+0000 7f24e445eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3cae9140 con 0x55df3cad7680 2023-11-06T01:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.947+0000 7f24e445eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3cacd500 con 0x55df3cad8ed0 2023-11-06T01:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.947+0000 7f24e445eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3cae9830 con 0x55df3cad7fe0 2023-11-06T01:46:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.947+0000 7f24d5869700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3cad92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.947+0000 7f24d5869700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3cad92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57612/0 (socket says 172.21.15.53:57612) 2023-11-06T01:46:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.947+0000 7f24d5869700 1 -- 172.21.15.53:0/4096449969 learned_addr learned my addr 172.21.15.53:0/4096449969 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.948+0000 7f24d686b700 1 --2- 172.21.15.53:0/4096449969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3cad3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.948+0000 7f24d606a700 1 --2- 172.21.15.53:0/4096449969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3caef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.948+0000 7f24d686b700 1 -- 172.21.15.53:0/4096449969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 msgr2=0x55df3cad92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.948+0000 7f24d686b700 1 --2- 172.21.15.53:0/4096449969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3cad92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.948+0000 7f24d686b700 1 -- 172.21.15.53:0/4096449969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 msgr2=0x55df3caef1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.948+0000 7f24d686b700 1 --2- 172.21.15.53:0/4096449969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3caef1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.948+0000 7f24d686b700 1 -- 172.21.15.53:0/4096449969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df3c87d1f0 con 0x55df3cad8ed0 2023-11-06T01:46:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.948+0000 7f24d686b700 1 --2- 172.21.15.53:0/4096449969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3cad3c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009fd0 tx=0x7f24c8004850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81168db6db0baa35 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.949+0000 7f24c55ab700 1 -- 172.21.15.53:0/4096449969 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8009920 con 0x55df3cad8ed0 2023-11-06T01:46:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.949+0000 7f24c55ab700 1 -- 172.21.15.53:0/4096449969 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24c800e030 con 0x55df3cad8ed0 2023-11-06T01:46:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.949+0000 7f24c55ab700 1 -- 172.21.15.53:0/4096449969 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8032630 con 0x55df3cad8ed0 2023-11-06T01:46:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.949+0000 7f24e445eb80 1 -- 172.21.15.53:0/4096449969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 msgr2=0x55df3cad3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.949+0000 7f24e445eb80 1 --2- 172.21.15.53:0/4096449969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3cad3c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009fd0 tx=0x7f24c8004850 comp rx=0 tx=0).stop 2023-11-06T01:46:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.950+0000 7f24e445eb80 1 -- 172.21.15.53:0/4096449969 shutdown_connections 2023-11-06T01:46:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.950+0000 7f24e445eb80 1 --2- 172.21.15.53:0/4096449969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3cad3c20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.950+0000 7f24e445eb80 1 --2- 172.21.15.53:0/4096449969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3caef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.950+0000 7f24e445eb80 1 --2- 172.21.15.53:0/4096449969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3cad92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.950+0000 7f24e445eb80 1 -- 172.21.15.53:0/4096449969 >> 172.21.15.53:0/4096449969 conn(0x55df3ca37070 msgr2=0x55df3cae8900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.950+0000 7f24e445eb80 1 -- 172.21.15.53:0/4096449969 shutdown_connections 2023-11-06T01:46:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.950+0000 7f24e445eb80 1 -- 172.21.15.53:0/4096449969 wait complete. 2023-11-06T01:46:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.950+0000 7f24e445eb80 1 -- start start 2023-11-06T01:46:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.951+0000 7f24e445eb80 1 Processor -- start 2023-11-06T01:46:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.951+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3caeb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.951+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3caeb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.951+0000 7f24e445eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.951+0000 7f24e445eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3caeee60 con 0x55df3cad7680 2023-11-06T01:46:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.951+0000 7f24e445eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3caeefd0 con 0x55df3cad8ed0 2023-11-06T01:46:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.951+0000 7f24e445eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3caef190 con 0x55df3cad7fe0 2023-11-06T01:46:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.951+0000 7f24e445eb80 10 client.0 _ll_register_callbacks cb 0x7f24c800e630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.951+0000 7f24d686b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.951+0000 7f24d686b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caee920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54814/0 (socket says 172.21.15.53:54814) 2023-11-06T01:46:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.951+0000 7f24d686b700 1 -- 172.21.15.53:0/1272155834 learned_addr learned my addr 172.21.15.53:0/1272155834 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.952+0000 7f24d5869700 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3caeb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.952+0000 7f24d606a700 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3caeb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.952+0000 7f24d686b700 1 -- 172.21.15.53:0/1272155834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 msgr2=0x55df3caeb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.952+0000 7f24d686b700 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3caeb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.952+0000 7f24d686b700 1 -- 172.21.15.53:0/1272155834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 msgr2=0x55df3caeb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.952+0000 7f24d686b700 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3caeb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.952+0000 7f24d686b700 1 -- 172.21.15.53:0/1272155834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df3c83fc00 con 0x55df3cad8ed0 2023-11-06T01:46:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.952+0000 7f24d686b700 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caee920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f24c8028af0 tx=0x7f24c8006070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a604c2f26d41a01 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.953+0000 7f24b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:46:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.953+0000 7f24b9ffb700 1 -- 172.21.15.53:0/1272155834 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8028680 con 0x55df3cad8ed0 2023-11-06T01:46:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.953+0000 7f24b9ffb700 1 -- 172.21.15.53:0/1272155834 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f24c80287e0 con 0x55df3cad8ed0 2023-11-06T01:46:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.953+0000 7f24e445eb80 1 -- 172.21.15.53:0/1272155834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24c800f040 con 0x55df3cad8ed0 2023-11-06T01:46:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.953+0000 7f24b9ffb700 1 -- 172.21.15.53:0/1272155834 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24c80323d0 con 0x55df3cad8ed0 2023-11-06T01:46:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.953+0000 7f24e445eb80 10 client.26336 fetch_fsmap learned FSMap version 48 2023-11-06T01:46:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.953+0000 7f24e445eb80 1 -- 172.21.15.53:0/1272155834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55df3c855250 con 0x55df3cad8ed0 2023-11-06T01:46:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.953+0000 7f24b9ffb700 1 -- 172.21.15.53:0/1272155834 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24c800f040 con 0x55df3cad8ed0 2023-11-06T01:46:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.954+0000 7f24b9ffb700 1 -- 172.21.15.53:0/1272155834 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 48) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f24c80078e0 con 0x55df3cad8ed0 2023-11-06T01:46:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.954+0000 7f24e445eb80 10 client.26336 fetch_fsmap finished waiting for FSMap version 48 2023-11-06T01:46:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.954+0000 7f24e445eb80 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.954+0000 7f24e445eb80 20 client.26336 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.954+0000 7f24e445eb80 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55df3cae4fc0 0x55df3cae73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.954+0000 7f24e445eb80 4 client.26336 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:46:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.954+0000 7f24e445eb80 1 -- 172.21.15.53:0/1272155834 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df3ca21ce0 con 0x55df3cae4fc0 2023-11-06T01:46:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.955+0000 7f24d606a700 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55df3cae4fc0 0x55df3cae73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.956+0000 7f24d606a700 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55df3cae4fc0 0x55df3cae73e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32b71766d9e73018 server_cookie=e033f4a4e5389d29 in_seq=0 out_seq=0 2023-11-06T01:46:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.956+0000 7f24b9ffb700 10 client.26336 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.964+0000 7f24b9ffb700 1 -- 172.21.15.53:0/1272155834 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df3ca21ce0 con 0x55df3cae4fc0 2023-11-06T01:46:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.964+0000 7f24b9ffb700 10 client.26336 handle_command_reply: tid=0 2023-11-06T01:46:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.964+0000 7f24e445eb80 1 client.26336 shutdown 2023-11-06T01:46:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24e445eb80 1 -- 172.21.15.53:0/1272155834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 msgr2=0x55df3caee920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24e445eb80 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caee920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f24c8028af0 tx=0x7f24c8006070 comp rx=0 tx=0).stop 2023-11-06T01:46:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24e445eb80 1 -- 172.21.15.53:0/1272155834 shutdown_connections 2023-11-06T01:46:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24e445eb80 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df3cad8ed0 0x55df3caee920 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24e445eb80 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55df3cae4fc0 0x55df3cae73e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24e445eb80 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df3cad7680 0x55df3caeb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24b9ffb700 0 client.26336 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24e445eb80 1 --2- 172.21.15.53:0/1272155834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df3cad7fe0 0x55df3caeb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24e445eb80 1 -- 172.21.15.53:0/1272155834 >> 172.21.15.53:0/1272155834 conn(0x55df3ca37070 msgr2=0x55df3cadf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24e445eb80 1 -- 172.21.15.53:0/1272155834 shutdown_connections 2023-11-06T01:46:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24e445eb80 1 -- 172.21.15.53:0/1272155834 wait complete. 2023-11-06T01:46:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24e445eb80 20 client.26336 trim_cache size 0 max 16384 2023-11-06T01:46:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:46:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:46:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:46:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:46:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24d82ce700 1 -- 172.21.15.53:0/1755006506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f24bc0a0620 msgr2=0x7f24bc0a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24d82ce700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f24bc0a0620 0x7f24bc0a2ae0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f24d0100ce0 tx=0x7f24cc00b800 comp rx=0 tx=0).stop 2023-11-06T01:46:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24d82ce700 1 -- 172.21.15.53:0/1755006506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d0101820 msgr2=0x7f24d0199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24d82ce700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d0101820 0x7f24d0199520 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009fd0 tx=0x7f24c000f690 comp rx=0 tx=0).stop 2023-11-06T01:46:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.966+0000 7f24d82ce700 1 -- 172.21.15.53:0/1755006506 shutdown_connections 2023-11-06T01:46:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.967+0000 7f24d82ce700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f24bc0a0620 0x7f24bc0a2ae0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.967+0000 7f24d82ce700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24d00ffc90 0x7f24d0198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.967+0000 7f24d82ce700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24d019a3a0 0x7f24d019a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.967+0000 7f24d82ce700 1 --2- 172.21.15.53:0/1755006506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d0101820 0x7f24d0199520 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.967+0000 7f24d82ce700 1 -- 172.21.15.53:0/1755006506 >> 172.21.15.53:0/1755006506 conn(0x7f24d00fb1e0 msgr2=0x7f24d00fbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.967+0000 7f24d82ce700 1 -- 172.21.15.53:0/1755006506 shutdown_connections 2023-11-06T01:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:34.967+0000 7f24d82ce700 1 -- 172.21.15.53:0/1755006506 wait complete. 2023-11-06T01:46:35.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:35 smithi053 ceph-mon[99472]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 813 op/s 2023-11-06T01:46:35.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:35 smithi062 ceph-mon[103924]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 813 op/s 2023-11-06T01:46:35.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:35 smithi163 ceph-mon[104416]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 813 op/s 2023-11-06T01:46:36.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:36 smithi053 ceph-mon[99472]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 776 op/s 2023-11-06T01:46:36.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:36 smithi062 ceph-mon[103924]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 776 op/s 2023-11-06T01:46:36.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:36 smithi163 ceph-mon[104416]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 776 op/s 2023-11-06T01:46:37.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 740.0 s, 1356.2 tps, lat 74.438 ms stddev 82.212 2023-11-06T01:46:39.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:38 smithi053 ceph-mon[99472]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 741 op/s 2023-11-06T01:46:39.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:38 smithi062 ceph-mon[103924]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 741 op/s 2023-11-06T01:46:39.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:38 smithi163 ceph-mon[104416]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 741 op/s 2023-11-06T01:46:40.888 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:46:41.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.100+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/3356317207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e640ffcb0 msgr2=0x7f9e641000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:41.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.100+0000 7f9e6a7ed700 1 --2- 172.21.15.53:0/3356317207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e640ffcb0 0x7f9e641000d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9e540099c0 tx=0x7f9e54009cd0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.102+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/3356317207 shutdown_connections 2023-11-06T01:46:41.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.102+0000 7f9e6a7ed700 1 --2- 172.21.15.53:0/3356317207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e64101840 0x7f9e641080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.102+0000 7f9e6a7ed700 1 --2- 172.21.15.53:0/3356317207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e640ffcb0 0x7f9e641000d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.102+0000 7f9e6a7ed700 1 --2- 172.21.15.53:0/3356317207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e64100ea0 0x7f9e64101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.102+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/3356317207 >> 172.21.15.53:0/3356317207 conn(0x7f9e640fb220 msgr2=0x7f9e640fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:41.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.102+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/3356317207 shutdown_connections 2023-11-06T01:46:41.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.102+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/3356317207 wait complete. 2023-11-06T01:46:41.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.102+0000 7f9e6a7ed700 1 Processor -- start 2023-11-06T01:46:41.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.102+0000 7f9e6a7ed700 1 -- start start 2023-11-06T01:46:41.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e6a7ed700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e640ffcb0 0x7f9e64103f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e6a7ed700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64100ea0 0x7f9e64102580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e6a7ed700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e64101840 0x7f9e64102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e6a7ed700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64103260 con 0x7f9e64100ea0 2023-11-06T01:46:41.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e6a7ed700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641033d0 con 0x7f9e640ffcb0 2023-11-06T01:46:41.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e6a7ed700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641a0b40 con 0x7f9e64101840 2023-11-06T01:46:41.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e637fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64100ea0 0x7f9e64102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e640ffcb0 0x7f9e64103f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e637fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64100ea0 0x7f9e64102580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43392/0 (socket says 172.21.15.53:43392) 2023-11-06T01:46:41.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e68d8a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e64101840 0x7f9e64102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e640ffcb0 0x7f9e64103f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54828/0 (socket says 172.21.15.53:54828) 2023-11-06T01:46:41.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e637fe700 1 -- 172.21.15.53:0/2207432395 learned_addr learned my addr 172.21.15.53:0/2207432395 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:41.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.103+0000 7f9e637fe700 1 -- 172.21.15.53:0/2207432395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e64101840 msgr2=0x7f9e64102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:41.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e637fe700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e64101840 0x7f9e64102ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e637fe700 1 -- 172.21.15.53:0/2207432395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e640ffcb0 msgr2=0x7f9e64103f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:41.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e637fe700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e640ffcb0 0x7f9e64103f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e637fe700 1 -- 172.21.15.53:0/2207432395 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e540095f0 con 0x7f9e64100ea0 2023-11-06T01:46:41.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e68d8a700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e64101840 0x7f9e64102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:46:41.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e63fff700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e640ffcb0 0x7f9e64103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:46:41.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e637fe700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64100ea0 0x7f9e64102580 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800e9a0 tx=0x7f9e5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:41.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e617fa700 1 -- 172.21.15.53:0/2207432395 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58009790 con 0x7f9e64100ea0 2023-11-06T01:46:41.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/2207432395 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e641a0e20 con 0x7f9e64100ea0 2023-11-06T01:46:41.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e617fa700 1 -- 172.21.15.53:0/2207432395 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e58010430 con 0x7f9e64100ea0 2023-11-06T01:46:41.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e617fa700 1 -- 172.21.15.53:0/2207432395 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58018680 con 0x7f9e64100ea0 2023-11-06T01:46:41.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.104+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/2207432395 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e641a12a0 con 0x7f9e64100ea0 2023-11-06T01:46:41.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.109+0000 7f9e7697db80 1 Processor -- start 2023-11-06T01:46:41.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.109+0000 7f9e7697db80 1 -- start start 2023-11-06T01:46:41.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.109+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42960 0x55ba68d42d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d41670 0x55ba68d41af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d498d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e7697db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68c90a10 con 0x55ba68d42960 2023-11-06T01:46:41.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e7697db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68c8e3d0 con 0x55ba68d42030 2023-11-06T01:46:41.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e7697db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68c89cc0 con 0x55ba68d41670 2023-11-06T01:46:41.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e68d8a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42960 0x55ba68d42d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e617fa700 1 -- 172.21.15.53:0/2207432395 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f9e580105a0 con 0x7f9e64100ea0 2023-11-06T01:46:41.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e68d8a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42960 0x55ba68d42d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43396/0 (socket says 172.21.15.53:43396) 2023-11-06T01:46:41.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e68d8a700 1 -- 172.21.15.53:0/1180700333 learned_addr learned my addr 172.21.15.53:0/1180700333 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:41.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e63fff700 1 --2- 172.21.15.53:0/1180700333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d41670 0x55ba68d41af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e637fe700 1 --2- 172.21.15.53:0/1180700333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e68d8a700 1 -- 172.21.15.53:0/1180700333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d41670 msgr2=0x55ba68d41af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:41.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e68d8a700 1 --2- 172.21.15.53:0/1180700333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d41670 0x55ba68d41af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e68d8a700 1 -- 172.21.15.53:0/1180700333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 msgr2=0x55ba68d498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:41.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e68d8a700 1 --2- 172.21.15.53:0/1180700333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e68d8a700 1 -- 172.21.15.53:0/1180700333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba68ab7de0 con 0x55ba68d42960 2023-11-06T01:46:41.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.110+0000 7f9e68d8a700 1 --2- 172.21.15.53:0/1180700333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42960 0x55ba68d42d80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c003e40 tx=0x7f9e5c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53c0c00fd82b1f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:41.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.111+0000 7f9e617fa700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e4c0a0520 0x7f9e4c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.111+0000 7f9e4a7fc700 1 -- 172.21.15.53:0/1180700333 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0092d0 con 0x55ba68d42960 2023-11-06T01:46:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.111+0000 7f9e4a7fc700 1 -- 172.21.15.53:0/1180700333 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e5c00f040 con 0x55ba68d42960 2023-11-06T01:46:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.111+0000 7f9e4a7fc700 1 -- 172.21.15.53:0/1180700333 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00e870 con 0x55ba68d42960 2023-11-06T01:46:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.111+0000 7f9e617fa700 1 -- 172.21.15.53:0/2207432395 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9e58014070 con 0x7f9e64100ea0 2023-11-06T01:46:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.111+0000 7f9e63fff700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e4c0a0520 0x7f9e4c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.112+0000 7f9e7697db80 1 -- 172.21.15.53:0/1180700333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42960 msgr2=0x55ba68d42d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.112+0000 7f9e7697db80 1 --2- 172.21.15.53:0/1180700333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42960 0x55ba68d42d80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c003e40 tx=0x7f9e5c009920 comp rx=0 tx=0).stop 2023-11-06T01:46:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.112+0000 7f9e7697db80 1 -- 172.21.15.53:0/1180700333 shutdown_connections 2023-11-06T01:46:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.112+0000 7f9e7697db80 1 --2- 172.21.15.53:0/1180700333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.112+0000 7f9e7697db80 1 --2- 172.21.15.53:0/1180700333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42960 0x55ba68d42d80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.112+0000 7f9e7697db80 1 --2- 172.21.15.53:0/1180700333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d41670 0x55ba68d41af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.112+0000 7f9e7697db80 1 -- 172.21.15.53:0/1180700333 >> 172.21.15.53:0/1180700333 conn(0x55ba68c980d0 msgr2=0x55ba68d33940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:41.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.112+0000 7f9e7697db80 1 -- 172.21.15.53:0/1180700333 shutdown_connections 2023-11-06T01:46:41.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.112+0000 7f9e63fff700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e4c0a0520 0x7f9e4c0a29e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009990 tx=0x7f9e5401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:41.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.112+0000 7f9e7697db80 1 -- 172.21.15.53:0/1180700333 wait complete. 2023-11-06T01:46:41.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.113+0000 7f9e7697db80 1 -- start start 2023-11-06T01:46:41.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.113+0000 7f9e7697db80 1 Processor -- start 2023-11-06T01:46:41.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 0x55ba68d3a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d3a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68d3f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e7697db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68c26510 con 0x55ba68d41670 2023-11-06T01:46:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e7697db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68c261b0 con 0x55ba68d42030 2023-11-06T01:46:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e7697db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68b0d920 con 0x55ba68d42960 2023-11-06T01:46:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e7697db80 10 client.0 _ll_register_callbacks cb 0x55ba68b73a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e68d8a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68d3f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 0x55ba68d3a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e68d8a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68d3f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57646/0 (socket says 172.21.15.53:57646) 2023-11-06T01:46:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e68d8a700 1 -- 172.21.15.53:0/2096362372 learned_addr learned my addr 172.21.15.53:0/2096362372 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e68d8a700 1 -- 172.21.15.53:0/2096362372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 msgr2=0x55ba68d3a7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:46:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e68d8a700 1 --2- 172.21.15.53:0/2096362372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d3a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e68d8a700 1 -- 172.21.15.53:0/2096362372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 msgr2=0x55ba68d3a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e68d8a700 1 --2- 172.21.15.53:0/2096362372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 0x55ba68d3a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.114+0000 7f9e68d8a700 1 -- 172.21.15.53:0/2096362372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c003b20 con 0x55ba68d42960 2023-11-06T01:46:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.115+0000 7f9e68d8a700 1 --2- 172.21.15.53:0/2096362372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68d3f810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c007730 tx=0x7f9e5c007760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7171a1c2f0a147fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.115+0000 7f9e437fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:46:41.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.115+0000 7f9e437fe700 1 -- 172.21.15.53:0/2096362372 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c004cd0 con 0x55ba68d42960 2023-11-06T01:46:41.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.115+0000 7f9e437fe700 1 -- 172.21.15.53:0/2096362372 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9e5c004e30 con 0x55ba68d42960 2023-11-06T01:46:41.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.115+0000 7f9e7697db80 1 -- 172.21.15.53:0/2096362372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba68cb4d10 con 0x55ba68d42960 2023-11-06T01:46:41.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.115+0000 7f9e437fe700 1 -- 172.21.15.53:0/2096362372 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e5c007d20 con 0x55ba68d42960 2023-11-06T01:46:41.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.115+0000 7f9e7697db80 10 client.26419 fetch_fsmap learned FSMap version 48 2023-11-06T01:46:41.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.115+0000 7f9e7697db80 1 -- 172.21.15.53:0/2096362372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55ba68aa0c60 con 0x55ba68d42960 2023-11-06T01:46:41.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.115+0000 7f9e437fe700 1 -- 172.21.15.53:0/2096362372 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba68cb4d10 con 0x55ba68d42960 2023-11-06T01:46:41.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.115+0000 7f9e437fe700 1 -- 172.21.15.53:0/2096362372 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 48) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f9e5c0157b0 con 0x55ba68d42960 2023-11-06T01:46:41.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.115+0000 7f9e7697db80 10 client.26419 fetch_fsmap finished waiting for FSMap version 48 2023-11-06T01:46:41.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.116+0000 7f9e7697db80 10 client.26419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:41.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.116+0000 7f9e7697db80 20 client.26419 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:41.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.116+0000 7f9e7697db80 1 --2- 172.21.15.53:0/2096362372 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ba68d5f710 0x55ba68d61b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.116+0000 7f9e7697db80 4 client.26419 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:46:41.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.116+0000 7f9e7697db80 1 -- 172.21.15.53:0/2096362372 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba68d07e10 con 0x55ba68d5f710 2023-11-06T01:46:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.116+0000 7f9e63fff700 1 --2- 172.21.15.53:0/2096362372 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ba68d5f710 0x55ba68d61b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.116+0000 7f9e437fe700 10 client.26419 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.116+0000 7f9e63fff700 1 --2- 172.21.15.53:0/2096362372 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ba68d5f710 0x55ba68d61b30 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7927277b8e8760b7 server_cookie=835b4ca0309c5888 in_seq=0 out_seq=0 2023-11-06T01:46:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e437fe700 1 -- 172.21.15.53:0/2096362372 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ba68d07e10 con 0x55ba68d5f710 2023-11-06T01:46:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e437fe700 10 client.26419 handle_command_reply: tid=0 2023-11-06T01:46:41.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e7697db80 1 client.26419 shutdown 2023-11-06T01:46:41.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e7697db80 1 -- 172.21.15.53:0/2096362372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 msgr2=0x55ba68d3f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:41.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e7697db80 1 --2- 172.21.15.53:0/2096362372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68d3f810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c007730 tx=0x7f9e5c007760 comp rx=0 tx=0).stop 2023-11-06T01:46:41.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e7697db80 1 -- 172.21.15.53:0/2096362372 shutdown_connections 2023-11-06T01:46:41.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e7697db80 1 --2- 172.21.15.53:0/2096362372 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ba68d5f710 0x55ba68d61b30 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e7697db80 1 --2- 172.21.15.53:0/2096362372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68d3f810 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e7697db80 1 --2- 172.21.15.53:0/2096362372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 0x55ba68d3a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e437fe700 0 client.26419 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:41.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e7697db80 1 --2- 172.21.15.53:0/2096362372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d3a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e7697db80 1 -- 172.21.15.53:0/2096362372 >> 172.21.15.53:0/2096362372 conn(0x55ba68c980d0 msgr2=0x55ba68d330b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:41.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.117+0000 7f9e7697db80 1 -- 172.21.15.53:0/2096362372 shutdown_connections 2023-11-06T01:46:41.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.118+0000 7f9e7697db80 1 -- 172.21.15.53:0/2096362372 wait complete. 2023-11-06T01:46:41.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.118+0000 7f9e7697db80 20 client.26419 trim_cache size 0 max 16384 2023-11-06T01:46:41.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e7697db80 1 Processor -- start 2023-11-06T01:46:41.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e7697db80 1 -- start start 2023-11-06T01:46:41.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 0x55ba68d3f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d40100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68d3a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e7697db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68d39e40 con 0x55ba68d41670 2023-11-06T01:46:41.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e7697db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68d42410 con 0x55ba68d42030 2023-11-06T01:46:41.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e7697db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68d5f2b0 con 0x55ba68d42960 2023-11-06T01:46:41.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 0x55ba68d3f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e637fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d40100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e637fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d40100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54850/0 (socket says 172.21.15.53:54850) 2023-11-06T01:46:41.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e637fe700 1 -- 172.21.15.53:0/1030207474 learned_addr learned my addr 172.21.15.53:0/1030207474 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:41.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 0x55ba68d3f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43410/0 (socket says 172.21.15.53:43410) 2023-11-06T01:46:41.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.142+0000 7f9e68d8a700 1 --2- 172.21.15.53:0/1030207474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68d3a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.143+0000 7f9e63fff700 1 -- 172.21.15.53:0/1030207474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 msgr2=0x55ba68d3a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:41.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.143+0000 7f9e63fff700 1 --2- 172.21.15.53:0/1030207474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68d3a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.143+0000 7f9e63fff700 1 -- 172.21.15.53:0/1030207474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 msgr2=0x55ba68d40100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:41.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.143+0000 7f9e63fff700 1 --2- 172.21.15.53:0/1030207474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d40100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.143+0000 7f9e63fff700 1 -- 172.21.15.53:0/1030207474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba68ade250 con 0x55ba68d41670 2023-11-06T01:46:41.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.143+0000 7f9e63fff700 1 --2- 172.21.15.53:0/1030207474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 0x55ba68d3f7d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9e540058e0 tx=0x7f9e54021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db333fd2967cdd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:41.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.143+0000 7f9e4affd700 1 -- 172.21.15.53:0/1030207474 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5401d040 con 0x55ba68d41670 2023-11-06T01:46:41.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.143+0000 7f9e4affd700 1 -- 172.21.15.53:0/1030207474 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e54003bb0 con 0x55ba68d41670 2023-11-06T01:46:41.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.143+0000 7f9e4affd700 1 -- 172.21.15.53:0/1030207474 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54025460 con 0x55ba68d41670 2023-11-06T01:46:41.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.144+0000 7f9e7697db80 1 -- 172.21.15.53:0/1030207474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 msgr2=0x55ba68d3f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:41.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.144+0000 7f9e7697db80 1 --2- 172.21.15.53:0/1030207474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 0x55ba68d3f7d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9e540058e0 tx=0x7f9e54021000 comp rx=0 tx=0).stop 2023-11-06T01:46:41.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.144+0000 7f9e7697db80 1 -- 172.21.15.53:0/1030207474 shutdown_connections 2023-11-06T01:46:41.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.144+0000 7f9e7697db80 1 --2- 172.21.15.53:0/1030207474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68d3a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.144+0000 7f9e7697db80 1 --2- 172.21.15.53:0/1030207474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d41670 0x55ba68d3f7d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.144+0000 7f9e7697db80 1 --2- 172.21.15.53:0/1030207474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d42030 0x55ba68d40100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.144+0000 7f9e7697db80 1 -- 172.21.15.53:0/1030207474 >> 172.21.15.53:0/1030207474 conn(0x55ba68c980d0 msgr2=0x55ba68d337e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:41.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.144+0000 7f9e7697db80 1 -- 172.21.15.53:0/1030207474 shutdown_connections 2023-11-06T01:46:41.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.146+0000 7f9e7697db80 1 -- 172.21.15.53:0/1030207474 wait complete. 2023-11-06T01:46:41.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.146+0000 7f9e7697db80 1 -- start start 2023-11-06T01:46:41.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.147+0000 7f9e7697db80 1 Processor -- start 2023-11-06T01:46:41.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d41670 0x55ba68da9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42030 0x55ba68da9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e7697db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68db0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e7697db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68db0a70 con 0x55ba68d42030 2023-11-06T01:46:41.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e7697db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68db0bb0 con 0x55ba68d41670 2023-11-06T01:46:41.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e7697db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba68db0d70 con 0x55ba68d42960 2023-11-06T01:46:41.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e7697db80 10 client.0 _ll_register_callbacks cb 0x7f9e5c003d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:41.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d41670 0x55ba68da9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d41670 0x55ba68da9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54856/0 (socket says 172.21.15.53:54856) 2023-11-06T01:46:41.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e63fff700 1 -- 172.21.15.53:0/3786975948 learned_addr learned my addr 172.21.15.53:0/3786975948 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:41.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e68d8a700 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68db0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e637fe700 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42030 0x55ba68da9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e63fff700 1 -- 172.21.15.53:0/3786975948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 msgr2=0x55ba68db0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:41.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e63fff700 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68db0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e63fff700 1 -- 172.21.15.53:0/3786975948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42030 msgr2=0x55ba68da9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:41.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e63fff700 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42030 0x55ba68da9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.148+0000 7f9e63fff700 1 -- 172.21.15.53:0/3786975948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e64108a40 con 0x55ba68d41670 2023-11-06T01:46:41.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.149+0000 7f9e68d8a700 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68db0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:46:41.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.149+0000 7f9e63fff700 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d41670 0x55ba68da9430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9e540059a0 tx=0x7f9e54005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83376c384a684253 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:41.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.149+0000 7f9e437fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:46:41.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.149+0000 7f9e437fe700 1 -- 172.21.15.53:0/3786975948 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5401d040 con 0x55ba68d41670 2023-11-06T01:46:41.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.149+0000 7f9e437fe700 1 -- 172.21.15.53:0/3786975948 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9e54033bc0 con 0x55ba68d41670 2023-11-06T01:46:41.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.149+0000 7f9e7697db80 1 -- 172.21.15.53:0/3786975948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e5c00f040 con 0x55ba68d41670 2023-11-06T01:46:41.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.149+0000 7f9e437fe700 1 -- 172.21.15.53:0/3786975948 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e54033d80 con 0x55ba68d41670 2023-11-06T01:46:41.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.149+0000 7f9e7697db80 10 client.26351 fetch_fsmap learned FSMap version 48 2023-11-06T01:46:41.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.149+0000 7f9e7697db80 1 -- 172.21.15.53:0/3786975948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55ba68ab62b0 con 0x55ba68d41670 2023-11-06T01:46:41.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.149+0000 7f9e437fe700 1 -- 172.21.15.53:0/3786975948 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e5c00f040 con 0x55ba68d41670 2023-11-06T01:46:41.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.150+0000 7f9e437fe700 1 -- 172.21.15.53:0/3786975948 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 48) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f9e54003d20 con 0x55ba68d41670 2023-11-06T01:46:41.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.150+0000 7f9e7697db80 10 client.26351 fetch_fsmap finished waiting for FSMap version 48 2023-11-06T01:46:41.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.150+0000 7f9e7697db80 10 client.26351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:41.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.150+0000 7f9e7697db80 20 client.26351 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:41.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.150+0000 7f9e7697db80 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ba68db22c0 0x55ba68db46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:41.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.150+0000 7f9e7697db80 4 client.26351 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:46:41.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.150+0000 7f9e7697db80 1 -- 172.21.15.53:0/3786975948 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba68c82d40 con 0x55ba68db22c0 2023-11-06T01:46:41.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.151+0000 7f9e637fe700 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ba68db22c0 0x55ba68db46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:41.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.151+0000 7f9e437fe700 10 client.26351 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:41.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.151+0000 7f9e637fe700 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ba68db22c0 0x55ba68db46e0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c5d5c45d95ccdc server_cookie=ec0347f466c8ae67 in_seq=0 out_seq=0 2023-11-06T01:46:41.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.151+0000 7f9e437fe700 1 -- 172.21.15.53:0/3786975948 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ba68c82d40 con 0x55ba68db22c0 2023-11-06T01:46:41.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.151+0000 7f9e437fe700 10 client.26351 handle_command_reply: tid=0 2023-11-06T01:46:41.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.151+0000 7f9e7697db80 1 client.26351 shutdown 2023-11-06T01:46:41.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e7697db80 1 -- 172.21.15.53:0/3786975948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d41670 msgr2=0x55ba68da9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:41.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e7697db80 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d41670 0x55ba68da9430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9e540059a0 tx=0x7f9e54005700 comp rx=0 tx=0).stop 2023-11-06T01:46:41.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e7697db80 1 -- 172.21.15.53:0/3786975948 shutdown_connections 2023-11-06T01:46:41.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e7697db80 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba68d41670 0x55ba68da9430 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e7697db80 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ba68db22c0 0x55ba68db46e0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e7697db80 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba68d42960 0x55ba68db0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e437fe700 0 client.26351 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:41.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e7697db80 1 --2- 172.21.15.53:0/3786975948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba68d42030 0x55ba68da9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e7697db80 1 -- 172.21.15.53:0/3786975948 >> 172.21.15.53:0/3786975948 conn(0x55ba68c980d0 msgr2=0x55ba68d3a240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e7697db80 1 -- 172.21.15.53:0/3786975948 shutdown_connections 2023-11-06T01:46:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e7697db80 1 -- 172.21.15.53:0/3786975948 wait complete. 2023-11-06T01:46:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.152+0000 7f9e7697db80 20 client.26351 trim_cache size 0 max 16384 2023-11-06T01:46:41.165 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:46:41.165 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:46:41.165 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:46:41.165 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:46:41.165 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:46:41.166 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:46:41.166 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:46:41.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:46:41.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:46:41.166 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:46:41.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/2207432395 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e4c0a0520 msgr2=0x7f9e4c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:41.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e4c0a0520 0x7f9e4c0a29e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009990 tx=0x7f9e5401a040 comp rx=0 tx=0).stop 2023-11-06T01:46:41.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/2207432395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64100ea0 msgr2=0x7f9e64102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:41.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64100ea0 0x7f9e64102580 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800e9a0 tx=0x7f9e5800ecb0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/2207432395 shutdown_connections 2023-11-06T01:46:41.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e4c0a0520 0x7f9e4c0a29e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e640ffcb0 0x7f9e64103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e64101840 0x7f9e64102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 --2- 172.21.15.53:0/2207432395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64100ea0 0x7f9e64102580 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/2207432395 >> 172.21.15.53:0/2207432395 conn(0x7f9e640fb220 msgr2=0x7f9e640fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/2207432395 shutdown_connections 2023-11-06T01:46:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:46:41.153+0000 7f9e6a7ed700 1 -- 172.21.15.53:0/2207432395 wait complete. 2023-11-06T01:46:41.171 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:46:41.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:41 smithi053 ceph-mon[99472]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 17 MiB/s wr, 797 op/s 2023-11-06T01:46:41.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:41 smithi062 ceph-mon[103924]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 17 MiB/s wr, 797 op/s 2023-11-06T01:46:41.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:41 smithi163 ceph-mon[104416]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 17 MiB/s wr, 797 op/s 2023-11-06T01:46:42.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 745.0 s, 1263.8 tps, lat 79.141 ms stddev 86.421 2023-11-06T01:46:42.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:42 smithi163 ceph-mon[104416]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 MiB/s wr, 784 op/s 2023-11-06T01:46:42.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:42 smithi053 ceph-mon[99472]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 MiB/s wr, 784 op/s 2023-11-06T01:46:42.897 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:42 smithi062 ceph-mon[103924]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 MiB/s wr, 784 op/s 2023-11-06T01:46:45.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:45 smithi053 ceph-mon[99472]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2023-11-06T01:46:45.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:45 smithi062 ceph-mon[103924]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2023-11-06T01:46:45.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:45 smithi163 ceph-mon[104416]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2023-11-06T01:46:47.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 750.0 s, 1124.8 tps, lat 88.582 ms stddev 162.142 2023-11-06T01:46:47.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:47 smithi053 ceph-mon[99472]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 29 MiB/s wr, 1.53k op/s 2023-11-06T01:46:47.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:47 smithi062 ceph-mon[103924]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 29 MiB/s wr, 1.53k op/s 2023-11-06T01:46:47.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:47 smithi163 ceph-mon[104416]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 29 MiB/s wr, 1.53k op/s 2023-11-06T01:46:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.154+0000 7f3934a0b700 1 -- 172.21.15.53:0/1131620402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3930106a70 msgr2=0x7f3930108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.154+0000 7f3934a0b700 1 --2- 172.21.15.53:0/1131620402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3930106a70 0x7f3930108ee0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f392800b0a0 tx=0x7f392800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.154+0000 7f3934a0b700 1 -- 172.21.15.53:0/1131620402 shutdown_connections 2023-11-06T01:46:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.154+0000 7f3934a0b700 1 --2- 172.21.15.53:0/1131620402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3930106a70 0x7f3930108ee0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.154+0000 7f3934a0b700 1 --2- 172.21.15.53:0/1131620402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3930101810 0x7f3930103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.154+0000 7f3934a0b700 1 --2- 172.21.15.53:0/1131620402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3930104140 0x7f3930106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.154+0000 7f3934a0b700 1 -- 172.21.15.53:0/1131620402 >> 172.21.15.53:0/1131620402 conn(0x7f39300fb1e0 msgr2=0x7f39300fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.154+0000 7f3934a0b700 1 -- 172.21.15.53:0/1131620402 shutdown_connections 2023-11-06T01:46:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.154+0000 7f3934a0b700 1 -- 172.21.15.53:0/1131620402 wait complete. 2023-11-06T01:46:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.155+0000 7f3934a0b700 1 Processor -- start 2023-11-06T01:46:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.155+0000 7f3934a0b700 1 -- start start 2023-11-06T01:46:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.155+0000 7f3934a0b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3930101810 0x7f39300ff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.155+0000 7f3934a0b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3930104140 0x7f39300ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.155+0000 7f3934a0b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3930106a70 0x7f3930100240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.155+0000 7f3934a0b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39301a0bc0 con 0x7f3930101810 2023-11-06T01:46:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.155+0000 7f3934a0b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39301a0d00 con 0x7f3930106a70 2023-11-06T01:46:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.155+0000 7f3934a0b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39301a0e40 con 0x7f3930104140 2023-11-06T01:46:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.155+0000 7f392e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3930101810 0x7f39300ff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3930104140 0x7f39300ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3930101810 0x7f39300ff7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43430/0 (socket says 172.21.15.53:43430) 2023-11-06T01:46:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392e59c700 1 -- 172.21.15.53:0/4175722646 learned_addr learned my addr 172.21.15.53:0/4175722646 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392ed9d700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3930106a70 0x7f3930100240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392e59c700 1 -- 172.21.15.53:0/4175722646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3930104140 msgr2=0x7f39300ffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392e59c700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3930104140 0x7f39300ffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392e59c700 1 -- 172.21.15.53:0/4175722646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3930106a70 msgr2=0x7f3930100240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392e59c700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3930106a70 0x7f3930100240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392e59c700 1 -- 172.21.15.53:0/4175722646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3928009d00 con 0x7f3930101810 2023-11-06T01:46:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392ed9d700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3930106a70 0x7f3930100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:46:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392dd9b700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3930104140 0x7f39300ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:46:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f392e59c700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3930101810 0x7f39300ff7c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f392000e9a0 tx=0x7f392000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f391f7fe700 1 -- 172.21.15.53:0/4175722646 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920009790 con 0x7f3930101810 2023-11-06T01:46:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f391f7fe700 1 -- 172.21.15.53:0/4175722646 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3920010430 con 0x7f3930101810 2023-11-06T01:46:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f3934a0b700 1 -- 172.21.15.53:0/4175722646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39301a10d0 con 0x7f3930101810 2023-11-06T01:46:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f391f7fe700 1 -- 172.21.15.53:0/4175722646 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920018680 con 0x7f3930101810 2023-11-06T01:46:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.156+0000 7f3934a0b700 1 -- 172.21.15.53:0/4175722646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39301a15e0 con 0x7f3930101810 2023-11-06T01:46:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f391f7fe700 1 -- 172.21.15.53:0/4175722646 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f39200105a0 con 0x7f3930101810 2023-11-06T01:46:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f3940b9bb80 1 Processor -- start 2023-11-06T01:46:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f3940b9bb80 1 -- start start 2023-11-06T01:46:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 0x557e39c8daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c8e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8eed0 0x557e39c8f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39be59b0 con 0x557e39c8d680 2023-11-06T01:46:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39be3370 con 0x557e39c8dfe0 2023-11-06T01:46:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39bdec60 con 0x557e39c8eed0 2023-11-06T01:46:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f392ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 0x557e39c8daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f392ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 0x557e39c8daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43436/0 (socket says 172.21.15.53:43436) 2023-11-06T01:46:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.161+0000 7f392ed9d700 1 -- 172.21.15.53:0/2130209443 learned_addr learned my addr 172.21.15.53:0/2130209443 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f391f7fe700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39180a05a0 0x7f39180a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f392ed9d700 1 -- 172.21.15.53:0/2130209443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8eed0 msgr2=0x557e39c8f350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:46:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f392ed9d700 1 --2- 172.21.15.53:0/2130209443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8eed0 0x557e39c8f350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f392ed9d700 1 -- 172.21.15.53:0/2130209443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 msgr2=0x557e39c8e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f392ed9d700 1 --2- 172.21.15.53:0/2130209443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c8e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f392ed9d700 1 -- 172.21.15.53:0/2130209443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e39a0cd80 con 0x557e39c8d680 2023-11-06T01:46:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f391f7fe700 1 -- 172.21.15.53:0/4175722646 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3920014070 con 0x7f3930101810 2023-11-06T01:46:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f392ed9d700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39180a05a0 0x7f39180a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f392ed9d700 1 --2- 172.21.15.53:0/2130209443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 0x557e39c8daa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f392800b0a0 tx=0x7f3928009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0952fffe2bc1f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f391cff9700 1 -- 172.21.15.53:0/2130209443 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392800e040 con 0x557e39c8d680 2023-11-06T01:46:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f391cff9700 1 -- 172.21.15.53:0/2130209443 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f392800be90 con 0x557e39c8d680 2023-11-06T01:46:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.162+0000 7f391cff9700 1 -- 172.21.15.53:0/2130209443 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39280059f0 con 0x557e39c8d680 2023-11-06T01:46:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.163+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2130209443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 msgr2=0x557e39c8daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.163+0000 7f392ed9d700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39180a05a0 0x7f39180a2a60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f392802b000 tx=0x7f392802c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.163+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/2130209443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 0x557e39c8daa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f392800b0a0 tx=0x7f3928009340 comp rx=0 tx=0).stop 2023-11-06T01:46:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.163+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2130209443 shutdown_connections 2023-11-06T01:46:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.163+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/2130209443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8eed0 0x557e39c8f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.163+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/2130209443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 0x557e39c8daa0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.163+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/2130209443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c8e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.163+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2130209443 >> 172.21.15.53:0/2130209443 conn(0x557e39bed070 msgr2=0x557e39c8a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.163+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2130209443 shutdown_connections 2023-11-06T01:46:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.163+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2130209443 wait complete. 2023-11-06T01:46:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f3940b9bb80 1 -- start start 2023-11-06T01:46:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f3940b9bb80 1 Processor -- start 2023-11-06T01:46:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 0x557e39c97580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c97ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8eed0 0x557e39c9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39b7b4b0 con 0x557e39c8d680 2023-11-06T01:46:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39b7b150 con 0x557e39c8dfe0 2023-11-06T01:46:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39a628c0 con 0x557e39c8eed0 2023-11-06T01:46:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f3940b9bb80 10 client.0 _ll_register_callbacks cb 0x557e39ac8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f392dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c97ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f392dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c97ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54884/0 (socket says 172.21.15.53:54884) 2023-11-06T01:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f392e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 0x557e39c97580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f392dd9b700 1 -- 172.21.15.53:0/2849829129 learned_addr learned my addr 172.21.15.53:0/2849829129 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f392ed9d700 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8eed0 0x557e39c9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f392dd9b700 1 -- 172.21.15.53:0/2849829129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8eed0 msgr2=0x557e39c9eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.164+0000 7f392dd9b700 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8eed0 0x557e39c9eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.165+0000 7f392dd9b700 1 -- 172.21.15.53:0/2849829129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 msgr2=0x557e39c97580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.165+0000 7f392dd9b700 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 0x557e39c97580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.165+0000 7f392dd9b700 1 -- 172.21.15.53:0/2849829129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3930109ad0 con 0x557e39c8dfe0 2023-11-06T01:46:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.165+0000 7f392dd9b700 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c97ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3924011a00 tx=0x7f3924011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d2e79eb17d4db4a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.165+0000 7f3915ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.165+0000 7f3915ffb700 1 -- 172.21.15.53:0/2849829129 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392400fd00 con 0x557e39c8dfe0 2023-11-06T01:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.165+0000 7f3915ffb700 1 -- 172.21.15.53:0/2849829129 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f392400fe60 con 0x557e39c8dfe0 2023-11-06T01:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.165+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2849829129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e39c09cb0 con 0x557e39c8dfe0 2023-11-06T01:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.165+0000 7f3915ffb700 1 -- 172.21.15.53:0/2849829129 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3924019430 con 0x557e39c8dfe0 2023-11-06T01:46:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.167+0000 7f3940b9bb80 10 client.26360 fetch_fsmap learned FSMap version 48 2023-11-06T01:46:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.167+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2849829129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x557e399f5c00 con 0x557e39c8dfe0 2023-11-06T01:46:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.168+0000 7f3915ffb700 1 -- 172.21.15.53:0/2849829129 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e39c09cb0 con 0x557e39c8dfe0 2023-11-06T01:46:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.168+0000 7f3915ffb700 1 -- 172.21.15.53:0/2849829129 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 48) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f3924021070 con 0x557e39c8dfe0 2023-11-06T01:46:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.168+0000 7f3940b9bb80 10 client.26360 fetch_fsmap finished waiting for FSMap version 48 2023-11-06T01:46:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.168+0000 7f3940b9bb80 10 client.26360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.168+0000 7f3940b9bb80 20 client.26360 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.168+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557e39ca3340 0x557e39ca5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.168+0000 7f3940b9bb80 4 client.26360 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:46:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.168+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2849829129 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e39c5cdb0 con 0x557e39ca3340 2023-11-06T01:46:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.168+0000 7f392e59c700 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557e39ca3340 0x557e39ca5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.169+0000 7f3915ffb700 10 client.26360 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.169+0000 7f392e59c700 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557e39ca3340 0x557e39ca5760 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bfd8ad3d554fba5 server_cookie=692f1c4903193c3d in_seq=0 out_seq=0 2023-11-06T01:46:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3915ffb700 1 -- 172.21.15.53:0/2849829129 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557e39c5cdb0 con 0x557e39ca3340 2023-11-06T01:46:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3915ffb700 10 client.26360 handle_command_reply: tid=0 2023-11-06T01:46:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 1 client.26360 shutdown 2023-11-06T01:46:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2849829129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 msgr2=0x557e39c97ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c97ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3924011a00 tx=0x7f3924011d10 comp rx=0 tx=0).stop 2023-11-06T01:46:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2849829129 shutdown_connections 2023-11-06T01:46:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557e39ca3340 0x557e39ca5760 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8eed0 0x557e39c9eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3915ffb700 0 client.26360 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8d680 0x557e39c97580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/2849829129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c97ac0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2849829129 >> 172.21.15.53:0/2849829129 conn(0x557e39bed070 msgr2=0x557e39c883c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2849829129 shutdown_connections 2023-11-06T01:46:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 1 -- 172.21.15.53:0/2849829129 wait complete. 2023-11-06T01:46:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.170+0000 7f3940b9bb80 20 client.26360 trim_cache size 0 max 16384 2023-11-06T01:46:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.190+0000 7f3940b9bb80 1 Processor -- start 2023-11-06T01:46:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f3940b9bb80 1 -- start start 2023-11-06T01:46:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8d680 0x557e39c9cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c9f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8eed0 0x557e39c961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39c809f0 con 0x557e39c8eed0 2023-11-06T01:46:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39c83500 con 0x557e39c8dfe0 2023-11-06T01:46:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39c970f0 con 0x557e39c8d680 2023-11-06T01:46:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c9f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8d680 0x557e39c9cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c9f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54892/0 (socket says 172.21.15.53:54892) 2023-11-06T01:46:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8d680 0x557e39c9cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57692/0 (socket says 172.21.15.53:57692) 2023-11-06T01:46:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392dd9b700 1 -- 172.21.15.53:0/1375165643 learned_addr learned my addr 172.21.15.53:0/1375165643 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392ed9d700 1 --2- 172.21.15.53:0/1375165643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8eed0 0x557e39c961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392e59c700 1 -- 172.21.15.53:0/1375165643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 msgr2=0x557e39c9f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392e59c700 1 --2- 172.21.15.53:0/1375165643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c9f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392e59c700 1 -- 172.21.15.53:0/1375165643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8eed0 msgr2=0x557e39c961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392e59c700 1 --2- 172.21.15.53:0/1375165643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8eed0 0x557e39c961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392e59c700 1 -- 172.21.15.53:0/1375165643 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e39a331f0 con 0x557e39c8d680 2023-11-06T01:46:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.191+0000 7f392dd9b700 1 --2- 172.21.15.53:0/1375165643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c9f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:46:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.192+0000 7f392e59c700 1 --2- 172.21.15.53:0/1375165643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8d680 0x557e39c9cab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f39200958b0 tx=0x7f3920010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69611c3927b373 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.192+0000 7f391d7fa700 1 -- 172.21.15.53:0/1375165643 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39200840b0 con 0x557e39c8d680 2023-11-06T01:46:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.192+0000 7f391d7fa700 1 -- 172.21.15.53:0/1375165643 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3920099ab0 con 0x557e39c8d680 2023-11-06T01:46:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.192+0000 7f391d7fa700 1 -- 172.21.15.53:0/1375165643 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920010880 con 0x557e39c8d680 2023-11-06T01:46:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.193+0000 7f3940b9bb80 1 -- 172.21.15.53:0/1375165643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8d680 msgr2=0x557e39c9cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.193+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/1375165643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8d680 0x557e39c9cab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f39200958b0 tx=0x7f3920010c80 comp rx=0 tx=0).stop 2023-11-06T01:46:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.193+0000 7f3940b9bb80 1 -- 172.21.15.53:0/1375165643 shutdown_connections 2023-11-06T01:46:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.193+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/1375165643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8eed0 0x557e39c961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.193+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/1375165643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8d680 0x557e39c9cab0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.193+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/1375165643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8dfe0 0x557e39c9f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.193+0000 7f3940b9bb80 1 -- 172.21.15.53:0/1375165643 >> 172.21.15.53:0/1375165643 conn(0x557e39bed070 msgr2=0x557e39c920a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.193+0000 7f3940b9bb80 1 -- 172.21.15.53:0/1375165643 shutdown_connections 2023-11-06T01:46:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.193+0000 7f3940b9bb80 1 -- 172.21.15.53:0/1375165643 wait complete. 2023-11-06T01:46:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.193+0000 7f3940b9bb80 1 -- start start 2023-11-06T01:46:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.193+0000 7f3940b9bb80 1 Processor -- start 2023-11-06T01:46:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8d680 0x557e39ced060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8dfe0 0x557e39ced5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f3940b9bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8eed0 0x557e39cf3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39cf44c0 con 0x557e39c8eed0 2023-11-06T01:46:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39cf4600 con 0x557e39c8d680 2023-11-06T01:46:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f3940b9bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e39cf47c0 con 0x557e39c8dfe0 2023-11-06T01:46:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f3940b9bb80 10 client.0 _ll_register_callbacks cb 0x7f39240114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:46:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8d680 0x557e39ced060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8dfe0 0x557e39ced5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8d680 0x557e39ced060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54896/0 (socket says 172.21.15.53:54896) 2023-11-06T01:46:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8dfe0 0x557e39ced5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57700/0 (socket says 172.21.15.53:57700) 2023-11-06T01:46:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392e59c700 1 -- 172.21.15.53:0/596070526 learned_addr learned my addr 172.21.15.53:0/596070526 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:46:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392ed9d700 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8eed0 0x557e39cf3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392e59c700 1 -- 172.21.15.53:0/596070526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8dfe0 msgr2=0x557e39ced5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392e59c700 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8dfe0 0x557e39ced5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392e59c700 1 -- 172.21.15.53:0/596070526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8eed0 msgr2=0x557e39cf3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:46:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392e59c700 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8eed0 0x557e39cf3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392e59c700 1 -- 172.21.15.53:0/596070526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e399f5c00 con 0x557e39c8d680 2023-11-06T01:46:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392dd9b700 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8dfe0 0x557e39ced5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:46:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f392e59c700 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8d680 0x557e39ced060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3920009230 tx=0x7f39200956f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e30ccf192fc291c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:46:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.194+0000 7f3915ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:46:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.195+0000 7f3915ffb700 1 -- 172.21.15.53:0/596070526 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39200840b0 con 0x557e39c8d680 2023-11-06T01:46:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.195+0000 7f3915ffb700 1 -- 172.21.15.53:0/596070526 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f392009d1e0 con 0x557e39c8d680 2023-11-06T01:46:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.196+0000 7f3940b9bb80 1 -- 172.21.15.53:0/596070526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f392800be90 con 0x557e39c8d680 2023-11-06T01:46:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.196+0000 7f3915ffb700 1 -- 172.21.15.53:0/596070526 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39200651f0 con 0x557e39c8d680 2023-11-06T01:46:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.196+0000 7f3940b9bb80 10 client.26366 fetch_fsmap learned FSMap version 48 2023-11-06T01:46:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.196+0000 7f3940b9bb80 1 -- 172.21.15.53:0/596070526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f39301a15e0 con 0x557e39c8d680 2023-11-06T01:46:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.197+0000 7f3915ffb700 1 -- 172.21.15.53:0/596070526 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f392800be90 con 0x557e39c8d680 2023-11-06T01:46:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.197+0000 7f3915ffb700 1 -- 172.21.15.53:0/596070526 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 48) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f39200a1020 con 0x557e39c8d680 2023-11-06T01:46:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.197+0000 7f3940b9bb80 10 client.26366 fetch_fsmap finished waiting for FSMap version 48 2023-11-06T01:46:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.197+0000 7f3940b9bb80 10 client.26366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:46:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.197+0000 7f3940b9bb80 20 client.26366 populate_metadata read hostname 'smithi053' 2023-11-06T01:46:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.197+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557e39cf5d20 0x557e39cf8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:46:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.197+0000 7f3940b9bb80 4 client.26366 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:46:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.197+0000 7f3940b9bb80 1 -- 172.21.15.53:0/596070526 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557e39bd7ce0 con 0x557e39cf5d20 2023-11-06T01:46:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.198+0000 7f392dd9b700 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557e39cf5d20 0x557e39cf8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:46:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.198+0000 7f3915ffb700 10 client.26366 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.198+0000 7f392dd9b700 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557e39cf5d20 0x557e39cf8140 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d7a7337ee66bf7 server_cookie=a9647a9761c9b13d in_seq=0 out_seq=0 2023-11-06T01:46:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3915ffb700 1 -- 172.21.15.53:0/596070526 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557e39bd7ce0 con 0x557e39cf5d20 2023-11-06T01:46:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3915ffb700 10 client.26366 handle_command_reply: tid=0 2023-11-06T01:46:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3940b9bb80 1 client.26366 shutdown 2023-11-06T01:46:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3940b9bb80 1 -- 172.21.15.53:0/596070526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8d680 msgr2=0x557e39ced060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8d680 0x557e39ced060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3920009230 tx=0x7f39200956f0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3940b9bb80 1 -- 172.21.15.53:0/596070526 shutdown_connections 2023-11-06T01:46:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e39c8d680 0x557e39ced060 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x557e39cf5d20 0x557e39cf8140 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e39c8eed0 0x557e39cf3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3940b9bb80 1 --2- 172.21.15.53:0/596070526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557e39c8dfe0 0x557e39ced5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3915ffb700 0 client.26366 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:46:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3940b9bb80 1 -- 172.21.15.53:0/596070526 >> 172.21.15.53:0/596070526 conn(0x557e39bed070 msgr2=0x557e39c9f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3940b9bb80 1 -- 172.21.15.53:0/596070526 shutdown_connections 2023-11-06T01:46:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.206+0000 7f3940b9bb80 1 -- 172.21.15.53:0/596070526 wait complete. 2023-11-06T01:46:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3940b9bb80 20 client.26366 trim_cache size 0 max 16384 2023-11-06T01:46:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:46:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:46:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:46:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:46:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 -- 172.21.15.53:0/4175722646 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39180a05a0 msgr2=0x7f39180a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39180a05a0 0x7f39180a2a60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f392802b000 tx=0x7f392802c000 comp rx=0 tx=0).stop 2023-11-06T01:46:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 -- 172.21.15.53:0/4175722646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3930101810 msgr2=0x7f39300ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:46:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3930101810 0x7f39300ff7c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f392000e9a0 tx=0x7f392000ecb0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 -- 172.21.15.53:0/4175722646 shutdown_connections 2023-11-06T01:46:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f39180a05a0 0x7f39180a2a60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3930106a70 0x7f3930100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3930101810 0x7f39300ff7c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 --2- 172.21.15.53:0/4175722646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3930104140 0x7f39300ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:46:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 -- 172.21.15.53:0/4175722646 >> 172.21.15.53:0/4175722646 conn(0x7f39300fb1e0 msgr2=0x7f3930102500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:46:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 -- 172.21.15.53:0/4175722646 shutdown_connections 2023-11-06T01:46:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:46:48.207+0000 7f3934a0b700 1 -- 172.21.15.53:0/4175722646 wait complete. 2023-11-06T01:46:49.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:49 smithi053 ceph-mon[99472]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 1.87k op/s 2023-11-06T01:46:49.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:49 smithi062 ceph-mon[103924]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 1.87k op/s 2023-11-06T01:46:49.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:49 smithi163 ceph-mon[104416]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 1.87k op/s 2023-11-06T01:46:50.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:50 smithi163 ceph-mon[104416]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2023-11-06T01:46:50.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:50 smithi053 ceph-mon[99472]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2023-11-06T01:46:50.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:50 smithi062 ceph-mon[103924]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2023-11-06T01:46:52.182 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 755.0 s, 1512.2 tps, lat 66.443 ms stddev 75.183 2023-11-06T01:46:52.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:52 smithi163 ceph-mon[104416]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 1.80k op/s 2023-11-06T01:46:52.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:52 smithi053 ceph-mon[99472]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 1.80k op/s 2023-11-06T01:46:52.901 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:52 smithi062 ceph-mon[103924]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 1.80k op/s 2023-11-06T01:46:55.351 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:55 smithi053 ceph-mon[99472]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.77k op/s 2023-11-06T01:46:55.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:55 smithi062 ceph-mon[103924]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.77k op/s 2023-11-06T01:46:55.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:55 smithi163 ceph-mon[104416]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.77k op/s 2023-11-06T01:46:57.186 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 760.0 s, 1473.2 tps, lat 67.427 ms stddev 72.505 2023-11-06T01:46:57.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:57 smithi053 ceph-mon[99472]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 MiB/s wr, 1.50k op/s 2023-11-06T01:46:57.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:46:57.348 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:46:57.349 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:57 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:46:57.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:57 smithi062 ceph-mon[103924]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 MiB/s wr, 1.50k op/s 2023-11-06T01:46:57.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:46:57.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:46:57.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:57 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:46:57.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:57 smithi163 ceph-mon[104416]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 MiB/s wr, 1.50k op/s 2023-11-06T01:46:57.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:46:57.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:46:57.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:57 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:46:58.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:46:58.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:46:58.345 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:58 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:46:58.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:58 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:46:58.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:58 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:46:58.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:58 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:46:58.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:58 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:46:58.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:58 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:46:58.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:58 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:46:59.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:46:59 smithi053 ceph-mon[99472]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2023-11-06T01:46:59.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:46:59 smithi062 ceph-mon[103924]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2023-11-06T01:46:59.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:46:59 smithi163 ceph-mon[104416]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2023-11-06T01:47:01.351 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:01 smithi053 ceph-mon[99472]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 596 op/s 2023-11-06T01:47:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.378+0000 7efeca0b7700 1 -- 172.21.15.53:0/951553186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4104990 msgr2=0x7efec4104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.378+0000 7efeca0b7700 1 --2- 172.21.15.53:0/951553186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4104990 0x7efec4104e10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efebc00b0a0 tx=0x7efebc00b3b0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.380+0000 7efeca0b7700 1 -- 172.21.15.53:0/951553186 shutdown_connections 2023-11-06T01:47:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.380+0000 7efeca0b7700 1 --2- 172.21.15.53:0/951553186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4104990 0x7efec4104e10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.380+0000 7efeca0b7700 1 --2- 172.21.15.53:0/951553186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec40ff650 0x7efec4101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.380+0000 7efeca0b7700 1 --2- 172.21.15.53:0/951553186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4101fc0 0x7efec4104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.380+0000 7efeca0b7700 1 -- 172.21.15.53:0/951553186 >> 172.21.15.53:0/951553186 conn(0x7efec40fb250 msgr2=0x7efec40fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.389+0000 7efeca0b7700 1 -- 172.21.15.53:0/951553186 shutdown_connections 2023-11-06T01:47:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.389+0000 7efeca0b7700 1 -- 172.21.15.53:0/951553186 wait complete. 2023-11-06T01:47:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.390+0000 7efeca0b7700 1 Processor -- start 2023-11-06T01:47:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.390+0000 7efeca0b7700 1 -- start start 2023-11-06T01:47:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.391+0000 7efeca0b7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec40ff650 0x7efec406f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.391+0000 7efeca0b7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec4101fc0 0x7efec406fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.391+0000 7efeca0b7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4104990 0x7efec4070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.391+0000 7efeca0b7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4071220 con 0x7efec40ff650 2023-11-06T01:47:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.391+0000 7efeca0b7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4071390 con 0x7efec4104990 2023-11-06T01:47:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.391+0000 7efeca0b7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4071550 con 0x7efec4101fc0 2023-11-06T01:47:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.391+0000 7efec3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4104990 0x7efec4070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.391+0000 7efec3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4104990 0x7efec4070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54910/0 (socket says 172.21.15.53:54910) 2023-11-06T01:47:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.391+0000 7efec3fff700 1 -- 172.21.15.53:0/2918775065 learned_addr learned my addr 172.21.15.53:0/2918775065 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:01.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:01 smithi062 ceph-mon[103924]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 596 op/s 2023-11-06T01:47:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.395+0000 7efec37fe700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec40ff650 0x7efec406f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.395+0000 7efec37fe700 1 -- 172.21.15.53:0/2918775065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec4101fc0 msgr2=0x7efec406fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.395+0000 7efec37fe700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec4101fc0 0x7efec406fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.395+0000 7efec37fe700 1 -- 172.21.15.53:0/2918775065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4104990 msgr2=0x7efec4070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.395+0000 7efec37fe700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4104990 0x7efec4070bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.395+0000 7efec37fe700 1 -- 172.21.15.53:0/2918775065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc009d00 con 0x7efec40ff650 2023-11-06T01:47:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.396+0000 7efec37fe700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec40ff650 0x7efec406f7b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7efeb400d950 tx=0x7efeb400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.396+0000 7efec0ff9700 1 -- 172.21.15.53:0/2918775065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4009b40 con 0x7efec40ff650 2023-11-06T01:47:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.396+0000 7efeca0b7700 1 -- 172.21.15.53:0/2918775065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec4071830 con 0x7efec40ff650 2023-11-06T01:47:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.396+0000 7efeca0b7700 1 -- 172.21.15.53:0/2918775065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec4072060 con 0x7efec40ff650 2023-11-06T01:47:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.397+0000 7efec0ff9700 1 -- 172.21.15.53:0/2918775065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb400be10 con 0x7efec40ff650 2023-11-06T01:47:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.397+0000 7efec0ff9700 1 -- 172.21.15.53:0/2918775065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb400f620 con 0x7efec40ff650 2023-11-06T01:47:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.403+0000 7efec0ff9700 1 -- 172.21.15.53:0/2918775065 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7efeb4009ca0 con 0x7efec40ff650 2023-11-06T01:47:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.404+0000 7efec0ff9700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efeac0a0670 0x7efeac0a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.404+0000 7efec0ff9700 1 -- 172.21.15.53:0/2918775065 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efeb40cb610 con 0x7efec40ff650 2023-11-06T01:47:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.404+0000 7efec2ffd700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efeac0a0670 0x7efeac0a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.417+0000 7efec2ffd700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efeac0a0670 0x7efeac0a2b30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efec4070800 tx=0x7efeb8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.424+0000 7efed6247b80 1 Processor -- start 2023-11-06T01:47:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.424+0000 7efed6247b80 1 -- start start 2023-11-06T01:47:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.424+0000 7efed6247b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592680 0x55ad2a592aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.425+0000 7efed6247b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592fe0 0x55ad2a5938f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.425+0000 7efed6247b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a593ed0 0x55ad2a594350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.425+0000 7efed6247b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a4ea9b0 con 0x55ad2a592680 2023-11-06T01:47:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.425+0000 7efed6247b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a4e8370 con 0x55ad2a593ed0 2023-11-06T01:47:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.425+0000 7efed6247b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a4e3c60 con 0x55ad2a592fe0 2023-11-06T01:47:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.426+0000 7efec2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592fe0 0x55ad2a5938f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.426+0000 7efec2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592fe0 0x55ad2a5938f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57724/0 (socket says 172.21.15.53:57724) 2023-11-06T01:47:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.426+0000 7efec2ffd700 1 -- 172.21.15.53:0/1704107896 learned_addr learned my addr 172.21.15.53:0/1704107896 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.427+0000 7efec2ffd700 1 -- 172.21.15.53:0/1704107896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a593ed0 msgr2=0x55ad2a594350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.427+0000 7efec2ffd700 1 --2- 172.21.15.53:0/1704107896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a593ed0 0x55ad2a594350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.427+0000 7efec2ffd700 1 -- 172.21.15.53:0/1704107896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592680 msgr2=0x55ad2a592aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.427+0000 7efec2ffd700 1 --2- 172.21.15.53:0/1704107896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592680 0x55ad2a592aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.427+0000 7efec2ffd700 1 -- 172.21.15.53:0/1704107896 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad2a311d80 con 0x55ad2a592fe0 2023-11-06T01:47:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.428+0000 7efec2ffd700 1 --2- 172.21.15.53:0/1704107896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592fe0 0x55ad2a5938f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efeb8015910 tx=0x7efeb8006f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df499ffcb82a2ba9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.434+0000 7efea9ffb700 1 -- 172.21.15.53:0/1704107896 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb801a070 con 0x55ad2a592fe0 2023-11-06T01:47:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.434+0000 7efea9ffb700 1 -- 172.21.15.53:0/1704107896 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb80129d0 con 0x55ad2a592fe0 2023-11-06T01:47:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.434+0000 7efea9ffb700 1 -- 172.21.15.53:0/1704107896 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb8013910 con 0x55ad2a592fe0 2023-11-06T01:47:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.435+0000 7efed6247b80 1 -- 172.21.15.53:0/1704107896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592fe0 msgr2=0x55ad2a5938f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.435+0000 7efed6247b80 1 --2- 172.21.15.53:0/1704107896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592fe0 0x55ad2a5938f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efeb8015910 tx=0x7efeb8006f90 comp rx=0 tx=0).stop 2023-11-06T01:47:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.435+0000 7efed6247b80 1 -- 172.21.15.53:0/1704107896 shutdown_connections 2023-11-06T01:47:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.435+0000 7efed6247b80 1 --2- 172.21.15.53:0/1704107896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a593ed0 0x55ad2a594350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.435+0000 7efed6247b80 1 --2- 172.21.15.53:0/1704107896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592680 0x55ad2a592aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.435+0000 7efed6247b80 1 --2- 172.21.15.53:0/1704107896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592fe0 0x55ad2a5938f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.435+0000 7efed6247b80 1 -- 172.21.15.53:0/1704107896 >> 172.21.15.53:0/1704107896 conn(0x55ad2a4f2070 msgr2=0x55ad2a58f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.435+0000 7efed6247b80 1 -- 172.21.15.53:0/1704107896 shutdown_connections 2023-11-06T01:47:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.435+0000 7efed6247b80 1 -- 172.21.15.53:0/1704107896 wait complete. 2023-11-06T01:47:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.435+0000 7efed6247b80 1 -- start start 2023-11-06T01:47:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efed6247b80 1 Processor -- start 2023-11-06T01:47:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efed6247b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592680 0x55ad2a5ad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efed6247b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592fe0 0x55ad2a5adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efed6247b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a593ed0 0x55ad2a5b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efed6247b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a4804b0 con 0x55ad2a592fe0 2023-11-06T01:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efed6247b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a480150 con 0x55ad2a592680 2023-11-06T01:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efed6247b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a3678c0 con 0x55ad2a593ed0 2023-11-06T01:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efed6247b80 10 client.0 _ll_register_callbacks cb 0x55ad2a3cda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efec2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592fe0 0x55ad2a5adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efec2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592fe0 0x55ad2a5adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43486/0 (socket says 172.21.15.53:43486) 2023-11-06T01:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efec2ffd700 1 -- 172.21.15.53:0/598177795 learned_addr learned my addr 172.21.15.53:0/598177795 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efec37fe700 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592680 0x55ad2a5ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.436+0000 7efec3fff700 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a593ed0 0x55ad2a5b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efec2ffd700 1 -- 172.21.15.53:0/598177795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a593ed0 msgr2=0x55ad2a5b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efec2ffd700 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a593ed0 0x55ad2a5b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efec2ffd700 1 -- 172.21.15.53:0/598177795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592680 msgr2=0x55ad2a5ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efec2ffd700 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592680 0x55ad2a5ad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efec2ffd700 1 -- 172.21.15.53:0/598177795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb80155f0 con 0x55ad2a592fe0 2023-11-06T01:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efec37fe700 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592680 0x55ad2a5ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efec2ffd700 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592fe0 0x55ad2a5adaa0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7efeb8006ba0 tx=0x7efeb80153d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb6c5a633fb6fdd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efea2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efea2ffd700 1 -- 172.21.15.53:0/598177795 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb801a030 con 0x55ad2a592fe0 2023-11-06T01:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efea2ffd700 1 -- 172.21.15.53:0/598177795 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efeb8013670 con 0x55ad2a592fe0 2023-11-06T01:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efed6247b80 1 -- 172.21.15.53:0/598177795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad2a50ecb0 con 0x55ad2a592fe0 2023-11-06T01:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efea2ffd700 1 -- 172.21.15.53:0/598177795 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeb8013830 con 0x55ad2a592fe0 2023-11-06T01:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efed6247b80 10 client.16872 fetch_fsmap learned FSMap version 48 2023-11-06T01:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.437+0000 7efed6247b80 1 -- 172.21.15.53:0/598177795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55ad2a2fac00 con 0x55ad2a592fe0 2023-11-06T01:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.438+0000 7efea2ffd700 1 -- 172.21.15.53:0/598177795 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad2a50ecb0 con 0x55ad2a592fe0 2023-11-06T01:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.438+0000 7efea2ffd700 1 -- 172.21.15.53:0/598177795 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 48) v1 ==== 7458+0+0 (secure 0 0 0) 0x7efeb8006680 con 0x55ad2a592fe0 2023-11-06T01:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.438+0000 7efed6247b80 10 client.16872 fetch_fsmap finished waiting for FSMap version 48 2023-11-06T01:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.438+0000 7efed6247b80 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.438+0000 7efed6247b80 20 client.16872 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.438+0000 7efed6247b80 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad2a5b9280 0x55ad2a5bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.438+0000 7efed6247b80 4 client.16872 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.438+0000 7efed6247b80 1 -- 172.21.15.53:0/598177795 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad2a561db0 con 0x55ad2a5b9280 2023-11-06T01:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.438+0000 7efec37fe700 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad2a5b9280 0x55ad2a5bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.438+0000 7efea2ffd700 10 client.16872 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.438+0000 7efec37fe700 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad2a5b9280 0x55ad2a5bb6a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62a4a99ac68d481 server_cookie=db3562a3aeeb5d8 in_seq=0 out_seq=0 2023-11-06T01:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efea2ffd700 1 -- 172.21.15.53:0/598177795 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ad2a561db0 con 0x55ad2a5b9280 2023-11-06T01:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efea2ffd700 10 client.16872 handle_command_reply: tid=0 2023-11-06T01:47:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efed6247b80 1 client.16872 shutdown 2023-11-06T01:47:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efed6247b80 1 -- 172.21.15.53:0/598177795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592fe0 msgr2=0x55ad2a5adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efed6247b80 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592fe0 0x55ad2a5adaa0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7efeb8006ba0 tx=0x7efeb80153d0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efed6247b80 1 -- 172.21.15.53:0/598177795 shutdown_connections 2023-11-06T01:47:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efed6247b80 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592680 0x55ad2a5ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efed6247b80 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad2a5b9280 0x55ad2a5bb6a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efed6247b80 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a593ed0 0x55ad2a5b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efea2ffd700 0 client.16872 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efed6247b80 1 --2- 172.21.15.53:0/598177795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592fe0 0x55ad2a5adaa0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.439+0000 7efed6247b80 1 -- 172.21.15.53:0/598177795 >> 172.21.15.53:0/598177795 conn(0x55ad2a4f2070 msgr2=0x55ad2a58d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.440+0000 7efed6247b80 1 -- 172.21.15.53:0/598177795 shutdown_connections 2023-11-06T01:47:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.440+0000 7efed6247b80 1 -- 172.21.15.53:0/598177795 wait complete. 2023-11-06T01:47:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.440+0000 7efed6247b80 20 client.16872 trim_cache size 0 max 16384 2023-11-06T01:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.455+0000 7efed6247b80 1 Processor -- start 2023-11-06T01:47:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.455+0000 7efed6247b80 1 -- start start 2023-11-06T01:47:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.455+0000 7efed6247b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592680 0x55ad2a58e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.455+0000 7efed6247b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 0x55ad2a5b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.455+0000 7efed6247b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a593ed0 0x55ad2a5acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.455+0000 7efed6247b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a5ae460 con 0x55ad2a593ed0 2023-11-06T01:47:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.455+0000 7efed6247b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a588500 con 0x55ad2a592fe0 2023-11-06T01:47:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.455+0000 7efed6247b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a5abfa0 con 0x55ad2a592680 2023-11-06T01:47:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec37fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592680 0x55ad2a58e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 0x55ad2a5b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a593ed0 0x55ad2a5acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 0x55ad2a5b29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54934/0 (socket says 172.21.15.53:54934) 2023-11-06T01:47:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec37fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592680 0x55ad2a58e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57734/0 (socket says 172.21.15.53:57734) 2023-11-06T01:47:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec2ffd700 1 -- 172.21.15.53:0/20785262 learned_addr learned my addr 172.21.15.53:0/20785262 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec2ffd700 1 -- 172.21.15.53:0/20785262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592680 msgr2=0x55ad2a58e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec2ffd700 1 --2- 172.21.15.53:0/20785262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592680 0x55ad2a58e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec2ffd700 1 -- 172.21.15.53:0/20785262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a593ed0 msgr2=0x55ad2a5acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec2ffd700 1 --2- 172.21.15.53:0/20785262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a593ed0 0x55ad2a5acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec2ffd700 1 -- 172.21.15.53:0/20785262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad2a3381f0 con 0x55ad2a592fe0 2023-11-06T01:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec37fe700 1 --2- 172.21.15.53:0/20785262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592680 0x55ad2a58e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efec2ffd700 1 --2- 172.21.15.53:0/20785262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 0x55ad2a5b29f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efeb80069e0 tx=0x7efeb800a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de547bcf7fcd3f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efeaa7fc700 1 -- 172.21.15.53:0/20785262 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb801a040 con 0x55ad2a592fe0 2023-11-06T01:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efeaa7fc700 1 -- 172.21.15.53:0/20785262 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb801b040 con 0x55ad2a592fe0 2023-11-06T01:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.456+0000 7efeaa7fc700 1 -- 172.21.15.53:0/20785262 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb8032880 con 0x55ad2a592fe0 2023-11-06T01:47:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.457+0000 7efed6247b80 1 -- 172.21.15.53:0/20785262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 msgr2=0x55ad2a5b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.457+0000 7efed6247b80 1 --2- 172.21.15.53:0/20785262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 0x55ad2a5b29f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efeb80069e0 tx=0x7efeb800a040 comp rx=0 tx=0).stop 2023-11-06T01:47:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.457+0000 7efed6247b80 1 -- 172.21.15.53:0/20785262 shutdown_connections 2023-11-06T01:47:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.457+0000 7efed6247b80 1 --2- 172.21.15.53:0/20785262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a593ed0 0x55ad2a5acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.457+0000 7efed6247b80 1 --2- 172.21.15.53:0/20785262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a592680 0x55ad2a58e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.457+0000 7efed6247b80 1 --2- 172.21.15.53:0/20785262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 0x55ad2a5b29f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.457+0000 7efed6247b80 1 -- 172.21.15.53:0/20785262 >> 172.21.15.53:0/20785262 conn(0x55ad2a4f2070 msgr2=0x55ad2a58ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.457+0000 7efed6247b80 1 -- 172.21.15.53:0/20785262 shutdown_connections 2023-11-06T01:47:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.457+0000 7efed6247b80 1 -- 172.21.15.53:0/20785262 wait complete. 2023-11-06T01:47:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.457+0000 7efed6247b80 1 -- start start 2023-11-06T01:47:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efed6247b80 1 Processor -- start 2023-11-06T01:47:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efed6247b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592680 0x55ad2a5b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efed6247b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 0x55ad2a5b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efed6247b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a593ed0 0x55ad2a609300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efed6247b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a6098b0 con 0x55ad2a592680 2023-11-06T01:47:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efed6247b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a609a20 con 0x55ad2a592fe0 2023-11-06T01:47:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efed6247b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2a609be0 con 0x55ad2a593ed0 2023-11-06T01:47:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efed6247b80 10 client.0 _ll_register_callbacks cb 0x7efeb8015810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efec37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592680 0x55ad2a5b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efec3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a593ed0 0x55ad2a609300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efec2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 0x55ad2a5b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efec37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592680 0x55ad2a5b2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43500/0 (socket says 172.21.15.53:43500) 2023-11-06T01:47:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.458+0000 7efec37fe700 1 -- 172.21.15.53:0/3940230613 learned_addr learned my addr 172.21.15.53:0/3940230613 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.459+0000 7efec37fe700 1 -- 172.21.15.53:0/3940230613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a593ed0 msgr2=0x55ad2a609300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.459+0000 7efec37fe700 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a593ed0 0x55ad2a609300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.459+0000 7efec37fe700 1 -- 172.21.15.53:0/3940230613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 msgr2=0x55ad2a5b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.459+0000 7efec37fe700 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 0x55ad2a5b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.459+0000 7efec37fe700 1 -- 172.21.15.53:0/3940230613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec4072060 con 0x55ad2a592680 2023-11-06T01:47:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.459+0000 7efec3fff700 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a593ed0 0x55ad2a609300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:47:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.460+0000 7efec37fe700 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592680 0x55ad2a5b2020 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005950 tx=0x7efeb40b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142c577085902e5d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.460+0000 7efea2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:47:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.460+0000 7efea2ffd700 1 -- 172.21.15.53:0/3940230613 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4014070 con 0x55ad2a592680 2023-11-06T01:47:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.460+0000 7efea2ffd700 1 -- 172.21.15.53:0/3940230613 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efeb40c91a0 con 0x55ad2a592680 2023-11-06T01:47:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.461+0000 7efed6247b80 1 -- 172.21.15.53:0/3940230613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efeb80129d0 con 0x55ad2a592680 2023-11-06T01:47:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.461+0000 7efea2ffd700 1 -- 172.21.15.53:0/3940230613 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeb40b11a0 con 0x55ad2a592680 2023-11-06T01:47:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.461+0000 7efed6247b80 10 client.16878 fetch_fsmap learned FSMap version 48 2023-11-06T01:47:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.461+0000 7efed6247b80 1 -- 172.21.15.53:0/3940230613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55ad2a2fac00 con 0x55ad2a592680 2023-11-06T01:47:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.462+0000 7efea2ffd700 1 -- 172.21.15.53:0/3940230613 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb407b030 con 0x55ad2a592680 2023-11-06T01:47:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.463+0000 7efea2ffd700 1 -- 172.21.15.53:0/3940230613 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 48) v1 ==== 7458+0+0 (secure 0 0 0) 0x7efeb40a09d0 con 0x55ad2a592680 2023-11-06T01:47:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.463+0000 7efed6247b80 10 client.16878 fetch_fsmap finished waiting for FSMap version 48 2023-11-06T01:47:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.463+0000 7efed6247b80 10 client.16878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.463+0000 7efed6247b80 20 client.16878 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.463+0000 7efed6247b80 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad2a60b550 0x55ad2a60d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.463+0000 7efed6247b80 4 client.16878 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:47:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.463+0000 7efed6247b80 1 -- 172.21.15.53:0/3940230613 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ad2a4dcce0 con 0x55ad2a60b550 2023-11-06T01:47:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.463+0000 7efec2ffd700 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad2a60b550 0x55ad2a60d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.463+0000 7efea2ffd700 10 client.16878 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.463+0000 7efec2ffd700 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad2a60b550 0x55ad2a60d970 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d29a7d946eeeb2f1 server_cookie=7795c72db639e035 in_seq=0 out_seq=0 2023-11-06T01:47:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.472+0000 7efea2ffd700 1 -- 172.21.15.53:0/3940230613 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ad2a4dcce0 con 0x55ad2a60b550 2023-11-06T01:47:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.472+0000 7efea2ffd700 10 client.16878 handle_command_reply: tid=0 2023-11-06T01:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.472+0000 7efed6247b80 1 client.16878 shutdown 2023-11-06T01:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efed6247b80 1 -- 172.21.15.53:0/3940230613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592680 msgr2=0x55ad2a5b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efed6247b80 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592680 0x55ad2a5b2020 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005950 tx=0x7efeb40b8040 comp rx=0 tx=0).stop 2023-11-06T01:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efed6247b80 1 -- 172.21.15.53:0/3940230613 shutdown_connections 2023-11-06T01:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efed6247b80 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55ad2a60b550 0x55ad2a60d970 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efed6247b80 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad2a593ed0 0x55ad2a609300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efed6247b80 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ad2a592680 0x55ad2a5b2020 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efea2ffd700 0 client.16878 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efed6247b80 1 --2- 172.21.15.53:0/3940230613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad2a592fe0 0x55ad2a5b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efed6247b80 1 -- 172.21.15.53:0/3940230613 >> 172.21.15.53:0/3940230613 conn(0x55ad2a4f2070 msgr2=0x55ad2a5970a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efed6247b80 1 -- 172.21.15.53:0/3940230613 shutdown_connections 2023-11-06T01:47:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efed6247b80 1 -- 172.21.15.53:0/3940230613 wait complete. 2023-11-06T01:47:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efed6247b80 20 client.16878 trim_cache size 0 max 16384 2023-11-06T01:47:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:47:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:47:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:47:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:47:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efeca0b7700 1 -- 172.21.15.53:0/2918775065 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efeac0a0670 msgr2=0x7efeac0a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efeca0b7700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efeac0a0670 0x7efeac0a2b30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efec4070800 tx=0x7efeb8006d20 comp rx=0 tx=0).stop 2023-11-06T01:47:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efeca0b7700 1 -- 172.21.15.53:0/2918775065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec40ff650 msgr2=0x7efec406f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efeca0b7700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec40ff650 0x7efec406f7b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7efeb400d950 tx=0x7efeb400dd10 comp rx=0 tx=0).stop 2023-11-06T01:47:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efeca0b7700 1 -- 172.21.15.53:0/2918775065 shutdown_connections 2023-11-06T01:47:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efeca0b7700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7efeac0a0670 0x7efeac0a2b30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efeca0b7700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4104990 0x7efec4070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efeca0b7700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec40ff650 0x7efec406f7b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efeca0b7700 1 --2- 172.21.15.53:0/2918775065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec4101fc0 0x7efec406fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.473+0000 7efeca0b7700 1 -- 172.21.15.53:0/2918775065 >> 172.21.15.53:0/2918775065 conn(0x7efec40fb250 msgr2=0x7efec4100300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.474+0000 7efeca0b7700 1 -- 172.21.15.53:0/2918775065 shutdown_connections 2023-11-06T01:47:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:01.474+0000 7efeca0b7700 1 -- 172.21.15.53:0/2918775065 wait complete. 2023-11-06T01:47:01.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:01 smithi163 ceph-mon[104416]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 596 op/s 2023-11-06T01:47:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 765.0 s, 1368.2 tps, lat 73.359 ms stddev 80.656 2023-11-06T01:47:03.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:03 smithi053 ceph-mon[99472]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 663 op/s 2023-11-06T01:47:03.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:03 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:47:03.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:03 smithi062 ceph-mon[103924]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 663 op/s 2023-11-06T01:47:03.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:03 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:47:03.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:03 smithi163 ceph-mon[104416]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 663 op/s 2023-11-06T01:47:03.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:03 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:47:04.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:04 smithi053 ceph-mon[99472]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 628 op/s 2023-11-06T01:47:04.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:04 smithi062 ceph-mon[103924]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 628 op/s 2023-11-06T01:47:04.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:04 smithi163 ceph-mon[104416]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 628 op/s 2023-11-06T01:47:07.185 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 770.0 s, 1598.2 tps, lat 62.929 ms stddev 71.364 2023-11-06T01:47:07.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:07 smithi053 ceph-mon[99472]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 697 op/s 2023-11-06T01:47:07.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:07 smithi062 ceph-mon[103924]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 697 op/s 2023-11-06T01:47:07.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:07 smithi163 ceph-mon[104416]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 697 op/s 2023-11-06T01:47:08.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:08 smithi062 ceph-mon[103924]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 701 op/s 2023-11-06T01:47:08.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:08 smithi163 ceph-mon[104416]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 701 op/s 2023-11-06T01:47:08.595 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:08 smithi053 ceph-mon[99472]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 701 op/s 2023-11-06T01:47:11.172 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:47:11.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:11 smithi053 ceph-mon[99472]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 664 op/s 2023-11-06T01:47:11.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.345+0000 7febdb513700 1 -- 172.21.15.53:0/1351503633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd40ff5f0 msgr2=0x7febd40ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:11.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.345+0000 7febdb513700 1 --2- 172.21.15.53:0/1351503633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd40ff5f0 0x7febd40ffa10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7febd0009990 tx=0x7febd0009ca0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.345+0000 7febdb513700 1 -- 172.21.15.53:0/1351503633 shutdown_connections 2023-11-06T01:47:11.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.345+0000 7febdb513700 1 --2- 172.21.15.53:0/1351503633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd4100910 0x7febd4100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.345+0000 7febdb513700 1 --2- 172.21.15.53:0/1351503633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd40ff5f0 0x7febd40ffa10 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.345+0000 7febdb513700 1 --2- 172.21.15.53:0/1351503633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd40fff50 0x7febd41003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.345+0000 7febdb513700 1 -- 172.21.15.53:0/1351503633 >> 172.21.15.53:0/1351503633 conn(0x7febd40fb1f0 msgr2=0x7febd40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:11.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febdb513700 1 -- 172.21.15.53:0/1351503633 shutdown_connections 2023-11-06T01:47:11.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febdb513700 1 -- 172.21.15.53:0/1351503633 wait complete. 2023-11-06T01:47:11.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febdb513700 1 Processor -- start 2023-11-06T01:47:11.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febdb513700 1 -- start start 2023-11-06T01:47:11.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febdb513700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd40ff5f0 0x7febd4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febdb513700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd40fff50 0x7febd41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febdb513700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd4100910 0x7febd41a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febdb513700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd41a0800 con 0x7febd40ff5f0 2023-11-06T01:47:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febdb513700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd41a0970 con 0x7febd4100910 2023-11-06T01:47:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febdb513700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd41a0b30 con 0x7febd40fff50 2023-11-06T01:47:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febd92af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd40ff5f0 0x7febd4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febd92af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd40ff5f0 0x7febd4198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43518/0 (socket says 172.21.15.53:43518) 2023-11-06T01:47:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.346+0000 7febd92af700 1 -- 172.21.15.53:0/4251901959 learned_addr learned my addr 172.21.15.53:0/4251901959 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febd9ab0700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd4100910 0x7febd41a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febd8aae700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd40fff50 0x7febd41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febd92af700 1 -- 172.21.15.53:0/4251901959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd40fff50 msgr2=0x7febd41991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febd92af700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd40fff50 0x7febd41991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febd92af700 1 -- 172.21.15.53:0/4251901959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd4100910 msgr2=0x7febd41a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febd92af700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd4100910 0x7febd41a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febd92af700 1 -- 172.21.15.53:0/4251901959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd00095f0 con 0x7febd40ff5f0 2023-11-06T01:47:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febd92af700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd40ff5f0 0x7febd4198c90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7febd0000c00 tx=0x7febd000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febca7fc700 1 -- 172.21.15.53:0/4251901959 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd001d070 con 0x7febd40ff5f0 2023-11-06T01:47:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febca7fc700 1 -- 172.21.15.53:0/4251901959 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd000fc00 con 0x7febd40ff5f0 2023-11-06T01:47:11.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febdb513700 1 -- 172.21.15.53:0/4251901959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd41a0db0 con 0x7febd40ff5f0 2023-11-06T01:47:11.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febca7fc700 1 -- 172.21.15.53:0/4251901959 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0017650 con 0x7febd40ff5f0 2023-11-06T01:47:11.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.347+0000 7febdb513700 1 -- 172.21.15.53:0/4251901959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd41a11b0 con 0x7febd40ff5f0 2023-11-06T01:47:11.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.351+0000 7febca7fc700 1 -- 172.21.15.53:0/4251901959 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7febd0021410 con 0x7febd40ff5f0 2023-11-06T01:47:11.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.352+0000 7febca7fc700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0520 0x7febc00a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.352+0000 7febe76a3b80 1 Processor -- start 2023-11-06T01:47:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.352+0000 7febca7fc700 1 -- 172.21.15.53:0/4251901959 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7febd00ccf00 con 0x7febd40ff5f0 2023-11-06T01:47:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.352+0000 7febd8aae700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0520 0x7febc00a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.352+0000 7febe76a3b80 1 -- start start 2023-11-06T01:47:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.352+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5889530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.352+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a5889a70 0x5604a5887e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.352+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5888870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.352+0000 7febe76a3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a57dfa10 con 0x5604a58890f0 2023-11-06T01:47:11.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.352+0000 7febe76a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a57dd3d0 con 0x5604a5889a70 2023-11-06T01:47:11.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.352+0000 7febe76a3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a57d8cc0 con 0x5604a58883f0 2023-11-06T01:47:11.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd9ab0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5888870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd92af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5889530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd9ab0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5888870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57768/0 (socket says 172.21.15.53:57768) 2023-11-06T01:47:11.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd8aae700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0520 0x7febc00a29e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7febd419a190 tx=0x7febc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:11.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd92af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5889530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43526/0 (socket says 172.21.15.53:43526) 2023-11-06T01:47:11.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd9ab0700 1 -- 172.21.15.53:0/1522028440 learned_addr learned my addr 172.21.15.53:0/1522028440 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:11.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd8aae700 1 --2- 172.21.15.53:0/1522028440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a5889a70 0x5604a5887e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd9ab0700 1 -- 172.21.15.53:0/1522028440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a5889a70 msgr2=0x5604a5887e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:11.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd9ab0700 1 --2- 172.21.15.53:0/1522028440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a5889a70 0x5604a5887e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd9ab0700 1 -- 172.21.15.53:0/1522028440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 msgr2=0x5604a5889530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:11.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd9ab0700 1 --2- 172.21.15.53:0/1522028440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5889530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd9ab0700 1 -- 172.21.15.53:0/1522028440 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604a5606de0 con 0x5604a58883f0 2023-11-06T01:47:11.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd92af700 1 --2- 172.21.15.53:0/1522028440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5889530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:11.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febd9ab0700 1 --2- 172.21.15.53:0/1522028440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5888870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7febcc003f90 tx=0x7febcc009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39be502214054e5b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:11.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febbf7fe700 1 -- 172.21.15.53:0/1522028440 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc0046f0 con 0x5604a58883f0 2023-11-06T01:47:11.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febbf7fe700 1 -- 172.21.15.53:0/1522028440 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febcc00f040 con 0x5604a58883f0 2023-11-06T01:47:11.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.353+0000 7febbf7fe700 1 -- 172.21.15.53:0/1522028440 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc0156e0 con 0x5604a58883f0 2023-11-06T01:47:11.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.354+0000 7febe76a3b80 1 -- 172.21.15.53:0/1522028440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 msgr2=0x5604a5888870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:11.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.354+0000 7febe76a3b80 1 --2- 172.21.15.53:0/1522028440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5888870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7febcc003f90 tx=0x7febcc009c20 comp rx=0 tx=0).stop 2023-11-06T01:47:11.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.354+0000 7febe76a3b80 1 -- 172.21.15.53:0/1522028440 shutdown_connections 2023-11-06T01:47:11.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.354+0000 7febe76a3b80 1 --2- 172.21.15.53:0/1522028440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5888870 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.354+0000 7febe76a3b80 1 --2- 172.21.15.53:0/1522028440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5889530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.354+0000 7febe76a3b80 1 --2- 172.21.15.53:0/1522028440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a5889a70 0x5604a5887e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.354+0000 7febe76a3b80 1 -- 172.21.15.53:0/1522028440 >> 172.21.15.53:0/1522028440 conn(0x5604a57e70d0 msgr2=0x5604a5882940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:11.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.355+0000 7febe76a3b80 1 -- 172.21.15.53:0/1522028440 shutdown_connections 2023-11-06T01:47:11.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.355+0000 7febe76a3b80 1 -- 172.21.15.53:0/1522028440 wait complete. 2023-11-06T01:47:11.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.355+0000 7febe76a3b80 1 -- start start 2023-11-06T01:47:11.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.355+0000 7febe76a3b80 1 Processor -- start 2023-11-06T01:47:11.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5891940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5891e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a5889a70 0x5604a5898ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febe76a3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a5775510 con 0x5604a58890f0 2023-11-06T01:47:11.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febe76a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a57751b0 con 0x5604a5889a70 2023-11-06T01:47:11.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febe76a3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a565c920 con 0x5604a58883f0 2023-11-06T01:47:11.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febe76a3b80 10 client.0 _ll_register_callbacks cb 0x5604a56c2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:11.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd8aae700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5891e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd92af700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5891940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd8aae700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5891e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43534/0 (socket says 172.21.15.53:43534) 2023-11-06T01:47:11.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd9ab0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a5889a70 0x5604a5898ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd8aae700 1 -- 172.21.15.53:0/1962544902 learned_addr learned my addr 172.21.15.53:0/1962544902 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:11.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd92af700 1 -- 172.21.15.53:0/1962544902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a5889a70 msgr2=0x5604a5898ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:11.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd92af700 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a5889a70 0x5604a5898ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd92af700 1 -- 172.21.15.53:0/1962544902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 msgr2=0x5604a5891e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:11.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd92af700 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5891e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd92af700 1 -- 172.21.15.53:0/1962544902 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd41a11b0 con 0x5604a58883f0 2023-11-06T01:47:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd8aae700 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5891e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febd92af700 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5891940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7febd00caa30 tx=0x7febd009ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bff6368c7b3e2e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.356+0000 7febbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:47:11.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febbcff9700 1 -- 172.21.15.53:0/1962544902 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd00caea0 con 0x5604a58883f0 2023-11-06T01:47:11.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febbcff9700 1 -- 172.21.15.53:0/1962544902 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7febd009a7d0 con 0x5604a58883f0 2023-11-06T01:47:11.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febe76a3b80 1 -- 172.21.15.53:0/1962544902 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604a5803d10 con 0x5604a58883f0 2023-11-06T01:47:11.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febbcff9700 1 -- 172.21.15.53:0/1962544902 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7febd009e600 con 0x5604a58883f0 2023-11-06T01:47:11.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febe76a3b80 10 client.26467 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:11.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febe76a3b80 1 -- 172.21.15.53:0/1962544902 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7febcc003bf0 con 0x5604a58883f0 2023-11-06T01:47:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febbcff9700 1 -- 172.21.15.53:0/1962544902 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604a5803d10 con 0x5604a58883f0 2023-11-06T01:47:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febbcff9700 1 -- 172.21.15.53:0/1962544902 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7febd00a04b0 con 0x5604a58883f0 2023-11-06T01:47:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febe76a3b80 10 client.26467 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febe76a3b80 10 client.26467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:11.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febe76a3b80 20 client.26467 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febe76a3b80 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5604a589d2b0 0x5604a589f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febe76a3b80 4 client.26467 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:47:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.357+0000 7febe76a3b80 1 -- 172.21.15.53:0/1962544902 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604a5856e10 con 0x5604a589d2b0 2023-11-06T01:47:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.358+0000 7febd8aae700 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5604a589d2b0 0x5604a589f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.358+0000 7febd8aae700 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5604a589d2b0 0x5604a589f6a0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e99cc13ebe49101 server_cookie=add1dd1b22d17a4a in_seq=0 out_seq=0 2023-11-06T01:47:11.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.358+0000 7febbcff9700 10 client.26467 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:11.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.358+0000 7febbcff9700 1 -- 172.21.15.53:0/1962544902 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5604a5856e10 con 0x5604a589d2b0 2023-11-06T01:47:11.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.358+0000 7febbcff9700 10 client.26467 handle_command_reply: tid=0 2023-11-06T01:47:11.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.359+0000 7febe76a3b80 1 client.26467 shutdown 2023-11-06T01:47:11.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.359+0000 7febe76a3b80 1 -- 172.21.15.53:0/1962544902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 msgr2=0x5604a5891940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:11.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.359+0000 7febe76a3b80 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5891940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7febd00caa30 tx=0x7febd009ad10 comp rx=0 tx=0).stop 2023-11-06T01:47:11.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.360+0000 7febe76a3b80 1 -- 172.21.15.53:0/1962544902 shutdown_connections 2023-11-06T01:47:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.360+0000 7febe76a3b80 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a5889a70 0x5604a5898ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.360+0000 7febe76a3b80 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5604a589d2b0 0x5604a589f6a0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.360+0000 7febe76a3b80 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5891940 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.360+0000 7febe76a3b80 1 --2- 172.21.15.53:0/1962544902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58890f0 0x5604a5891e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.360+0000 7febe76a3b80 1 -- 172.21.15.53:0/1962544902 >> 172.21.15.53:0/1962544902 conn(0x5604a57e70d0 msgr2=0x5604a58820b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:11.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.360+0000 7febbcff9700 0 client.26467 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:11.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.361+0000 7febe76a3b80 1 -- 172.21.15.53:0/1962544902 shutdown_connections 2023-11-06T01:47:11.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.361+0000 7febe76a3b80 1 -- 172.21.15.53:0/1962544902 wait complete. 2023-11-06T01:47:11.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.361+0000 7febe76a3b80 20 client.26467 trim_cache size 0 max 16384 2023-11-06T01:47:11.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.381+0000 7febe76a3b80 1 Processor -- start 2023-11-06T01:47:11.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.381+0000 7febe76a3b80 1 -- start start 2023-11-06T01:47:11.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.381+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5896e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.381+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a58890f0 0x5604a58997b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.381+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a5889a70 0x5604a58916f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.381+0000 7febe76a3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a5891510 con 0x5604a5889a70 2023-11-06T01:47:11.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.381+0000 7febe76a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a5856e10 con 0x5604a58890f0 2023-11-06T01:47:11.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.381+0000 7febe76a3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a5883df0 con 0x5604a58883f0 2023-11-06T01:47:11.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.381+0000 7febd92af700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5896e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febd92af700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5896e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57782/0 (socket says 172.21.15.53:57782) 2023-11-06T01:47:11.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febd92af700 1 -- 172.21.15.53:0/2571573044 learned_addr learned my addr 172.21.15.53:0/2571573044 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:11.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febd8aae700 1 --2- 172.21.15.53:0/2571573044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a58890f0 0x5604a58997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febd92af700 1 -- 172.21.15.53:0/2571573044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a58890f0 msgr2=0x5604a58997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:11.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febd92af700 1 --2- 172.21.15.53:0/2571573044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a58890f0 0x5604a58997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febd92af700 1 -- 172.21.15.53:0/2571573044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a5889a70 msgr2=0x5604a58916f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:11.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febd92af700 1 --2- 172.21.15.53:0/2571573044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a5889a70 0x5604a58916f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febd92af700 1 -- 172.21.15.53:0/2571573044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604a5882be0 con 0x5604a58883f0 2023-11-06T01:47:11.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febd92af700 1 --2- 172.21.15.53:0/2571573044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5896e70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7febd000b680 tx=0x7febd0096510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed8b009acb81020 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:11.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febbffff700 1 -- 172.21.15.53:0/2571573044 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd00caea0 con 0x5604a58883f0 2023-11-06T01:47:11.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febbffff700 1 -- 172.21.15.53:0/2571573044 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd009a7d0 con 0x5604a58883f0 2023-11-06T01:47:11.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.382+0000 7febbffff700 1 -- 172.21.15.53:0/2571573044 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd00a05d0 con 0x5604a58883f0 2023-11-06T01:47:11.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.383+0000 7febe76a3b80 1 -- 172.21.15.53:0/2571573044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 msgr2=0x5604a5896e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:11.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.383+0000 7febe76a3b80 1 --2- 172.21.15.53:0/2571573044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5896e70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7febd000b680 tx=0x7febd0096510 comp rx=0 tx=0).stop 2023-11-06T01:47:11.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.383+0000 7febe76a3b80 1 -- 172.21.15.53:0/2571573044 shutdown_connections 2023-11-06T01:47:11.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.383+0000 7febe76a3b80 1 --2- 172.21.15.53:0/2571573044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a5889a70 0x5604a58916f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.383+0000 7febe76a3b80 1 --2- 172.21.15.53:0/2571573044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a58883f0 0x5604a5896e70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.383+0000 7febe76a3b80 1 --2- 172.21.15.53:0/2571573044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a58890f0 0x5604a58997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.383+0000 7febe76a3b80 1 -- 172.21.15.53:0/2571573044 >> 172.21.15.53:0/2571573044 conn(0x5604a57e70d0 msgr2=0x5604a58906b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:11.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.383+0000 7febe76a3b80 1 -- 172.21.15.53:0/2571573044 shutdown_connections 2023-11-06T01:47:11.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febe76a3b80 1 -- 172.21.15.53:0/2571573044 wait complete. 2023-11-06T01:47:11.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febe76a3b80 1 -- start start 2023-11-06T01:47:11.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febe76a3b80 1 Processor -- start 2023-11-06T01:47:11.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58883f0 0x5604a5896810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a58890f0 0x5604a5896d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febe76a3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a5889a70 0x5604a5894e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febe76a3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a58954c0 con 0x5604a58883f0 2023-11-06T01:47:11.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febe76a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a5895630 con 0x5604a58890f0 2023-11-06T01:47:11.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febe76a3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604a58957f0 con 0x5604a5889a70 2023-11-06T01:47:11.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febe76a3b80 10 client.0 _ll_register_callbacks cb 0x7febd009a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:11.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febd92af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58883f0 0x5604a5896810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febd8aae700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a58890f0 0x5604a5896d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febd9ab0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a5889a70 0x5604a5894e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febd8aae700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a58890f0 0x5604a5896d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:54984/0 (socket says 172.21.15.53:54984) 2023-11-06T01:47:11.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.384+0000 7febd92af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58883f0 0x5604a5896810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43546/0 (socket says 172.21.15.53:43546) 2023-11-06T01:47:11.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febd8aae700 1 -- 172.21.15.53:0/4088012443 learned_addr learned my addr 172.21.15.53:0/4088012443 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:11.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febd9ab0700 1 -- 172.21.15.53:0/4088012443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a58890f0 msgr2=0x5604a5896d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:11.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febd9ab0700 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a58890f0 0x5604a5896d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febd9ab0700 1 -- 172.21.15.53:0/4088012443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58883f0 msgr2=0x5604a5896810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:11.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febd9ab0700 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58883f0 0x5604a5896810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febd9ab0700 1 -- 172.21.15.53:0/4088012443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc003bf0 con 0x5604a5889a70 2023-11-06T01:47:11.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febd92af700 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58883f0 0x5604a5896810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:11.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febd9ab0700 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a5889a70 0x5604a5894e20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7febcc009f70 tx=0x7febcc007730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f7f89cb1e74328 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:11.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:47:11.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febbcff9700 1 -- 172.21.15.53:0/4088012443 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc004900 con 0x5604a5889a70 2023-11-06T01:47:11.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febbcff9700 1 -- 172.21.15.53:0/4088012443 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7febcc0078a0 con 0x5604a5889a70 2023-11-06T01:47:11.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febe76a3b80 1 -- 172.21.15.53:0/4088012443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febcc00f040 con 0x5604a5889a70 2023-11-06T01:47:11.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febbcff9700 1 -- 172.21.15.53:0/4088012443 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7febcc007e20 con 0x5604a5889a70 2023-11-06T01:47:11.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febe76a3b80 10 client.26473 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:11.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.385+0000 7febe76a3b80 1 -- 172.21.15.53:0/4088012443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5604a55efc60 con 0x5604a5889a70 2023-11-06T01:47:11.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febbcff9700 1 -- 172.21.15.53:0/4088012443 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febcc00f040 con 0x5604a5889a70 2023-11-06T01:47:11.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febbcff9700 1 -- 172.21.15.53:0/4088012443 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7febcc015b80 con 0x5604a5889a70 2023-11-06T01:47:11.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febe76a3b80 10 client.26473 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:11.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febe76a3b80 10 client.26473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:11.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febe76a3b80 20 client.26473 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:11.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febe76a3b80 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5604a58f88f0 0x5604a58fad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:11.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febe76a3b80 4 client.26473 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:47:11.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febe76a3b80 1 -- 172.21.15.53:0/4088012443 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5604a57d1d40 con 0x5604a58f88f0 2023-11-06T01:47:11.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febd92af700 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5604a58f88f0 0x5604a58fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:11.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febbcff9700 10 client.26473 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:11.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febd92af700 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5604a58f88f0 0x5604a58fad10 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aae3dd7e004cc0d server_cookie=4bf515df5f0b059e in_seq=0 out_seq=0 2023-11-06T01:47:11.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febbcff9700 1 -- 172.21.15.53:0/4088012443 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5604a57d1d40 con 0x5604a58f88f0 2023-11-06T01:47:11.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.386+0000 7febbcff9700 10 client.26473 handle_command_reply: tid=0 2023-11-06T01:47:11.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 1 client.26473 shutdown 2023-11-06T01:47:11.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 1 -- 172.21.15.53:0/4088012443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a5889a70 msgr2=0x5604a5894e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:11.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a5889a70 0x5604a5894e20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7febcc009f70 tx=0x7febcc007730 comp rx=0 tx=0).stop 2023-11-06T01:47:11.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 1 -- 172.21.15.53:0/4088012443 shutdown_connections 2023-11-06T01:47:11.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5604a58f88f0 0x5604a58fad10 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5604a5889a70 0x5604a5894e20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604a58883f0 0x5604a5896810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 1 --2- 172.21.15.53:0/4088012443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604a58890f0 0x5604a5896d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febbcff9700 0 client.26473 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:11.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 1 -- 172.21.15.53:0/4088012443 >> 172.21.15.53:0/4088012443 conn(0x5604a57e70d0 msgr2=0x5604a588c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:11.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 1 -- 172.21.15.53:0/4088012443 shutdown_connections 2023-11-06T01:47:11.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 1 -- 172.21.15.53:0/4088012443 wait complete. 2023-11-06T01:47:11.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febe76a3b80 20 client.26473 trim_cache size 0 max 16384 2023-11-06T01:47:11.402 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:47:11.402 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:47:11.402 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:47:11.402 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:47:11.403 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:47:11.403 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:47:11.403 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:47:11.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:47:11.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:47:11.404 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:47:11.404 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:11 smithi062 ceph-mon[103924]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 664 op/s 2023-11-06T01:47:11.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febdb513700 1 -- 172.21.15.53:0/4251901959 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0520 msgr2=0x7febc00a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:11.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febdb513700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0520 0x7febc00a29e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7febd419a190 tx=0x7febc4009260 comp rx=0 tx=0).stop 2023-11-06T01:47:11.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febdb513700 1 -- 172.21.15.53:0/4251901959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd40ff5f0 msgr2=0x7febd4198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:11.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.387+0000 7febdb513700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd40ff5f0 0x7febd4198c90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7febd0000c00 tx=0x7febd000f460 comp rx=0 tx=0).stop 2023-11-06T01:47:11.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.388+0000 7febdb513700 1 -- 172.21.15.53:0/4251901959 shutdown_connections 2023-11-06T01:47:11.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.388+0000 7febdb513700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7febc00a0520 0x7febc00a29e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.388+0000 7febdb513700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febd4100910 0x7febd41a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.388+0000 7febdb513700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febd40ff5f0 0x7febd4198c90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.388+0000 7febdb513700 1 --2- 172.21.15.53:0/4251901959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febd40fff50 0x7febd41991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:11.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.388+0000 7febdb513700 1 -- 172.21.15.53:0/4251901959 >> 172.21.15.53:0/4251901959 conn(0x7febd40fb1f0 msgr2=0x7febd40fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:11.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.388+0000 7febdb513700 1 -- 172.21.15.53:0/4251901959 shutdown_connections 2023-11-06T01:47:11.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:11.388+0000 7febdb513700 1 -- 172.21.15.53:0/4251901959 wait complete. 2023-11-06T01:47:11.408 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:47:11.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:11 smithi163 ceph-mon[104416]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 664 op/s 2023-11-06T01:47:12.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 775.0 s, 1518.5 tps, lat 66.002 ms stddev 73.522 2023-11-06T01:47:12.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:12 smithi163 ceph-mon[104416]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 772 op/s 2023-11-06T01:47:12.843 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:12 smithi053 ceph-mon[99472]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 772 op/s 2023-11-06T01:47:12.900 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:12 smithi062 ceph-mon[103924]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 772 op/s 2023-11-06T01:47:14.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:14 smithi062 ceph-mon[103924]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 713 op/s 2023-11-06T01:47:14.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:14 smithi163 ceph-mon[104416]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 713 op/s 2023-11-06T01:47:14.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:14 smithi053 ceph-mon[99472]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 713 op/s 2023-11-06T01:47:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.613+0000 7febb8ddb700 1 -- 172.21.15.53:0/610771826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb4100ed0 msgr2=0x7febb4101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.613+0000 7febb8ddb700 1 --2- 172.21.15.53:0/610771826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb4100ed0 0x7febb4101330 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7feba80099c0 tx=0x7feba8009cd0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.613+0000 7febb8ddb700 1 -- 172.21.15.53:0/610771826 shutdown_connections 2023-11-06T01:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.613+0000 7febb8ddb700 1 --2- 172.21.15.53:0/610771826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febb40ffce0 0x7febb4100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.613+0000 7febb8ddb700 1 --2- 172.21.15.53:0/610771826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febb4101870 0x7febb4108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.613+0000 7febb8ddb700 1 --2- 172.21.15.53:0/610771826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb4100ed0 0x7febb4101330 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.613+0000 7febb8ddb700 1 -- 172.21.15.53:0/610771826 >> 172.21.15.53:0/610771826 conn(0x7febb40fb250 msgr2=0x7febb40fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.613+0000 7febb8ddb700 1 -- 172.21.15.53:0/610771826 shutdown_connections 2023-11-06T01:47:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.613+0000 7febb8ddb700 1 -- 172.21.15.53:0/610771826 wait complete. 2023-11-06T01:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.614+0000 7febb8ddb700 1 Processor -- start 2023-11-06T01:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.614+0000 7febb8ddb700 1 -- start start 2023-11-06T01:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.614+0000 7febb8ddb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb40ffce0 0x7febb4190fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.614+0000 7febb8ddb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febb4100ed0 0x7febb41914f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.614+0000 7febb8ddb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febb4101870 0x7febb41923c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.614+0000 7febb8ddb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb4192a20 con 0x7febb40ffce0 2023-11-06T01:47:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.614+0000 7febb8ddb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb4192b90 con 0x7febb4101870 2023-11-06T01:47:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.614+0000 7febb8ddb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb4192d50 con 0x7febb4100ed0 2023-11-06T01:47:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.615+0000 7febb2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febb4101870 0x7febb41923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.615+0000 7febb2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febb4101870 0x7febb41923c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55000/0 (socket says 172.21.15.53:55000) 2023-11-06T01:47:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.615+0000 7febb2d9d700 1 -- 172.21.15.53:0/1849344486 learned_addr learned my addr 172.21.15.53:0/1849344486 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.615+0000 7febb2d9d700 1 -- 172.21.15.53:0/1849344486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febb4100ed0 msgr2=0x7febb41914f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.615+0000 7febb2d9d700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febb4100ed0 0x7febb41914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.615+0000 7febb2d9d700 1 -- 172.21.15.53:0/1849344486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb40ffce0 msgr2=0x7febb4190fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.615+0000 7febb2d9d700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb40ffce0 0x7febb4190fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.615+0000 7febb2d9d700 1 -- 172.21.15.53:0/1849344486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba80095f0 con 0x7febb4101870 2023-11-06T01:47:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.616+0000 7febb2d9d700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febb4101870 0x7febb41923c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7febac00ef30 tx=0x7febac00c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.617+0000 7feba37fe700 1 -- 172.21.15.53:0/1849344486 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac018070 con 0x7febb4101870 2023-11-06T01:47:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.617+0000 7feba37fe700 1 -- 172.21.15.53:0/1849344486 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febac00f040 con 0x7febb4101870 2023-11-06T01:47:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.617+0000 7feba37fe700 1 -- 172.21.15.53:0/1849344486 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac003dc0 con 0x7febb4101870 2023-11-06T01:47:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.618+0000 7febb8ddb700 1 -- 172.21.15.53:0/1849344486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb4192fd0 con 0x7febb4101870 2023-11-06T01:47:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.618+0000 7febb8ddb700 1 -- 172.21.15.53:0/1849344486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb41937e0 con 0x7febb4101870 2023-11-06T01:47:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.620+0000 7feba37fe700 1 -- 172.21.15.53:0/1849344486 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7febac00caa0 con 0x7febb4101870 2023-11-06T01:47:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.621+0000 7feba37fe700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb9c0a0670 0x7feb9c0a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.621+0000 7feba37fe700 1 -- 172.21.15.53:0/1849344486 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7febac0cade0 con 0x7febb4101870 2023-11-06T01:47:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.621+0000 7febb259c700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb9c0a0670 0x7feb9c0a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.621+0000 7febc4f6bb80 1 Processor -- start 2023-11-06T01:47:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.621+0000 7febc4f6bb80 1 -- start start 2023-11-06T01:47:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.621+0000 7febb259c700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb9c0a0670 0x7feb9c0a2b30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7febb4190d20 tx=0x7feba40096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 0x563c2b517aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517fe0 0x563c2b5188f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b518ed0 0x563c2b519350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b46f9b0 con 0x563c2b518ed0 2023-11-06T01:47:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b46d370 con 0x563c2b517fe0 2023-11-06T01:47:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b468c60 con 0x563c2b517680 2023-11-06T01:47:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febb2d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b518ed0 0x563c2b519350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febb1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517fe0 0x563c2b5188f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febb1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517fe0 0x563c2b5188f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55008/0 (socket says 172.21.15.53:55008) 2023-11-06T01:47:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febb1d9b700 1 -- 172.21.15.53:0/3220933214 learned_addr learned my addr 172.21.15.53:0/3220933214 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febb259c700 1 --2- 172.21.15.53:0/3220933214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 0x563c2b517aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febb2d9d700 1 -- 172.21.15.53:0/3220933214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 msgr2=0x563c2b517aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febb2d9d700 1 --2- 172.21.15.53:0/3220933214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 0x563c2b517aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febb2d9d700 1 -- 172.21.15.53:0/3220933214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517fe0 msgr2=0x563c2b5188f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febb2d9d700 1 --2- 172.21.15.53:0/3220933214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517fe0 0x563c2b5188f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febb2d9d700 1 -- 172.21.15.53:0/3220933214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c2b296d80 con 0x563c2b518ed0 2023-11-06T01:47:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.622+0000 7febb1d9b700 1 --2- 172.21.15.53:0/3220933214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517fe0 0x563c2b5188f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.623+0000 7febb2d9d700 1 --2- 172.21.15.53:0/3220933214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b518ed0 0x563c2b519350 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7febac0c8910 tx=0x7febac098270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a7e4b474974f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.623+0000 7feba0ff9700 1 -- 172.21.15.53:0/3220933214 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac09c450 con 0x563c2b518ed0 2023-11-06T01:47:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.623+0000 7feba0ff9700 1 -- 172.21.15.53:0/3220933214 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febac0a0690 con 0x563c2b518ed0 2023-11-06T01:47:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.623+0000 7feba0ff9700 1 -- 172.21.15.53:0/3220933214 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac00cd50 con 0x563c2b518ed0 2023-11-06T01:47:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.623+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3220933214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b518ed0 msgr2=0x563c2b519350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.623+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3220933214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b518ed0 0x563c2b519350 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7febac0c8910 tx=0x7febac098270 comp rx=0 tx=0).stop 2023-11-06T01:47:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3220933214 shutdown_connections 2023-11-06T01:47:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3220933214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b518ed0 0x563c2b519350 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3220933214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 0x563c2b517aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3220933214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517fe0 0x563c2b5188f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3220933214 >> 172.21.15.53:0/3220933214 conn(0x563c2b477070 msgr2=0x563c2b514040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3220933214 shutdown_connections 2023-11-06T01:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3220933214 wait complete. 2023-11-06T01:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 -- start start 2023-11-06T01:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 Processor -- start 2023-11-06T01:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517680 0x563c2b521520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 0x563c2b521a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b518ed0 0x563c2b528aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b4054b0 con 0x563c2b517fe0 2023-11-06T01:47:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b405150 con 0x563c2b517680 2023-11-06T01:47:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b2ec8c0 con 0x563c2b518ed0 2023-11-06T01:47:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.624+0000 7febc4f6bb80 10 client.0 _ll_register_callbacks cb 0x563c2b352a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb1d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 0x563c2b521a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b518ed0 0x563c2b528aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517680 0x563c2b521520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517680 0x563c2b521520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55012/0 (socket says 172.21.15.53:55012) 2023-11-06T01:47:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb1d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 0x563c2b521a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43576/0 (socket says 172.21.15.53:43576) 2023-11-06T01:47:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b518ed0 0x563c2b528aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57818/0 (socket says 172.21.15.53:57818) 2023-11-06T01:47:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb259c700 1 -- 172.21.15.53:0/3530284131 learned_addr learned my addr 172.21.15.53:0/3530284131 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb2d9d700 1 -- 172.21.15.53:0/3530284131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517680 msgr2=0x563c2b521520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb2d9d700 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517680 0x563c2b521520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb2d9d700 1 -- 172.21.15.53:0/3530284131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 msgr2=0x563c2b521a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb2d9d700 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 0x563c2b521a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb2d9d700 1 -- 172.21.15.53:0/3530284131 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb4108a70 con 0x563c2b518ed0 2023-11-06T01:47:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb1d9b700 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 0x563c2b521a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb259c700 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517680 0x563c2b521520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febb2d9d700 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b518ed0 0x563c2b528aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x563c2b5186b0 tx=0x7febac00c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=379fe9cb68dd73ae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7feb99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:47:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7feb99ffb700 1 -- 172.21.15.53:0/3530284131 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac09c450 con 0x563c2b518ed0 2023-11-06T01:47:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7feb99ffb700 1 -- 172.21.15.53:0/3530284131 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7febac00cd50 con 0x563c2b518ed0 2023-11-06T01:47:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3530284131 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c2b493cb0 con 0x563c2b518ed0 2023-11-06T01:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7feb99ffb700 1 -- 172.21.15.53:0/3530284131 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7febac0a0200 con 0x563c2b518ed0 2023-11-06T01:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.625+0000 7febc4f6bb80 10 client.26482 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.626+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3530284131 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7febb41937e0 con 0x563c2b518ed0 2023-11-06T01:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.626+0000 7feb99ffb700 1 -- 172.21.15.53:0/3530284131 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c2b493cb0 con 0x563c2b518ed0 2023-11-06T01:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.626+0000 7feb99ffb700 1 -- 172.21.15.53:0/3530284131 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7febac09d190 con 0x563c2b518ed0 2023-11-06T01:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.626+0000 7febc4f6bb80 10 client.26482 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.626+0000 7febc4f6bb80 10 client.26482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.626+0000 7febc4f6bb80 20 client.26482 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.626+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563c2b52ce40 0x563c2b52f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.626+0000 7febc4f6bb80 4 client.26482 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.626+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3530284131 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c2b4e6db0 con 0x563c2b52ce40 2023-11-06T01:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.626+0000 7febb259c700 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563c2b52ce40 0x563c2b52f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7feb99ffb700 10 client.26482 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7febb259c700 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563c2b52ce40 0x563c2b52f230 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f72b637ca9207 server_cookie=83ba835685218d2b in_seq=0 out_seq=0 2023-11-06T01:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7feb99ffb700 1 -- 172.21.15.53:0/3530284131 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563c2b4e6db0 con 0x563c2b52ce40 2023-11-06T01:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7feb99ffb700 10 client.26482 handle_command_reply: tid=0 2023-11-06T01:47:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7febc4f6bb80 1 client.26482 shutdown 2023-11-06T01:47:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3530284131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b518ed0 msgr2=0x563c2b528aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b518ed0 0x563c2b528aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x563c2b5186b0 tx=0x7febac00c970 comp rx=0 tx=0).stop 2023-11-06T01:47:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3530284131 shutdown_connections 2023-11-06T01:47:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517680 0x563c2b521520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563c2b52ce40 0x563c2b52f230 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b518ed0 0x563c2b528aa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3530284131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 0x563c2b521a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3530284131 >> 172.21.15.53:0/3530284131 conn(0x563c2b477070 msgr2=0x563c2b5123c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.627+0000 7feb99ffb700 0 client.26482 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.630+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3530284131 shutdown_connections 2023-11-06T01:47:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.632+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3530284131 wait complete. 2023-11-06T01:47:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.632+0000 7febc4f6bb80 20 client.26482 trim_cache size 0 max 16384 2023-11-06T01:47:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.669+0000 7febc4f6bb80 1 Processor -- start 2023-11-06T01:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.671+0000 7febc4f6bb80 1 -- start start 2023-11-06T01:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.671+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 0x563c2b526a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.671+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 0x563c2b519310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.671+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b518ed0 0x563c2b51f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.671+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b50a7a0 con 0x563c2b517fe0 2023-11-06T01:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.671+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b51fbe0 con 0x563c2b518ed0 2023-11-06T01:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.671+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b5209d0 con 0x563c2b517680 2023-11-06T01:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.677+0000 7febb259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 0x563c2b526a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.677+0000 7febb259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 0x563c2b526a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57822/0 (socket says 172.21.15.53:57822) 2023-11-06T01:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.677+0000 7febb259c700 1 -- 172.21.15.53:0/1895814023 learned_addr learned my addr 172.21.15.53:0/1895814023 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.685+0000 7febb2d9d700 1 --2- 172.21.15.53:0/1895814023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b518ed0 0x563c2b51f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.686+0000 7febb1d9b700 1 --2- 172.21.15.53:0/1895814023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 0x563c2b519310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.687+0000 7febb259c700 1 -- 172.21.15.53:0/1895814023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b518ed0 msgr2=0x563c2b51f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.687+0000 7febb259c700 1 --2- 172.21.15.53:0/1895814023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b518ed0 0x563c2b51f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.687+0000 7febb259c700 1 -- 172.21.15.53:0/1895814023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 msgr2=0x563c2b519310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.687+0000 7febb259c700 1 --2- 172.21.15.53:0/1895814023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 0x563c2b519310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.687+0000 7febb259c700 1 -- 172.21.15.53:0/1895814023 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c2b27fc00 con 0x563c2b517680 2023-11-06T01:47:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.693+0000 7febb259c700 1 --2- 172.21.15.53:0/1895814023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 0x563c2b526a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x563c2b52fb40 tx=0x7feba4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2588d921e24184 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.696+0000 7feba17fa700 1 -- 172.21.15.53:0/1895814023 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4022040 con 0x563c2b517680 2023-11-06T01:47:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.696+0000 7feba17fa700 1 -- 172.21.15.53:0/1895814023 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feba4028410 con 0x563c2b517680 2023-11-06T01:47:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.696+0000 7feba17fa700 1 -- 172.21.15.53:0/1895814023 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4010690 con 0x563c2b517680 2023-11-06T01:47:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.697+0000 7febc4f6bb80 1 -- 172.21.15.53:0/1895814023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 msgr2=0x563c2b526a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.697+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/1895814023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 0x563c2b526a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x563c2b52fb40 tx=0x7feba4009f90 comp rx=0 tx=0).stop 2023-11-06T01:47:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.697+0000 7febc4f6bb80 1 -- 172.21.15.53:0/1895814023 shutdown_connections 2023-11-06T01:47:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.697+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/1895814023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b518ed0 0x563c2b51f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.697+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/1895814023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b517680 0x563c2b526a50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.697+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/1895814023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517fe0 0x563c2b519310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.697+0000 7febc4f6bb80 1 -- 172.21.15.53:0/1895814023 >> 172.21.15.53:0/1895814023 conn(0x563c2b477070 msgr2=0x563c2b520140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.701+0000 7febc4f6bb80 1 -- 172.21.15.53:0/1895814023 shutdown_connections 2023-11-06T01:47:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.706+0000 7febc4f6bb80 1 -- 172.21.15.53:0/1895814023 wait complete. 2023-11-06T01:47:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.706+0000 7febc4f6bb80 1 -- start start 2023-11-06T01:47:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.706+0000 7febc4f6bb80 1 Processor -- start 2023-11-06T01:47:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.706+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517680 0x563c2b522c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.706+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517fe0 0x563c2b5231c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.706+0000 7febc4f6bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b518ed0 0x563c2b525e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.706+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b5263b0 con 0x563c2b517680 2023-11-06T01:47:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.706+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b526520 con 0x563c2b517fe0 2023-11-06T01:47:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.706+0000 7febc4f6bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2b5266e0 con 0x563c2b518ed0 2023-11-06T01:47:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.706+0000 7febc4f6bb80 10 client.0 _ll_register_callbacks cb 0x7febac0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.707+0000 7febb259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517680 0x563c2b522c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.707+0000 7febb259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517680 0x563c2b522c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43588/0 (socket says 172.21.15.53:43588) 2023-11-06T01:47:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.707+0000 7febb259c700 1 -- 172.21.15.53:0/3469101278 learned_addr learned my addr 172.21.15.53:0/3469101278 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.707+0000 7febb259c700 1 -- 172.21.15.53:0/3469101278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b518ed0 msgr2=0x563c2b525e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.707+0000 7febb259c700 1 --2- 172.21.15.53:0/3469101278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b518ed0 0x563c2b525e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.707+0000 7febb259c700 1 -- 172.21.15.53:0/3469101278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517fe0 msgr2=0x563c2b5231c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:47:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.707+0000 7febb259c700 1 --2- 172.21.15.53:0/3469101278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517fe0 0x563c2b5231c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.707+0000 7febb259c700 1 -- 172.21.15.53:0/3469101278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba40287f0 con 0x563c2b517680 2023-11-06T01:47:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.707+0000 7febb259c700 1 --2- 172.21.15.53:0/3469101278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517680 0x563c2b522c80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feba4000c00 tx=0x7feba401d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47680961e0e0ea86 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.714+0000 7feb99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:47:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.714+0000 7feb99ffb700 1 -- 172.21.15.53:0/3469101278 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4022050 con 0x563c2b517680 2023-11-06T01:47:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.714+0000 7feb99ffb700 1 -- 172.21.15.53:0/3469101278 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7feba401d9a0 con 0x563c2b517680 2023-11-06T01:47:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.714+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3469101278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febac0a0690 con 0x563c2b517680 2023-11-06T01:47:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.719+0000 7feb99ffb700 1 -- 172.21.15.53:0/3469101278 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7feba401db60 con 0x563c2b517680 2023-11-06T01:47:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.719+0000 7febc4f6bb80 10 client.16908 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.719+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3469101278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x563c2b2bd1f0 con 0x563c2b517680 2023-11-06T01:47:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.720+0000 7feb99ffb700 1 -- 172.21.15.53:0/3469101278 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febac0a0690 con 0x563c2b517680 2023-11-06T01:47:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.728+0000 7feb99ffb700 1 -- 172.21.15.53:0/3469101278 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7feba4010430 con 0x563c2b517680 2023-11-06T01:47:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.728+0000 7febc4f6bb80 10 client.16908 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.728+0000 7febc4f6bb80 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.728+0000 7febc4f6bb80 20 client.16908 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.728+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3469101278 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563c2b590910 0x563c2b592d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.728+0000 7febc4f6bb80 4 client.16908 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:47:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.728+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3469101278 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563c2b461ce0 con 0x563c2b590910 2023-11-06T01:47:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.728+0000 7febb1d9b700 1 --2- 172.21.15.53:0/3469101278 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563c2b590910 0x563c2b592d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.735+0000 7feb99ffb700 10 client.16908 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.735+0000 7febb1d9b700 1 --2- 172.21.15.53:0/3469101278 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563c2b590910 0x563c2b592d30 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb9ba4f0e09b9b5 server_cookie=238b857aaba53b9 in_seq=0 out_seq=0 2023-11-06T01:47:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7feb99ffb700 1 -- 172.21.15.53:0/3469101278 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563c2b461ce0 con 0x563c2b590910 2023-11-06T01:47:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7feb99ffb700 10 client.16908 handle_command_reply: tid=0 2023-11-06T01:47:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7febc4f6bb80 1 client.16908 shutdown 2023-11-06T01:47:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3469101278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517680 msgr2=0x563c2b522c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3469101278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517680 0x563c2b522c80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feba4000c00 tx=0x7feba401d430 comp rx=0 tx=0).stop 2023-11-06T01:47:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3469101278 shutdown_connections 2023-11-06T01:47:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3469101278 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x563c2b590910 0x563c2b592d30 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3469101278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563c2b518ed0 0x563c2b525e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7feb99ffb700 0 client.16908 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3469101278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c2b517680 0x563c2b522c80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7febc4f6bb80 1 --2- 172.21.15.53:0/3469101278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c2b517fe0 0x563c2b5231c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3469101278 >> 172.21.15.53:0/3469101278 conn(0x563c2b477070 msgr2=0x563c2b51f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.752+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3469101278 shutdown_connections 2023-11-06T01:47:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febc4f6bb80 1 -- 172.21.15.53:0/3469101278 wait complete. 2023-11-06T01:47:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febc4f6bb80 20 client.16908 trim_cache size 0 max 16384 2023-11-06T01:47:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:47:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:47:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:47:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:47:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 -- 172.21.15.53:0/1849344486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb9c0a0670 msgr2=0x7feb9c0a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb9c0a0670 0x7feb9c0a2b30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7febb4190d20 tx=0x7feba40096c0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 -- 172.21.15.53:0/1849344486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febb4101870 msgr2=0x7febb41923c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febb4101870 0x7febb41923c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7febac00ef30 tx=0x7febac00c450 comp rx=0 tx=0).stop 2023-11-06T01:47:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 -- 172.21.15.53:0/1849344486 shutdown_connections 2023-11-06T01:47:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7feb9c0a0670 0x7feb9c0a2b30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febb4101870 0x7febb41923c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb40ffce0 0x7febb4190fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 --2- 172.21.15.53:0/1849344486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febb4100ed0 0x7febb41914f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 -- 172.21.15.53:0/1849344486 >> 172.21.15.53:0/1849344486 conn(0x7febb40fb250 msgr2=0x7febb40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 -- 172.21.15.53:0/1849344486 shutdown_connections 2023-11-06T01:47:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:14.753+0000 7febb8ddb700 1 -- 172.21.15.53:0/1849344486 wait complete. 2023-11-06T01:47:17.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 780.0 s, 1652.6 tps, lat 60.589 ms stddev 65.030 2023-11-06T01:47:17.352 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:17 smithi053 ceph-mon[99472]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 761 op/s 2023-11-06T01:47:17.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:17 smithi062 ceph-mon[103924]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 761 op/s 2023-11-06T01:47:17.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:17 smithi163 ceph-mon[104416]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 761 op/s 2023-11-06T01:47:19.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:19 smithi053 ceph-mon[99472]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 745 op/s 2023-11-06T01:47:19.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:19 smithi062 ceph-mon[103924]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 745 op/s 2023-11-06T01:47:19.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:19 smithi163 ceph-mon[104416]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 745 op/s 2023-11-06T01:47:21.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:21 smithi053 ceph-mon[99472]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 664 op/s 2023-11-06T01:47:21.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:21 smithi062 ceph-mon[103924]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 664 op/s 2023-11-06T01:47:21.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:21 smithi163 ceph-mon[104416]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 664 op/s 2023-11-06T01:47:22.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 785.0 s, 1309.6 tps, lat 75.801 ms stddev 85.268 2023-11-06T01:47:22.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:22 smithi163 ceph-mon[104416]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 736 op/s 2023-11-06T01:47:22.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:22 smithi053 ceph-mon[99472]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 736 op/s 2023-11-06T01:47:22.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:22 smithi062 ceph-mon[103924]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 736 op/s 2023-11-06T01:47:25.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:25 smithi053 ceph-mon[99472]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 12 MiB/s wr, 645 op/s 2023-11-06T01:47:25.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:25 smithi062 ceph-mon[103924]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 12 MiB/s wr, 645 op/s 2023-11-06T01:47:25.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:25 smithi163 ceph-mon[104416]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 12 MiB/s wr, 645 op/s 2023-11-06T01:47:27.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 790.0 s, 1183.3 tps, lat 84.475 ms stddev 96.986 2023-11-06T01:47:27.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:27 smithi053 ceph-mon[99472]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 698 op/s 2023-11-06T01:47:27.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:27 smithi062 ceph-mon[103924]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 698 op/s 2023-11-06T01:47:27.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:27 smithi163 ceph-mon[104416]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 698 op/s 2023-11-06T01:47:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.943+0000 7fcd97359700 1 -- 172.21.15.53:0/467050308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd9010d0d0 msgr2=0x7fcd9010d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.943+0000 7fcd97359700 1 --2- 172.21.15.53:0/467050308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd9010d0d0 0x7fcd9010d550 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800b0a0 tx=0x7fcd8800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:47:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.943+0000 7fcd97359700 1 -- 172.21.15.53:0/467050308 shutdown_connections 2023-11-06T01:47:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.943+0000 7fcd97359700 1 --2- 172.21.15.53:0/467050308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd9010d0d0 0x7fcd9010d550 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.943+0000 7fcd97359700 1 --2- 172.21.15.53:0/467050308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd90107d90 0x7fcd9010a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.943+0000 7fcd97359700 1 --2- 172.21.15.53:0/467050308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9010a700 0x7fcd9010cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.943+0000 7fcd97359700 1 -- 172.21.15.53:0/467050308 >> 172.21.15.53:0/467050308 conn(0x7fcd9006dbc0 msgr2=0x7fcd90070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.944+0000 7fcd97359700 1 -- 172.21.15.53:0/467050308 shutdown_connections 2023-11-06T01:47:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.944+0000 7fcd97359700 1 -- 172.21.15.53:0/467050308 wait complete. 2023-11-06T01:47:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.944+0000 7fcd97359700 1 Processor -- start 2023-11-06T01:47:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.944+0000 7fcd97359700 1 -- start start 2023-11-06T01:47:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd97359700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd90107d90 0x7fcd901a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd97359700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9010a700 0x7fcd901a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd97359700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd9010d0d0 0x7fcd901a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd97359700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd901a74e0 con 0x7fcd9010a700 2023-11-06T01:47:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd97359700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd901a7650 con 0x7fcd9010d0d0 2023-11-06T01:47:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd97359700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd901a7810 con 0x7fcd90107d90 2023-11-06T01:47:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd950f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd90107d90 0x7fcd901a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd948f4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9010a700 0x7fcd901a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd950f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd90107d90 0x7fcd901a5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57842/0 (socket says 172.21.15.53:57842) 2023-11-06T01:47:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd948f4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9010a700 0x7fcd901a6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43604/0 (socket says 172.21.15.53:43604) 2023-11-06T01:47:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd950f5700 1 -- 172.21.15.53:0/4035454624 learned_addr learned my addr 172.21.15.53:0/4035454624 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd958f6700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd9010d0d0 0x7fcd901a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd950f5700 1 -- 172.21.15.53:0/4035454624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd9010d0d0 msgr2=0x7fcd901a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd950f5700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd9010d0d0 0x7fcd901a6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd950f5700 1 -- 172.21.15.53:0/4035454624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9010a700 msgr2=0x7fcd901a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd950f5700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9010a700 0x7fcd901a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd950f5700 1 -- 172.21.15.53:0/4035454624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd88009d00 con 0x7fcd90107d90 2023-11-06T01:47:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.945+0000 7fcd948f4700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9010a700 0x7fcd901a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.946+0000 7fcd950f5700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd90107d90 0x7fcd901a5ac0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b550 tx=0x7fcd8000b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.946+0000 7fcd867fc700 1 -- 172.21.15.53:0/4035454624 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80011830 con 0x7fcd90107d90 2023-11-06T01:47:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.946+0000 7fcd97359700 1 -- 172.21.15.53:0/4035454624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd901a7af0 con 0x7fcd90107d90 2023-11-06T01:47:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.946+0000 7fcd867fc700 1 -- 172.21.15.53:0/4035454624 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd80011e50 con 0x7fcd90107d90 2023-11-06T01:47:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.946+0000 7fcd867fc700 1 -- 172.21.15.53:0/4035454624 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80010dc0 con 0x7fcd90107d90 2023-11-06T01:47:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.946+0000 7fcd97359700 1 -- 172.21.15.53:0/4035454624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd901ae330 con 0x7fcd90107d90 2023-11-06T01:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.949+0000 7fcd867fc700 1 -- 172.21.15.53:0/4035454624 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fcd80010380 con 0x7fcd90107d90 2023-11-06T01:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.950+0000 7fcd867fc700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcd7c0a0520 0x7fcd7c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.951+0000 7fcd867fc700 1 -- 172.21.15.53:0/4035454624 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcd800cb3a0 con 0x7fcd90107d90 2023-11-06T01:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.951+0000 7fcda34e9b80 1 Processor -- start 2023-11-06T01:47:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.957+0000 7fcd948f4700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcd7c0a0520 0x7fcd7c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.958+0000 7fcd948f4700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcd7c0a0520 0x7fcd7c0a29e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009de0 tx=0x7fcd8c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.959+0000 7fcda34e9b80 1 -- start start 2023-11-06T01:47:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.959+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359e680 0x56491359eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.959+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x56491359f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.959+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 0x5649135a0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.959+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649134f69b0 con 0x56491359fed0 2023-11-06T01:47:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.959+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649134f4370 con 0x56491359efe0 2023-11-06T01:47:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.959+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649134efc60 con 0x56491359e680 2023-11-06T01:47:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.962+0000 7fcd948f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x56491359f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.962+0000 7fcd948f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x56491359f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55050/0 (socket says 172.21.15.53:55050) 2023-11-06T01:47:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.962+0000 7fcd948f4700 1 -- 172.21.15.53:0/2902435078 learned_addr learned my addr 172.21.15.53:0/2902435078 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.963+0000 7fcd958f6700 1 --2- 172.21.15.53:0/2902435078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 0x5649135a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.968+0000 7fcd958f6700 1 -- 172.21.15.53:0/2902435078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359e680 msgr2=0x56491359eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.968+0000 7fcd958f6700 1 --2- 172.21.15.53:0/2902435078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359e680 0x56491359eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.968+0000 7fcd958f6700 1 -- 172.21.15.53:0/2902435078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 msgr2=0x56491359f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.968+0000 7fcd958f6700 1 --2- 172.21.15.53:0/2902435078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x56491359f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.968+0000 7fcd958f6700 1 -- 172.21.15.53:0/2902435078 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56491331dd80 con 0x56491359fed0 2023-11-06T01:47:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.979+0000 7fcd958f6700 1 --2- 172.21.15.53:0/2902435078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 0x5649135a0350 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800b070 tx=0x7fcd88007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e13376486f8c48 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.980+0000 7fcd7b7fe700 1 -- 172.21.15.53:0/2902435078 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800e040 con 0x56491359fed0 2023-11-06T01:47:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.980+0000 7fcd7b7fe700 1 -- 172.21.15.53:0/2902435078 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd880074e0 con 0x56491359fed0 2023-11-06T01:47:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.980+0000 7fcd7b7fe700 1 -- 172.21.15.53:0/2902435078 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd880053f0 con 0x56491359fed0 2023-11-06T01:47:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.981+0000 7fcda34e9b80 1 -- 172.21.15.53:0/2902435078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 msgr2=0x5649135a0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.981+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/2902435078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 0x5649135a0350 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800b070 tx=0x7fcd88007f20 comp rx=0 tx=0).stop 2023-11-06T01:47:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.992+0000 7fcda34e9b80 1 -- 172.21.15.53:0/2902435078 shutdown_connections 2023-11-06T01:47:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.992+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/2902435078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 0x5649135a0350 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:27.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.992+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/2902435078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359e680 0x56491359eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:27.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.992+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/2902435078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x56491359f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:27.997 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.992+0000 7fcda34e9b80 1 -- 172.21.15.53:0/2902435078 >> 172.21.15.53:0/2902435078 conn(0x5649134fe070 msgr2=0x56491359b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:27.998+0000 7fcda34e9b80 1 -- 172.21.15.53:0/2902435078 shutdown_connections 2023-11-06T01:47:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.005+0000 7fcda34e9b80 1 -- 172.21.15.53:0/2902435078 wait complete. 2023-11-06T01:47:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.005+0000 7fcda34e9b80 1 -- start start 2023-11-06T01:47:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.005+0000 7fcda34e9b80 1 Processor -- start 2023-11-06T01:47:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.006+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359e680 0x5649135b9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.006+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x5649135b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.006+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 0x5649135c0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.006+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491348c4b0 con 0x56491359fed0 2023-11-06T01:47:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.006+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491348c150 con 0x56491359efe0 2023-11-06T01:47:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.006+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649133738c0 con 0x56491359e680 2023-11-06T01:47:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.006+0000 7fcda34e9b80 10 client.0 _ll_register_callbacks cb 0x5649133d9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.008+0000 7fcd950f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359e680 0x5649135b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.008+0000 7fcd950f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359e680 0x5649135b9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57856/0 (socket says 172.21.15.53:57856) 2023-11-06T01:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.008+0000 7fcd950f5700 1 -- 172.21.15.53:0/1564923873 learned_addr learned my addr 172.21.15.53:0/1564923873 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.008+0000 7fcd958f6700 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 0x5649135c0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.008+0000 7fcd958f6700 1 -- 172.21.15.53:0/1564923873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359e680 msgr2=0x5649135b9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.008+0000 7fcd958f6700 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359e680 0x5649135b9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.008+0000 7fcd958f6700 1 -- 172.21.15.53:0/1564923873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 msgr2=0x5649135b9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.008+0000 7fcd958f6700 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x5649135b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.008+0000 7fcd958f6700 1 -- 172.21.15.53:0/1564923873 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9010e0d0 con 0x56491359fed0 2023-11-06T01:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.008+0000 7fcd958f6700 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 0x5649135c0aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcd88007ba0 tx=0x7fcd88007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6101302825fe0f4f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.008+0000 7fcd78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.009+0000 7fcd78ff9700 1 -- 172.21.15.53:0/1564923873 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800e040 con 0x56491359fed0 2023-11-06T01:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.009+0000 7fcd78ff9700 1 -- 172.21.15.53:0/1564923873 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcd88005370 con 0x56491359fed0 2023-11-06T01:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.012+0000 7fcda34e9b80 1 -- 172.21.15.53:0/1564923873 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56491351acb0 con 0x56491359fed0 2023-11-06T01:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.012+0000 7fcd78ff9700 1 -- 172.21.15.53:0/1564923873 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd88005530 con 0x56491359fed0 2023-11-06T01:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.014+0000 7fcda34e9b80 10 client.16917 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.014+0000 7fcda34e9b80 1 -- 172.21.15.53:0/1564923873 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564913306c00 con 0x56491359fed0 2023-11-06T01:47:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.020+0000 7fcd78ff9700 1 -- 172.21.15.53:0/1564923873 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56491351acb0 con 0x56491359fed0 2023-11-06T01:47:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.020+0000 7fcd78ff9700 1 -- 172.21.15.53:0/1564923873 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fcd88030070 con 0x56491359fed0 2023-11-06T01:47:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.020+0000 7fcda34e9b80 10 client.16917 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.020+0000 7fcda34e9b80 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.020+0000 7fcda34e9b80 20 client.16917 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.020+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5649135c4db0 0x5649135c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.020+0000 7fcda34e9b80 4 client.16917 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:47:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.020+0000 7fcda34e9b80 1 -- 172.21.15.53:0/1564923873 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56491356ddb0 con 0x5649135c4db0 2023-11-06T01:47:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.021+0000 7fcd950f5700 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5649135c4db0 0x5649135c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.022+0000 7fcd950f5700 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5649135c4db0 0x5649135c71a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1abc4e9d96a9f0c0 server_cookie=6851cf10049f901 in_seq=0 out_seq=0 2023-11-06T01:47:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.022+0000 7fcd78ff9700 10 client.16917 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:28.028 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.029+0000 7fcd78ff9700 1 -- 172.21.15.53:0/1564923873 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56491356ddb0 con 0x5649135c4db0 2023-11-06T01:47:28.028 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.029+0000 7fcd78ff9700 10 client.16917 handle_command_reply: tid=0 2023-11-06T01:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.029+0000 7fcda34e9b80 1 client.16917 shutdown 2023-11-06T01:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.029+0000 7fcda34e9b80 1 -- 172.21.15.53:0/1564923873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 msgr2=0x5649135c0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.029+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 0x5649135c0aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcd88007ba0 tx=0x7fcd88007700 comp rx=0 tx=0).stop 2023-11-06T01:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.034+0000 7fcda34e9b80 1 -- 172.21.15.53:0/1564923873 shutdown_connections 2023-11-06T01:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.034+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5649135c4db0 0x5649135c71a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.034+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359fed0 0x5649135c0aa0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.034+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359e680 0x5649135b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.034+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/1564923873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x5649135b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.034+0000 7fcda34e9b80 1 -- 172.21.15.53:0/1564923873 >> 172.21.15.53:0/1564923873 conn(0x5649134fe070 msgr2=0x5649135993c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.035+0000 7fcd78ff9700 0 client.16917 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.036+0000 7fcda34e9b80 1 -- 172.21.15.53:0/1564923873 shutdown_connections 2023-11-06T01:47:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.037+0000 7fcda34e9b80 1 -- 172.21.15.53:0/1564923873 wait complete. 2023-11-06T01:47:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.039+0000 7fcda34e9b80 20 client.16917 trim_cache size 0 max 16384 2023-11-06T01:47:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.060+0000 7fcda34e9b80 1 Processor -- start 2023-11-06T01:47:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.060+0000 7fcda34e9b80 1 -- start start 2023-11-06T01:47:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359e680 0x5649135bea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359efe0 0x5649135a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 0x5649135c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564913596710 con 0x56491359efe0 2023-11-06T01:47:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649135917a0 con 0x56491359e680 2023-11-06T01:47:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649135b89d0 con 0x56491359fed0 2023-11-06T01:47:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd948f4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359efe0 0x5649135a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd950f5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359e680 0x5649135bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd948f4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359efe0 0x5649135a02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43626/0 (socket says 172.21.15.53:43626) 2023-11-06T01:47:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd950f5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359e680 0x5649135bea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55062/0 (socket says 172.21.15.53:55062) 2023-11-06T01:47:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd948f4700 1 -- 172.21.15.53:0/3726383047 learned_addr learned my addr 172.21.15.53:0/3726383047 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd958f6700 1 --2- 172.21.15.53:0/3726383047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 0x5649135c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd958f6700 1 -- 172.21.15.53:0/3726383047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359e680 msgr2=0x5649135bea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd958f6700 1 --2- 172.21.15.53:0/3726383047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359e680 0x5649135bea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd958f6700 1 -- 172.21.15.53:0/3726383047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359efe0 msgr2=0x5649135a02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd958f6700 1 --2- 172.21.15.53:0/3726383047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359efe0 0x5649135a02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd958f6700 1 -- 172.21.15.53:0/3726383047 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649133441f0 con 0x56491359fed0 2023-11-06T01:47:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.061+0000 7fcd950f5700 1 --2- 172.21.15.53:0/3726383047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359e680 0x5649135bea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.062+0000 7fcd948f4700 1 --2- 172.21.15.53:0/3726383047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359efe0 0x5649135a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.062+0000 7fcd958f6700 1 --2- 172.21.15.53:0/3726383047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 0x5649135c12d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcd88029ee0 tx=0x7fcd88000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=682e46964b80dc77 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.062+0000 7fcd7bfff700 1 -- 172.21.15.53:0/3726383047 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800e030 con 0x56491359fed0 2023-11-06T01:47:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.062+0000 7fcd7bfff700 1 -- 172.21.15.53:0/3726383047 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd8801cb00 con 0x56491359fed0 2023-11-06T01:47:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.062+0000 7fcd7bfff700 1 -- 172.21.15.53:0/3726383047 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88012e60 con 0x56491359fed0 2023-11-06T01:47:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.063+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3726383047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 msgr2=0x5649135c12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.063+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/3726383047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 0x5649135c12d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcd88029ee0 tx=0x7fcd88000f40 comp rx=0 tx=0).stop 2023-11-06T01:47:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.063+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3726383047 shutdown_connections 2023-11-06T01:47:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.063+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/3726383047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359e680 0x5649135bea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.063+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/3726383047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 0x5649135c12d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.063+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/3726383047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359efe0 0x5649135a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.063+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3726383047 >> 172.21.15.53:0/3726383047 conn(0x5649134fe070 msgr2=0x56491359ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.063+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3726383047 shutdown_connections 2023-11-06T01:47:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.063+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3726383047 wait complete. 2023-11-06T01:47:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.064+0000 7fcda34e9b80 1 -- start start 2023-11-06T01:47:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.064+0000 7fcda34e9b80 1 Processor -- start 2023-11-06T01:47:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.064+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359e680 0x5649135be1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x5649135be710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcda34e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 0x564913614f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649136154a0 con 0x56491359e680 2023-11-06T01:47:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564913615610 con 0x56491359efe0 2023-11-06T01:47:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcda34e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649136157d0 con 0x56491359fed0 2023-11-06T01:47:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcda34e9b80 10 client.0 _ll_register_callbacks cb 0x7fcd88007e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd958f6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 0x564913614f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd948f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x5649135be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd958f6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 0x564913614f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57872/0 (socket says 172.21.15.53:57872) 2023-11-06T01:47:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd958f6700 1 -- 172.21.15.53:0/3401031445 learned_addr learned my addr 172.21.15.53:0/3401031445 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd950f5700 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359e680 0x5649135be1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd958f6700 1 -- 172.21.15.53:0/3401031445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 msgr2=0x5649135be710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd958f6700 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x5649135be710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd958f6700 1 -- 172.21.15.53:0/3401031445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359e680 msgr2=0x5649135be1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd958f6700 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359e680 0x5649135be1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd958f6700 1 -- 172.21.15.53:0/3401031445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd901ae330 con 0x56491359fed0 2023-11-06T01:47:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd958f6700 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 0x564913614f60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800d010 tx=0x7fcd88012d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26896a28f50c55ae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.065+0000 7fcd78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:47:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcd78ff9700 1 -- 172.21.15.53:0/3401031445 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8801c850 con 0x56491359fed0 2023-11-06T01:47:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcd78ff9700 1 -- 172.21.15.53:0/3401031445 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcd8801cc70 con 0x56491359fed0 2023-11-06T01:47:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3401031445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd880074e0 con 0x56491359fed0 2023-11-06T01:47:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcd78ff9700 1 -- 172.21.15.53:0/3401031445 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd880050c0 con 0x56491359fed0 2023-11-06T01:47:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcda34e9b80 10 client.26497 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3401031445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fcd8c00ea50 con 0x56491359fed0 2023-11-06T01:47:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcd78ff9700 1 -- 172.21.15.53:0/3401031445 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd880074e0 con 0x56491359fed0 2023-11-06T01:47:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcd78ff9700 1 -- 172.21.15.53:0/3401031445 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fcd88030070 con 0x56491359fed0 2023-11-06T01:47:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcda34e9b80 10 client.26497 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcda34e9b80 10 client.26497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcda34e9b80 20 client.26497 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564913616c20 0x564913619040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcda34e9b80 4 client.26497 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:47:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.066+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3401031445 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5649134e8ce0 con 0x564913616c20 2023-11-06T01:47:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.067+0000 7fcd950f5700 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564913616c20 0x564913619040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.067+0000 7fcd78ff9700 10 client.26497 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.067+0000 7fcd950f5700 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564913616c20 0x564913619040 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90dc986cc1c636a5 server_cookie=7bd77b8490804913 in_seq=0 out_seq=0 2023-11-06T01:47:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcd78ff9700 1 -- 172.21.15.53:0/3401031445 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5649134e8ce0 con 0x564913616c20 2023-11-06T01:47:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcd78ff9700 10 client.26497 handle_command_reply: tid=0 2023-11-06T01:47:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 1 client.26497 shutdown 2023-11-06T01:47:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3401031445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 msgr2=0x564913614f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 0x564913614f60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800d010 tx=0x7fcd88012d90 comp rx=0 tx=0).stop 2023-11-06T01:47:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3401031445 shutdown_connections 2023-11-06T01:47:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x564913616c20 0x564913619040 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56491359fed0 0x564913614f60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcd78ff9700 0 client.26497 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56491359e680 0x5649135be1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 1 --2- 172.21.15.53:0/3401031445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56491359efe0 0x5649135be710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3401031445 >> 172.21.15.53:0/3401031445 conn(0x5649134fe070 msgr2=0x5649135c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3401031445 shutdown_connections 2023-11-06T01:47:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 1 -- 172.21.15.53:0/3401031445 wait complete. 2023-11-06T01:47:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.078+0000 7fcda34e9b80 20 client.26497 trim_cache size 0 max 16384 2023-11-06T01:47:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:47:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:47:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:47:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:47:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 -- 172.21.15.53:0/4035454624 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcd7c0a0520 msgr2=0x7fcd7c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcd7c0a0520 0x7fcd7c0a29e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009de0 tx=0x7fcd8c009190 comp rx=0 tx=0).stop 2023-11-06T01:47:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 -- 172.21.15.53:0/4035454624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd90107d90 msgr2=0x7fcd901a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd90107d90 0x7fcd901a5ac0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b550 tx=0x7fcd8000b860 comp rx=0 tx=0).stop 2023-11-06T01:47:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 -- 172.21.15.53:0/4035454624 shutdown_connections 2023-11-06T01:47:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fcd7c0a0520 0x7fcd7c0a29e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd9010d0d0 0x7fcd901a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd90107d90 0x7fcd901a5ac0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 --2- 172.21.15.53:0/4035454624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9010a700 0x7fcd901a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 -- 172.21.15.53:0/4035454624 >> 172.21.15.53:0/4035454624 conn(0x7fcd9006dbc0 msgr2=0x7fcd90109630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 -- 172.21.15.53:0/4035454624 shutdown_connections 2023-11-06T01:47:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:28.079+0000 7fcd97359700 1 -- 172.21.15.53:0/4035454624 wait complete. 2023-11-06T01:47:29.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:29 smithi053 ceph-mon[99472]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 662 op/s 2023-11-06T01:47:29.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:29 smithi062 ceph-mon[103924]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 662 op/s 2023-11-06T01:47:29.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:29 smithi163 ceph-mon[104416]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 662 op/s 2023-11-06T01:47:31.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:31 smithi053 ceph-mon[99472]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 680 op/s 2023-11-06T01:47:31.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:31 smithi062 ceph-mon[103924]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 680 op/s 2023-11-06T01:47:31.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:31 smithi163 ceph-mon[104416]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 680 op/s 2023-11-06T01:47:32.186 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 795.0 s, 1527.4 tps, lat 65.417 ms stddev 74.414 2023-11-06T01:47:32.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:32 smithi163 ceph-mon[104416]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 731 op/s 2023-11-06T01:47:32.841 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:32 smithi053 ceph-mon[99472]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 731 op/s 2023-11-06T01:47:32.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:32 smithi062 ceph-mon[103924]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 731 op/s 2023-11-06T01:47:35.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:35 smithi053 ceph-mon[99472]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 654 op/s 2023-11-06T01:47:35.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:35 smithi062 ceph-mon[103924]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 654 op/s 2023-11-06T01:47:35.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:35 smithi163 ceph-mon[104416]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 654 op/s 2023-11-06T01:47:37.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 800.0 s, 1450.5 tps, lat 69.535 ms stddev 78.175 2023-11-06T01:47:37.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:37 smithi053 ceph-mon[99472]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 14 MiB/s wr, 699 op/s 2023-11-06T01:47:37.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:37 smithi062 ceph-mon[103924]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 14 MiB/s wr, 699 op/s 2023-11-06T01:47:37.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:37 smithi163 ceph-mon[104416]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 14 MiB/s wr, 699 op/s 2023-11-06T01:47:39.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:39 smithi062 ceph-mon[103924]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 15 MiB/s wr, 691 op/s 2023-11-06T01:47:39.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:39 smithi163 ceph-mon[104416]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 15 MiB/s wr, 691 op/s 2023-11-06T01:47:39.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:39 smithi053 ceph-mon[99472]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 15 MiB/s wr, 691 op/s 2023-11-06T01:47:40.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:40 smithi062 ceph-mon[103924]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 15 MiB/s wr, 741 op/s 2023-11-06T01:47:40.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:40 smithi163 ceph-mon[104416]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 15 MiB/s wr, 741 op/s 2023-11-06T01:47:40.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:40 smithi053 ceph-mon[99472]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 15 MiB/s wr, 741 op/s 2023-11-06T01:47:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.227+0000 7f87c6c03700 1 -- 172.21.15.53:0/2441357140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87c0101fa0 msgr2=0x7f87c0102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.227+0000 7f87c6c03700 1 --2- 172.21.15.53:0/2441357140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87c0101fa0 0x7f87c0102400 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0098f0 tx=0x7f87ac009c00 comp rx=0 tx=0).stop 2023-11-06T01:47:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.228+0000 7f87c6c03700 1 -- 172.21.15.53:0/2441357140 shutdown_connections 2023-11-06T01:47:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.228+0000 7f87c6c03700 1 --2- 172.21.15.53:0/2441357140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87c0100db0 0x7f87c01011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.228+0000 7f87c6c03700 1 --2- 172.21.15.53:0/2441357140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0102940 0x7f87c01091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.228+0000 7f87c6c03700 1 --2- 172.21.15.53:0/2441357140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87c0101fa0 0x7f87c0102400 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.228+0000 7f87c6c03700 1 -- 172.21.15.53:0/2441357140 >> 172.21.15.53:0/2441357140 conn(0x7f87c00fc340 msgr2=0x7f87c00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.228+0000 7f87c6c03700 1 -- 172.21.15.53:0/2441357140 shutdown_connections 2023-11-06T01:47:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.228+0000 7f87c6c03700 1 -- 172.21.15.53:0/2441357140 wait complete. 2023-11-06T01:47:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87c6c03700 1 Processor -- start 2023-11-06T01:47:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87c6c03700 1 -- start start 2023-11-06T01:47:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87c6c03700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0100db0 0x7f87c006f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87c6c03700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87c0101fa0 0x7f87c006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87c6c03700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87c0102940 0x7f87c0070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87c6c03700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0071180 con 0x7f87c0100db0 2023-11-06T01:47:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87c6c03700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00712f0 con 0x7f87c0101fa0 2023-11-06T01:47:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87c6c03700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00714b0 con 0x7f87c0102940 2023-11-06T01:47:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87c499f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0100db0 0x7f87c006f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87bffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87c0101fa0 0x7f87c006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87bffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87c0101fa0 0x7f87c006fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55092/0 (socket says 172.21.15.53:55092) 2023-11-06T01:47:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87bffff700 1 -- 172.21.15.53:0/109921750 learned_addr learned my addr 172.21.15.53:0/109921750 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.229+0000 7f87c51a0700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87c0102940 0x7f87c0070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.230+0000 7f87bffff700 1 -- 172.21.15.53:0/109921750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87c0102940 msgr2=0x7f87c0070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.230+0000 7f87bffff700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87c0102940 0x7f87c0070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.230+0000 7f87bffff700 1 -- 172.21.15.53:0/109921750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0100db0 msgr2=0x7f87c006f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.230+0000 7f87bffff700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0100db0 0x7f87c006f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.230+0000 7f87bffff700 1 -- 172.21.15.53:0/109921750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac009520 con 0x7f87c0101fa0 2023-11-06T01:47:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.230+0000 7f87bffff700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87c0101fa0 0x7f87c006fc50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00c010 tx=0x7f87ac00b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.230+0000 7f87bdffb700 1 -- 172.21.15.53:0/109921750 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac01c070 con 0x7f87c0101fa0 2023-11-06T01:47:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.230+0000 7f87bdffb700 1 -- 172.21.15.53:0/109921750 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87ac00bbc0 con 0x7f87c0101fa0 2023-11-06T01:47:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.230+0000 7f87c6c03700 1 -- 172.21.15.53:0/109921750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c0071730 con 0x7f87c0101fa0 2023-11-06T01:47:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.230+0000 7f87bdffb700 1 -- 172.21.15.53:0/109921750 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac004a90 con 0x7f87c0101fa0 2023-11-06T01:47:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.230+0000 7f87c6c03700 1 -- 172.21.15.53:0/109921750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c0071f40 con 0x7f87c0101fa0 2023-11-06T01:47:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.233+0000 7f87bdffb700 1 -- 172.21.15.53:0/109921750 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f87ac020070 con 0x7f87c0101fa0 2023-11-06T01:47:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.234+0000 7f87bdffb700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f87b00a0520 0x7f87b00a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.234+0000 7f87bdffb700 1 -- 172.21.15.53:0/109921750 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f87ac0cc700 con 0x7f87c0101fa0 2023-11-06T01:47:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.234+0000 7f87c499f700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f87b00a0520 0x7f87b00a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.234+0000 7f87d2d93b80 1 Processor -- start 2023-11-06T01:47:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87d2d93b80 1 -- start start 2023-11-06T01:47:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 0x5631b501f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 0x5631b5020100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b50206e0 0x5631b5020b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b4f6e9b0 con 0x5631b50206e0 2023-11-06T01:47:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b4f6c370 con 0x5631b501ee90 2023-11-06T01:47:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b4f67c60 con 0x5631b501f7f0 2023-11-06T01:47:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87c499f700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f87b00a0520 0x7f87b00a29e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f87c006f480 tx=0x7f87b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87c499f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 0x5631b501f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87c499f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 0x5631b501f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55100/0 (socket says 172.21.15.53:55100) 2023-11-06T01:47:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87c499f700 1 -- 172.21.15.53:0/1898420843 learned_addr learned my addr 172.21.15.53:0/1898420843 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87bffff700 1 --2- 172.21.15.53:0/1898420843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 0x5631b5020100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.235+0000 7f87c51a0700 1 --2- 172.21.15.53:0/1898420843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b50206e0 0x5631b5020b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.236+0000 7f87c499f700 1 -- 172.21.15.53:0/1898420843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 msgr2=0x5631b5020100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.236+0000 7f87c499f700 1 --2- 172.21.15.53:0/1898420843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 0x5631b5020100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.236+0000 7f87c499f700 1 -- 172.21.15.53:0/1898420843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b50206e0 msgr2=0x5631b5020b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.236+0000 7f87c499f700 1 --2- 172.21.15.53:0/1898420843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b50206e0 0x5631b5020b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.236+0000 7f87c499f700 1 -- 172.21.15.53:0/1898420843 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631b4d95d80 con 0x5631b501ee90 2023-11-06T01:47:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.236+0000 7f87c499f700 1 --2- 172.21.15.53:0/1898420843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 0x5631b501f2b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f87b4011d30 tx=0x7f87b4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c751cefa7ab9968 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.236+0000 7f87aaffd700 1 -- 172.21.15.53:0/1898420843 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400eb20 con 0x5631b501ee90 2023-11-06T01:47:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.236+0000 7f87aaffd700 1 -- 172.21.15.53:0/1898420843 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87b4021070 con 0x5631b501ee90 2023-11-06T01:47:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.236+0000 7f87aaffd700 1 -- 172.21.15.53:0/1898420843 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b401cea0 con 0x5631b501ee90 2023-11-06T01:47:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.237+0000 7f87d2d93b80 1 -- 172.21.15.53:0/1898420843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 msgr2=0x5631b501f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.237+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/1898420843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 0x5631b501f2b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f87b4011d30 tx=0x7f87b4014000 comp rx=0 tx=0).stop 2023-11-06T01:47:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.237+0000 7f87d2d93b80 1 -- 172.21.15.53:0/1898420843 shutdown_connections 2023-11-06T01:47:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.237+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/1898420843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 0x5631b501f2b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.237+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/1898420843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b50206e0 0x5631b5020b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.237+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/1898420843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 0x5631b5020100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.237+0000 7f87d2d93b80 1 -- 172.21.15.53:0/1898420843 >> 172.21.15.53:0/1898420843 conn(0x5631b4f76070 msgr2=0x5631b5013040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.237+0000 7f87d2d93b80 1 -- 172.21.15.53:0/1898420843 shutdown_connections 2023-11-06T01:47:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.237+0000 7f87d2d93b80 1 -- 172.21.15.53:0/1898420843 wait complete. 2023-11-06T01:47:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.238+0000 7f87d2d93b80 1 -- start start 2023-11-06T01:47:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.238+0000 7f87d2d93b80 1 Processor -- start 2023-11-06T01:47:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.238+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 0x5631b5017c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.238+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501f7f0 0x5631b5018150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.238+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b50206e0 0x5631b501d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.238+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b4f044b0 con 0x5631b501f7f0 2023-11-06T01:47:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.238+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b4f04150 con 0x5631b501ee90 2023-11-06T01:47:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.238+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b4deb8c0 con 0x5631b50206e0 2023-11-06T01:47:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87d2d93b80 10 client.0 _ll_register_callbacks cb 0x5631b4e51a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87bffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501f7f0 0x5631b5018150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87c499f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 0x5631b5017c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87bffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501f7f0 0x5631b5018150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43668/0 (socket says 172.21.15.53:43668) 2023-11-06T01:47:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87bffff700 1 -- 172.21.15.53:0/2631893566 learned_addr learned my addr 172.21.15.53:0/2631893566 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87c51a0700 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b50206e0 0x5631b501d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87c499f700 1 -- 172.21.15.53:0/2631893566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b50206e0 msgr2=0x5631b501d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87c499f700 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b50206e0 0x5631b501d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87c499f700 1 -- 172.21.15.53:0/2631893566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501f7f0 msgr2=0x5631b5018150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87c499f700 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501f7f0 0x5631b5018150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87c499f700 1 -- 172.21.15.53:0/2631893566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b4017070 con 0x5631b501ee90 2023-11-06T01:47:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87bffff700 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501f7f0 0x5631b5018150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87c499f700 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 0x5631b5017c10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009340 tx=0x7f87b401ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373f6e486dc5b66d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f879ffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:47:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f879ffff700 1 -- 172.21.15.53:0/2631893566 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4011e20 con 0x5631b501ee90 2023-11-06T01:47:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f87d2d93b80 1 -- 172.21.15.53:0/2631893566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631b4f92cb0 con 0x5631b501ee90 2023-11-06T01:47:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.239+0000 7f879ffff700 1 -- 172.21.15.53:0/2631893566 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f87b400ed30 con 0x5631b501ee90 2023-11-06T01:47:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.240+0000 7f879ffff700 1 -- 172.21.15.53:0/2631893566 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87b401c410 con 0x5631b501ee90 2023-11-06T01:47:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.240+0000 7f87d2d93b80 10 client.26435 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.240+0000 7f87d2d93b80 1 -- 172.21.15.53:0/2631893566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5631b4d7ec00 con 0x5631b501ee90 2023-11-06T01:47:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.240+0000 7f879ffff700 1 -- 172.21.15.53:0/2631893566 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5631b4f92cb0 con 0x5631b501ee90 2023-11-06T01:47:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.240+0000 7f879ffff700 1 -- 172.21.15.53:0/2631893566 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f87b4004780 con 0x5631b501ee90 2023-11-06T01:47:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.240+0000 7f87d2d93b80 10 client.26435 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.240+0000 7f87d2d93b80 10 client.26435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.240+0000 7f87d2d93b80 20 client.26435 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.240+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5631b503d180 0x5631b503f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.240+0000 7f87d2d93b80 4 client.26435 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:47:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.240+0000 7f87d2d93b80 1 -- 172.21.15.53:0/2631893566 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631b4fe5db0 con 0x5631b503d180 2023-11-06T01:47:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.241+0000 7f87bffff700 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5631b503d180 0x5631b503f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.241+0000 7f87bffff700 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5631b503d180 0x5631b503f5a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=265d3aad4bfa27a8 server_cookie=cbbabb5c71dddfb4 in_seq=0 out_seq=0 2023-11-06T01:47:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.241+0000 7f879ffff700 10 client.26435 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f879ffff700 1 -- 172.21.15.53:0/2631893566 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5631b4fe5db0 con 0x5631b503d180 2023-11-06T01:47:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f879ffff700 10 client.26435 handle_command_reply: tid=0 2023-11-06T01:47:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 1 client.26435 shutdown 2023-11-06T01:47:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 1 -- 172.21.15.53:0/2631893566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 msgr2=0x5631b5017c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 0x5631b5017c10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009340 tx=0x7f87b401ce20 comp rx=0 tx=0).stop 2023-11-06T01:47:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 1 -- 172.21.15.53:0/2631893566 shutdown_connections 2023-11-06T01:47:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b501ee90 0x5631b5017c10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5631b503d180 0x5631b503f5a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b50206e0 0x5631b501d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f879ffff700 0 client.26435 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/2631893566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501f7f0 0x5631b5018150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 1 -- 172.21.15.53:0/2631893566 >> 172.21.15.53:0/2631893566 conn(0x5631b4f76070 msgr2=0x5631b50113c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 1 -- 172.21.15.53:0/2631893566 shutdown_connections 2023-11-06T01:47:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 1 -- 172.21.15.53:0/2631893566 wait complete. 2023-11-06T01:47:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.242+0000 7f87d2d93b80 20 client.26435 trim_cache size 0 max 16384 2023-11-06T01:47:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.277+0000 7f87d2d93b80 1 Processor -- start 2023-11-06T01:47:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.280+0000 7f87d2d93b80 1 -- start start 2023-11-06T01:47:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.281+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501ee90 0x5631b501d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.281+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 0x5631b5020ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.281+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b50206e0 0x5631b5016830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.281+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b5012880 con 0x5631b501ee90 2023-11-06T01:47:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.281+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b500c500 con 0x5631b50206e0 2023-11-06T01:47:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.281+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b5017780 con 0x5631b501f7f0 2023-11-06T01:47:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.281+0000 7f87c51a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b50206e0 0x5631b5016830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.281+0000 7f87c51a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b50206e0 0x5631b5016830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55112/0 (socket says 172.21.15.53:55112) 2023-11-06T01:47:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.281+0000 7f87c51a0700 1 -- 172.21.15.53:0/1403700512 learned_addr learned my addr 172.21.15.53:0/1403700512 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.290+0000 7f87c51a0700 1 -- 172.21.15.53:0/1403700512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 msgr2=0x5631b5020ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.290+0000 7f87c51a0700 1 --2- 172.21.15.53:0/1403700512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 0x5631b5020ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.290+0000 7f87c51a0700 1 -- 172.21.15.53:0/1403700512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501ee90 msgr2=0x5631b501d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.290+0000 7f87c51a0700 1 --2- 172.21.15.53:0/1403700512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501ee90 0x5631b501d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.290+0000 7f87c51a0700 1 -- 172.21.15.53:0/1403700512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631b4dbc1f0 con 0x5631b50206e0 2023-11-06T01:47:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.292+0000 7f87c51a0700 1 --2- 172.21.15.53:0/1403700512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b50206e0 0x5631b5016830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009610 tx=0x7f87b800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=562853478a4a5da server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.296+0000 7f87ab7fe700 1 -- 172.21.15.53:0/1403700512 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b800e040 con 0x5631b50206e0 2023-11-06T01:47:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.296+0000 7f87ab7fe700 1 -- 172.21.15.53:0/1403700512 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87b800f040 con 0x5631b50206e0 2023-11-06T01:47:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.296+0000 7f87ab7fe700 1 -- 172.21.15.53:0/1403700512 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8039680 con 0x5631b50206e0 2023-11-06T01:47:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.301+0000 7f87d2d93b80 1 -- 172.21.15.53:0/1403700512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b50206e0 msgr2=0x5631b5016830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.301+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/1403700512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b50206e0 0x5631b5016830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009610 tx=0x7f87b800b040 comp rx=0 tx=0).stop 2023-11-06T01:47:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.301+0000 7f87d2d93b80 1 -- 172.21.15.53:0/1403700512 shutdown_connections 2023-11-06T01:47:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.301+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/1403700512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b50206e0 0x5631b5016830 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.301+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/1403700512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501ee90 0x5631b501d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.301+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/1403700512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 0x5631b5020ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.301+0000 7f87d2d93b80 1 -- 172.21.15.53:0/1403700512 >> 172.21.15.53:0/1403700512 conn(0x5631b4f76070 msgr2=0x5631b5012c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.314+0000 7f87d2d93b80 1 -- 172.21.15.53:0/1403700512 shutdown_connections 2023-11-06T01:47:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.320+0000 7f87d2d93b80 1 -- 172.21.15.53:0/1403700512 wait complete. 2023-11-06T01:47:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.320+0000 7f87d2d93b80 1 -- start start 2023-11-06T01:47:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.321+0000 7f87d2d93b80 1 Processor -- start 2023-11-06T01:47:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.321+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501ee90 0x5631b5026de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.321+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 0x5631b5027320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.321+0000 7f87d2d93b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b50206e0 0x5631b502e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.321+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b502e8a0 con 0x5631b501ee90 2023-11-06T01:47:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.321+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b502ea10 con 0x5631b50206e0 2023-11-06T01:47:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.321+0000 7f87d2d93b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631b502ebd0 con 0x5631b501f7f0 2023-11-06T01:47:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.321+0000 7f87d2d93b80 10 client.0 _ll_register_callbacks cb 0x7f87ac005b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.321+0000 7f87c499f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501ee90 0x5631b5026de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.321+0000 7f87c499f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501ee90 0x5631b5026de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43680/0 (socket says 172.21.15.53:43680) 2023-11-06T01:47:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.321+0000 7f87c499f700 1 -- 172.21.15.53:0/121587506 learned_addr learned my addr 172.21.15.53:0/121587506 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.322+0000 7f87c499f700 1 -- 172.21.15.53:0/121587506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 msgr2=0x5631b5027320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.322+0000 7f87c499f700 1 --2- 172.21.15.53:0/121587506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 0x5631b5027320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.322+0000 7f87c499f700 1 -- 172.21.15.53:0/121587506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b50206e0 msgr2=0x5631b502e360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:47:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.322+0000 7f87c499f700 1 --2- 172.21.15.53:0/121587506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b50206e0 0x5631b502e360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.322+0000 7f87c499f700 1 -- 172.21.15.53:0/121587506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b80092c0 con 0x5631b501ee90 2023-11-06T01:47:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.322+0000 7f87c499f700 1 --2- 172.21.15.53:0/121587506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501ee90 0x5631b5026de0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f87b4016040 tx=0x7f87b4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994a95eefa4b0cde server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.322+0000 7f879ffff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:47:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.322+0000 7f879ffff700 1 -- 172.21.15.53:0/121587506 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4004710 con 0x5631b501ee90 2023-11-06T01:47:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.322+0000 7f879ffff700 1 -- 172.21.15.53:0/121587506 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f87b4004870 con 0x5631b501ee90 2023-11-06T01:47:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.322+0000 7f87d2d93b80 1 -- 172.21.15.53:0/121587506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87b4021070 con 0x5631b501ee90 2023-11-06T01:47:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.322+0000 7f879ffff700 1 -- 172.21.15.53:0/121587506 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87b401c410 con 0x5631b501ee90 2023-11-06T01:47:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.323+0000 7f87d2d93b80 10 client.16938 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.323+0000 7f87d2d93b80 1 -- 172.21.15.53:0/121587506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5631b4d94250 con 0x5631b501ee90 2023-11-06T01:47:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.323+0000 7f879ffff700 1 -- 172.21.15.53:0/121587506 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87b4021070 con 0x5631b501ee90 2023-11-06T01:47:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.323+0000 7f879ffff700 1 -- 172.21.15.53:0/121587506 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f87b404a020 con 0x5631b501ee90 2023-11-06T01:47:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.323+0000 7f87d2d93b80 10 client.16938 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.323+0000 7f87d2d93b80 10 client.16938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.323+0000 7f87d2d93b80 20 client.16938 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.323+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/121587506 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5631b5030260 0x5631b50326a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.323+0000 7f87d2d93b80 4 client.16938 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:47:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.323+0000 7f87d2d93b80 1 -- 172.21.15.53:0/121587506 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5631b4f60ce0 con 0x5631b5030260 2023-11-06T01:47:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.330+0000 7f87bffff700 1 --2- 172.21.15.53:0/121587506 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5631b5030260 0x5631b50326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.330+0000 7f87bffff700 1 --2- 172.21.15.53:0/121587506 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5631b5030260 0x5631b50326a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d29e916a0a1ae6 server_cookie=c3d13c3dac523ba0 in_seq=0 out_seq=0 2023-11-06T01:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.330+0000 7f879ffff700 10 client.16938 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f879ffff700 1 -- 172.21.15.53:0/121587506 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5631b4f60ce0 con 0x5631b5030260 2023-11-06T01:47:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f879ffff700 10 client.16938 handle_command_reply: tid=0 2023-11-06T01:47:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f87d2d93b80 1 client.16938 shutdown 2023-11-06T01:47:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f87d2d93b80 1 -- 172.21.15.53:0/121587506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501ee90 msgr2=0x5631b5026de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/121587506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501ee90 0x5631b5026de0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f87b4016040 tx=0x7f87b4005d50 comp rx=0 tx=0).stop 2023-11-06T01:47:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f87d2d93b80 1 -- 172.21.15.53:0/121587506 shutdown_connections 2023-11-06T01:47:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/121587506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631b50206e0 0x5631b502e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/121587506 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5631b5030260 0x5631b50326a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/121587506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5631b501ee90 0x5631b5026de0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f879ffff700 0 client.16938 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f87d2d93b80 1 --2- 172.21.15.53:0/121587506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5631b501f7f0 0x5631b5027320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.361+0000 7f87d2d93b80 1 -- 172.21.15.53:0/121587506 >> 172.21.15.53:0/121587506 conn(0x5631b4f76070 msgr2=0x5631b5016830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.362+0000 7f87d2d93b80 1 -- 172.21.15.53:0/121587506 shutdown_connections 2023-11-06T01:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.362+0000 7f87d2d93b80 1 -- 172.21.15.53:0/121587506 wait complete. 2023-11-06T01:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.362+0000 7f87d2d93b80 20 client.16938 trim_cache size 0 max 16384 2023-11-06T01:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:47:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:47:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.362+0000 7f87c6c03700 1 -- 172.21.15.53:0/109921750 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f87b00a0520 msgr2=0x7f87b00a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.362+0000 7f87c6c03700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f87b00a0520 0x7f87b00a29e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f87c006f480 tx=0x7f87b4009260 comp rx=0 tx=0).stop 2023-11-06T01:47:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.362+0000 7f87c6c03700 1 -- 172.21.15.53:0/109921750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87c0101fa0 msgr2=0x7f87c006fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.362+0000 7f87c6c03700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87c0101fa0 0x7f87c006fc50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00c010 tx=0x7f87ac00b560 comp rx=0 tx=0).stop 2023-11-06T01:47:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.362+0000 7f87c6c03700 1 -- 172.21.15.53:0/109921750 shutdown_connections 2023-11-06T01:47:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.362+0000 7f87c6c03700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f87b00a0520 0x7f87b00a29e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.363+0000 7f87c6c03700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87c0102940 0x7f87c0070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.363+0000 7f87c6c03700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0100db0 0x7f87c006f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.363+0000 7f87c6c03700 1 --2- 172.21.15.53:0/109921750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87c0101fa0 0x7f87c006fc50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.363+0000 7f87c6c03700 1 -- 172.21.15.53:0/109921750 >> 172.21.15.53:0/109921750 conn(0x7f87c00fc340 msgr2=0x7f87c00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.363+0000 7f87c6c03700 1 -- 172.21.15.53:0/109921750 shutdown_connections 2023-11-06T01:47:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:41.363+0000 7f87c6c03700 1 -- 172.21.15.53:0/109921750 wait complete. 2023-11-06T01:47:41.410 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:47:41.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.602+0000 7f5a14039700 1 -- 172.21.15.53:0/2902417351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a0c1041b0 msgr2=0x7f5a0c1065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.602+0000 7f5a14039700 1 --2- 172.21.15.53:0/2902417351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a0c1041b0 0x7f5a0c1065a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009990 tx=0x7f59fc009ca0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.602+0000 7f5a14039700 1 -- 172.21.15.53:0/2902417351 shutdown_connections 2023-11-06T01:47:41.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.602+0000 7f5a14039700 1 --2- 172.21.15.53:0/2902417351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c106ae0 0x7f5a0c108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.602+0000 7f5a14039700 1 --2- 172.21.15.53:0/2902417351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0c101880 0x7f5a0c103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.602+0000 7f5a14039700 1 --2- 172.21.15.53:0/2902417351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a0c1041b0 0x7f5a0c1065a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.602+0000 7f5a14039700 1 -- 172.21.15.53:0/2902417351 >> 172.21.15.53:0/2902417351 conn(0x7f5a0c0fb1f0 msgr2=0x7f5a0c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.602+0000 7f5a14039700 1 -- 172.21.15.53:0/2902417351 shutdown_connections 2023-11-06T01:47:41.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.602+0000 7f5a14039700 1 -- 172.21.15.53:0/2902417351 wait complete. 2023-11-06T01:47:41.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a14039700 1 Processor -- start 2023-11-06T01:47:41.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a14039700 1 -- start start 2023-11-06T01:47:41.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a14039700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0c101880 0x7f5a0c196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a14039700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c1041b0 0x7f5a0c197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a14039700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a0c106ae0 0x7f5a0c19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a14039700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c19e680 con 0x7f5a0c106ae0 2023-11-06T01:47:41.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a14039700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c19e7c0 con 0x7f5a0c1041b0 2023-11-06T01:47:41.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a14039700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c19e950 con 0x7f5a0c101880 2023-11-06T01:47:41.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a11dd5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0c101880 0x7f5a0c196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a125d6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a0c106ae0 0x7f5a0c19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a115d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c1041b0 0x7f5a0c197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a125d6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a0c106ae0 0x7f5a0c19e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43698/0 (socket says 172.21.15.53:43698) 2023-11-06T01:47:41.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a11dd5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0c101880 0x7f5a0c196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57934/0 (socket says 172.21.15.53:57934) 2023-11-06T01:47:41.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a115d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c1041b0 0x7f5a0c197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55132/0 (socket says 172.21.15.53:55132) 2023-11-06T01:47:41.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a125d6700 1 -- 172.21.15.53:0/2390531368 learned_addr learned my addr 172.21.15.53:0/2390531368 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:41.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a11dd5700 1 -- 172.21.15.53:0/2390531368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c1041b0 msgr2=0x7f5a0c197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a11dd5700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c1041b0 0x7f5a0c197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a11dd5700 1 -- 172.21.15.53:0/2390531368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a0c106ae0 msgr2=0x7f5a0c19e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a11dd5700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a0c106ae0 0x7f5a0c19e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a11dd5700 1 -- 172.21.15.53:0/2390531368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a04009c40 con 0x7f5a0c101880 2023-11-06T01:47:41.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.603+0000 7f5a125d6700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a0c106ae0 0x7f5a0c19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:41.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.604+0000 7f5a11dd5700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0c101880 0x7f5a0c196b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800e9f0 tx=0x7f5a0800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.604+0000 7f5a02ffd700 1 -- 172.21.15.53:0/2390531368 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800c940 con 0x7f5a0c101880 2023-11-06T01:47:41.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.604+0000 7f5a14039700 1 -- 172.21.15.53:0/2390531368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc0095f0 con 0x7f5a0c101880 2023-11-06T01:47:41.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.604+0000 7f5a02ffd700 1 -- 172.21.15.53:0/2390531368 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a08010430 con 0x7f5a0c101880 2023-11-06T01:47:41.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.604+0000 7f5a02ffd700 1 -- 172.21.15.53:0/2390531368 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08018680 con 0x7f5a0c101880 2023-11-06T01:47:41.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.604+0000 7f5a14039700 1 -- 172.21.15.53:0/2390531368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c19efb0 con 0x7f5a0c101880 2023-11-06T01:47:41.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.607+0000 7f5a02ffd700 1 -- 172.21.15.53:0/2390531368 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5a080105a0 con 0x7f5a0c101880 2023-11-06T01:47:41.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.607+0000 7f5a02ffd700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59f80a0570 0x7f59f80a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.608+0000 7f5a201c9b80 1 Processor -- start 2023-11-06T01:47:41.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.608+0000 7f5a02ffd700 1 -- 172.21.15.53:0/2390531368 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5a08014070 con 0x7f5a0c101880 2023-11-06T01:47:41.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.608+0000 7f5a201c9b80 1 -- start start 2023-11-06T01:47:41.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.609+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec050 0x55dddf0ec470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.609+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec9b0 0x55dddf0f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.609+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0f9fc0 0x55dddf0fc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.609+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddf043a10 con 0x55dddf0f9fc0 2023-11-06T01:47:41.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.609+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddf0413d0 con 0x55dddf0ec050 2023-11-06T01:47:41.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.609+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddf03ccc0 con 0x55dddf0ec9b0 2023-11-06T01:47:41.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.609+0000 7f5a115d4700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59f80a0570 0x7f59f80a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.609+0000 7f5a115d4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec9b0 0x55dddf0f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.609+0000 7f5a125d6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0f9fc0 0x55dddf0fc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.609+0000 7f5a125d6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0f9fc0 0x55dddf0fc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43702/0 (socket says 172.21.15.53:43702) 2023-11-06T01:47:41.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.609+0000 7f5a125d6700 1 -- 172.21.15.53:0/3685032879 learned_addr learned my addr 172.21.15.53:0/3685032879 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:41.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.610+0000 7f5a125d6700 1 -- 172.21.15.53:0/3685032879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec9b0 msgr2=0x55dddf0f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.610+0000 7f5a125d6700 1 --2- 172.21.15.53:0/3685032879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec9b0 0x55dddf0f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.610+0000 7f5a125d6700 1 -- 172.21.15.53:0/3685032879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec050 msgr2=0x55dddf0ec470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.610+0000 7f5a125d6700 1 --2- 172.21.15.53:0/3685032879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec050 0x55dddf0ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.610+0000 7f5a125d6700 1 -- 172.21.15.53:0/3685032879 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dddee6ade0 con 0x55dddf0f9fc0 2023-11-06T01:47:41.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.610+0000 7f5a125d6700 1 --2- 172.21.15.53:0/3685032879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0f9fc0 0x55dddf0fc3b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e700 tx=0x7f5a04009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bb298bfc5dadd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.610+0000 7f59f7fff700 1 -- 172.21.15.53:0/3685032879 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04004bb0 con 0x55dddf0f9fc0 2023-11-06T01:47:41.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.611+0000 7f59f7fff700 1 -- 172.21.15.53:0/3685032879 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a0400f040 con 0x55dddf0f9fc0 2023-11-06T01:47:41.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.611+0000 7f5a115d4700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59f80a0570 0x7f59f80a2a30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b2e0 tx=0x7f59fc00b9b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.611+0000 7f59f7fff700 1 -- 172.21.15.53:0/3685032879 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04007d80 con 0x55dddf0f9fc0 2023-11-06T01:47:41.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.611+0000 7f5a201c9b80 1 -- 172.21.15.53:0/3685032879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0f9fc0 msgr2=0x55dddf0fc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.611+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/3685032879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0f9fc0 0x55dddf0fc3b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e700 tx=0x7f5a04009920 comp rx=0 tx=0).stop 2023-11-06T01:47:41.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.612+0000 7f5a201c9b80 1 -- 172.21.15.53:0/3685032879 shutdown_connections 2023-11-06T01:47:41.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.612+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/3685032879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec050 0x55dddf0ec470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.612+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/3685032879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0f9fc0 0x55dddf0fc3b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.612+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/3685032879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec9b0 0x55dddf0f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.612+0000 7f5a201c9b80 1 -- 172.21.15.53:0/3685032879 >> 172.21.15.53:0/3685032879 conn(0x55dddf04b0d0 msgr2=0x55dddf0e6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.612+0000 7f5a201c9b80 1 -- 172.21.15.53:0/3685032879 shutdown_connections 2023-11-06T01:47:41.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.612+0000 7f5a201c9b80 1 -- 172.21.15.53:0/3685032879 wait complete. 2023-11-06T01:47:41.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.612+0000 7f5a201c9b80 1 -- start start 2023-11-06T01:47:41.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.612+0000 7f5a201c9b80 1 Processor -- start 2023-11-06T01:47:41.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf0fe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec9b0 0x55dddf0fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0f9fc0 0x55dddf1057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddefd9510 con 0x55dddf0ec050 2023-11-06T01:47:41.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddefd91b0 con 0x55dddf0f9fc0 2023-11-06T01:47:41.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddeec0920 con 0x55dddf0ec9b0 2023-11-06T01:47:41.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a201c9b80 10 client.0 _ll_register_callbacks cb 0x55dddef26a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:41.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a11dd5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf0fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a11dd5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf0fe260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43708/0 (socket says 172.21.15.53:43708) 2023-11-06T01:47:41.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a11dd5700 1 -- 172.21.15.53:0/1651032732 learned_addr learned my addr 172.21.15.53:0/1651032732 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:41.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a125d6700 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0f9fc0 0x55dddf1057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a11dd5700 1 -- 172.21.15.53:0/1651032732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec9b0 msgr2=0x55dddf0fe7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:47:41.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a11dd5700 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec9b0 0x55dddf0fe7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a11dd5700 1 -- 172.21.15.53:0/1651032732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0f9fc0 msgr2=0x55dddf1057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a11dd5700 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0f9fc0 0x55dddf1057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.613+0000 7f5a11dd5700 1 -- 172.21.15.53:0/1651032732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a04009260 con 0x55dddf0ec050 2023-11-06T01:47:41.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.614+0000 7f5a11dd5700 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf0fe260 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5a0801e000 tx=0x7f5a08099810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb00329927f5f9e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.614+0000 7f59f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:47:41.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.614+0000 7f59f57fa700 1 -- 172.21.15.53:0/1651032732 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0809d100 con 0x55dddf0ec050 2023-11-06T01:47:41.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.614+0000 7f59f57fa700 1 -- 172.21.15.53:0/1651032732 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5a080a1340 con 0x55dddf0ec050 2023-11-06T01:47:41.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.614+0000 7f5a201c9b80 1 -- 172.21.15.53:0/1651032732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dddf067d10 con 0x55dddf0ec050 2023-11-06T01:47:41.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.614+0000 7f59f57fa700 1 -- 172.21.15.53:0/1651032732 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a080a0eb0 con 0x55dddf0ec050 2023-11-06T01:47:41.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.614+0000 7f5a201c9b80 10 client.16950 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:41.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.614+0000 7f5a201c9b80 1 -- 172.21.15.53:0/1651032732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55dddee53c60 con 0x55dddf0ec050 2023-11-06T01:47:41.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.614+0000 7f59f57fa700 1 -- 172.21.15.53:0/1651032732 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dddf067d10 con 0x55dddf0ec050 2023-11-06T01:47:41.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.614+0000 7f59f57fa700 1 -- 172.21.15.53:0/1651032732 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f5a080951e0 con 0x55dddf0ec050 2023-11-06T01:47:41.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.615+0000 7f5a201c9b80 10 client.16950 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:41.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.615+0000 7f5a201c9b80 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:41.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.615+0000 7f5a201c9b80 20 client.16950 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:41.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.615+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dddf109f20 0x55dddf10c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.615+0000 7f5a201c9b80 4 client.16950 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:47:41.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.615+0000 7f5a201c9b80 1 -- 172.21.15.53:0/1651032732 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dddf0bae10 con 0x55dddf109f20 2023-11-06T01:47:41.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.617+0000 7f5a115d4700 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dddf109f20 0x55dddf10c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.617+0000 7f59f57fa700 10 client.16950 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:41.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.617+0000 7f5a115d4700 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dddf109f20 0x55dddf10c310 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd17d4d8dad26779 server_cookie=a8582d1dc895dce5 in_seq=0 out_seq=0 2023-11-06T01:47:41.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f59f57fa700 1 -- 172.21.15.53:0/1651032732 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55dddf0bae10 con 0x55dddf109f20 2023-11-06T01:47:41.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f59f57fa700 10 client.16950 handle_command_reply: tid=0 2023-11-06T01:47:41.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 1 client.16950 shutdown 2023-11-06T01:47:41.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 1 -- 172.21.15.53:0/1651032732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 msgr2=0x55dddf0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf0fe260 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5a0801e000 tx=0x7f5a08099810 comp rx=0 tx=0).stop 2023-11-06T01:47:41.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 1 -- 172.21.15.53:0/1651032732 shutdown_connections 2023-11-06T01:47:41.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0f9fc0 0x55dddf1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dddf109f20 0x55dddf10c310 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf0fe260 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/1651032732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec9b0 0x55dddf0fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f59f57fa700 0 client.16950 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:41.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 1 -- 172.21.15.53:0/1651032732 >> 172.21.15.53:0/1651032732 conn(0x55dddf04b0d0 msgr2=0x55dddf0e60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 1 -- 172.21.15.53:0/1651032732 shutdown_connections 2023-11-06T01:47:41.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 1 -- 172.21.15.53:0/1651032732 wait complete. 2023-11-06T01:47:41.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.618+0000 7f5a201c9b80 20 client.16950 trim_cache size 0 max 16384 2023-11-06T01:47:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.646+0000 7f5a201c9b80 1 Processor -- start 2023-11-06T01:47:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.646+0000 7f5a201c9b80 1 -- start start 2023-11-06T01:47:41.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.646+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf103790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.646+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf0f6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.646+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0f9fc0 0x55dddf0f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.646+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddf107510 con 0x55dddf0ec050 2023-11-06T01:47:41.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.646+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddf0fcd50 con 0x55dddf0ec9b0 2023-11-06T01:47:41.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.646+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddf0fd710 con 0x55dddf0f9fc0 2023-11-06T01:47:41.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a11dd5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a115d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf0f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a115d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf0f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55154/0 (socket says 172.21.15.53:55154) 2023-11-06T01:47:41.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a115d4700 1 -- 172.21.15.53:0/323389527 learned_addr learned my addr 172.21.15.53:0/323389527 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:41.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a11dd5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf103790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43716/0 (socket says 172.21.15.53:43716) 2023-11-06T01:47:41.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a125d6700 1 --2- 172.21.15.53:0/323389527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0f9fc0 0x55dddf0f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a115d4700 1 -- 172.21.15.53:0/323389527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0f9fc0 msgr2=0x55dddf0f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a115d4700 1 --2- 172.21.15.53:0/323389527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0f9fc0 0x55dddf0f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a115d4700 1 -- 172.21.15.53:0/323389527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 msgr2=0x55dddf103790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a115d4700 1 --2- 172.21.15.53:0/323389527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf103790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a115d4700 1 -- 172.21.15.53:0/323389527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dddee91250 con 0x55dddf0ec9b0 2023-11-06T01:47:41.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a11dd5700 1 --2- 172.21.15.53:0/323389527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:41.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a115d4700 1 --2- 172.21.15.53:0/323389527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf0f6160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b680 tx=0x7f59fc029000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0858b149cc917ce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a00ff9700 1 -- 172.21.15.53:0/323389527 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc01d040 con 0x55dddf0ec9b0 2023-11-06T01:47:41.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a00ff9700 1 -- 172.21.15.53:0/323389527 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59fc004d20 con 0x55dddf0ec9b0 2023-11-06T01:47:41.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.647+0000 7f5a00ff9700 1 -- 172.21.15.53:0/323389527 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc031850 con 0x55dddf0ec9b0 2023-11-06T01:47:41.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.648+0000 7f5a201c9b80 1 -- 172.21.15.53:0/323389527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 msgr2=0x55dddf0f6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.648+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/323389527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf0f6160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b680 tx=0x7f59fc029000 comp rx=0 tx=0).stop 2023-11-06T01:47:41.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.648+0000 7f5a201c9b80 1 -- 172.21.15.53:0/323389527 shutdown_connections 2023-11-06T01:47:41.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.648+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/323389527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0f9fc0 0x55dddf0f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.648+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/323389527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0ec050 0x55dddf103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.648+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/323389527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf0f6160 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.648+0000 7f5a201c9b80 1 -- 172.21.15.53:0/323389527 >> 172.21.15.53:0/323389527 conn(0x55dddf04b0d0 msgr2=0x55dddf0fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.648+0000 7f5a201c9b80 1 -- 172.21.15.53:0/323389527 shutdown_connections 2023-11-06T01:47:41.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.648+0000 7f5a201c9b80 1 -- 172.21.15.53:0/323389527 wait complete. 2023-11-06T01:47:41.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a201c9b80 1 -- start start 2023-11-06T01:47:41.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a201c9b80 1 Processor -- start 2023-11-06T01:47:41.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec050 0x55dddf153d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a201c9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0f9fc0 0x55dddf15b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddf15b850 con 0x55dddf0f9fc0 2023-11-06T01:47:41.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddf15b990 con 0x55dddf0ec9b0 2023-11-06T01:47:41.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a201c9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddf15bb20 con 0x55dddf0ec050 2023-11-06T01:47:41.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a201c9b80 10 client.0 _ll_register_callbacks cb 0x55dddef26a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:41.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a115d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a11dd5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec050 0x55dddf153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a115d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf1542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55162/0 (socket says 172.21.15.53:55162) 2023-11-06T01:47:41.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a11dd5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec050 0x55dddf153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57962/0 (socket says 172.21.15.53:57962) 2023-11-06T01:47:41.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.649+0000 7f5a115d4700 1 -- 172.21.15.53:0/2024833337 learned_addr learned my addr 172.21.15.53:0/2024833337 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:41.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.650+0000 7f5a115d4700 1 -- 172.21.15.53:0/2024833337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec050 msgr2=0x55dddf153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.650+0000 7f5a115d4700 1 --2- 172.21.15.53:0/2024833337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec050 0x55dddf153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.650+0000 7f5a115d4700 1 -- 172.21.15.53:0/2024833337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0f9fc0 msgr2=0x55dddf15b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:41.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.650+0000 7f5a115d4700 1 --2- 172.21.15.53:0/2024833337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0f9fc0 0x55dddf15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.650+0000 7f5a115d4700 1 -- 172.21.15.53:0/2024833337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dddee53c60 con 0x55dddf0ec9b0 2023-11-06T01:47:41.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.650+0000 7f5a115d4700 1 --2- 172.21.15.53:0/2024833337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf1542d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009350 tx=0x7f59fc00bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db09c3fed02195e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:41.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.650+0000 7f59f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:47:41.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.650+0000 7f59f57fa700 1 -- 172.21.15.53:0/2024833337 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc01d040 con 0x55dddf0ec9b0 2023-11-06T01:47:41.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.650+0000 7f59f57fa700 1 -- 172.21.15.53:0/2024833337 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f59fc004860 con 0x55dddf0ec9b0 2023-11-06T01:47:41.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.650+0000 7f5a201c9b80 1 -- 172.21.15.53:0/2024833337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a0400f040 con 0x55dddf0ec9b0 2023-11-06T01:47:41.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.654+0000 7f59f57fa700 1 -- 172.21.15.53:0/2024833337 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59fc004a20 con 0x55dddf0ec9b0 2023-11-06T01:47:41.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.654+0000 7f5a201c9b80 10 client.26453 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:41.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.654+0000 7f5a201c9b80 1 -- 172.21.15.53:0/2024833337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f59fc020040 con 0x55dddf0ec9b0 2023-11-06T01:47:41.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.654+0000 7f59f57fa700 1 -- 172.21.15.53:0/2024833337 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a0400f040 con 0x55dddf0ec9b0 2023-11-06T01:47:41.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.654+0000 7f59f57fa700 1 -- 172.21.15.53:0/2024833337 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f59fc0296e0 con 0x55dddf0ec9b0 2023-11-06T01:47:41.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.654+0000 7f5a201c9b80 10 client.26453 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:41.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.654+0000 7f5a201c9b80 10 client.26453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:41.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.654+0000 7f5a201c9b80 20 client.26453 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:41.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.654+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/2024833337 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dddf15d0f0 0x55dddf15f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:41.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.655+0000 7f5a201c9b80 4 client.26453 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:47:41.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.655+0000 7f5a201c9b80 1 -- 172.21.15.53:0/2024833337 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dddf035d40 con 0x55dddf15d0f0 2023-11-06T01:47:41.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.657+0000 7f5a11dd5700 1 --2- 172.21.15.53:0/2024833337 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dddf15d0f0 0x55dddf15f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:41.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.682+0000 7f5a11dd5700 1 --2- 172.21.15.53:0/2024833337 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dddf15d0f0 0x55dddf15f510 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ceba7d89304060d server_cookie=8336f69f8e193abe in_seq=0 out_seq=0 2023-11-06T01:47:41.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.689+0000 7f59f57fa700 10 client.26453 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:41.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.700+0000 7f59f57fa700 1 -- 172.21.15.53:0/2024833337 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dddf035d40 con 0x55dddf15d0f0 2023-11-06T01:47:41.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.700+0000 7f59f57fa700 10 client.26453 handle_command_reply: tid=0 2023-11-06T01:47:41.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.709+0000 7f5a201c9b80 1 client.26453 shutdown 2023-11-06T01:47:41.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.712+0000 7f5a201c9b80 1 -- 172.21.15.53:0/2024833337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 msgr2=0x55dddf1542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.712+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/2024833337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf1542d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009350 tx=0x7f59fc00bfb0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.718+0000 7f5a201c9b80 1 -- 172.21.15.53:0/2024833337 shutdown_connections 2023-11-06T01:47:41.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.718+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/2024833337 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55dddf15d0f0 0x55dddf15f510 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.718+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/2024833337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dddf0f9fc0 0x55dddf15b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.718+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/2024833337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dddf0ec050 0x55dddf153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.718+0000 7f5a201c9b80 1 --2- 172.21.15.53:0/2024833337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddf0ec9b0 0x55dddf1542d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.718+0000 7f5a201c9b80 1 -- 172.21.15.53:0/2024833337 >> 172.21.15.53:0/2024833337 conn(0x55dddf04b0d0 msgr2=0x55dddf0f6160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.722+0000 7f59f57fa700 0 client.26453 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:41.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.722+0000 7f5a201c9b80 1 -- 172.21.15.53:0/2024833337 shutdown_connections 2023-11-06T01:47:41.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.728+0000 7f5a201c9b80 1 -- 172.21.15.53:0/2024833337 wait complete. 2023-11-06T01:47:41.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.737+0000 7f5a201c9b80 20 client.26453 trim_cache size 0 max 16384 2023-11-06T01:47:41.741 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:47:41.742 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:47:41.742 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:47:41.742 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:47:41.742 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:47:41.742 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:47:41.743 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:47:41.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:47:41.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:47:41.743 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:47:41.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.749+0000 7f5a14039700 1 -- 172.21.15.53:0/2390531368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59f80a0570 msgr2=0x7f59f80a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.749+0000 7f5a14039700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59f80a0570 0x7f59f80a2a30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b2e0 tx=0x7f59fc00b9b0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.749+0000 7f5a14039700 1 -- 172.21.15.53:0/2390531368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0c101880 msgr2=0x7f5a0c196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:41.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.749+0000 7f5a14039700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0c101880 0x7f5a0c196b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800e9f0 tx=0x7f5a0800ed00 comp rx=0 tx=0).stop 2023-11-06T01:47:41.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.755+0000 7f5a14039700 1 -- 172.21.15.53:0/2390531368 shutdown_connections 2023-11-06T01:47:41.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.755+0000 7f5a14039700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f59f80a0570 0x7f59f80a2a30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.755+0000 7f5a14039700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a0c106ae0 0x7f5a0c19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.755+0000 7f5a14039700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0c101880 0x7f5a0c196b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.755+0000 7f5a14039700 1 --2- 172.21.15.53:0/2390531368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a0c1041b0 0x7f5a0c197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:41.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.755+0000 7f5a14039700 1 -- 172.21.15.53:0/2390531368 >> 172.21.15.53:0/2390531368 conn(0x7f5a0c0fb1f0 msgr2=0x7f5a0c102530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:41.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.764+0000 7f5a14039700 1 -- 172.21.15.53:0/2390531368 shutdown_connections 2023-11-06T01:47:41.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:47:41.766+0000 7f5a14039700 1 -- 172.21.15.53:0/2390531368 wait complete. 2023-11-06T01:47:41.814 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:47:42.185 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 805.0 s, 1247.6 tps, lat 76.396 ms stddev 87.308 2023-11-06T01:47:42.847 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:42 smithi053 ceph-mon[99472]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 760 op/s 2023-11-06T01:47:42.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:42 smithi062 ceph-mon[103924]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 760 op/s 2023-11-06T01:47:42.989 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:42 smithi163 ceph-mon[104416]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 760 op/s 2023-11-06T01:47:45.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:45 smithi053 ceph-mon[99472]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 19 MiB/s wr, 1.06k op/s 2023-11-06T01:47:45.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:45 smithi062 ceph-mon[103924]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 19 MiB/s wr, 1.06k op/s 2023-11-06T01:47:45.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:45 smithi163 ceph-mon[104416]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 19 MiB/s wr, 1.06k op/s 2023-11-06T01:47:47.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 810.0 s, 1403.1 tps, lat 74.619 ms stddev 144.484 2023-11-06T01:47:47.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:47 smithi053 ceph-mon[99472]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 1.54k op/s 2023-11-06T01:47:47.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:47 smithi062 ceph-mon[103924]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 1.54k op/s 2023-11-06T01:47:47.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:47 smithi163 ceph-mon[104416]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 1.54k op/s 2023-11-06T01:47:49.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:49 smithi062 ceph-mon[103924]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 1.92k op/s 2023-11-06T01:47:49.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:49 smithi163 ceph-mon[104416]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 1.92k op/s 2023-11-06T01:47:49.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:49 smithi053 ceph-mon[99472]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 1.92k op/s 2023-11-06T01:47:50.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:50 smithi062 ceph-mon[103924]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.96k op/s 2023-11-06T01:47:50.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:50 smithi163 ceph-mon[104416]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.96k op/s 2023-11-06T01:47:50.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:50 smithi053 ceph-mon[99472]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.96k op/s 2023-11-06T01:47:52.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 815.0 s, 1479.6 tps, lat 67.461 ms stddev 77.963 2023-11-06T01:47:52.841 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:52 smithi053 ceph-mon[99472]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.93k op/s 2023-11-06T01:47:52.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:52 smithi062 ceph-mon[103924]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.93k op/s 2023-11-06T01:47:52.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:52 smithi163 ceph-mon[104416]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.93k op/s 2023-11-06T01:47:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.534+0000 7fe082e7e700 1 -- 172.21.15.53:0/2987069821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe07c101820 msgr2=0x7fe07c1080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.534+0000 7fe082e7e700 1 --2- 172.21.15.53:0/2987069821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe07c101820 0x7fe07c1080d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe0640099c0 tx=0x7fe064009cd0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.534+0000 7fe082e7e700 1 -- 172.21.15.53:0/2987069821 shutdown_connections 2023-11-06T01:47:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.534+0000 7fe082e7e700 1 --2- 172.21.15.53:0/2987069821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe07c101820 0x7fe07c1080d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.534+0000 7fe082e7e700 1 --2- 172.21.15.53:0/2987069821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe07c0ffc90 0x7fe07c1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.534+0000 7fe082e7e700 1 --2- 172.21.15.53:0/2987069821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe07c100e80 0x7fe07c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.534+0000 7fe082e7e700 1 -- 172.21.15.53:0/2987069821 >> 172.21.15.53:0/2987069821 conn(0x7fe07c0fb1e0 msgr2=0x7fe07c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.538+0000 7fe082e7e700 1 -- 172.21.15.53:0/2987069821 shutdown_connections 2023-11-06T01:47:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.538+0000 7fe082e7e700 1 -- 172.21.15.53:0/2987069821 wait complete. 2023-11-06T01:47:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.539+0000 7fe082e7e700 1 Processor -- start 2023-11-06T01:47:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe082e7e700 1 -- start start 2023-11-06T01:47:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe082e7e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe07c0ffc90 0x7fe07c194910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe082e7e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe07c100e80 0x7fe07c194e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe082e7e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe07c101820 0x7fe07c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe082e7e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c19d5f0 con 0x7fe07c100e80 2023-11-06T01:47:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe082e7e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c195d20 con 0x7fe07c0ffc90 2023-11-06T01:47:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe082e7e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c195ee0 con 0x7fe07c101820 2023-11-06T01:47:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe080c1a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe07c0ffc90 0x7fe07c194910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe080c1a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe07c0ffc90 0x7fe07c194910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55174/0 (socket says 172.21.15.53:55174) 2023-11-06T01:47:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe080c1a700 1 -- 172.21.15.53:0/438315771 learned_addr learned my addr 172.21.15.53:0/438315771 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe07bfff700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe07c100e80 0x7fe07c194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe080c1a700 1 -- 172.21.15.53:0/438315771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe07c101820 msgr2=0x7fe07c19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe080c1a700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe07c101820 0x7fe07c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe080c1a700 1 -- 172.21.15.53:0/438315771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe07c100e80 msgr2=0x7fe07c194e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe080c1a700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe07c100e80 0x7fe07c194e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.541+0000 7fe080c1a700 1 -- 172.21.15.53:0/438315771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0640095f0 con 0x7fe07c0ffc90 2023-11-06T01:47:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.542+0000 7fe080c1a700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe07c0ffc90 0x7fe07c194910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe07400e970 tx=0x7fe07400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.542+0000 7fe079ffb700 1 -- 172.21.15.53:0/438315771 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074009790 con 0x7fe07c0ffc90 2023-11-06T01:47:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.542+0000 7fe079ffb700 1 -- 172.21.15.53:0/438315771 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe07400eec0 con 0x7fe07c0ffc90 2023-11-06T01:47:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.542+0000 7fe082e7e700 1 -- 172.21.15.53:0/438315771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07c1961c0 con 0x7fe07c0ffc90 2023-11-06T01:47:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.542+0000 7fe079ffb700 1 -- 172.21.15.53:0/438315771 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074003cf0 con 0x7fe07c0ffc90 2023-11-06T01:47:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.543+0000 7fe082e7e700 1 -- 172.21.15.53:0/438315771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07c196720 con 0x7fe07c0ffc90 2023-11-06T01:47:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.544+0000 7fe079ffb700 1 -- 172.21.15.53:0/438315771 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe07400ccd0 con 0x7fe07c0ffc90 2023-11-06T01:47:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.545+0000 7fe079ffb700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0680a05a0 0x7fe0680a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.545+0000 7fe079ffb700 1 -- 172.21.15.53:0/438315771 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe074014070 con 0x7fe07c0ffc90 2023-11-06T01:47:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.545+0000 7fe07bfff700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0680a05a0 0x7fe0680a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.546+0000 7fe07bfff700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0680a05a0 0x7fe0680a2a60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe07c195960 tx=0x7fe06c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.548+0000 7fe08f00eb80 1 Processor -- start 2023-11-06T01:47:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.548+0000 7fe08f00eb80 1 -- start start 2023-11-06T01:47:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.548+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1680 0x5585134a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.548+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1fe0 0x5585134a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.548+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134a3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.548+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585133f99b0 con 0x5585134a1680 2023-11-06T01:47:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.548+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585133f7370 con 0x5585134a2ed0 2023-11-06T01:47:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.548+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585133f2c60 con 0x5585134a1fe0 2023-11-06T01:47:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe07bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1fe0 0x5585134a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe08141b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe08141b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134a3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55186/0 (socket says 172.21.15.53:55186) 2023-11-06T01:47:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe07bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1fe0 0x5585134a28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57986/0 (socket says 172.21.15.53:57986) 2023-11-06T01:47:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe08141b700 1 -- 172.21.15.53:0/3676762504 learned_addr learned my addr 172.21.15.53:0/3676762504 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe08141b700 1 -- 172.21.15.53:0/3676762504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1fe0 msgr2=0x5585134a28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe08141b700 1 --2- 172.21.15.53:0/3676762504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1fe0 0x5585134a28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe08141b700 1 -- 172.21.15.53:0/3676762504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1680 msgr2=0x5585134a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe08141b700 1 --2- 172.21.15.53:0/3676762504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1680 0x5585134a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe08141b700 1 -- 172.21.15.53:0/3676762504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558513220d80 con 0x5585134a2ed0 2023-11-06T01:47:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe08141b700 1 --2- 172.21.15.53:0/3676762504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134a3350 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe064005950 tx=0x7fe064005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ccc824359c88dfb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe072ffd700 1 -- 172.21.15.53:0/3676762504 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06401c070 con 0x5585134a2ed0 2023-11-06T01:47:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe072ffd700 1 -- 172.21.15.53:0/3676762504 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe06400bc50 con 0x5585134a2ed0 2023-11-06T01:47:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.549+0000 7fe072ffd700 1 -- 172.21.15.53:0/3676762504 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064022420 con 0x5585134a2ed0 2023-11-06T01:47:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.550+0000 7fe08f00eb80 1 -- 172.21.15.53:0/3676762504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 msgr2=0x5585134a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.550+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/3676762504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134a3350 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe064005950 tx=0x7fe064005c00 comp rx=0 tx=0).stop 2023-11-06T01:47:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.550+0000 7fe08f00eb80 1 -- 172.21.15.53:0/3676762504 shutdown_connections 2023-11-06T01:47:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.550+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/3676762504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134a3350 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.550+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/3676762504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1680 0x5585134a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.550+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/3676762504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1fe0 0x5585134a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.550+0000 7fe08f00eb80 1 -- 172.21.15.53:0/3676762504 >> 172.21.15.53:0/3676762504 conn(0x558513401070 msgr2=0x55851349e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.551+0000 7fe08f00eb80 1 -- 172.21.15.53:0/3676762504 shutdown_connections 2023-11-06T01:47:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.551+0000 7fe08f00eb80 1 -- 172.21.15.53:0/3676762504 wait complete. 2023-11-06T01:47:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.551+0000 7fe08f00eb80 1 -- start start 2023-11-06T01:47:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.551+0000 7fe08f00eb80 1 Processor -- start 2023-11-06T01:47:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134ab490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 0x5585134ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55851338f4b0 con 0x5585134a1fe0 2023-11-06T01:47:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55851338f150 con 0x5585134a2ed0 2023-11-06T01:47:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585132768c0 con 0x5585134a1680 2023-11-06T01:47:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe08f00eb80 10 client.0 _ll_register_callbacks cb 0x5585132dca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe07bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 0x5585134ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe080c1a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe080c1a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134ab490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57990/0 (socket says 172.21.15.53:57990) 2023-11-06T01:47:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe07bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 0x5585134ab9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43752/0 (socket says 172.21.15.53:43752) 2023-11-06T01:47:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe080c1a700 1 -- 172.21.15.53:0/2582824064 learned_addr learned my addr 172.21.15.53:0/2582824064 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe08141b700 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe080c1a700 1 -- 172.21.15.53:0/2582824064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 msgr2=0x5585134b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe080c1a700 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe080c1a700 1 -- 172.21.15.53:0/2582824064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 msgr2=0x5585134ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe080c1a700 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 0x5585134ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe080c1a700 1 -- 172.21.15.53:0/2582824064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07c108a20 con 0x5585134a1680 2023-11-06T01:47:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe080c1a700 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134ab490 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe074095d30 tx=0x7fe074098ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12cf077560a99d69 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe05bfff700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:47:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.552+0000 7fe05bfff700 1 -- 172.21.15.53:0/2582824064 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074099720 con 0x5585134a1680 2023-11-06T01:47:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe05bfff700 1 -- 172.21.15.53:0/2582824064 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe07409d0f0 con 0x5585134a1680 2023-11-06T01:47:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2582824064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55851341dcb0 con 0x5585134a1680 2023-11-06T01:47:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe05bfff700 1 -- 172.21.15.53:0/2582824064 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0740a1330 con 0x5585134a1680 2023-11-06T01:47:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe08f00eb80 10 client.26530 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2582824064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x558513209c00 con 0x5585134a1680 2023-11-06T01:47:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe05bfff700 1 -- 172.21.15.53:0/2582824064 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe07407c020 con 0x5585134a1680 2023-11-06T01:47:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe05bfff700 1 -- 172.21.15.53:0/2582824064 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe0740a0ea0 con 0x5585134a1680 2023-11-06T01:47:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe08f00eb80 10 client.26530 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe08f00eb80 10 client.26530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe08f00eb80 20 client.26530 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5585134b7250 0x5585134b9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe08f00eb80 4 client.26530 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:47:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.553+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2582824064 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558513470db0 con 0x5585134b7250 2023-11-06T01:47:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.554+0000 7fe07bfff700 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5585134b7250 0x5585134b9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.554+0000 7fe07bfff700 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5585134b7250 0x5585134b9670 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9bf23288041b585 server_cookie=f8f716e36826d9c6 in_seq=0 out_seq=0 2023-11-06T01:47:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.554+0000 7fe05bfff700 10 client.26530 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe05bfff700 1 -- 172.21.15.53:0/2582824064 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558513470db0 con 0x5585134b7250 2023-11-06T01:47:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe05bfff700 10 client.26530 handle_command_reply: tid=0 2023-11-06T01:47:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe08f00eb80 1 client.26530 shutdown 2023-11-06T01:47:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2582824064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 msgr2=0x5585134ab490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134ab490 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe074095d30 tx=0x7fe074098ca0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2582824064 shutdown_connections 2023-11-06T01:47:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5585134b7250 0x5585134b9670 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134ab490 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe05bfff700 0 client.26530 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2582824064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 0x5585134ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2582824064 >> 172.21.15.53:0/2582824064 conn(0x558513401070 msgr2=0x55851349c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2582824064 shutdown_connections 2023-11-06T01:47:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.555+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2582824064 wait complete. 2023-11-06T01:47:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.556+0000 7fe08f00eb80 20 client.26530 trim_cache size 0 max 16384 2023-11-06T01:47:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe08f00eb80 1 Processor -- start 2023-11-06T01:47:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe08f00eb80 1 -- start start 2023-11-06T01:47:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 0x5585134b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558513497500 con 0x5585134a1fe0 2023-11-06T01:47:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585134ab000 con 0x5585134a2ed0 2023-11-06T01:47:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585134a23c0 con 0x5585134a1680 2023-11-06T01:47:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe080c1a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe07bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 0x5585134b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe08141b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe080c1a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134aa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:57998/0 (socket says 172.21.15.53:57998) 2023-11-06T01:47:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.574+0000 7fe080c1a700 1 -- 172.21.15.53:0/3308401459 learned_addr learned my addr 172.21.15.53:0/3308401459 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.575+0000 7fe080c1a700 1 -- 172.21.15.53:0/3308401459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 msgr2=0x5585134a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.575+0000 7fe080c1a700 1 --2- 172.21.15.53:0/3308401459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.575+0000 7fe080c1a700 1 -- 172.21.15.53:0/3308401459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 msgr2=0x5585134b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.575+0000 7fe080c1a700 1 --2- 172.21.15.53:0/3308401459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 0x5585134b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.575+0000 7fe080c1a700 1 -- 172.21.15.53:0/3308401459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585132471f0 con 0x5585134a1680 2023-11-06T01:47:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.575+0000 7fe07bfff700 1 --2- 172.21.15.53:0/3308401459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 0x5585134b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.575+0000 7fe08141b700 1 --2- 172.21.15.53:0/3308401459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.575+0000 7fe080c1a700 1 --2- 172.21.15.53:0/3308401459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134aa140 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c5b0 tx=0x7fe074097870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dff4dee30afadfe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.575+0000 7fe0737fe700 1 -- 172.21.15.53:0/3308401459 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074004530 con 0x5585134a1680 2023-11-06T01:47:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.575+0000 7fe0737fe700 1 -- 172.21.15.53:0/3308401459 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe07407c030 con 0x5585134a1680 2023-11-06T01:47:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.575+0000 7fe0737fe700 1 -- 172.21.15.53:0/3308401459 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074099520 con 0x5585134a1680 2023-11-06T01:47:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.576+0000 7fe08f00eb80 1 -- 172.21.15.53:0/3308401459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 msgr2=0x5585134aa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.576+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/3308401459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134aa140 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c5b0 tx=0x7fe074097870 comp rx=0 tx=0).stop 2023-11-06T01:47:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.579+0000 7fe08f00eb80 1 -- 172.21.15.53:0/3308401459 shutdown_connections 2023-11-06T01:47:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.579+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/3308401459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a2ed0 0x5585134a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.579+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/3308401459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a1680 0x5585134aa140 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.579+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/3308401459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1fe0 0x5585134b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.579+0000 7fe08f00eb80 1 -- 172.21.15.53:0/3308401459 >> 172.21.15.53:0/3308401459 conn(0x558513401070 msgr2=0x55851349dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.579+0000 7fe08f00eb80 1 -- 172.21.15.53:0/3308401459 shutdown_connections 2023-11-06T01:47:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.579+0000 7fe08f00eb80 1 -- 172.21.15.53:0/3308401459 wait complete. 2023-11-06T01:47:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.579+0000 7fe08f00eb80 1 -- start start 2023-11-06T01:47:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.579+0000 7fe08f00eb80 1 Processor -- start 2023-11-06T01:47:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1680 0x5585134b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a1fe0 0x5585134b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe08f00eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a2ed0 0x5585134b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585134b24d0 con 0x5585134a1680 2023-11-06T01:47:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585134b2640 con 0x5585134a1fe0 2023-11-06T01:47:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe08f00eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585134b2800 con 0x5585134a2ed0 2023-11-06T01:47:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe08f00eb80 10 client.0 _ll_register_callbacks cb 0x7fe074099090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:47:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe080c1a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1680 0x5585134b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe07bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a1fe0 0x5585134b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe080c1a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1680 0x5585134b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43764/0 (socket says 172.21.15.53:43764) 2023-11-06T01:47:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe07bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a1fe0 0x5585134b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55204/0 (socket says 172.21.15.53:55204) 2023-11-06T01:47:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe080c1a700 1 -- 172.21.15.53:0/2242258941 learned_addr learned my addr 172.21.15.53:0/2242258941 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:47:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe08141b700 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a2ed0 0x5585134b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe07bfff700 1 -- 172.21.15.53:0/2242258941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a2ed0 msgr2=0x5585134b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe07bfff700 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a2ed0 0x5585134b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe07bfff700 1 -- 172.21.15.53:0/2242258941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1680 msgr2=0x5585134b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:47:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe07bfff700 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1680 0x5585134b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe07bfff700 1 -- 172.21.15.53:0/2242258941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06c016800 con 0x5585134a1fe0 2023-11-06T01:47:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.580+0000 7fe080c1a700 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1680 0x5585134b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:47:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.581+0000 7fe07bfff700 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a1fe0 0x5585134b10b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5585134b7630 tx=0x7fe06c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76cc8fbe84b1caab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:47:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.581+0000 7fe05bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:47:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.581+0000 7fe05bfff700 1 -- 172.21.15.53:0/2242258941 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c010040 con 0x5585134a1fe0 2023-11-06T01:47:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.581+0000 7fe05bfff700 1 -- 172.21.15.53:0/2242258941 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe06c006e40 con 0x5585134a1fe0 2023-11-06T01:47:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.581+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2242258941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe06400bc50 con 0x5585134a1fe0 2023-11-06T01:47:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.581+0000 7fe05bfff700 1 -- 172.21.15.53:0/2242258941 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe06c029820 con 0x5585134a1fe0 2023-11-06T01:47:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.581+0000 7fe08f00eb80 10 client.26471 fetch_fsmap learned FSMap version 49 2023-11-06T01:47:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.581+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2242258941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x558513220d80 con 0x5585134a1fe0 2023-11-06T01:47:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.581+0000 7fe05bfff700 1 -- 172.21.15.53:0/2242258941 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe06400bc50 con 0x5585134a1fe0 2023-11-06T01:47:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.581+0000 7fe05bfff700 1 -- 172.21.15.53:0/2242258941 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe06c03a020 con 0x5585134a1fe0 2023-11-06T01:47:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.582+0000 7fe08f00eb80 10 client.26471 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:47:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.582+0000 7fe08f00eb80 10 client.26471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:47:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.582+0000 7fe08f00eb80 20 client.26471 populate_metadata read hostname 'smithi053' 2023-11-06T01:47:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.582+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55851350a060 0x55851350c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:47:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.582+0000 7fe08f00eb80 4 client.26471 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:47:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.582+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2242258941 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5585133ebce0 con 0x55851350a060 2023-11-06T01:47:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.582+0000 7fe080c1a700 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55851350a060 0x55851350c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:47:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.583+0000 7fe080c1a700 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55851350a060 0x55851350c480 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9779f14e2fbd531d server_cookie=9a1728fee4aec050 in_seq=0 out_seq=0 2023-11-06T01:47:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.583+0000 7fe05bfff700 10 client.26471 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe05bfff700 1 -- 172.21.15.53:0/2242258941 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5585133ebce0 con 0x55851350a060 2023-11-06T01:47:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe05bfff700 10 client.26471 handle_command_reply: tid=0 2023-11-06T01:47:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 1 client.26471 shutdown 2023-11-06T01:47:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2242258941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a1fe0 msgr2=0x5585134b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a1fe0 0x5585134b10b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5585134b7630 tx=0x7fe06c004ad0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2242258941 shutdown_connections 2023-11-06T01:47:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55851350a060 0x55851350c480 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5585134a2ed0 0x5585134b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe05bfff700 0 client.26471 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:47:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585134a1680 0x5585134b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 1 --2- 172.21.15.53:0/2242258941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585134a1fe0 0x5585134b10b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2242258941 >> 172.21.15.53:0/2242258941 conn(0x558513401070 msgr2=0x5585134aa140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2242258941 shutdown_connections 2023-11-06T01:47:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 1 -- 172.21.15.53:0/2242258941 wait complete. 2023-11-06T01:47:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.593+0000 7fe08f00eb80 20 client.26471 trim_cache size 0 max 16384 2023-11-06T01:47:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:47:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:47:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:47:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:47:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 -- 172.21.15.53:0/438315771 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0680a05a0 msgr2=0x7fe0680a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0680a05a0 0x7fe0680a2a60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe07c195960 tx=0x7fe06c00d040 comp rx=0 tx=0).stop 2023-11-06T01:47:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 -- 172.21.15.53:0/438315771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe07c0ffc90 msgr2=0x7fe07c194910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:47:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe07c0ffc90 0x7fe07c194910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe07400e970 tx=0x7fe07400ec80 comp rx=0 tx=0).stop 2023-11-06T01:47:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 -- 172.21.15.53:0/438315771 shutdown_connections 2023-11-06T01:47:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe0680a05a0 0x7fe0680a2a60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe07c0ffc90 0x7fe07c194910 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe07c101820 0x7fe07c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 --2- 172.21.15.53:0/438315771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe07c100e80 0x7fe07c194e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:47:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 -- 172.21.15.53:0/438315771 >> 172.21.15.53:0/438315771 conn(0x7fe07c0fb1e0 msgr2=0x7fe07c068280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:47:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 -- 172.21.15.53:0/438315771 shutdown_connections 2023-11-06T01:47:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:47:54.594+0000 7fe082e7e700 1 -- 172.21.15.53:0/438315771 wait complete. 2023-11-06T01:47:55.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:55 smithi053 ceph-mon[99472]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 1.83k op/s 2023-11-06T01:47:55.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:55 smithi062 ceph-mon[103924]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 1.83k op/s 2023-11-06T01:47:55.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:55 smithi163 ceph-mon[104416]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 1.83k op/s 2023-11-06T01:47:57.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 820.0 s, 1508.6 tps, lat 65.775 ms stddev 73.660 2023-11-06T01:47:57.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:57 smithi053 ceph-mon[99472]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 1.50k op/s 2023-11-06T01:47:57.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:57 smithi062 ceph-mon[103924]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 1.50k op/s 2023-11-06T01:47:57.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:57 smithi163 ceph-mon[104416]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 1.50k op/s 2023-11-06T01:47:59.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:59 smithi053 ceph-mon[99472]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2023-11-06T01:47:59.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:47:59.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:47:59.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:47:59.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:47:59.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:47:59 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:47:59.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:59 smithi062 ceph-mon[103924]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2023-11-06T01:47:59.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:47:59.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:47:59.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:47:59.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:47:59.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:47:59 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:47:59.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:59 smithi163 ceph-mon[104416]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2023-11-06T01:47:59.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:47:59.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:47:59.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:47:59.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:47:59.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:47:59 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:48:00.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:48:00.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:48:00.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:48:00.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:00 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:48:00.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:00 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:48:00.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:00 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:48:00.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:00 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:48:00.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:00 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:48:00.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:00 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:48:01.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:01 smithi053 ceph-mon[99472]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 674 op/s 2023-11-06T01:48:01.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:01 smithi062 ceph-mon[103924]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 674 op/s 2023-11-06T01:48:01.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:01 smithi163 ceph-mon[104416]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 674 op/s 2023-11-06T01:48:02.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 825.0 s, 1400.6 tps, lat 71.597 ms stddev 76.182 2023-11-06T01:48:02.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:02 smithi163 ceph-mon[104416]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 639 op/s 2023-11-06T01:48:02.841 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:02 smithi053 ceph-mon[99472]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 639 op/s 2023-11-06T01:48:02.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:02 smithi062 ceph-mon[103924]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 639 op/s 2023-11-06T01:48:05.347 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:05 smithi053 ceph-mon[99472]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.6 MiB/s wr, 584 op/s 2023-11-06T01:48:05.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:05 smithi062 ceph-mon[103924]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.6 MiB/s wr, 584 op/s 2023-11-06T01:48:05.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:05 smithi163 ceph-mon[104416]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.6 MiB/s wr, 584 op/s 2023-11-06T01:48:07.187 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 830.0 s, 1634.3 tps, lat 61.167 ms stddev 70.344 2023-11-06T01:48:07.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:07 smithi053 ceph-mon[99472]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 10 MiB/s wr, 630 op/s 2023-11-06T01:48:07.398 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:07 smithi062 ceph-mon[103924]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 10 MiB/s wr, 630 op/s 2023-11-06T01:48:07.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:07 smithi163 ceph-mon[104416]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 10 MiB/s wr, 630 op/s 2023-11-06T01:48:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.750+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/4115032997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a54107d90 msgr2=0x7f4a541081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.750+0000 7f4a5b9e2700 1 --2- 172.21.15.53:0/4115032997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a54107d90 0x7f4a541081b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009990 tx=0x7f4a50009ca0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.750+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/4115032997 shutdown_connections 2023-11-06T01:48:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.750+0000 7f4a5b9e2700 1 --2- 172.21.15.53:0/4115032997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a5410f610 0x7f4a54111a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.750+0000 7f4a5b9e2700 1 --2- 172.21.15.53:0/4115032997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a54107d90 0x7f4a541081b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.750+0000 7f4a5b9e2700 1 --2- 172.21.15.53:0/4115032997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a541086f0 0x7f4a5410f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.750+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/4115032997 >> 172.21.15.53:0/4115032997 conn(0x7f4a5406dbc0 msgr2=0x7f4a54070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.750+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/4115032997 shutdown_connections 2023-11-06T01:48:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.750+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/4115032997 wait complete. 2023-11-06T01:48:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a5b9e2700 1 Processor -- start 2023-11-06T01:48:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a5b9e2700 1 -- start start 2023-11-06T01:48:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a5b9e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a54107d90 0x7f4a5419f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a5b9e2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541086f0 0x7f4a5419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a5b9e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a5410f610 0x7f4a541a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a5b9e2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a541a0d60 con 0x7f4a541086f0 2023-11-06T01:48:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a5b9e2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a541a0ed0 con 0x7f4a5410f610 2023-11-06T01:48:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a5b9e2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a541a1090 con 0x7f4a54107d90 2023-11-06T01:48:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a58f7d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541086f0 0x7f4a5419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a58f7d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541086f0 0x7f4a5419f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43780/0 (socket says 172.21.15.53:43780) 2023-11-06T01:48:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a58f7d700 1 -- 172.21.15.53:0/620412380 learned_addr learned my addr 172.21.15.53:0/620412380 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a5410f610 0x7f4a541a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.751+0000 7f4a5977e700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a54107d90 0x7f4a5419f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a58f7d700 1 -- 172.21.15.53:0/620412380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a54107d90 msgr2=0x7f4a5419f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a58f7d700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a54107d90 0x7f4a5419f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a58f7d700 1 -- 172.21.15.53:0/620412380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a5410f610 msgr2=0x7f4a541a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a58f7d700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a5410f610 0x7f4a541a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a58f7d700 1 -- 172.21.15.53:0/620412380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500095f0 con 0x7f4a541086f0 2023-11-06T01:48:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a5410f610 0x7f4a541a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:48:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a58f7d700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541086f0 0x7f4a5419f880 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400e9a0 tx=0x7f4a4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a4a7fc700 1 -- 172.21.15.53:0/620412380 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44009790 con 0x7f4a541086f0 2023-11-06T01:48:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/620412380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a541a1370 con 0x7f4a541086f0 2023-11-06T01:48:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a4a7fc700 1 -- 172.21.15.53:0/620412380 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a44010430 con 0x7f4a541086f0 2023-11-06T01:48:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a4a7fc700 1 -- 172.21.15.53:0/620412380 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44018680 con 0x7f4a541086f0 2023-11-06T01:48:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.752+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/620412380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a541a7b30 con 0x7f4a541086f0 2023-11-06T01:48:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.755+0000 7f4a4a7fc700 1 -- 172.21.15.53:0/620412380 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4a440105a0 con 0x7f4a541086f0 2023-11-06T01:48:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.757+0000 7f4a67b72b80 1 Processor -- start 2023-11-06T01:48:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.757+0000 7f4a67b72b80 1 -- start start 2023-11-06T01:48:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.757+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 0x559f144ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.757+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144ccfe0 0x559f144cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.757+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cded0 0x559f144ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.757+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f144249b0 con 0x559f144cc680 2023-11-06T01:48:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.757+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14422370 con 0x559f144cded0 2023-11-06T01:48:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.757+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f1441dc60 con 0x559f144ccfe0 2023-11-06T01:48:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.757+0000 7f4a4a7fc700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a400a0520 0x7f4a400a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.758+0000 7f4a59f7f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 0x559f144ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.758+0000 7f4a4a7fc700 1 -- 172.21.15.53:0/620412380 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4a44014070 con 0x7f4a541086f0 2023-11-06T01:48:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.758+0000 7f4a59f7f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 0x559f144ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43784/0 (socket says 172.21.15.53:43784) 2023-11-06T01:48:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.758+0000 7f4a59f7f700 1 -- 172.21.15.53:0/1849429913 learned_addr learned my addr 172.21.15.53:0/1849429913 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.758+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a400a0520 0x7f4a400a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.758+0000 7f4a59f7f700 1 -- 172.21.15.53:0/1849429913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144ccfe0 msgr2=0x559f144cd8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:48:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.758+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/1849429913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144ccfe0 0x559f144cd8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.758+0000 7f4a59f7f700 1 -- 172.21.15.53:0/1849429913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cded0 msgr2=0x559f144ce350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:48:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.758+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/1849429913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cded0 0x559f144ce350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.758+0000 7f4a59f7f700 1 -- 172.21.15.53:0/1849429913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f1424bd80 con 0x559f144cc680 2023-11-06T01:48:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.759+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a400a0520 0x7f4a400a29e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009fd0 tx=0x7f4a4c0090c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.759+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/1849429913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 0x559f144ccaa0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c004a40 tx=0x7f4a4c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e903cdee4d878ef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.759+0000 7f4a3f7fe700 1 -- 172.21.15.53:0/1849429913 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00ee00 con 0x559f144cc680 2023-11-06T01:48:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.759+0000 7f4a3f7fe700 1 -- 172.21.15.53:0/1849429913 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c01e070 con 0x559f144cc680 2023-11-06T01:48:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.759+0000 7f4a3f7fe700 1 -- 172.21.15.53:0/1849429913 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c022ab0 con 0x559f144cc680 2023-11-06T01:48:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.760+0000 7f4a67b72b80 1 -- 172.21.15.53:0/1849429913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 msgr2=0x559f144ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.760+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/1849429913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 0x559f144ccaa0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c004a40 tx=0x7f4a4c01e000 comp rx=0 tx=0).stop 2023-11-06T01:48:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.776+0000 7f4a67b72b80 1 -- 172.21.15.53:0/1849429913 shutdown_connections 2023-11-06T01:48:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.776+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/1849429913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cded0 0x559f144ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.776+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/1849429913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 0x559f144ccaa0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.776+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/1849429913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144ccfe0 0x559f144cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.776+0000 7f4a67b72b80 1 -- 172.21.15.53:0/1849429913 >> 172.21.15.53:0/1849429913 conn(0x559f1442c070 msgr2=0x559f144c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.788+0000 7f4a67b72b80 1 -- 172.21.15.53:0/1849429913 shutdown_connections 2023-11-06T01:48:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.792+0000 7f4a67b72b80 1 -- 172.21.15.53:0/1849429913 wait complete. 2023-11-06T01:48:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.792+0000 7f4a67b72b80 1 -- start start 2023-11-06T01:48:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.792+0000 7f4a67b72b80 1 Processor -- start 2023-11-06T01:48:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.793+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cc680 0x559f144d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.793+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144ccfe0 0x559f144d2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.793+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.793+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f143ba4b0 con 0x559f144ccfe0 2023-11-06T01:48:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.793+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f143ba150 con 0x559f144cc680 2023-11-06T01:48:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.793+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f142a18c0 con 0x559f144cded0 2023-11-06T01:48:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.793+0000 7f4a67b72b80 10 client.0 _ll_register_callbacks cb 0x559f14307a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.793+0000 7f4a59f7f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.793+0000 7f4a59f7f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144d3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58032/0 (socket says 172.21.15.53:58032) 2023-11-06T01:48:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.793+0000 7f4a59f7f700 1 -- 172.21.15.53:0/3818841463 learned_addr learned my addr 172.21.15.53:0/3818841463 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.795+0000 7f4a59f7f700 1 -- 172.21.15.53:0/3818841463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cc680 msgr2=0x559f144d1f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:48:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.795+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/3818841463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cc680 0x559f144d1f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.795+0000 7f4a59f7f700 1 -- 172.21.15.53:0/3818841463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144ccfe0 msgr2=0x559f144d2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.795+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/3818841463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144ccfe0 0x559f144d2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.795+0000 7f4a59f7f700 1 -- 172.21.15.53:0/3818841463 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c008de0 con 0x559f144cded0 2023-11-06T01:48:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.795+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/3818841463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144d3490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c004a10 tx=0x7f4a4c004da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c5012ddc8ce83df server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.795+0000 7f4a3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:48:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.796+0000 7f4a3cff9700 1 -- 172.21.15.53:0/3818841463 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c023d30 con 0x559f144cded0 2023-11-06T01:48:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.796+0000 7f4a3cff9700 1 -- 172.21.15.53:0/3818841463 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4a4c023e90 con 0x559f144cded0 2023-11-06T01:48:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.796+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3818841463 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f14448cb0 con 0x559f144cded0 2023-11-06T01:48:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.796+0000 7f4a3cff9700 1 -- 172.21.15.53:0/3818841463 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a4c01a720 con 0x559f144cded0 2023-11-06T01:48:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.797+0000 7f4a67b72b80 10 client.26545 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.797+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3818841463 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x559f14234c00 con 0x559f144cded0 2023-11-06T01:48:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.797+0000 7f4a3cff9700 1 -- 172.21.15.53:0/3818841463 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f14448cb0 con 0x559f144cded0 2023-11-06T01:48:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.798+0000 7f4a3cff9700 1 -- 172.21.15.53:0/3818841463 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f4a4c03e050 con 0x559f144cded0 2023-11-06T01:48:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.798+0000 7f4a67b72b80 10 client.26545 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.798+0000 7f4a67b72b80 10 client.26545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.798+0000 7f4a67b72b80 20 client.26545 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.799+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/3818841463 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x559f144e22b0 0x559f144e46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.799+0000 7f4a67b72b80 4 client.26545 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:48:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.799+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3818841463 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f1449bdb0 con 0x559f144e22b0 2023-11-06T01:48:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.799+0000 7f4a5977e700 1 --2- 172.21.15.53:0/3818841463 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x559f144e22b0 0x559f144e46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.800+0000 7f4a3cff9700 10 client.26545 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.800+0000 7f4a5977e700 1 --2- 172.21.15.53:0/3818841463 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x559f144e22b0 0x559f144e46d0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adcb0d9524a7eb87 server_cookie=62676d0f5eb5e084 in_seq=0 out_seq=0 2023-11-06T01:48:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.801+0000 7f4a3cff9700 1 -- 172.21.15.53:0/3818841463 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559f1449bdb0 con 0x559f144e22b0 2023-11-06T01:48:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.801+0000 7f4a3cff9700 10 client.26545 handle_command_reply: tid=0 2023-11-06T01:48:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.801+0000 7f4a67b72b80 1 client.26545 shutdown 2023-11-06T01:48:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.803+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3818841463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 msgr2=0x559f144d3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.803+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/3818841463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144d3490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c004a10 tx=0x7f4a4c004da0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.806+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3818841463 shutdown_connections 2023-11-06T01:48:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.806+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/3818841463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cc680 0x559f144d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.806+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/3818841463 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x559f144e22b0 0x559f144e46d0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.806+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/3818841463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144d3490 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.806+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/3818841463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144ccfe0 0x559f144d2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.806+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3818841463 >> 172.21.15.53:0/3818841463 conn(0x559f1442c070 msgr2=0x559f144c73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.806+0000 7f4a3cff9700 0 client.26545 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.807+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3818841463 shutdown_connections 2023-11-06T01:48:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.807+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3818841463 wait complete. 2023-11-06T01:48:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.810+0000 7f4a67b72b80 20 client.26545 trim_cache size 0 max 16384 2023-11-06T01:48:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a67b72b80 1 Processor -- start 2023-11-06T01:48:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a67b72b80 1 -- start start 2023-11-06T01:48:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cc680 0x559f144d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144ccfe0 0x559f144d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f144c8490 con 0x559f144ccfe0 2023-11-06T01:48:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f144bf7a0 con 0x559f144cc680 2023-11-06T01:48:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f144cd480 con 0x559f144cded0 2023-11-06T01:48:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a58f7d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144ccfe0 0x559f144d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a5977e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cc680 0x559f144d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a58f7d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144ccfe0 0x559f144d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43800/0 (socket says 172.21.15.53:43800) 2023-11-06T01:48:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a5977e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cc680 0x559f144d3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55236/0 (socket says 172.21.15.53:55236) 2023-11-06T01:48:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.830+0000 7f4a58f7d700 1 -- 172.21.15.53:0/3185561254 learned_addr learned my addr 172.21.15.53:0/3185561254 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.831+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/3185561254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.831+0000 7f4a58f7d700 1 -- 172.21.15.53:0/3185561254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 msgr2=0x559f144e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.831+0000 7f4a58f7d700 1 --2- 172.21.15.53:0/3185561254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.831+0000 7f4a58f7d700 1 -- 172.21.15.53:0/3185561254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cc680 msgr2=0x559f144d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.831+0000 7f4a58f7d700 1 --2- 172.21.15.53:0/3185561254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cc680 0x559f144d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.831+0000 7f4a58f7d700 1 -- 172.21.15.53:0/3185561254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f142721f0 con 0x559f144ccfe0 2023-11-06T01:48:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.831+0000 7f4a5977e700 1 --2- 172.21.15.53:0/3185561254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cc680 0x559f144d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:48:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.831+0000 7f4a58f7d700 1 --2- 172.21.15.53:0/3185561254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144ccfe0 0x559f144d15f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4a44099340 tx=0x7f4a44098f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befa34493d18d32 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.831+0000 7f4a3ffff700 1 -- 172.21.15.53:0/3185561254 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440840e0 con 0x559f144ccfe0 2023-11-06T01:48:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.831+0000 7f4a3ffff700 1 -- 172.21.15.53:0/3185561254 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a44099ab0 con 0x559f144ccfe0 2023-11-06T01:48:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.831+0000 7f4a3ffff700 1 -- 172.21.15.53:0/3185561254 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44097870 con 0x559f144ccfe0 2023-11-06T01:48:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.832+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3185561254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144ccfe0 msgr2=0x559f144d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.832+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/3185561254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144ccfe0 0x559f144d15f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4a44099340 tx=0x7f4a44098f70 comp rx=0 tx=0).stop 2023-11-06T01:48:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.832+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3185561254 shutdown_connections 2023-11-06T01:48:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.832+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/3185561254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144cc680 0x559f144d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.832+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/3185561254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.832+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/3185561254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144ccfe0 0x559f144d15f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.832+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3185561254 >> 172.21.15.53:0/3185561254 conn(0x559f1442c070 msgr2=0x559f144c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.832+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3185561254 shutdown_connections 2023-11-06T01:48:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.832+0000 7f4a67b72b80 1 -- 172.21.15.53:0/3185561254 wait complete. 2023-11-06T01:48:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.832+0000 7f4a67b72b80 1 -- start start 2023-11-06T01:48:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.832+0000 7f4a67b72b80 1 Processor -- start 2023-11-06T01:48:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.833+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 0x559f144f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.833+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144ccfe0 0x559f144f1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.833+0000 7f4a67b72b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.833+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14554be0 con 0x559f144cc680 2023-11-06T01:48:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.833+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14554d50 con 0x559f144ccfe0 2023-11-06T01:48:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.833+0000 7f4a67b72b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14554f10 con 0x559f144cded0 2023-11-06T01:48:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.833+0000 7f4a67b72b80 10 client.0 _ll_register_callbacks cb 0x7f4a4c008630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.833+0000 7f4a58f7d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144ccfe0 0x559f144f1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.833+0000 7f4a58f7d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144ccfe0 0x559f144f1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55244/0 (socket says 172.21.15.53:55244) 2023-11-06T01:48:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.833+0000 7f4a58f7d700 1 -- 172.21.15.53:0/2982909637 learned_addr learned my addr 172.21.15.53:0/2982909637 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.833+0000 7f4a5977e700 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 0x559f144f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.834+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.834+0000 7f4a58f7d700 1 -- 172.21.15.53:0/2982909637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 msgr2=0x559f144f17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.834+0000 7f4a58f7d700 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144f17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.834+0000 7f4a58f7d700 1 -- 172.21.15.53:0/2982909637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 msgr2=0x559f144f0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.834+0000 7f4a58f7d700 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 0x559f144f0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.834+0000 7f4a58f7d700 1 -- 172.21.15.53:0/2982909637 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a54112300 con 0x559f144ccfe0 2023-11-06T01:48:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.834+0000 7f4a59f7f700 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:48:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.834+0000 7f4a58f7d700 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144ccfe0 0x559f144f1270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4a440958b0 tx=0x7f4a44097850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4754c8ce814506f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.834+0000 7f4a3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:48:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.834+0000 7f4a3cff9700 1 -- 172.21.15.53:0/2982909637 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440a1190 con 0x559f144ccfe0 2023-11-06T01:48:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.834+0000 7f4a3cff9700 1 -- 172.21.15.53:0/2982909637 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4a4409d1e0 con 0x559f144ccfe0 2023-11-06T01:48:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a67b72b80 1 -- 172.21.15.53:0/2982909637 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a4c01e070 con 0x559f144ccfe0 2023-11-06T01:48:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a3cff9700 1 -- 172.21.15.53:0/2982909637 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a440993b0 con 0x559f144ccfe0 2023-11-06T01:48:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a67b72b80 10 client.26483 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a67b72b80 1 -- 172.21.15.53:0/2982909637 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f4a541a7b30 con 0x559f144ccfe0 2023-11-06T01:48:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a3cff9700 1 -- 172.21.15.53:0/2982909637 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c01e070 con 0x559f144ccfe0 2023-11-06T01:48:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a3cff9700 1 -- 172.21.15.53:0/2982909637 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f4a440951a0 con 0x559f144ccfe0 2023-11-06T01:48:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a67b72b80 10 client.26483 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a67b72b80 10 client.26483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a67b72b80 20 client.26483 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x559f14556670 0x559f14558a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a67b72b80 4 client.26483 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:48:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.835+0000 7f4a67b72b80 1 -- 172.21.15.53:0/2982909637 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559f14416ce0 con 0x559f14556670 2023-11-06T01:48:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.836+0000 7f4a5977e700 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x559f14556670 0x559f14558a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.836+0000 7f4a5977e700 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x559f14556670 0x559f14558a90 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23daa312bea642ec server_cookie=b8d07c5d9bea964a in_seq=0 out_seq=0 2023-11-06T01:48:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.836+0000 7f4a3cff9700 10 client.26483 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.845+0000 7f4a3cff9700 1 -- 172.21.15.53:0/2982909637 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559f14416ce0 con 0x559f14556670 2023-11-06T01:48:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.845+0000 7f4a3cff9700 10 client.26483 handle_command_reply: tid=0 2023-11-06T01:48:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.845+0000 7f4a67b72b80 1 client.26483 shutdown 2023-11-06T01:48:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a67b72b80 1 -- 172.21.15.53:0/2982909637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144ccfe0 msgr2=0x559f144f1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144ccfe0 0x559f144f1270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4a440958b0 tx=0x7f4a44097850 comp rx=0 tx=0).stop 2023-11-06T01:48:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a67b72b80 1 -- 172.21.15.53:0/2982909637 shutdown_connections 2023-11-06T01:48:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x559f14556670 0x559f14558a90 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559f144cded0 0x559f144f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f144cc680 0x559f144f0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a3cff9700 0 client.26483 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a67b72b80 1 --2- 172.21.15.53:0/2982909637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f144ccfe0 0x559f144f1270 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a67b72b80 1 -- 172.21.15.53:0/2982909637 >> 172.21.15.53:0/2982909637 conn(0x559f1442c070 msgr2=0x559f144e42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a67b72b80 1 -- 172.21.15.53:0/2982909637 shutdown_connections 2023-11-06T01:48:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a67b72b80 1 -- 172.21.15.53:0/2982909637 wait complete. 2023-11-06T01:48:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a67b72b80 20 client.26483 trim_cache size 0 max 16384 2023-11-06T01:48:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:48:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:48:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:48:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:48:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/620412380 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a400a0520 msgr2=0x7f4a400a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a5b9e2700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a400a0520 0x7f4a400a29e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009fd0 tx=0x7f4a4c0090c0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/620412380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541086f0 msgr2=0x7f4a5419f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.846+0000 7f4a5b9e2700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541086f0 0x7f4a5419f880 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400e9a0 tx=0x7f4a4400ecb0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.847+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/620412380 shutdown_connections 2023-11-06T01:48:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.847+0000 7f4a5b9e2700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f4a400a0520 0x7f4a400a29e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.847+0000 7f4a5b9e2700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a5410f610 0x7f4a541a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.847+0000 7f4a5b9e2700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a54107d90 0x7f4a5419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.847+0000 7f4a5b9e2700 1 --2- 172.21.15.53:0/620412380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541086f0 0x7f4a5419f880 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.847+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/620412380 >> 172.21.15.53:0/620412380 conn(0x7f4a5406dbc0 msgr2=0x7f4a5406eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.847+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/620412380 shutdown_connections 2023-11-06T01:48:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:07.847+0000 7f4a5b9e2700 1 -- 172.21.15.53:0/620412380 wait complete. 2023-11-06T01:48:09.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:09 smithi053 ceph-mon[99472]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 MiB/s wr, 657 op/s 2023-11-06T01:48:09.395 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:09 smithi062 ceph-mon[103924]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 MiB/s wr, 657 op/s 2023-11-06T01:48:09.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:09 smithi163 ceph-mon[104416]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 MiB/s wr, 657 op/s 2023-11-06T01:48:11.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:11 smithi053 ceph-mon[99472]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 10 MiB/s wr, 728 op/s 2023-11-06T01:48:11.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:11 smithi062 ceph-mon[103924]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 10 MiB/s wr, 728 op/s 2023-11-06T01:48:11.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:11 smithi163 ceph-mon[104416]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 10 MiB/s wr, 728 op/s 2023-11-06T01:48:11.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:48:12.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:11.995+0000 7f3f48fbb700 1 -- 172.21.15.53:0/392208463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f44100d80 msgr2=0x7f3f441011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:11.995+0000 7f3f48fbb700 1 --2- 172.21.15.53:0/392208463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f44100d80 0x7f3f441011a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3f340099c0 tx=0x7f3f34009cd0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.012+0000 7f3f48fbb700 1 -- 172.21.15.53:0/392208463 shutdown_connections 2023-11-06T01:48:12.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.012+0000 7f3f48fbb700 1 --2- 172.21.15.53:0/392208463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f44100d80 0x7f3f441011a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.012+0000 7f3f48fbb700 1 --2- 172.21.15.53:0/392208463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f44102910 0x7f3f441091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.012+0000 7f3f48fbb700 1 --2- 172.21.15.53:0/392208463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f44101f70 0x7f3f441023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.012+0000 7f3f48fbb700 1 -- 172.21.15.53:0/392208463 >> 172.21.15.53:0/392208463 conn(0x7f3f440fc2d0 msgr2=0x7f3f440fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:12.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.015+0000 7f3f48fbb700 1 -- 172.21.15.53:0/392208463 shutdown_connections 2023-11-06T01:48:12.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.015+0000 7f3f48fbb700 1 -- 172.21.15.53:0/392208463 wait complete. 2023-11-06T01:48:12.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.016+0000 7f3f48fbb700 1 Processor -- start 2023-11-06T01:48:12.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.016+0000 7f3f48fbb700 1 -- start start 2023-11-06T01:48:12.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.016+0000 7f3f48fbb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f44100d80 0x7f3f4410fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.016+0000 7f3f48fbb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f44101f70 0x7f3f44110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.016+0000 7f3f48fbb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f44102910 0x7f3f441ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.016+0000 7f3f48fbb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f44117220 con 0x7f3f44102910 2023-11-06T01:48:12.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.016+0000 7f3f48fbb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f441ad6d0 con 0x7f3f44100d80 2023-11-06T01:48:12.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.016+0000 7f3f48fbb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f441ad810 con 0x7f3f44101f70 2023-11-06T01:48:12.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.016+0000 7f3f42d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f44102910 0x7f3f441ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.016+0000 7f3f4259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f44100d80 0x7f3f4410fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.017+0000 7f3f42d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f44102910 0x7f3f441ad190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43818/0 (socket says 172.21.15.53:43818) 2023-11-06T01:48:12.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.017+0000 7f3f42d9d700 1 -- 172.21.15.53:0/638078492 learned_addr learned my addr 172.21.15.53:0/638078492 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:12.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.017+0000 7f3f42d9d700 1 -- 172.21.15.53:0/638078492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f44101f70 msgr2=0x7f3f44110120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:48:12.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.017+0000 7f3f42d9d700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f44101f70 0x7f3f44110120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.017+0000 7f3f42d9d700 1 -- 172.21.15.53:0/638078492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f44100d80 msgr2=0x7f3f4410fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:12.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.017+0000 7f3f42d9d700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f44100d80 0x7f3f4410fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.017+0000 7f3f42d9d700 1 -- 172.21.15.53:0/638078492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f340095f0 con 0x7f3f44102910 2023-11-06T01:48:12.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.017+0000 7f3f42d9d700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f44102910 0x7f3f441ad190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00ef30 tx=0x7f3f3c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:12.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.018+0000 7f3f337fe700 1 -- 172.21.15.53:0/638078492 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c018070 con 0x7f3f44102910 2023-11-06T01:48:12.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.018+0000 7f3f337fe700 1 -- 172.21.15.53:0/638078492 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f3c00f040 con 0x7f3f44102910 2023-11-06T01:48:12.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.018+0000 7f3f337fe700 1 -- 172.21.15.53:0/638078492 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00cda0 con 0x7f3f44102910 2023-11-06T01:48:12.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.018+0000 7f3f48fbb700 1 -- 172.21.15.53:0/638078492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f441ada10 con 0x7f3f44102910 2023-11-06T01:48:12.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.018+0000 7f3f48fbb700 1 -- 172.21.15.53:0/638078492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f441adea0 con 0x7f3f44102910 2023-11-06T01:48:12.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.022+0000 7f3f337fe700 1 -- 172.21.15.53:0/638078492 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f3f3c004ad0 con 0x7f3f44102910 2023-11-06T01:48:12.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.023+0000 7f3f337fe700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f2c0a05a0 0x7f3f2c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.023+0000 7f3f337fe700 1 -- 172.21.15.53:0/638078492 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3f3c0cb520 con 0x7f3f44102910 2023-11-06T01:48:12.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.026+0000 7f3f5514bb80 1 Processor -- start 2023-11-06T01:48:12.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.031+0000 7f3f5514bb80 1 -- start start 2023-11-06T01:48:12.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.032+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e21190f0 0x5652e2119530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.032+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2117e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.032+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 0x5652e2118870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.032+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e206fa10 con 0x5652e2119a70 2023-11-06T01:48:12.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.032+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e206d3d0 con 0x5652e21183f0 2023-11-06T01:48:12.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.032+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e2068cc0 con 0x5652e21190f0 2023-11-06T01:48:12.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.034+0000 7f3f42d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 0x5652e2118870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.034+0000 7f3f42d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 0x5652e2118870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55266/0 (socket says 172.21.15.53:55266) 2023-11-06T01:48:12.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.034+0000 7f3f42d9d700 1 -- 172.21.15.53:0/2208835345 learned_addr learned my addr 172.21.15.53:0/2208835345 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:12.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.035+0000 7f3f4259c700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f2c0a05a0 0x7f3f2c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.035+0000 7f3f42d9d700 1 -- 172.21.15.53:0/2208835345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e21190f0 msgr2=0x5652e2119530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:12.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.035+0000 7f3f42d9d700 1 --2- 172.21.15.53:0/2208835345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e21190f0 0x5652e2119530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.035+0000 7f3f42d9d700 1 -- 172.21.15.53:0/2208835345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 msgr2=0x5652e2117e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:12.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.035+0000 7f3f42d9d700 1 --2- 172.21.15.53:0/2208835345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2117e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.035+0000 7f3f42d9d700 1 -- 172.21.15.53:0/2208835345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652e1e96de0 con 0x5652e21183f0 2023-11-06T01:48:12.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.035+0000 7f3f42d9d700 1 --2- 172.21.15.53:0/2208835345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 0x5652e2118870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c095860 tx=0x7f3f3c098820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6c1bbe45728b425 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:12.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.043+0000 7f3f4259c700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f2c0a05a0 0x7f3f2c0a2a60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009990 tx=0x7f3f34005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:12.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.046+0000 7f3f30ff9700 1 -- 172.21.15.53:0/2208835345 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c098df0 con 0x5652e21183f0 2023-11-06T01:48:12.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.046+0000 7f3f30ff9700 1 -- 172.21.15.53:0/2208835345 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f3c09cc20 con 0x5652e21183f0 2023-11-06T01:48:12.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.046+0000 7f3f30ff9700 1 -- 172.21.15.53:0/2208835345 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0982e0 con 0x5652e21183f0 2023-11-06T01:48:12.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.047+0000 7f3f5514bb80 1 -- 172.21.15.53:0/2208835345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 msgr2=0x5652e2118870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:12.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.047+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/2208835345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 0x5652e2118870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c095860 tx=0x7f3f3c098820 comp rx=0 tx=0).stop 2023-11-06T01:48:12.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.047+0000 7f3f5514bb80 1 -- 172.21.15.53:0/2208835345 shutdown_connections 2023-11-06T01:48:12.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.047+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/2208835345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 0x5652e2118870 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.047+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/2208835345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e21190f0 0x5652e2119530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.047+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/2208835345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2117e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.047+0000 7f3f5514bb80 1 -- 172.21.15.53:0/2208835345 >> 172.21.15.53:0/2208835345 conn(0x5652e20770d0 msgr2=0x5652e2112940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:12.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.051+0000 7f3f5514bb80 1 -- 172.21.15.53:0/2208835345 shutdown_connections 2023-11-06T01:48:12.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.051+0000 7f3f5514bb80 1 -- 172.21.15.53:0/2208835345 wait complete. 2023-11-06T01:48:12.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.051+0000 7f3f5514bb80 1 -- start start 2023-11-06T01:48:12.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.051+0000 7f3f5514bb80 1 Processor -- start 2023-11-06T01:48:12.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.051+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 0x5652e2121a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.051+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e21190f0 0x5652e2121f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.051+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e2119a70 0x5652e2128fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.051+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e2005510 con 0x5652e21190f0 2023-11-06T01:48:12.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.051+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e20051b0 con 0x5652e21183f0 2023-11-06T01:48:12.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.052+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e1eec920 con 0x5652e2119a70 2023-11-06T01:48:12.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.052+0000 7f3f5514bb80 10 client.0 _ll_register_callbacks cb 0x5652e1f52a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:12.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.059+0000 7f3f42d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e2119a70 0x5652e2128fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.059+0000 7f3f42d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e2119a70 0x5652e2128fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58074/0 (socket says 172.21.15.53:58074) 2023-11-06T01:48:12.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.059+0000 7f3f42d9d700 1 -- 172.21.15.53:0/1985638647 learned_addr learned my addr 172.21.15.53:0/1985638647 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:12.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.062+0000 7f3f41d9b700 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e21190f0 0x5652e2121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.063+0000 7f3f4259c700 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 0x5652e2121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.063+0000 7f3f4259c700 1 -- 172.21.15.53:0/1985638647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e2119a70 msgr2=0x5652e2128fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:12.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.063+0000 7f3f4259c700 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e2119a70 0x5652e2128fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.063+0000 7f3f4259c700 1 -- 172.21.15.53:0/1985638647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e21190f0 msgr2=0x5652e2121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:12.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.063+0000 7f3f4259c700 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e21190f0 0x5652e2121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.063+0000 7f3f4259c700 1 -- 172.21.15.53:0/1985638647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f441adea0 con 0x5652e21183f0 2023-11-06T01:48:12.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.063+0000 7f3f4259c700 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 0x5652e2121a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3f340058e0 tx=0x7f3f34009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f0c274fd1a200af server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:12.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.063+0000 7f3f29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:48:12.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.064+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1985638647 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3402a070 con 0x5652e21183f0 2023-11-06T01:48:12.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.064+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1985638647 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3f34005420 con 0x5652e21183f0 2023-11-06T01:48:12.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.064+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1985638647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652e2093d10 con 0x5652e21183f0 2023-11-06T01:48:12.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.064+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1985638647 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f340055e0 con 0x5652e21183f0 2023-11-06T01:48:12.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.064+0000 7f3f5514bb80 10 client.26495 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:12.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.064+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1985638647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5652e1e7fc60 con 0x5652e21183f0 2023-11-06T01:48:12.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.064+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1985638647 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5652e2093d10 con 0x5652e21183f0 2023-11-06T01:48:12.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.064+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1985638647 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f3f34037020 con 0x5652e21183f0 2023-11-06T01:48:12.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.064+0000 7f3f5514bb80 10 client.26495 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:12.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.064+0000 7f3f5514bb80 10 client.26495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:12.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.064+0000 7f3f5514bb80 20 client.26495 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:12.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.065+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5652e213e7a0 0x5652e2140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.065+0000 7f3f5514bb80 4 client.26495 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:48:12.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.065+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1985638647 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652e20e6e10 con 0x5652e213e7a0 2023-11-06T01:48:12.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.067+0000 7f3f41d9b700 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5652e213e7a0 0x5652e2140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.069+0000 7f3f29ffb700 10 client.26495 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:12.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.069+0000 7f3f41d9b700 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5652e213e7a0 0x5652e2140bc0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea2316f1a312977 server_cookie=81a3018ee2ecaf35 in_seq=0 out_seq=0 2023-11-06T01:48:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.071+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1985638647 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5652e20e6e10 con 0x5652e213e7a0 2023-11-06T01:48:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.071+0000 7f3f29ffb700 10 client.26495 handle_command_reply: tid=0 2023-11-06T01:48:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.071+0000 7f3f5514bb80 1 client.26495 shutdown 2023-11-06T01:48:12.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.071+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1985638647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 msgr2=0x5652e2121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:12.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.071+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 0x5652e2121a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3f340058e0 tx=0x7f3f34009310 comp rx=0 tx=0).stop 2023-11-06T01:48:12.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.072+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1985638647 shutdown_connections 2023-11-06T01:48:12.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.072+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21183f0 0x5652e2121a20 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3f340058e0 tx=0x7f3f34009310 comp rx=0 tx=0).stop 2023-11-06T01:48:12.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.072+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5652e213e7a0 0x5652e2140bc0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.072+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e2119a70 0x5652e2128fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.072+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1985638647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e21190f0 0x5652e2121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.072+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1985638647 >> 172.21.15.53:0/1985638647 conn(0x5652e20770d0 msgr2=0x5652e21120b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:12.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.072+0000 7f3f29ffb700 0 client.26495 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:12.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.074+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1985638647 shutdown_connections 2023-11-06T01:48:12.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.074+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1985638647 wait complete. 2023-11-06T01:48:12.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.074+0000 7f3f5514bb80 20 client.26495 trim_cache size 0 max 16384 2023-11-06T01:48:12.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.099+0000 7f3f5514bb80 1 Processor -- start 2023-11-06T01:48:12.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.099+0000 7f3f5514bb80 1 -- start start 2023-11-06T01:48:12.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21190f0 0x5652e2117e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2128f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e210ed20 0x5652e21217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e20e6e10 con 0x5652e2119a70 2023-11-06T01:48:12.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e2093d10 con 0x5652e21190f0 2023-11-06T01:48:12.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e213e310 con 0x5652e210ed20 2023-11-06T01:48:12.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f41d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2128f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f41d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2128f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43838/0 (socket says 172.21.15.53:43838) 2023-11-06T01:48:12.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f41d9b700 1 -- 172.21.15.53:0/3718704694 learned_addr learned my addr 172.21.15.53:0/3718704694 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:12.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f42d9d700 1 --2- 172.21.15.53:0/3718704694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e210ed20 0x5652e21217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f4259c700 1 --2- 172.21.15.53:0/3718704694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21190f0 0x5652e2117e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f41d9b700 1 -- 172.21.15.53:0/3718704694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e210ed20 msgr2=0x5652e21217d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:12.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f41d9b700 1 --2- 172.21.15.53:0/3718704694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e210ed20 0x5652e21217d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f41d9b700 1 -- 172.21.15.53:0/3718704694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21190f0 msgr2=0x5652e2117e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:12.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f41d9b700 1 --2- 172.21.15.53:0/3718704694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21190f0 0x5652e2117e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f41d9b700 1 -- 172.21.15.53:0/3718704694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652e1ebd250 con 0x5652e2119a70 2023-11-06T01:48:12.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.100+0000 7f3f42d9d700 1 --2- 172.21.15.53:0/3718704694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e210ed20 0x5652e21217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:48:12.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.101+0000 7f3f41d9b700 1 --2- 172.21.15.53:0/3718704694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2128f60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3f380094f0 tx=0x7f3f380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145ad7d85eba1bc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:12.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.101+0000 7f3f317fa700 1 -- 172.21.15.53:0/3718704694 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38016040 con 0x5652e2119a70 2023-11-06T01:48:12.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.101+0000 7f3f317fa700 1 -- 172.21.15.53:0/3718704694 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f38005030 con 0x5652e2119a70 2023-11-06T01:48:12.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.101+0000 7f3f317fa700 1 -- 172.21.15.53:0/3718704694 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38004ad0 con 0x5652e2119a70 2023-11-06T01:48:12.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.102+0000 7f3f5514bb80 1 -- 172.21.15.53:0/3718704694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 msgr2=0x5652e2128f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:12.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.102+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/3718704694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2128f60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3f380094f0 tx=0x7f3f380095c0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.102+0000 7f3f5514bb80 1 -- 172.21.15.53:0/3718704694 shutdown_connections 2023-11-06T01:48:12.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.102+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/3718704694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21190f0 0x5652e2117e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.102+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/3718704694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e210ed20 0x5652e21217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.102+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/3718704694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2128f60 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.102+0000 7f3f5514bb80 1 -- 172.21.15.53:0/3718704694 >> 172.21.15.53:0/3718704694 conn(0x5652e20770d0 msgr2=0x5652e213a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:12.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.102+0000 7f3f5514bb80 1 -- 172.21.15.53:0/3718704694 shutdown_connections 2023-11-06T01:48:12.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.102+0000 7f3f5514bb80 1 -- 172.21.15.53:0/3718704694 wait complete. 2023-11-06T01:48:12.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.102+0000 7f3f5514bb80 1 -- start start 2023-11-06T01:48:12.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.102+0000 7f3f5514bb80 1 Processor -- start 2023-11-06T01:48:12.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e210ed20 0x5652e2128f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21190f0 0x5652e2127530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f5514bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2127b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e2128280 con 0x5652e2119a70 2023-11-06T01:48:12.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e21283f0 con 0x5652e21190f0 2023-11-06T01:48:12.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f5514bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e2190330 con 0x5652e210ed20 2023-11-06T01:48:12.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f5514bb80 10 client.0 _ll_register_callbacks cb 0x7f3f3400be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:12.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f42d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2127b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f4259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e210ed20 0x5652e2128f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f42d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2127b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43848/0 (socket says 172.21.15.53:43848) 2023-11-06T01:48:12.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f41d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21190f0 0x5652e2127530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f42d9d700 1 -- 172.21.15.53:0/1532458257 learned_addr learned my addr 172.21.15.53:0/1532458257 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:12.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f42d9d700 1 -- 172.21.15.53:0/1532458257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e210ed20 msgr2=0x5652e2128f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:12.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f42d9d700 1 --2- 172.21.15.53:0/1532458257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e210ed20 0x5652e2128f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f42d9d700 1 -- 172.21.15.53:0/1532458257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21190f0 msgr2=0x5652e2127530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:12.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f42d9d700 1 --2- 172.21.15.53:0/1532458257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21190f0 0x5652e2127530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f42d9d700 1 -- 172.21.15.53:0/1532458257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652e1e96de0 con 0x5652e2119a70 2023-11-06T01:48:12.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f42d9d700 1 --2- 172.21.15.53:0/1532458257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2127b40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0996a0 tx=0x7f3f3c09cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187cebc700c3574b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:12.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:48:12.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1532458257 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0a0e50 con 0x5652e2119a70 2023-11-06T01:48:12.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1532458257 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3f3c097190 con 0x5652e2119a70 2023-11-06T01:48:12.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.103+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1532458257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f3c09cc20 con 0x5652e2119a70 2023-11-06T01:48:12.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1532458257 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f3c0b1120 con 0x5652e2119a70 2023-11-06T01:48:12.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f5514bb80 10 client.16998 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:12.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1532458257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f3f38009910 con 0x5652e2119a70 2023-11-06T01:48:12.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1532458257 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f3c07b020 con 0x5652e2119a70 2023-11-06T01:48:12.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1532458257 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f3f3c0a09d0 con 0x5652e2119a70 2023-11-06T01:48:12.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f5514bb80 10 client.16998 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:12.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f5514bb80 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:12.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f5514bb80 20 client.16998 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:12.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1532458257 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5652e2191aa0 0x5652e2193ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:12.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f5514bb80 4 client.16998 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:48:12.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1532458257 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652e2061d40 con 0x5652e2191aa0 2023-11-06T01:48:12.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.104+0000 7f3f4259c700 1 --2- 172.21.15.53:0/1532458257 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5652e2191aa0 0x5652e2193ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:12.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.105+0000 7f3f29ffb700 10 client.16998 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:12.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.105+0000 7f3f4259c700 1 --2- 172.21.15.53:0/1532458257 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5652e2191aa0 0x5652e2193ec0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a90654d91965afd server_cookie=74e91d381557da5a in_seq=0 out_seq=0 2023-11-06T01:48:12.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.105+0000 7f3f29ffb700 1 -- 172.21.15.53:0/1532458257 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5652e2061d40 con 0x5652e2191aa0 2023-11-06T01:48:12.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.105+0000 7f3f29ffb700 10 client.16998 handle_command_reply: tid=0 2023-11-06T01:48:12.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.105+0000 7f3f5514bb80 1 client.16998 shutdown 2023-11-06T01:48:12.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.106+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1532458257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 msgr2=0x5652e2127b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:12.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.106+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1532458257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2127b40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0996a0 tx=0x7f3f3c09cd90 comp rx=0 tx=0).stop 2023-11-06T01:48:12.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.106+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1532458257 shutdown_connections 2023-11-06T01:48:12.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.106+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1532458257 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5652e2191aa0 0x5652e2193ec0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.106+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1532458257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652e2119a70 0x5652e2127b40 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.106+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1532458257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652e210ed20 0x5652e2128f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.106+0000 7f3f29ffb700 0 client.16998 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:12.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.106+0000 7f3f5514bb80 1 --2- 172.21.15.53:0/1532458257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e21190f0 0x5652e2127530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.106+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1532458257 >> 172.21.15.53:0/1532458257 conn(0x5652e20770d0 msgr2=0x5652e21217d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:12.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.107+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1532458257 shutdown_connections 2023-11-06T01:48:12.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.107+0000 7f3f5514bb80 1 -- 172.21.15.53:0/1532458257 wait complete. 2023-11-06T01:48:12.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.107+0000 7f3f5514bb80 20 client.16998 trim_cache size 0 max 16384 2023-11-06T01:48:12.123 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:48:12.123 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:48:12.123 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:48:12.123 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:48:12.124 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:48:12.124 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:48:12.124 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:48:12.124 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:48:12.124 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:48:12.125 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:48:12.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.107+0000 7f3f48fbb700 1 -- 172.21.15.53:0/638078492 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f2c0a05a0 msgr2=0x7f3f2c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:12.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.107+0000 7f3f48fbb700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f2c0a05a0 0x7f3f2c0a2a60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009990 tx=0x7f3f34005a90 comp rx=0 tx=0).stop 2023-11-06T01:48:12.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.107+0000 7f3f48fbb700 1 -- 172.21.15.53:0/638078492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f44102910 msgr2=0x7f3f441ad190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:12.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.107+0000 7f3f48fbb700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f44102910 0x7f3f441ad190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00ef30 tx=0x7f3f3c00c450 comp rx=0 tx=0).stop 2023-11-06T01:48:12.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.108+0000 7f3f48fbb700 1 -- 172.21.15.53:0/638078492 shutdown_connections 2023-11-06T01:48:12.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.108+0000 7f3f48fbb700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3f2c0a05a0 0x7f3f2c0a2a60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.108+0000 7f3f48fbb700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f44100d80 0x7f3f4410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.108+0000 7f3f48fbb700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f44102910 0x7f3f441ad190 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.108+0000 7f3f48fbb700 1 --2- 172.21.15.53:0/638078492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f44101f70 0x7f3f44110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:12.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.108+0000 7f3f48fbb700 1 -- 172.21.15.53:0/638078492 >> 172.21.15.53:0/638078492 conn(0x7f3f440fc2d0 msgr2=0x7f3f440fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:12.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.108+0000 7f3f48fbb700 1 -- 172.21.15.53:0/638078492 shutdown_connections 2023-11-06T01:48:12.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:12.108+0000 7f3f48fbb700 1 -- 172.21.15.53:0/638078492 wait complete. 2023-11-06T01:48:12.129 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:48:12.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 835.0 s, 1462.2 tps, lat 68.052 ms stddev 76.299 2023-11-06T01:48:12.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:12 smithi163 ceph-mon[104416]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 MiB/s wr, 681 op/s 2023-11-06T01:48:12.844 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:12 smithi053 ceph-mon[99472]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 MiB/s wr, 681 op/s 2023-11-06T01:48:12.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:12 smithi062 ceph-mon[103924]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 MiB/s wr, 681 op/s 2023-11-06T01:48:15.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:15 smithi053 ceph-mon[99472]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 725 op/s 2023-11-06T01:48:15.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:15 smithi062 ceph-mon[103924]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 725 op/s 2023-11-06T01:48:15.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:15 smithi163 ceph-mon[104416]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 725 op/s 2023-11-06T01:48:17.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 840.0 s, 1919.6 tps, lat 52.511 ms stddev 66.859 2023-11-06T01:48:17.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:17 smithi053 ceph-mon[99472]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 750 op/s 2023-11-06T01:48:17.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:17 smithi062 ceph-mon[103924]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 750 op/s 2023-11-06T01:48:17.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:17 smithi163 ceph-mon[104416]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 750 op/s 2023-11-06T01:48:19.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:19 smithi062 ceph-mon[103924]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 776 op/s 2023-11-06T01:48:19.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:19 smithi163 ceph-mon[104416]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 776 op/s 2023-11-06T01:48:19.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:19 smithi053 ceph-mon[99472]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 776 op/s 2023-11-06T01:48:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.003+0000 7ff2687b0700 1 -- 172.21.15.53:0/1586514499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff260101890 msgr2=0x7ff260108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.003+0000 7ff2687b0700 1 --2- 172.21.15.53:0/1586514499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff260101890 0x7ff260108140 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff25800b0d0 tx=0x7ff25800b3e0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.003+0000 7ff2687b0700 1 -- 172.21.15.53:0/1586514499 shutdown_connections 2023-11-06T01:48:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.003+0000 7ff2687b0700 1 --2- 172.21.15.53:0/1586514499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff260101890 0x7ff260108140 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.003+0000 7ff2687b0700 1 --2- 172.21.15.53:0/1586514499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2600ffd00 0x7ff260100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.003+0000 7ff2687b0700 1 --2- 172.21.15.53:0/1586514499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff260100ef0 0x7ff260101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.003+0000 7ff2687b0700 1 -- 172.21.15.53:0/1586514499 >> 172.21.15.53:0/1586514499 conn(0x7ff2600fb250 msgr2=0x7ff2600fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.003+0000 7ff2687b0700 1 -- 172.21.15.53:0/1586514499 shutdown_connections 2023-11-06T01:48:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.003+0000 7ff2687b0700 1 -- 172.21.15.53:0/1586514499 wait complete. 2023-11-06T01:48:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff2687b0700 1 Processor -- start 2023-11-06T01:48:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff2687b0700 1 -- start start 2023-11-06T01:48:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff2687b0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2600ffd00 0x7ff260199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff2687b0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260100ef0 0x7ff26019be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff2687b0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff260101890 0x7ff26019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff2687b0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26019e9a0 con 0x7ff260101890 2023-11-06T01:48:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff2687b0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26019eae0 con 0x7ff2600ffd00 2023-11-06T01:48:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff2687b0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26019eca0 con 0x7ff260100ef0 2023-11-06T01:48:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff265d4b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260100ef0 0x7ff26019be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff265d4b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260100ef0 0x7ff26019be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58100/0 (socket says 172.21.15.53:58100) 2023-11-06T01:48:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff265d4b700 1 -- 172.21.15.53:0/1444775636 learned_addr learned my addr 172.21.15.53:0/1444775636 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.004+0000 7ff26654c700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2600ffd00 0x7ff260199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff266d4d700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff260101890 0x7ff26019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff265d4b700 1 -- 172.21.15.53:0/1444775636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2600ffd00 msgr2=0x7ff260199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff265d4b700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2600ffd00 0x7ff260199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff265d4b700 1 -- 172.21.15.53:0/1444775636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff260101890 msgr2=0x7ff26019e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff265d4b700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff260101890 0x7ff26019e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff265d4b700 1 -- 172.21.15.53:0/1444775636 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff258009d00 con 0x7ff260100ef0 2023-11-06T01:48:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff266d4d700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff260101890 0x7ff26019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:48:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff265d4b700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260100ef0 0x7ff26019be80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff25000e9a0 tx=0x7ff25000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff2577fe700 1 -- 172.21.15.53:0/1444775636 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250009790 con 0x7ff260100ef0 2023-11-06T01:48:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff2577fe700 1 -- 172.21.15.53:0/1444775636 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff25000cbd0 con 0x7ff260100ef0 2023-11-06T01:48:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff2577fe700 1 -- 172.21.15.53:0/1444775636 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250018680 con 0x7ff260100ef0 2023-11-06T01:48:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff2687b0700 1 -- 172.21.15.53:0/1444775636 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26019ef80 con 0x7ff260100ef0 2023-11-06T01:48:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.005+0000 7ff2687b0700 1 -- 172.21.15.53:0/1444775636 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26019f3b0 con 0x7ff260100ef0 2023-11-06T01:48:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.008+0000 7ff2577fe700 1 -- 172.21.15.53:0/1444775636 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff25000cd40 con 0x7ff260100ef0 2023-11-06T01:48:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.009+0000 7ff2577fe700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff24c0a0520 0x7ff24c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.009+0000 7ff26654c700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff24c0a0520 0x7ff24c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.009+0000 7ff2577fe700 1 -- 172.21.15.53:0/1444775636 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff250014070 con 0x7ff260100ef0 2023-11-06T01:48:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.009+0000 7ff26654c700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff24c0a0520 0x7ff24c0a29e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff260069720 tx=0x7ff25c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.009+0000 7ff274940b80 1 Processor -- start 2023-11-06T01:48:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff274940b80 1 -- start start 2023-11-06T01:48:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff274940b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc23aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff274940b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 0x5635bc23b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff274940b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23bed0 0x5635bc23c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff274940b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc1929b0 con 0x5635bc23afe0 2023-11-06T01:48:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff274940b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc190370 con 0x5635bc23a680 2023-11-06T01:48:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff274940b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc18bc60 con 0x5635bc23bed0 2023-11-06T01:48:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff265d4b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 0x5635bc23b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff26654c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc23aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff265d4b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 0x5635bc23b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43868/0 (socket says 172.21.15.53:43868) 2023-11-06T01:48:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff265d4b700 1 -- 172.21.15.53:0/3819688796 learned_addr learned my addr 172.21.15.53:0/3819688796 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff266d4d700 1 --2- 172.21.15.53:0/3819688796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23bed0 0x5635bc23c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff26654c700 1 -- 172.21.15.53:0/3819688796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23bed0 msgr2=0x5635bc23c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff26654c700 1 --2- 172.21.15.53:0/3819688796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23bed0 0x5635bc23c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff26654c700 1 -- 172.21.15.53:0/3819688796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 msgr2=0x5635bc23b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff26654c700 1 --2- 172.21.15.53:0/3819688796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 0x5635bc23b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.010+0000 7ff26654c700 1 -- 172.21.15.53:0/3819688796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635bbfb9d80 con 0x5635bc23a680 2023-11-06T01:48:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.011+0000 7ff26654c700 1 --2- 172.21.15.53:0/3819688796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc23aaa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff260100d50 tx=0x7ff25c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1e465450bdafcbb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.011+0000 7ff254ff9700 1 -- 172.21.15.53:0/3819688796 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c004220 con 0x5635bc23a680 2023-11-06T01:48:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.011+0000 7ff254ff9700 1 -- 172.21.15.53:0/3819688796 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff25c025070 con 0x5635bc23a680 2023-11-06T01:48:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.011+0000 7ff254ff9700 1 -- 172.21.15.53:0/3819688796 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c021df0 con 0x5635bc23a680 2023-11-06T01:48:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.012+0000 7ff274940b80 1 -- 172.21.15.53:0/3819688796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 msgr2=0x5635bc23aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.012+0000 7ff274940b80 1 --2- 172.21.15.53:0/3819688796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc23aaa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff260100d50 tx=0x7ff25c014000 comp rx=0 tx=0).stop 2023-11-06T01:48:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.012+0000 7ff274940b80 1 -- 172.21.15.53:0/3819688796 shutdown_connections 2023-11-06T01:48:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.012+0000 7ff274940b80 1 --2- 172.21.15.53:0/3819688796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc23aaa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.012+0000 7ff274940b80 1 --2- 172.21.15.53:0/3819688796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23bed0 0x5635bc23c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.012+0000 7ff274940b80 1 --2- 172.21.15.53:0/3819688796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 0x5635bc23b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.012+0000 7ff274940b80 1 -- 172.21.15.53:0/3819688796 >> 172.21.15.53:0/3819688796 conn(0x5635bc19a070 msgr2=0x5635bc237040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.012+0000 7ff274940b80 1 -- 172.21.15.53:0/3819688796 shutdown_connections 2023-11-06T01:48:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.012+0000 7ff274940b80 1 -- 172.21.15.53:0/3819688796 wait complete. 2023-11-06T01:48:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff274940b80 1 -- start start 2023-11-06T01:48:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff274940b80 1 Processor -- start 2023-11-06T01:48:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff274940b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc2444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff274940b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23afe0 0x5635bc244a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff274940b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23bed0 0x5635bc24ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff274940b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc1284b0 con 0x5635bc23bed0 2023-11-06T01:48:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff274940b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc128150 con 0x5635bc23a680 2023-11-06T01:48:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff274940b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc00f8c0 con 0x5635bc23afe0 2023-11-06T01:48:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff266d4d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23bed0 0x5635bc24ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff26654c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc2444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff274940b80 10 client.0 _ll_register_callbacks cb 0x5635bc075a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff266d4d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23bed0 0x5635bc24ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43876/0 (socket says 172.21.15.53:43876) 2023-11-06T01:48:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff266d4d700 1 -- 172.21.15.53:0/1879322862 learned_addr learned my addr 172.21.15.53:0/1879322862 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.013+0000 7ff265d4b700 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23afe0 0x5635bc244a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff26654c700 1 -- 172.21.15.53:0/1879322862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23afe0 msgr2=0x5635bc244a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff26654c700 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23afe0 0x5635bc244a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff26654c700 1 -- 172.21.15.53:0/1879322862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23bed0 msgr2=0x5635bc24ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff26654c700 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23bed0 0x5635bc24ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff26654c700 1 -- 172.21.15.53:0/1879322862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c017070 con 0x5635bc23a680 2023-11-06T01:48:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff266d4d700 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23bed0 0x5635bc24ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:48:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff26654c700 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc2444c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009780 tx=0x7ff25c022c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddb207a22f62121 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff249ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:48:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff249ffb700 1 -- 172.21.15.53:0/1879322862 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c02e070 con 0x5635bc23a680 2023-11-06T01:48:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff274940b80 1 -- 172.21.15.53:0/1879322862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635bc1b6cb0 con 0x5635bc23a680 2023-11-06T01:48:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff249ffb700 1 -- 172.21.15.53:0/1879322862 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff25c00ea50 con 0x5635bc23a680 2023-11-06T01:48:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff249ffb700 1 -- 172.21.15.53:0/1879322862 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff25c00ec10 con 0x5635bc23a680 2023-11-06T01:48:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff274940b80 10 client.26513 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.014+0000 7ff274940b80 1 -- 172.21.15.53:0/1879322862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5635bbfa2c00 con 0x5635bc23a680 2023-11-06T01:48:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.015+0000 7ff249ffb700 1 -- 172.21.15.53:0/1879322862 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635bc1b6cb0 con 0x5635bc23a680 2023-11-06T01:48:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.015+0000 7ff249ffb700 1 -- 172.21.15.53:0/1879322862 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff25c03b020 con 0x5635bc23a680 2023-11-06T01:48:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.015+0000 7ff274940b80 10 client.26513 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.015+0000 7ff274940b80 10 client.26513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.015+0000 7ff274940b80 20 client.26513 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.015+0000 7ff274940b80 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5635bc250240 0x5635bc252660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.015+0000 7ff274940b80 4 client.26513 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:48:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.015+0000 7ff274940b80 1 -- 172.21.15.53:0/1879322862 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635bc209db0 con 0x5635bc250240 2023-11-06T01:48:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.015+0000 7ff265d4b700 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5635bc250240 0x5635bc252660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff249ffb700 10 client.26513 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff265d4b700 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5635bc250240 0x5635bc252660 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c575c3fc5757f5e0 server_cookie=26652da95f9d5888 in_seq=0 out_seq=0 2023-11-06T01:48:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff249ffb700 1 -- 172.21.15.53:0/1879322862 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5635bc209db0 con 0x5635bc250240 2023-11-06T01:48:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff249ffb700 10 client.26513 handle_command_reply: tid=0 2023-11-06T01:48:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff274940b80 1 client.26513 shutdown 2023-11-06T01:48:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff274940b80 1 -- 172.21.15.53:0/1879322862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 msgr2=0x5635bc2444c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff274940b80 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc2444c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009780 tx=0x7ff25c022c20 comp rx=0 tx=0).stop 2023-11-06T01:48:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff274940b80 1 -- 172.21.15.53:0/1879322862 shutdown_connections 2023-11-06T01:48:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff274940b80 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc2444c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff274940b80 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5635bc250240 0x5635bc252660 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff274940b80 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23bed0 0x5635bc24ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff249ffb700 0 client.26513 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff274940b80 1 --2- 172.21.15.53:0/1879322862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23afe0 0x5635bc244a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff274940b80 1 -- 172.21.15.53:0/1879322862 >> 172.21.15.53:0/1879322862 conn(0x5635bc19a070 msgr2=0x5635bc2353c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff274940b80 1 -- 172.21.15.53:0/1879322862 shutdown_connections 2023-11-06T01:48:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.016+0000 7ff274940b80 1 -- 172.21.15.53:0/1879322862 wait complete. 2023-11-06T01:48:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.017+0000 7ff274940b80 20 client.26513 trim_cache size 0 max 16384 2023-11-06T01:48:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.033+0000 7ff274940b80 1 Processor -- start 2023-11-06T01:48:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff274940b80 1 -- start start 2023-11-06T01:48:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff274940b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23a680 0x5635bc2499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff274940b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23afe0 0x5635bc23c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff274940b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23bed0 0x5635bc243b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff274940b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc230500 con 0x5635bc23a680 2023-11-06T01:48:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff274940b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc242f90 con 0x5635bc23bed0 2023-11-06T01:48:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff274940b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc23a280 con 0x5635bc23afe0 2023-11-06T01:48:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff26654c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23a680 0x5635bc2499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff265d4b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23afe0 0x5635bc23c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff26654c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23a680 0x5635bc2499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43880/0 (socket says 172.21.15.53:43880) 2023-11-06T01:48:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff26654c700 1 -- 172.21.15.53:0/2922219973 learned_addr learned my addr 172.21.15.53:0/2922219973 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff266d4d700 1 --2- 172.21.15.53:0/2922219973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23bed0 0x5635bc243b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff26654c700 1 -- 172.21.15.53:0/2922219973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23afe0 msgr2=0x5635bc23c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff26654c700 1 --2- 172.21.15.53:0/2922219973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23afe0 0x5635bc23c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff26654c700 1 -- 172.21.15.53:0/2922219973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23bed0 msgr2=0x5635bc243b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff26654c700 1 --2- 172.21.15.53:0/2922219973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23bed0 0x5635bc243b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff26654c700 1 -- 172.21.15.53:0/2922219973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635bbfe01f0 con 0x5635bc23a680 2023-11-06T01:48:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.034+0000 7ff265d4b700 1 --2- 172.21.15.53:0/2922219973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23afe0 0x5635bc23c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:48:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.035+0000 7ff26654c700 1 --2- 172.21.15.53:0/2922219973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23a680 0x5635bc2499f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00e710 tx=0x7ff25c03d510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a458283e1e1611b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.035+0000 7ff2557fa700 1 -- 172.21.15.53:0/2922219973 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c02e020 con 0x5635bc23a680 2023-11-06T01:48:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.035+0000 7ff2557fa700 1 -- 172.21.15.53:0/2922219973 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff25c022d40 con 0x5635bc23a680 2023-11-06T01:48:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.035+0000 7ff2557fa700 1 -- 172.21.15.53:0/2922219973 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c03dac0 con 0x5635bc23a680 2023-11-06T01:48:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.036+0000 7ff274940b80 1 -- 172.21.15.53:0/2922219973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23a680 msgr2=0x5635bc2499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.036+0000 7ff274940b80 1 --2- 172.21.15.53:0/2922219973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23a680 0x5635bc2499f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00e710 tx=0x7ff25c03d510 comp rx=0 tx=0).stop 2023-11-06T01:48:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.036+0000 7ff274940b80 1 -- 172.21.15.53:0/2922219973 shutdown_connections 2023-11-06T01:48:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.036+0000 7ff274940b80 1 --2- 172.21.15.53:0/2922219973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23bed0 0x5635bc243b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.036+0000 7ff274940b80 1 --2- 172.21.15.53:0/2922219973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23a680 0x5635bc2499f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.036+0000 7ff274940b80 1 --2- 172.21.15.53:0/2922219973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23afe0 0x5635bc23c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.036+0000 7ff274940b80 1 -- 172.21.15.53:0/2922219973 >> 172.21.15.53:0/2922219973 conn(0x5635bc19a070 msgr2=0x5635bc236c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.037+0000 7ff274940b80 1 -- 172.21.15.53:0/2922219973 shutdown_connections 2023-11-06T01:48:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.037+0000 7ff274940b80 1 -- 172.21.15.53:0/2922219973 wait complete. 2023-11-06T01:48:21.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.037+0000 7ff274940b80 1 -- start start 2023-11-06T01:48:21.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.037+0000 7ff274940b80 1 Processor -- start 2023-11-06T01:48:21.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.037+0000 7ff274940b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc248ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.037+0000 7ff274940b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 0x5635bc249420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.037+0000 7ff274940b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23bed0 0x5635bc2a0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.037+0000 7ff274940b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc2a0740 con 0x5635bc23afe0 2023-11-06T01:48:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.037+0000 7ff274940b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc2a0880 con 0x5635bc23a680 2023-11-06T01:48:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.037+0000 7ff274940b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bc2a0a40 con 0x5635bc23bed0 2023-11-06T01:48:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.037+0000 7ff274940b80 10 client.0 _ll_register_callbacks cb 0x7ff2500950b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff265d4b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 0x5635bc249420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff265d4b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 0x5635bc249420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43888/0 (socket says 172.21.15.53:43888) 2023-11-06T01:48:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff266d4d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23bed0 0x5635bc2a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff26654c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc248ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff265d4b700 1 -- 172.21.15.53:0/4002342225 learned_addr learned my addr 172.21.15.53:0/4002342225 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff265d4b700 1 -- 172.21.15.53:0/4002342225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23bed0 msgr2=0x5635bc2a0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff265d4b700 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23bed0 0x5635bc2a0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff265d4b700 1 -- 172.21.15.53:0/4002342225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 msgr2=0x5635bc248ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff265d4b700 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc248ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff265d4b700 1 -- 172.21.15.53:0/4002342225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26019f3b0 con 0x5635bc23afe0 2023-11-06T01:48:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff26654c700 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc248ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:48:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff265d4b700 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 0x5635bc249420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff25001e000 tx=0x7ff250084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a15d0a7d418cfd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff249ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:48:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff249ffb700 1 -- 172.21.15.53:0/4002342225 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2500a14a0 con 0x5635bc23afe0 2023-11-06T01:48:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff249ffb700 1 -- 172.21.15.53:0/4002342225 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff2500b0070 con 0x5635bc23afe0 2023-11-06T01:48:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff274940b80 1 -- 172.21.15.53:0/4002342225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635bc184ce0 con 0x5635bc23afe0 2023-11-06T01:48:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff249ffb700 1 -- 172.21.15.53:0/4002342225 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2500977d0 con 0x5635bc23afe0 2023-11-06T01:48:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff274940b80 10 client.17013 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.038+0000 7ff274940b80 1 -- 172.21.15.53:0/4002342225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5635bbfa2c00 con 0x5635bc23afe0 2023-11-06T01:48:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.039+0000 7ff249ffb700 1 -- 172.21.15.53:0/4002342225 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635bc184ce0 con 0x5635bc23afe0 2023-11-06T01:48:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.039+0000 7ff249ffb700 1 -- 172.21.15.53:0/4002342225 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff2500a1010 con 0x5635bc23afe0 2023-11-06T01:48:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.039+0000 7ff274940b80 10 client.17013 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.039+0000 7ff274940b80 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.039+0000 7ff274940b80 20 client.17013 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.039+0000 7ff274940b80 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5635bc2a2430 0x5635bc2a4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.039+0000 7ff274940b80 4 client.17013 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:48:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.039+0000 7ff274940b80 1 -- 172.21.15.53:0/4002342225 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5635bc184a00 con 0x5635bc2a2430 2023-11-06T01:48:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.039+0000 7ff26654c700 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5635bc2a2430 0x5635bc2a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.039+0000 7ff249ffb700 10 client.17013 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.039+0000 7ff26654c700 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5635bc2a2430 0x5635bc2a4850 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44afbd6548b8bb3f server_cookie=87fae7d5b32b9b3d in_seq=0 out_seq=0 2023-11-06T01:48:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff249ffb700 1 -- 172.21.15.53:0/4002342225 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5635bc184a00 con 0x5635bc2a2430 2023-11-06T01:48:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff249ffb700 10 client.17013 handle_command_reply: tid=0 2023-11-06T01:48:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff274940b80 1 client.17013 shutdown 2023-11-06T01:48:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff274940b80 1 -- 172.21.15.53:0/4002342225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 msgr2=0x5635bc249420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff274940b80 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 0x5635bc249420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff25001e000 tx=0x7ff250084040 comp rx=0 tx=0).stop 2023-11-06T01:48:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff274940b80 1 -- 172.21.15.53:0/4002342225 shutdown_connections 2023-11-06T01:48:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff274940b80 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635bc23a680 0x5635bc248ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff274940b80 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5635bc2a2430 0x5635bc2a4850 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff274940b80 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635bc23bed0 0x5635bc2a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff274940b80 1 --2- 172.21.15.53:0/4002342225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5635bc23afe0 0x5635bc249420 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff249ffb700 0 client.17013 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff274940b80 1 -- 172.21.15.53:0/4002342225 >> 172.21.15.53:0/4002342225 conn(0x5635bc19a070 msgr2=0x5635bc242240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.047+0000 7ff274940b80 1 -- 172.21.15.53:0/4002342225 shutdown_connections 2023-11-06T01:48:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.049+0000 7ff274940b80 1 -- 172.21.15.53:0/4002342225 wait complete. 2023-11-06T01:48:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.049+0000 7ff274940b80 20 client.17013 trim_cache size 0 max 16384 2023-11-06T01:48:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:48:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:48:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:48:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:48:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 -- 172.21.15.53:0/1444775636 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff24c0a0520 msgr2=0x7ff24c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff24c0a0520 0x7ff24c0a29e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff260069720 tx=0x7ff25c009260 comp rx=0 tx=0).stop 2023-11-06T01:48:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 -- 172.21.15.53:0/1444775636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260100ef0 msgr2=0x7ff26019be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260100ef0 0x7ff26019be80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff25000e9a0 tx=0x7ff25000ecb0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 -- 172.21.15.53:0/1444775636 shutdown_connections 2023-11-06T01:48:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ff24c0a0520 0x7ff24c0a29e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2600ffd00 0x7ff260199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff260101890 0x7ff26019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 --2- 172.21.15.53:0/1444775636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff260100ef0 0x7ff26019be80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 -- 172.21.15.53:0/1444775636 >> 172.21.15.53:0/1444775636 conn(0x7ff2600fb250 msgr2=0x7ff260104380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 -- 172.21.15.53:0/1444775636 shutdown_connections 2023-11-06T01:48:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:21.050+0000 7ff2687b0700 1 -- 172.21.15.53:0/1444775636 wait complete. 2023-11-06T01:48:21.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:21 smithi062 ceph-mon[103924]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 775 op/s 2023-11-06T01:48:21.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:21 smithi163 ceph-mon[104416]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 775 op/s 2023-11-06T01:48:21.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:21 smithi053 ceph-mon[99472]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 775 op/s 2023-11-06T01:48:22.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 845.0 s, 1527.2 tps, lat 65.496 ms stddev 74.323 2023-11-06T01:48:22.841 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:22 smithi053 ceph-mon[99472]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 690 op/s 2023-11-06T01:48:22.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:22 smithi062 ceph-mon[103924]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 690 op/s 2023-11-06T01:48:22.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:22 smithi163 ceph-mon[104416]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 690 op/s 2023-11-06T01:48:25.498 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:25 smithi163 ceph-mon[104416]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 710 op/s 2023-11-06T01:48:25.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:25 smithi053 ceph-mon[99472]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 710 op/s 2023-11-06T01:48:25.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:25 smithi062 ceph-mon[103924]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 710 op/s 2023-11-06T01:48:26.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:26 smithi163 ceph-mon[104416]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 653 op/s 2023-11-06T01:48:26.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:26 smithi053 ceph-mon[99472]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 653 op/s 2023-11-06T01:48:26.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:26 smithi062 ceph-mon[103924]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 653 op/s 2023-11-06T01:48:27.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 850.0 s, 1565.2 tps, lat 63.416 ms stddev 73.210 2023-11-06T01:48:29.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:29 smithi062 ceph-mon[103924]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 643 op/s 2023-11-06T01:48:29.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:29 smithi163 ceph-mon[104416]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 643 op/s 2023-11-06T01:48:29.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:29 smithi053 ceph-mon[99472]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 643 op/s 2023-11-06T01:48:31.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:31 smithi062 ceph-mon[103924]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 641 op/s 2023-11-06T01:48:31.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:31 smithi163 ceph-mon[104416]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 641 op/s 2023-11-06T01:48:31.596 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:31 smithi053 ceph-mon[99472]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 641 op/s 2023-11-06T01:48:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 855.0 s, 1166.6 tps, lat 84.295 ms stddev 126.489 2023-11-06T01:48:32.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:32 smithi163 ceph-mon[104416]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 608 op/s 2023-11-06T01:48:32.841 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:32 smithi053 ceph-mon[99472]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 608 op/s 2023-11-06T01:48:32.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:32 smithi062 ceph-mon[103924]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 608 op/s 2023-11-06T01:48:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.229+0000 7fbbe38ea700 1 -- 172.21.15.53:0/2368411575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbdc0ff600 msgr2=0x7fbbdc101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.229+0000 7fbbe38ea700 1 --2- 172.21.15.53:0/2368411575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbdc0ff600 0x7fbbdc101a30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009990 tx=0x7fbbcc009ca0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.229+0000 7fbbe38ea700 1 -- 172.21.15.53:0/2368411575 shutdown_connections 2023-11-06T01:48:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.229+0000 7fbbe38ea700 1 --2- 172.21.15.53:0/2368411575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbdc104940 0x7fbbdc104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.229+0000 7fbbe38ea700 1 --2- 172.21.15.53:0/2368411575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbdc0ff600 0x7fbbdc101a30 secure :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009990 tx=0x7fbbcc009ca0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.229+0000 7fbbe38ea700 1 --2- 172.21.15.53:0/2368411575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbdc101f70 0x7fbbdc104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.229+0000 7fbbe38ea700 1 -- 172.21.15.53:0/2368411575 >> 172.21.15.53:0/2368411575 conn(0x7fbbdc0fb1e0 msgr2=0x7fbbdc0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.232+0000 7fbbe38ea700 1 -- 172.21.15.53:0/2368411575 shutdown_connections 2023-11-06T01:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.232+0000 7fbbe38ea700 1 -- 172.21.15.53:0/2368411575 wait complete. 2023-11-06T01:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe38ea700 1 Processor -- start 2023-11-06T01:48:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe38ea700 1 -- start start 2023-11-06T01:48:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe38ea700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbdc101f70 0x7fbbdc19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe38ea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbdc104940 0x7fbbdc19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe38ea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbdc19e7d0 0x7fbbdc19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe38ea700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc19f220 con 0x7fbbdc101f70 2023-11-06T01:48:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe38ea700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc19f390 con 0x7fbbdc104940 2023-11-06T01:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe38ea700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc19f550 con 0x7fbbdc19e7d0 2023-11-06T01:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe1686700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbdc101f70 0x7fbbdc19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe1686700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbdc101f70 0x7fbbdc19d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43900/0 (socket says 172.21.15.53:43900) 2023-11-06T01:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe1686700 1 -- 172.21.15.53:0/1374060107 learned_addr learned my addr 172.21.15.53:0/1374060107 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.233+0000 7fbbe1e87700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbdc19e7d0 0x7fbbdc19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.234+0000 7fbbe1686700 1 -- 172.21.15.53:0/1374060107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbdc19e7d0 msgr2=0x7fbbdc19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.234+0000 7fbbe1686700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbdc19e7d0 0x7fbbdc19ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.234+0000 7fbbe1686700 1 -- 172.21.15.53:0/1374060107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbdc104940 msgr2=0x7fbbdc19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.234+0000 7fbbe1686700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbdc104940 0x7fbbdc19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.234+0000 7fbbe1686700 1 -- 172.21.15.53:0/1374060107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc0095f0 con 0x7fbbdc101f70 2023-11-06T01:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.234+0000 7fbbe1686700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbdc101f70 0x7fbbdc19d410 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00b680 tx=0x7fbbcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.234+0000 7fbbd27fc700 1 -- 172.21.15.53:0/1374060107 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc01d070 con 0x7fbbdc101f70 2023-11-06T01:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.234+0000 7fbbe38ea700 1 -- 172.21.15.53:0/1374060107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbdc1a5a20 con 0x7fbbdc101f70 2023-11-06T01:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.234+0000 7fbbe38ea700 1 -- 172.21.15.53:0/1374060107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbdc1a5ec0 con 0x7fbbdc101f70 2023-11-06T01:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.234+0000 7fbbd27fc700 1 -- 172.21.15.53:0/1374060107 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbcc0054f0 con 0x7fbbdc101f70 2023-11-06T01:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.234+0000 7fbbd27fc700 1 -- 172.21.15.53:0/1374060107 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc004c00 con 0x7fbbdc101f70 2023-11-06T01:48:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.238+0000 7fbbd27fc700 1 -- 172.21.15.53:0/1374060107 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fbbcc00fb80 con 0x7fbbdc101f70 2023-11-06T01:48:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.239+0000 7fbbd27fc700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbc80a05a0 0x7fbbc80a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.239+0000 7fbbefa7ab80 1 Processor -- start 2023-11-06T01:48:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.240+0000 7fbbefa7ab80 1 -- start start 2023-11-06T01:48:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.240+0000 7fbbd27fc700 1 -- 172.21.15.53:0/1374060107 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbbcc0cc9a0 con 0x7fbbdc101f70 2023-11-06T01:48:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.240+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a79690 0x55c575a79ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.240+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79ff0 0x55c575a7a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.240+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a7aee0 0x55c575a7b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.240+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5759c09b0 con 0x55c575a79690 2023-11-06T01:48:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.240+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5759be370 con 0x55c575a7aee0 2023-11-06T01:48:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.240+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5759b9c60 con 0x55c575a79ff0 2023-11-06T01:48:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.241+0000 7fbbe1e87700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a7aee0 0x55c575a7b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.241+0000 7fbbe1e87700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a7aee0 0x55c575a7b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55344/0 (socket says 172.21.15.53:55344) 2023-11-06T01:48:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.241+0000 7fbbe1e87700 1 -- 172.21.15.53:0/1697309899 learned_addr learned my addr 172.21.15.53:0/1697309899 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.241+0000 7fbbe1e87700 1 -- 172.21.15.53:0/1697309899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79ff0 msgr2=0x55c575a7a900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:48:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.241+0000 7fbbe1e87700 1 --2- 172.21.15.53:0/1697309899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79ff0 0x55c575a7a900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.241+0000 7fbbe1e87700 1 -- 172.21.15.53:0/1697309899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a79690 msgr2=0x55c575a79ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.241+0000 7fbbe1e87700 1 --2- 172.21.15.53:0/1697309899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a79690 0x55c575a79ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.241+0000 7fbbe1e87700 1 -- 172.21.15.53:0/1697309899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5757e7d80 con 0x55c575a7aee0 2023-11-06T01:48:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.244+0000 7fbbe0e85700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbc80a05a0 0x7fbbc80a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.245+0000 7fbbe0e85700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbc80a05a0 0x7fbbc80a2a60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009710 tx=0x7fbbd8006cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.246+0000 7fbbe1e87700 1 --2- 172.21.15.53:0/1697309899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a7aee0 0x55c575a7b360 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4011680 tx=0x7fbbd4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aebfc13ea3773d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.251+0000 7fbbc77fe700 1 -- 172.21.15.53:0/1697309899 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400e040 con 0x55c575a7aee0 2023-11-06T01:48:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.251+0000 7fbbc77fe700 1 -- 172.21.15.53:0/1697309899 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbd400f040 con 0x55c575a7aee0 2023-11-06T01:48:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.251+0000 7fbbc77fe700 1 -- 172.21.15.53:0/1697309899 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40048c0 con 0x55c575a7aee0 2023-11-06T01:48:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.251+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1697309899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a7aee0 msgr2=0x55c575a7b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.252+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/1697309899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a7aee0 0x55c575a7b360 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4011680 tx=0x7fbbd4009920 comp rx=0 tx=0).stop 2023-11-06T01:48:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.252+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1697309899 shutdown_connections 2023-11-06T01:48:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.252+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/1697309899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a7aee0 0x55c575a7b360 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.252+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/1697309899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a79690 0x55c575a79ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.252+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/1697309899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79ff0 0x55c575a7a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.252+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1697309899 >> 172.21.15.53:0/1697309899 conn(0x55c5759c8070 msgr2=0x55c575a65040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.252+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1697309899 shutdown_connections 2023-11-06T01:48:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.252+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1697309899 wait complete. 2023-11-06T01:48:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.252+0000 7fbbefa7ab80 1 -- start start 2023-11-06T01:48:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.252+0000 7fbbefa7ab80 1 Processor -- start 2023-11-06T01:48:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.253+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a79690 0x55c575a69a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.253+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a69fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.253+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a7aee0 0x55c575a71010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.253+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5759564b0 con 0x55c575a79690 2023-11-06T01:48:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.253+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c575956150 con 0x55c575a79ff0 2023-11-06T01:48:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.253+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57583d8c0 con 0x55c575a7aee0 2023-11-06T01:48:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.253+0000 7fbbefa7ab80 10 client.0 _ll_register_callbacks cb 0x55c5758a3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.255+0000 7fbbe1e87700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a7aee0 0x55c575a71010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.255+0000 7fbbe1e87700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a7aee0 0x55c575a71010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58154/0 (socket says 172.21.15.53:58154) 2023-11-06T01:48:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.255+0000 7fbbe1e87700 1 -- 172.21.15.53:0/3789523680 learned_addr learned my addr 172.21.15.53:0/3789523680 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.255+0000 7fbbe0e85700 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a69fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.255+0000 7fbbe1e87700 1 -- 172.21.15.53:0/3789523680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 msgr2=0x55c575a69fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.255+0000 7fbbe1e87700 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a69fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.255+0000 7fbbe1e87700 1 -- 172.21.15.53:0/3789523680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a79690 msgr2=0x55c575a69a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.255+0000 7fbbe1e87700 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a79690 0x55c575a69a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.256+0000 7fbbe1e87700 1 -- 172.21.15.53:0/3789523680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd4011330 con 0x55c575a7aee0 2023-11-06T01:48:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.256+0000 7fbbe1e87700 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a7aee0 0x55c575a71010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400b010 tx=0x7fbbd4003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1601050361dc95bc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.256+0000 7fbbc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:48:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.256+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/3789523680 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400e030 con 0x55c575a7aee0 2023-11-06T01:48:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.256+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/3789523680 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbbd40093a0 con 0x55c575a7aee0 2023-11-06T01:48:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.256+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/3789523680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5759e4cb0 con 0x55c575a7aee0 2023-11-06T01:48:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.256+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/3789523680 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbd4004ad0 con 0x55c575a7aee0 2023-11-06T01:48:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.256+0000 7fbbefa7ab80 10 client.26587 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.256+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/3789523680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c5757d0c00 con 0x55c575a7aee0 2023-11-06T01:48:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.257+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/3789523680 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5759e4cb0 con 0x55c575a7aee0 2023-11-06T01:48:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.257+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/3789523680 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fbbd4024020 con 0x55c575a7aee0 2023-11-06T01:48:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.257+0000 7fbbefa7ab80 10 client.26587 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.257+0000 7fbbefa7ab80 10 client.26587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.257+0000 7fbbefa7ab80 20 client.26587 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.257+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c575a75830 0x55c575a77c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.257+0000 7fbbefa7ab80 4 client.26587 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:48:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.257+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/3789523680 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c575a37db0 con 0x55c575a75830 2023-11-06T01:48:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.260+0000 7fbbe1686700 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c575a75830 0x55c575a77c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.260+0000 7fbbc4ff9700 10 client.26587 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.260+0000 7fbbe1686700 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c575a75830 0x55c575a77c50 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de223f2dbc62abc server_cookie=5a52a128e45bf419 in_seq=0 out_seq=0 2023-11-06T01:48:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.261+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/3789523680 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c575a37db0 con 0x55c575a75830 2023-11-06T01:48:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.261+0000 7fbbc4ff9700 10 client.26587 handle_command_reply: tid=0 2023-11-06T01:48:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.261+0000 7fbbefa7ab80 1 client.26587 shutdown 2023-11-06T01:48:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/3789523680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a7aee0 msgr2=0x55c575a71010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a7aee0 0x55c575a71010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400b010 tx=0x7fbbd4003c30 comp rx=0 tx=0).stop 2023-11-06T01:48:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/3789523680 shutdown_connections 2023-11-06T01:48:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c575a75830 0x55c575a77c50 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a7aee0 0x55c575a71010 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a79690 0x55c575a69a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/3789523680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a69fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/3789523680 >> 172.21.15.53:0/3789523680 conn(0x55c5759c8070 msgr2=0x55c575a633c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbc4ff9700 0 client.26587 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/3789523680 shutdown_connections 2023-11-06T01:48:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/3789523680 wait complete. 2023-11-06T01:48:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.262+0000 7fbbefa7ab80 20 client.26587 trim_cache size 0 max 16384 2023-11-06T01:48:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.286+0000 7fbbefa7ab80 1 Processor -- start 2023-11-06T01:48:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.286+0000 7fbbefa7ab80 1 -- start start 2023-11-06T01:48:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.286+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79690 0x55c575a6ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.286+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a7e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.286+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a7aee0 0x55c575a77840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.286+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c575a5e500 con 0x55c575a7aee0 2023-11-06T01:48:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.286+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c575a68440 con 0x55c575a79ff0 2023-11-06T01:48:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.286+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c575a65330 con 0x55c575a79690 2023-11-06T01:48:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe0e85700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a7e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe0e85700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a7e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55358/0 (socket says 172.21.15.53:55358) 2023-11-06T01:48:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe0e85700 1 -- 172.21.15.53:0/316250077 learned_addr learned my addr 172.21.15.53:0/316250077 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe1686700 1 --2- 172.21.15.53:0/316250077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79690 0x55c575a6ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe1e87700 1 --2- 172.21.15.53:0/316250077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a7aee0 0x55c575a77840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe0e85700 1 -- 172.21.15.53:0/316250077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79690 msgr2=0x55c575a6ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe0e85700 1 --2- 172.21.15.53:0/316250077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79690 0x55c575a6ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe0e85700 1 -- 172.21.15.53:0/316250077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a7aee0 msgr2=0x55c575a77840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe0e85700 1 --2- 172.21.15.53:0/316250077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a7aee0 0x55c575a77840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe0e85700 1 -- 172.21.15.53:0/316250077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c57580e1f0 con 0x55c575a79ff0 2023-11-06T01:48:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe1e87700 1 --2- 172.21.15.53:0/316250077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a7aee0 0x55c575a77840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:48:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbe0e85700 1 --2- 172.21.15.53:0/316250077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a7e0b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8015b10 tx=0x7fbbd8012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6c8c34591fb20c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.287+0000 7fbbc7fff700 1 -- 172.21.15.53:0/316250077 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd801a050 con 0x55c575a79ff0 2023-11-06T01:48:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.288+0000 7fbbc7fff700 1 -- 172.21.15.53:0/316250077 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbd801b040 con 0x55c575a79ff0 2023-11-06T01:48:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.288+0000 7fbbc7fff700 1 -- 172.21.15.53:0/316250077 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd801fc10 con 0x55c575a79ff0 2023-11-06T01:48:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/316250077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 msgr2=0x55c575a7e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/316250077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a7e0b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8015b10 tx=0x7fbbd8012000 comp rx=0 tx=0).stop 2023-11-06T01:48:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/316250077 shutdown_connections 2023-11-06T01:48:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/316250077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a7aee0 0x55c575a77840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/316250077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79690 0x55c575a6ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/316250077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a7e0b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/316250077 >> 172.21.15.53:0/316250077 conn(0x55c5759c8070 msgr2=0x55c575a64c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/316250077 shutdown_connections 2023-11-06T01:48:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/316250077 wait complete. 2023-11-06T01:48:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 -- start start 2023-11-06T01:48:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 Processor -- start 2023-11-06T01:48:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.289+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79690 0x55c575a6e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a6e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbefa7ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a7aee0 0x55c575b01dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c575b02300 con 0x55c575a7aee0 2023-11-06T01:48:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c575b02470 con 0x55c575a79ff0 2023-11-06T01:48:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbefa7ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c575b02630 con 0x55c575a79690 2023-11-06T01:48:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbefa7ab80 10 client.0 _ll_register_callbacks cb 0x7fbbd4011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbe0e85700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a6e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbe0e85700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a6e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55364/0 (socket says 172.21.15.53:55364) 2023-11-06T01:48:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbe0e85700 1 -- 172.21.15.53:0/1701144493 learned_addr learned my addr 172.21.15.53:0/1701144493 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbe1686700 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79690 0x55c575a6e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbe1e87700 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a7aee0 0x55c575b01dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbe0e85700 1 -- 172.21.15.53:0/1701144493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79690 msgr2=0x55c575a6e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbe0e85700 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79690 0x55c575a6e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbe0e85700 1 -- 172.21.15.53:0/1701144493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a7aee0 msgr2=0x55c575b01dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbe0e85700 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a7aee0 0x55c575b01dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbe0e85700 1 -- 172.21.15.53:0/1701144493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd8015850 con 0x55c575a79ff0 2023-11-06T01:48:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbe0e85700 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a6e6f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80094f0 tx=0x7fbbd80109e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73f106ad5a1875d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.290+0000 7fbbc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:48:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/1701144493 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8010e40 con 0x55c575a79ff0 2023-11-06T01:48:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/1701144493 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbbd8015670 con 0x55c575a79ff0 2023-11-06T01:48:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1701144493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbd400f040 con 0x55c575a79ff0 2023-11-06T01:48:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/1701144493 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbd801a070 con 0x55c575a79ff0 2023-11-06T01:48:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbefa7ab80 10 client.26534 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1701144493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c5757e6250 con 0x55c575a79ff0 2023-11-06T01:48:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/1701144493 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbd400f040 con 0x55c575a79ff0 2023-11-06T01:48:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/1701144493 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fbbd8013400 con 0x55c575a79ff0 2023-11-06T01:48:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbefa7ab80 10 client.26534 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbefa7ab80 10 client.26534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbefa7ab80 20 client.26534 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c575b04030 0x55c575b06450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbefa7ab80 4 client.26534 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:48:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.291+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1701144493 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c5759b2ce0 con 0x55c575b04030 2023-11-06T01:48:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.292+0000 7fbbe1686700 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c575b04030 0x55c575b06450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.292+0000 7fbbe1686700 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c575b04030 0x55c575b06450 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ba4ede6e36a57c8 server_cookie=24858f93e714a3e5 in_seq=0 out_seq=0 2023-11-06T01:48:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.292+0000 7fbbc4ff9700 10 client.26534 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.332+0000 7fbbc4ff9700 1 -- 172.21.15.53:0/1701144493 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c5759b2ce0 con 0x55c575b04030 2023-11-06T01:48:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.332+0000 7fbbc4ff9700 10 client.26534 handle_command_reply: tid=0 2023-11-06T01:48:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.332+0000 7fbbefa7ab80 1 client.26534 shutdown 2023-11-06T01:48:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1701144493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 msgr2=0x55c575a6e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a6e6f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80094f0 tx=0x7fbbd80109e0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1701144493 shutdown_connections 2023-11-06T01:48:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55c575b04030 0x55c575b06450 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c575a7aee0 0x55c575b01dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c575a79690 0x55c575a6e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbefa7ab80 1 --2- 172.21.15.53:0/1701144493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c575a79ff0 0x55c575a6e6f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbc4ff9700 0 client.26534 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1701144493 >> 172.21.15.53:0/1701144493 conn(0x55c5759c8070 msgr2=0x55c575a7e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1701144493 shutdown_connections 2023-11-06T01:48:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbefa7ab80 1 -- 172.21.15.53:0/1701144493 wait complete. 2023-11-06T01:48:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.333+0000 7fbbefa7ab80 20 client.26534 trim_cache size 0 max 16384 2023-11-06T01:48:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:48:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:48:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:48:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:48:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 -- 172.21.15.53:0/1374060107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbc80a05a0 msgr2=0x7fbbc80a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbc80a05a0 0x7fbbc80a2a60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009710 tx=0x7fbbd8006cc0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 -- 172.21.15.53:0/1374060107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbdc101f70 msgr2=0x7fbbdc19d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbdc101f70 0x7fbbdc19d410 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00b680 tx=0x7fbbcc004730 comp rx=0 tx=0).stop 2023-11-06T01:48:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 -- 172.21.15.53:0/1374060107 shutdown_connections 2023-11-06T01:48:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbbc80a05a0 0x7fbbc80a2a60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbdc19e7d0 0x7fbbdc19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbdc101f70 0x7fbbdc19d410 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 --2- 172.21.15.53:0/1374060107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbdc104940 0x7fbbdc19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 -- 172.21.15.53:0/1374060107 >> 172.21.15.53:0/1374060107 conn(0x7fbbdc0fb1e0 msgr2=0x7fbbdc102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 -- 172.21.15.53:0/1374060107 shutdown_connections 2023-11-06T01:48:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:34.334+0000 7fbbe38ea700 1 -- 172.21.15.53:0/1374060107 wait complete. 2023-11-06T01:48:35.343 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:35 smithi053 ceph-mon[99472]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 704 op/s 2023-11-06T01:48:35.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:35 smithi062 ceph-mon[103924]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 704 op/s 2023-11-06T01:48:35.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:35 smithi163 ceph-mon[104416]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 704 op/s 2023-11-06T01:48:37.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 860.0 s, 1042.4 tps, lat 98.346 ms stddev 112.702 2023-11-06T01:48:37.396 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:37 smithi062 ceph-mon[103924]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 MiB/s wr, 720 op/s 2023-11-06T01:48:37.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:37 smithi163 ceph-mon[104416]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 MiB/s wr, 720 op/s 2023-11-06T01:48:37.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:37 smithi053 ceph-mon[99472]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 MiB/s wr, 720 op/s 2023-11-06T01:48:39.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:39 smithi062 ceph-mon[103924]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 742 op/s 2023-11-06T01:48:39.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:39 smithi163 ceph-mon[104416]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 742 op/s 2023-11-06T01:48:39.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:39 smithi053 ceph-mon[99472]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 742 op/s 2023-11-06T01:48:41.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:41 smithi062 ceph-mon[103924]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 835 op/s 2023-11-06T01:48:41.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:41 smithi163 ceph-mon[104416]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 835 op/s 2023-11-06T01:48:41.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:41 smithi053 ceph-mon[99472]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 835 op/s 2023-11-06T01:48:42.130 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:48:42.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 865.0 s, 1457.4 tps, lat 68.355 ms stddev 75.438 2023-11-06T01:48:42.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.308+0000 7f481218b700 1 -- 172.21.15.53:0/1910250473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c108f40 msgr2=0x7f480c10f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:42.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.308+0000 7f481218b700 1 --2- 172.21.15.53:0/1910250473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c108f40 0x7f480c10f7f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f480400b0d0 tx=0x7f480400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.308+0000 7f481218b700 1 -- 172.21.15.53:0/1910250473 shutdown_connections 2023-11-06T01:48:42.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.309+0000 7f481218b700 1 --2- 172.21.15.53:0/1910250473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480c1073b0 0x7f480c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.309+0000 7f481218b700 1 --2- 172.21.15.53:0/1910250473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c108f40 0x7f480c10f7f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.309+0000 7f481218b700 1 --2- 172.21.15.53:0/1910250473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480c1085a0 0x7f480c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.309+0000 7f481218b700 1 -- 172.21.15.53:0/1910250473 >> 172.21.15.53:0/1910250473 conn(0x7f480c076350 msgr2=0x7f480c078790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:42.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.309+0000 7f481218b700 1 -- 172.21.15.53:0/1910250473 shutdown_connections 2023-11-06T01:48:42.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.309+0000 7f481218b700 1 -- 172.21.15.53:0/1910250473 wait complete. 2023-11-06T01:48:42.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.309+0000 7f481218b700 1 Processor -- start 2023-11-06T01:48:42.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.310+0000 7f481218b700 1 -- start start 2023-11-06T01:48:42.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.310+0000 7f481218b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480c1073b0 0x7f480c19d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.310+0000 7f481218b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480c1085a0 0x7f480c19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.310+0000 7f481218b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c108f40 0x7f480c1a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.310+0000 7f481218b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c1a4d40 con 0x7f480c108f40 2023-11-06T01:48:42.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.310+0000 7f481218b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c0795e0 con 0x7f480c1073b0 2023-11-06T01:48:42.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.310+0000 7f481218b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c079770 con 0x7f480c1085a0 2023-11-06T01:48:42.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.310+0000 7f480affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480c1085a0 0x7f480c19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.310+0000 7f480bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c108f40 0x7f480c1a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.310+0000 7f480affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480c1085a0 0x7f480c19d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58178/0 (socket says 172.21.15.53:58178) 2023-11-06T01:48:42.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.310+0000 7f480affd700 1 -- 172.21.15.53:0/1732500111 learned_addr learned my addr 172.21.15.53:0/1732500111 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:42.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.311+0000 7f480affd700 1 -- 172.21.15.53:0/1732500111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480c1073b0 msgr2=0x7f480c19d1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-06T01:48:42.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.311+0000 7f480affd700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480c1073b0 0x7f480c19d1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.311+0000 7f480affd700 1 -- 172.21.15.53:0/1732500111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c108f40 msgr2=0x7f480c1a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:42.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.311+0000 7f480affd700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c108f40 0x7f480c1a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.311+0000 7f480affd700 1 -- 172.21.15.53:0/1732500111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc009520 con 0x7f480c1085a0 2023-11-06T01:48:42.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.311+0000 7f480affd700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480c1085a0 0x7f480c19d700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f480000e9a0 tx=0x7f480000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:42.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.312+0000 7f4808ff9700 1 -- 172.21.15.53:0/1732500111 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480000cbd0 con 0x7f480c1085a0 2023-11-06T01:48:42.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.312+0000 7f4808ff9700 1 -- 172.21.15.53:0/1732500111 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f480000cd30 con 0x7f480c1085a0 2023-11-06T01:48:42.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.312+0000 7f4808ff9700 1 -- 172.21.15.53:0/1732500111 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800018680 con 0x7f480c1085a0 2023-11-06T01:48:42.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.312+0000 7f481218b700 1 -- 172.21.15.53:0/1732500111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4804009d00 con 0x7f480c1085a0 2023-11-06T01:48:42.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.312+0000 7f481218b700 1 -- 172.21.15.53:0/1732500111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480c079d80 con 0x7f480c1085a0 2023-11-06T01:48:42.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.314+0000 7f4808ff9700 1 -- 172.21.15.53:0/1732500111 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4800010430 con 0x7f480c1085a0 2023-11-06T01:48:42.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.315+0000 7f4808ff9700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47f40a05a0 0x7f47f40a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.315+0000 7f4808ff9700 1 -- 172.21.15.53:0/1732500111 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4800014070 con 0x7f480c1085a0 2023-11-06T01:48:42.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.315+0000 7f480b7fe700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47f40a05a0 0x7f47f40a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.315+0000 7f480b7fe700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47f40a05a0 0x7f47f40a2a60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f480c108400 tx=0x7f47fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:42.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f481e31bb80 1 Processor -- start 2023-11-06T01:48:42.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f481e31bb80 1 -- start start 2023-11-06T01:48:42.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3b0f0 0x5614e7d3b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3ba70 0x5614e7d39e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3a3f0 0x5614e7d3a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f481e31bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7c91a10 con 0x5614e7d3b0f0 2023-11-06T01:48:42.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f481e31bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7c8f3d0 con 0x5614e7d3a3f0 2023-11-06T01:48:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f481e31bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7c8acc0 con 0x5614e7d3ba70 2023-11-06T01:48:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f480b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3b0f0 0x5614e7d3b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f480b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3b0f0 0x5614e7d3b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43946/0 (socket says 172.21.15.53:43946) 2023-11-06T01:48:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f480b7fe700 1 -- 172.21.15.53:0/273042132 learned_addr learned my addr 172.21.15.53:0/273042132 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f480affd700 1 --2- 172.21.15.53:0/273042132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3ba70 0x5614e7d39e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f480bfff700 1 --2- 172.21.15.53:0/273042132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3a3f0 0x5614e7d3a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f480b7fe700 1 -- 172.21.15.53:0/273042132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3ba70 msgr2=0x5614e7d39e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:42.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f480b7fe700 1 --2- 172.21.15.53:0/273042132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3ba70 0x5614e7d39e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f480b7fe700 1 -- 172.21.15.53:0/273042132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3a3f0 msgr2=0x5614e7d3a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:42.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f480b7fe700 1 --2- 172.21.15.53:0/273042132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3a3f0 0x5614e7d3a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.316+0000 7f480b7fe700 1 -- 172.21.15.53:0/273042132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e7ab8de0 con 0x5614e7d3b0f0 2023-11-06T01:48:42.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.317+0000 7f480b7fe700 1 --2- 172.21.15.53:0/273042132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3b0f0 0x5614e7d3b530 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00ece0 tx=0x7f47fc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a28864aec887f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:42.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.318+0000 7f47f1ffb700 1 -- 172.21.15.53:0/273042132 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc009950 con 0x5614e7d3b0f0 2023-11-06T01:48:42.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.318+0000 7f47f1ffb700 1 -- 172.21.15.53:0/273042132 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47fc010d80 con 0x5614e7d3b0f0 2023-11-06T01:48:42.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.318+0000 7f47f1ffb700 1 -- 172.21.15.53:0/273042132 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc003d60 con 0x5614e7d3b0f0 2023-11-06T01:48:42.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.319+0000 7f481e31bb80 1 -- 172.21.15.53:0/273042132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3b0f0 msgr2=0x5614e7d3b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:42.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.319+0000 7f481e31bb80 1 --2- 172.21.15.53:0/273042132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3b0f0 0x5614e7d3b530 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00ece0 tx=0x7f47fc011000 comp rx=0 tx=0).stop 2023-11-06T01:48:42.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.319+0000 7f481e31bb80 1 -- 172.21.15.53:0/273042132 shutdown_connections 2023-11-06T01:48:42.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.319+0000 7f481e31bb80 1 --2- 172.21.15.53:0/273042132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3a3f0 0x5614e7d3a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.319+0000 7f481e31bb80 1 --2- 172.21.15.53:0/273042132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3b0f0 0x5614e7d3b530 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.319+0000 7f481e31bb80 1 --2- 172.21.15.53:0/273042132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3ba70 0x5614e7d39e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.319+0000 7f481e31bb80 1 -- 172.21.15.53:0/273042132 >> 172.21.15.53:0/273042132 conn(0x5614e7c990d0 msgr2=0x5614e7d34940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:42.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.319+0000 7f481e31bb80 1 -- 172.21.15.53:0/273042132 shutdown_connections 2023-11-06T01:48:42.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.319+0000 7f481e31bb80 1 -- 172.21.15.53:0/273042132 wait complete. 2023-11-06T01:48:42.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.319+0000 7f481e31bb80 1 -- start start 2023-11-06T01:48:42.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.319+0000 7f481e31bb80 1 Processor -- start 2023-11-06T01:48:42.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3a3f0 0x5614e7d43a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3b0f0 0x5614e7d43f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3ba70 0x5614e7d4afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f481e31bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7c27510 con 0x5614e7d3a3f0 2023-11-06T01:48:42.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f481e31bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7c271b0 con 0x5614e7d3ba70 2023-11-06T01:48:42.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f481e31bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7b0e920 con 0x5614e7d3b0f0 2023-11-06T01:48:42.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f481e31bb80 10 client.0 _ll_register_callbacks cb 0x5614e7b74a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:42.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3a3f0 0x5614e7d43a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3ba70 0x5614e7d4afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3a3f0 0x5614e7d43a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43952/0 (socket says 172.21.15.53:43952) 2023-11-06T01:48:42.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480b7fe700 1 -- 172.21.15.53:0/1216819157 learned_addr learned my addr 172.21.15.53:0/1216819157 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:42.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480affd700 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3b0f0 0x5614e7d43f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480b7fe700 1 -- 172.21.15.53:0/1216819157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3b0f0 msgr2=0x5614e7d43f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:42.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480b7fe700 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3b0f0 0x5614e7d43f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480b7fe700 1 -- 172.21.15.53:0/1216819157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3ba70 msgr2=0x5614e7d4afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:42.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480b7fe700 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3ba70 0x5614e7d4afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480b7fe700 1 -- 172.21.15.53:0/1216819157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc009e70 con 0x5614e7d3a3f0 2023-11-06T01:48:42.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480bfff700 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3ba70 0x5614e7d4afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:48:42.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f480b7fe700 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3a3f0 0x5614e7d43a50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00f010 tx=0x7f47fc010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3251c7de651cd6f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:42.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.320+0000 7f47eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:48:42.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f47eaffd700 1 -- 172.21.15.53:0/1216819157 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc033070 con 0x5614e7d3a3f0 2023-11-06T01:48:42.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f47eaffd700 1 -- 172.21.15.53:0/1216819157 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f47fc003cf0 con 0x5614e7d3a3f0 2023-11-06T01:48:42.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f481e31bb80 1 -- 172.21.15.53:0/1216819157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614e7cb5d10 con 0x5614e7d3a3f0 2023-11-06T01:48:42.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f47eaffd700 1 -- 172.21.15.53:0/1216819157 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47fc003eb0 con 0x5614e7d3a3f0 2023-11-06T01:48:42.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f481e31bb80 10 client.17037 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:42.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f481e31bb80 1 -- 172.21.15.53:0/1216819157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5614e7aa1c60 con 0x5614e7d3a3f0 2023-11-06T01:48:42.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f47eaffd700 1 -- 172.21.15.53:0/1216819157 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614e7cb5d10 con 0x5614e7d3a3f0 2023-11-06T01:48:42.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f47eaffd700 1 -- 172.21.15.53:0/1216819157 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f47fc036030 con 0x5614e7d3a3f0 2023-11-06T01:48:42.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f481e31bb80 10 client.17037 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:42.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f481e31bb80 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:42.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f481e31bb80 20 client.17037 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:42.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f481e31bb80 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5614e7d4f5f0 0x5614e7d51a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f481e31bb80 4 client.17037 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:48:42.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f481e31bb80 1 -- 172.21.15.53:0/1216819157 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614e7d08e10 con 0x5614e7d4f5f0 2023-11-06T01:48:42.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.321+0000 7f480affd700 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5614e7d4f5f0 0x5614e7d51a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.322+0000 7f47eaffd700 10 client.17037 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:42.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.322+0000 7f480affd700 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5614e7d4f5f0 0x5614e7d51a10 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b996587597232e server_cookie=f0f3dcf8f0951445 in_seq=0 out_seq=0 2023-11-06T01:48:42.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.322+0000 7f47eaffd700 1 -- 172.21.15.53:0/1216819157 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5614e7d08e10 con 0x5614e7d4f5f0 2023-11-06T01:48:42.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.322+0000 7f47eaffd700 10 client.17037 handle_command_reply: tid=0 2023-11-06T01:48:42.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.322+0000 7f481e31bb80 1 client.17037 shutdown 2023-11-06T01:48:42.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.322+0000 7f481e31bb80 1 -- 172.21.15.53:0/1216819157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3a3f0 msgr2=0x5614e7d43a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:42.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.322+0000 7f481e31bb80 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3a3f0 0x5614e7d43a50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00f010 tx=0x7f47fc010cf0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.323+0000 7f481e31bb80 1 -- 172.21.15.53:0/1216819157 shutdown_connections 2023-11-06T01:48:42.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.323+0000 7f481e31bb80 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3ba70 0x5614e7d4afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.323+0000 7f481e31bb80 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5614e7d4f5f0 0x5614e7d51a10 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.323+0000 7f481e31bb80 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3a3f0 0x5614e7d43a50 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.323+0000 7f47eaffd700 0 client.17037 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:42.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.323+0000 7f481e31bb80 1 --2- 172.21.15.53:0/1216819157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3b0f0 0x5614e7d43f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.323+0000 7f481e31bb80 1 -- 172.21.15.53:0/1216819157 >> 172.21.15.53:0/1216819157 conn(0x5614e7c990d0 msgr2=0x5614e7d340b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:42.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.323+0000 7f481e31bb80 1 -- 172.21.15.53:0/1216819157 shutdown_connections 2023-11-06T01:48:42.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.323+0000 7f481e31bb80 1 -- 172.21.15.53:0/1216819157 wait complete. 2023-11-06T01:48:42.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.323+0000 7f481e31bb80 20 client.17037 trim_cache size 0 max 16384 2023-11-06T01:48:42.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f481e31bb80 1 Processor -- start 2023-11-06T01:48:42.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f481e31bb80 1 -- start start 2023-11-06T01:48:42.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3a3f0 0x5614e7d39e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 0x5614e7d48f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7d43800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f481e31bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7d35860 con 0x5614e7d3ba70 2023-11-06T01:48:42.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f481e31bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7d08e10 con 0x5614e7d3b0f0 2023-11-06T01:48:42.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f481e31bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7d2cc50 con 0x5614e7d3a3f0 2023-11-06T01:48:42.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7d43800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 0x5614e7d48f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7d43800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43964/0 (socket says 172.21.15.53:43964) 2023-11-06T01:48:42.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 0x5614e7d48f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55400/0 (socket says 172.21.15.53:55400) 2023-11-06T01:48:42.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480bfff700 1 -- 172.21.15.53:0/1860612431 learned_addr learned my addr 172.21.15.53:0/1860612431 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:42.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480b7fe700 1 --2- 172.21.15.53:0/1860612431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3a3f0 0x5614e7d39e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480bfff700 1 -- 172.21.15.53:0/1860612431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3a3f0 msgr2=0x5614e7d39e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:42.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480bfff700 1 --2- 172.21.15.53:0/1860612431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3a3f0 0x5614e7d39e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480bfff700 1 -- 172.21.15.53:0/1860612431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 msgr2=0x5614e7d48f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:42.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480bfff700 1 --2- 172.21.15.53:0/1860612431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 0x5614e7d48f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480bfff700 1 -- 172.21.15.53:0/1860612431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e7adf250 con 0x5614e7d3ba70 2023-11-06T01:48:42.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.344+0000 7f480affd700 1 --2- 172.21.15.53:0/1860612431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 0x5614e7d48f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:48:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.345+0000 7f480bfff700 1 --2- 172.21.15.53:0/1860612431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7d43800 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f480400b0d0 tx=0x7f4804008c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536ecfcd525f9615 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.345+0000 7f47f27fc700 1 -- 172.21.15.53:0/1860612431 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804021070 con 0x5614e7d3ba70 2023-11-06T01:48:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.345+0000 7f47f27fc700 1 -- 172.21.15.53:0/1860612431 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4804008dc0 con 0x5614e7d3ba70 2023-11-06T01:48:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.345+0000 7f47f27fc700 1 -- 172.21.15.53:0/1860612431 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480400e040 con 0x5614e7d3ba70 2023-11-06T01:48:42.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.345+0000 7f481e31bb80 1 -- 172.21.15.53:0/1860612431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 msgr2=0x5614e7d43800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:42.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.346+0000 7f481e31bb80 1 --2- 172.21.15.53:0/1860612431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7d43800 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f480400b0d0 tx=0x7f4804008c10 comp rx=0 tx=0).stop 2023-11-06T01:48:42.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.346+0000 7f481e31bb80 1 -- 172.21.15.53:0/1860612431 shutdown_connections 2023-11-06T01:48:42.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.346+0000 7f481e31bb80 1 --2- 172.21.15.53:0/1860612431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7d43800 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.346+0000 7f481e31bb80 1 --2- 172.21.15.53:0/1860612431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3a3f0 0x5614e7d39e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.346+0000 7f481e31bb80 1 --2- 172.21.15.53:0/1860612431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 0x5614e7d48f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.346+0000 7f481e31bb80 1 -- 172.21.15.53:0/1860612431 >> 172.21.15.53:0/1860612431 conn(0x5614e7c990d0 msgr2=0x5614e7d41890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:42.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.346+0000 7f481e31bb80 1 -- 172.21.15.53:0/1860612431 shutdown_connections 2023-11-06T01:48:42.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.346+0000 7f481e31bb80 1 -- 172.21.15.53:0/1860612431 wait complete. 2023-11-06T01:48:42.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.346+0000 7f481e31bb80 1 -- start start 2023-11-06T01:48:42.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.346+0000 7f481e31bb80 1 Processor -- start 2023-11-06T01:48:42.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3a3f0 0x5614e7d99370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 0x5614e7d998b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f481e31bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7da08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f481e31bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7da0de0 con 0x5614e7d3ba70 2023-11-06T01:48:42.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f481e31bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7da0f20 con 0x5614e7d3b0f0 2023-11-06T01:48:42.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f481e31bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e7da10e0 con 0x5614e7d3a3f0 2023-11-06T01:48:42.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f481e31bb80 10 client.0 _ll_register_callbacks cb 0x7f47fc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:42.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f480bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7da08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f480affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 0x5614e7d998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f480bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7da08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43970/0 (socket says 172.21.15.53:43970) 2023-11-06T01:48:42.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f480bfff700 1 -- 172.21.15.53:0/62791706 learned_addr learned my addr 172.21.15.53:0/62791706 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:42.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f480b7fe700 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3a3f0 0x5614e7d99370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f480affd700 1 -- 172.21.15.53:0/62791706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3a3f0 msgr2=0x5614e7d99370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:42.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f480affd700 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3a3f0 0x5614e7d99370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f480affd700 1 -- 172.21.15.53:0/62791706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 msgr2=0x5614e7da08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:42.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f480affd700 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7da08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.347+0000 7f480affd700 1 -- 172.21.15.53:0/62791706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480c079d80 con 0x5614e7d3b0f0 2023-11-06T01:48:42.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.348+0000 7f480bfff700 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7da08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:48:42.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.348+0000 7f480affd700 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 0x5614e7d998b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f48000c96a0 tx=0x7f4800084040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69336273b44266 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:42.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.348+0000 7f47eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:48:42.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.348+0000 7f47eaffd700 1 -- 172.21.15.53:0/62791706 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000977d0 con 0x5614e7d3b0f0 2023-11-06T01:48:42.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.348+0000 7f47eaffd700 1 -- 172.21.15.53:0/62791706 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4800099690 con 0x5614e7d3b0f0 2023-11-06T01:48:42.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.348+0000 7f481e31bb80 1 -- 172.21.15.53:0/62791706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47fc010d80 con 0x5614e7d3b0f0 2023-11-06T01:48:42.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f47eaffd700 1 -- 172.21.15.53:0/62791706 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4800099850 con 0x5614e7d3b0f0 2023-11-06T01:48:42.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f481e31bb80 10 client.26549 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:42.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f481e31bb80 1 -- 172.21.15.53:0/62791706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f480c110140 con 0x5614e7d3b0f0 2023-11-06T01:48:42.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f47eaffd700 1 -- 172.21.15.53:0/62791706 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47fc010d80 con 0x5614e7d3b0f0 2023-11-06T01:48:42.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f47eaffd700 1 -- 172.21.15.53:0/62791706 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f48000a1010 con 0x5614e7d3b0f0 2023-11-06T01:48:42.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f481e31bb80 10 client.26549 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:42.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f481e31bb80 10 client.26549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f481e31bb80 20 client.26549 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f481e31bb80 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5614e7da2a70 0x5614e7da4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f481e31bb80 4 client.26549 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:48:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f481e31bb80 1 -- 172.21.15.53:0/62791706 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614e7c83d40 con 0x5614e7da2a70 2023-11-06T01:48:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.349+0000 7f480b7fe700 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5614e7da2a70 0x5614e7da4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f47eaffd700 10 client.26549 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f480b7fe700 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5614e7da2a70 0x5614e7da4e90 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89097ebff89d3b9c server_cookie=b72f797b3ee6194b in_seq=0 out_seq=0 2023-11-06T01:48:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f47eaffd700 1 -- 172.21.15.53:0/62791706 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5614e7c83d40 con 0x5614e7da2a70 2023-11-06T01:48:42.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f47eaffd700 10 client.26549 handle_command_reply: tid=0 2023-11-06T01:48:42.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f481e31bb80 1 client.26549 shutdown 2023-11-06T01:48:42.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f481e31bb80 1 -- 172.21.15.53:0/62791706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 msgr2=0x5614e7d998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:42.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f481e31bb80 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 0x5614e7d998b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f48000c96a0 tx=0x7f4800084040 comp rx=0 tx=0).stop 2023-11-06T01:48:42.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f481e31bb80 1 -- 172.21.15.53:0/62791706 shutdown_connections 2023-11-06T01:48:42.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f481e31bb80 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x5614e7da2a70 0x5614e7da4e90 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f481e31bb80 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614e7d3ba70 0x5614e7da08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f481e31bb80 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5614e7d3a3f0 0x5614e7d99370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f481e31bb80 1 --2- 172.21.15.53:0/62791706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614e7d3b0f0 0x5614e7d998b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f47eaffd700 0 client.26549 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:42.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f481e31bb80 1 -- 172.21.15.53:0/62791706 >> 172.21.15.53:0/62791706 conn(0x5614e7c990d0 msgr2=0x5614e7d43800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:42.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f481e31bb80 1 -- 172.21.15.53:0/62791706 shutdown_connections 2023-11-06T01:48:42.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.350+0000 7f481e31bb80 1 -- 172.21.15.53:0/62791706 wait complete. 2023-11-06T01:48:42.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481e31bb80 20 client.26549 trim_cache size 0 max 16384 2023-11-06T01:48:42.372 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:48:42.372 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:48:42.372 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:48:42.373 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:48:42.373 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:48:42.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:48:42.373 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:48:42.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:48:42.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:48:42.374 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:48:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 -- 172.21.15.53:0/1732500111 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47f40a05a0 msgr2=0x7f47f40a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47f40a05a0 0x7f47f40a2a60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f480c108400 tx=0x7f47fc009260 comp rx=0 tx=0).stop 2023-11-06T01:48:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 -- 172.21.15.53:0/1732500111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480c1085a0 msgr2=0x7f480c19d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:42.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480c1085a0 0x7f480c19d700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f480000e9a0 tx=0x7f480000ecb0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 -- 172.21.15.53:0/1732500111 shutdown_connections 2023-11-06T01:48:42.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f47f40a05a0 0x7f47f40a2a60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480c1073b0 0x7f480c19d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c108f40 0x7f480c1a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 --2- 172.21.15.53:0/1732500111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480c1085a0 0x7f480c19d700 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:42.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 -- 172.21.15.53:0/1732500111 >> 172.21.15.53:0/1732500111 conn(0x7f480c076350 msgr2=0x7f480c10b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:42.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 -- 172.21.15.53:0/1732500111 shutdown_connections 2023-11-06T01:48:42.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:48:42.351+0000 7f481218b700 1 -- 172.21.15.53:0/1732500111 wait complete. 2023-11-06T01:48:42.379 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:48:42.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:42 smithi053 ceph-mon[99472]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 816 op/s 2023-11-06T01:48:42.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:42 smithi062 ceph-mon[103924]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 816 op/s 2023-11-06T01:48:42.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:42 smithi163 ceph-mon[104416]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 816 op/s 2023-11-06T01:48:45.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:45 smithi053 ceph-mon[99472]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 895 op/s 2023-11-06T01:48:45.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:45 smithi062 ceph-mon[103924]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 895 op/s 2023-11-06T01:48:45.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:45 smithi163 ceph-mon[104416]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 895 op/s 2023-11-06T01:48:47.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 870.0 s, 1326.6 tps, lat 75.082 ms stddev 82.197 2023-11-06T01:48:47.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:47 smithi062 ceph-mon[103924]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 832 op/s 2023-11-06T01:48:47.495 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:47 smithi163 ceph-mon[104416]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 832 op/s 2023-11-06T01:48:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.519+0000 7f7b57d60700 1 -- 172.21.15.53:0/2772081554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b500ff600 msgr2=0x7f7b50101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.519+0000 7f7b57d60700 1 --2- 172.21.15.53:0/2772081554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b500ff600 0x7f7b50101a30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800b0a0 tx=0x7f7b4800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.523+0000 7f7b57d60700 1 -- 172.21.15.53:0/2772081554 shutdown_connections 2023-11-06T01:48:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.523+0000 7f7b57d60700 1 --2- 172.21.15.53:0/2772081554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b50104940 0x7f7b50104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.523+0000 7f7b57d60700 1 --2- 172.21.15.53:0/2772081554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b500ff600 0x7f7b50101a30 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.523+0000 7f7b57d60700 1 --2- 172.21.15.53:0/2772081554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b50101f70 0x7f7b50104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.523+0000 7f7b57d60700 1 -- 172.21.15.53:0/2772081554 >> 172.21.15.53:0/2772081554 conn(0x7f7b500fb1e0 msgr2=0x7f7b500fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.531+0000 7f7b57d60700 1 -- 172.21.15.53:0/2772081554 shutdown_connections 2023-11-06T01:48:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.536+0000 7f7b57d60700 1 -- 172.21.15.53:0/2772081554 wait complete. 2023-11-06T01:48:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.537+0000 7f7b57d60700 1 Processor -- start 2023-11-06T01:48:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.537+0000 7f7b57d60700 1 -- start start 2023-11-06T01:48:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.537+0000 7f7b57d60700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b500ff600 0x7f7b501a7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.537+0000 7f7b57d60700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b50101f70 0x7f7b501a7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.537+0000 7f7b57d60700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b50104940 0x7f7b500717c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.537+0000 7f7b57d60700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50071d00 con 0x7f7b50104940 2023-11-06T01:48:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.537+0000 7f7b57d60700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50071e40 con 0x7f7b500ff600 2023-11-06T01:48:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.537+0000 7f7b57d60700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50072000 con 0x7f7b50101f70 2023-11-06T01:48:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.539+0000 7f7b55afc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b500ff600 0x7f7b501a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.539+0000 7f7b55afc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b500ff600 0x7f7b501a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55418/0 (socket says 172.21.15.53:55418) 2023-11-06T01:48:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.539+0000 7f7b55afc700 1 -- 172.21.15.53:0/965121800 learned_addr learned my addr 172.21.15.53:0/965121800 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.540+0000 7f7b55afc700 1 -- 172.21.15.53:0/965121800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b50101f70 msgr2=0x7f7b501a7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.540+0000 7f7b55afc700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b50101f70 0x7f7b501a7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.540+0000 7f7b55afc700 1 -- 172.21.15.53:0/965121800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b50104940 msgr2=0x7f7b500717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.540+0000 7f7b55afc700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b50104940 0x7f7b500717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.540+0000 7f7b55afc700 1 -- 172.21.15.53:0/965121800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b48009d00 con 0x7f7b500ff600 2023-11-06T01:48:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.540+0000 7f7b55afc700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b500ff600 0x7f7b501a7110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7b48007b60 tx=0x7f7b48005b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.540+0000 7f7b46ffd700 1 -- 172.21.15.53:0/965121800 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4800e040 con 0x7f7b500ff600 2023-11-06T01:48:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.540+0000 7f7b57d60700 1 -- 172.21.15.53:0/965121800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b50072280 con 0x7f7b500ff600 2023-11-06T01:48:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.540+0000 7f7b46ffd700 1 -- 172.21.15.53:0/965121800 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b4801d480 con 0x7f7b500ff600 2023-11-06T01:48:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.540+0000 7f7b46ffd700 1 -- 172.21.15.53:0/965121800 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4801b690 con 0x7f7b500ff600 2023-11-06T01:48:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.541+0000 7f7b57d60700 1 -- 172.21.15.53:0/965121800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b50105e00 con 0x7f7b500ff600 2023-11-06T01:48:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.544+0000 7f7b46ffd700 1 -- 172.21.15.53:0/965121800 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f7b4801d5f0 con 0x7f7b500ff600 2023-11-06T01:48:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.545+0000 7f7b46ffd700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b3c0a05a0 0x7f7b3c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.545+0000 7f7b46ffd700 1 -- 172.21.15.53:0/965121800 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7b480cd8d0 con 0x7f7b500ff600 2023-11-06T01:48:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.545+0000 7f7b63ef0b80 1 Processor -- start 2023-11-06T01:48:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b63ef0b80 1 -- start start 2023-11-06T01:48:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f096baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f096c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096ced0 0x55d7f096d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f08c39b0 con 0x55d7f096bfe0 2023-11-06T01:48:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f08c1370 con 0x55d7f096b680 2023-11-06T01:48:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f08bcc60 con 0x55d7f096ced0 2023-11-06T01:48:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b55afc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f096baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b55afc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f096baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55426/0 (socket says 172.21.15.53:55426) 2023-11-06T01:48:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b55afc700 1 -- 172.21.15.53:0/2816193978 learned_addr learned my addr 172.21.15.53:0/2816193978 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b552fb700 1 --2- 172.21.15.53:0/2816193978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f096c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.546+0000 7f7b562fd700 1 --2- 172.21.15.53:0/2816193978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096ced0 0x55d7f096d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.547+0000 7f7b55afc700 1 -- 172.21.15.53:0/2816193978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096ced0 msgr2=0x55d7f096d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.547+0000 7f7b55afc700 1 --2- 172.21.15.53:0/2816193978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096ced0 0x55d7f096d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.547+0000 7f7b55afc700 1 -- 172.21.15.53:0/2816193978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 msgr2=0x55d7f096c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.547+0000 7f7b55afc700 1 --2- 172.21.15.53:0/2816193978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f096c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.547+0000 7f7b55afc700 1 -- 172.21.15.53:0/2816193978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7f06ead80 con 0x55d7f096b680 2023-11-06T01:48:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.547+0000 7f7b55afc700 1 --2- 172.21.15.53:0/2816193978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f096baa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7b480a2bf0 tx=0x7f7b4800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83b4d6161bd57700 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.547+0000 7f7b3bfff700 1 -- 172.21.15.53:0/2816193978 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4809ef70 con 0x55d7f096b680 2023-11-06T01:48:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.547+0000 7f7b3bfff700 1 -- 172.21.15.53:0/2816193978 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b480a3120 con 0x55d7f096b680 2023-11-06T01:48:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.547+0000 7f7b3bfff700 1 -- 172.21.15.53:0/2816193978 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4809b600 con 0x55d7f096b680 2023-11-06T01:48:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.548+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/2816193978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 msgr2=0x55d7f096baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.548+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/2816193978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f096baa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7b480a2bf0 tx=0x7f7b4800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.548+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/2816193978 shutdown_connections 2023-11-06T01:48:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.548+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/2816193978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f096baa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.548+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/2816193978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096ced0 0x55d7f096d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.548+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/2816193978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f096c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.548+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/2816193978 >> 172.21.15.53:0/2816193978 conn(0x55d7f08cb070 msgr2=0x55d7f0968040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.548+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/2816193978 shutdown_connections 2023-11-06T01:48:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.548+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/2816193978 wait complete. 2023-11-06T01:48:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.549+0000 7f7b63ef0b80 1 -- start start 2023-11-06T01:48:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.549+0000 7f7b63ef0b80 1 Processor -- start 2023-11-06T01:48:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.549+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096b680 0x55d7f0975290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.549+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096bfe0 0x55d7f09757d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.549+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 0x55d7f097c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.549+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f08594b0 con 0x55d7f096b680 2023-11-06T01:48:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.549+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f0859150 con 0x55d7f096ced0 2023-11-06T01:48:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.549+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f07408c0 con 0x55d7f096bfe0 2023-11-06T01:48:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.549+0000 7f7b63ef0b80 10 client.0 _ll_register_callbacks cb 0x55d7f07a6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b552fb700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b3c0a05a0 0x7f7b3c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b562fd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 0x55d7f097c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b562fd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 0x55d7f097c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55434/0 (socket says 172.21.15.53:55434) 2023-11-06T01:48:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b562fd700 1 -- 172.21.15.53:0/3959071553 learned_addr learned my addr 172.21.15.53:0/3959071553 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b55afc700 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096b680 0x55d7f0975290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b552fb700 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096bfe0 0x55d7f09757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b562fd700 1 -- 172.21.15.53:0/3959071553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096bfe0 msgr2=0x55d7f09757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b562fd700 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096bfe0 0x55d7f09757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b562fd700 1 -- 172.21.15.53:0/3959071553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096b680 msgr2=0x55d7f0975290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b562fd700 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096b680 0x55d7f0975290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b562fd700 1 -- 172.21.15.53:0/3959071553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b50105e00 con 0x55d7f096ced0 2023-11-06T01:48:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b562fd700 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 0x55d7f097c810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00e480 tx=0x7f7b4c00e840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=467b91811b2e612f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.550+0000 7f7b397fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:48:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.551+0000 7f7b397fa700 1 -- 172.21.15.53:0/3959071553 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c00ee50 con 0x55d7f096ced0 2023-11-06T01:48:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.551+0000 7f7b397fa700 1 -- 172.21.15.53:0/3959071553 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7b4c013680 con 0x55d7f096ced0 2023-11-06T01:48:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.551+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/3959071553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7f08e7cb0 con 0x55d7f096ced0 2023-11-06T01:48:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.551+0000 7f7b397fa700 1 -- 172.21.15.53:0/3959071553 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b4c013840 con 0x55d7f096ced0 2023-11-06T01:48:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.554+0000 7f7b63ef0b80 10 client.26561 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.554+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/3959071553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d7f06d3c00 con 0x55d7f096ced0 2023-11-06T01:48:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.555+0000 7f7b397fa700 1 -- 172.21.15.53:0/3959071553 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7f08e7cb0 con 0x55d7f096ced0 2023-11-06T01:48:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.555+0000 7f7b397fa700 1 -- 172.21.15.53:0/3959071553 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7b4c011070 con 0x55d7f096ced0 2023-11-06T01:48:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.555+0000 7f7b63ef0b80 10 client.26561 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.555+0000 7f7b63ef0b80 10 client.26561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.555+0000 7f7b63ef0b80 20 client.26561 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.555+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7f09810a0 0x55d7f09834c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.555+0000 7f7b63ef0b80 4 client.26561 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:48:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.555+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/3959071553 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7f093adb0 con 0x55d7f09810a0 2023-11-06T01:48:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.556+0000 7f7b552fb700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b3c0a05a0 0x7f7b3c0a2a60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7b40005d90 tx=0x7f7b4000a5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.556+0000 7f7b55afc700 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7f09810a0 0x55d7f09834c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.557+0000 7f7b55afc700 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7f09810a0 0x55d7f09834c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e1bc92e2c1288b server_cookie=616e9f6f45918f45 in_seq=0 out_seq=0 2023-11-06T01:48:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.557+0000 7f7b397fa700 10 client.26561 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b397fa700 1 -- 172.21.15.53:0/3959071553 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d7f093adb0 con 0x55d7f09810a0 2023-11-06T01:48:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b397fa700 10 client.26561 handle_command_reply: tid=0 2023-11-06T01:48:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 1 client.26561 shutdown 2023-11-06T01:48:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/3959071553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 msgr2=0x55d7f097c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 0x55d7f097c810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00e480 tx=0x7f7b4c00e840 comp rx=0 tx=0).stop 2023-11-06T01:48:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/3959071553 shutdown_connections 2023-11-06T01:48:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 0x55d7f097c810 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7f09810a0 0x55d7f09834c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096b680 0x55d7f0975290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/3959071553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096bfe0 0x55d7f09757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/3959071553 >> 172.21.15.53:0/3959071553 conn(0x55d7f08cb070 msgr2=0x55d7f09663c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b397fa700 0 client.26561 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/3959071553 shutdown_connections 2023-11-06T01:48:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/3959071553 wait complete. 2023-11-06T01:48:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.558+0000 7f7b63ef0b80 20 client.26561 trim_cache size 0 max 16384 2023-11-06T01:48:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.577+0000 7f7b63ef0b80 1 Processor -- start 2023-11-06T01:48:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.578+0000 7f7b63ef0b80 1 -- start start 2023-11-06T01:48:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.578+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f0987de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.578+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f097a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.578+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096ced0 0x55d7f096d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.578+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f095edd0 con 0x55d7f096bfe0 2023-11-06T01:48:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.578+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f095e7a0 con 0x55d7f096b680 2023-11-06T01:48:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.578+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f0974e00 con 0x55d7f096ced0 2023-11-06T01:48:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.578+0000 7f7b55afc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f0987de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.578+0000 7f7b55afc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f0987de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55440/0 (socket says 172.21.15.53:55440) 2023-11-06T01:48:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.578+0000 7f7b55afc700 1 -- 172.21.15.53:0/2809059835 learned_addr learned my addr 172.21.15.53:0/2809059835 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.578+0000 7f7b562fd700 1 --2- 172.21.15.53:0/2809059835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096ced0 0x55d7f096d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.579+0000 7f7b552fb700 1 --2- 172.21.15.53:0/2809059835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f097a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.579+0000 7f7b562fd700 1 -- 172.21.15.53:0/2809059835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 msgr2=0x55d7f0987de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.579+0000 7f7b562fd700 1 --2- 172.21.15.53:0/2809059835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f0987de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.579+0000 7f7b562fd700 1 -- 172.21.15.53:0/2809059835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 msgr2=0x55d7f097a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.579+0000 7f7b562fd700 1 --2- 172.21.15.53:0/2809059835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f097a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.579+0000 7f7b562fd700 1 -- 172.21.15.53:0/2809059835 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7f07111f0 con 0x55d7f096ced0 2023-11-06T01:48:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.579+0000 7f7b55afc700 1 --2- 172.21.15.53:0/2809059835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f0987de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:48:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.579+0000 7f7b552fb700 1 --2- 172.21.15.53:0/2809059835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f097a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:48:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.580+0000 7f7b562fd700 1 --2- 172.21.15.53:0/2809059835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096ced0 0x55d7f096d2b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c012c10 tx=0x7f7b4c00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2c01f4a5b8fa41d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.580+0000 7f7b44ff9700 1 -- 172.21.15.53:0/2809059835 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c005cc0 con 0x55d7f096ced0 2023-11-06T01:48:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.580+0000 7f7b44ff9700 1 -- 172.21.15.53:0/2809059835 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b4c009280 con 0x55d7f096ced0 2023-11-06T01:48:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.580+0000 7f7b44ff9700 1 -- 172.21.15.53:0/2809059835 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c013ad0 con 0x55d7f096ced0 2023-11-06T01:48:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.581+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/2809059835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096ced0 msgr2=0x55d7f096d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.581+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/2809059835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096ced0 0x55d7f096d2b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c012c10 tx=0x7f7b4c00ecf0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.581+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/2809059835 shutdown_connections 2023-11-06T01:48:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.581+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/2809059835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096b680 0x55d7f0987de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.581+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/2809059835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096ced0 0x55d7f096d2b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.581+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/2809059835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f097a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.581+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/2809059835 >> 172.21.15.53:0/2809059835 conn(0x55d7f08cb070 msgr2=0x55d7f0967c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.581+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/2809059835 shutdown_connections 2023-11-06T01:48:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.581+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/2809059835 wait complete. 2023-11-06T01:48:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.581+0000 7f7b63ef0b80 1 -- start start 2023-11-06T01:48:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.581+0000 7f7b63ef0b80 1 Processor -- start 2023-11-06T01:48:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096b680 0x55d7f09cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f09cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b63ef0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 0x55d7f09d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f09d21e0 con 0x55d7f096bfe0 2023-11-06T01:48:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f09d2350 con 0x55d7f096ced0 2023-11-06T01:48:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b63ef0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f09d2510 con 0x55d7f096b680 2023-11-06T01:48:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b63ef0b80 10 client.0 _ll_register_callbacks cb 0x7f7b4c0099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:48:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b55afc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096b680 0x55d7f09cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b55afc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096b680 0x55d7f09cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58248/0 (socket says 172.21.15.53:58248) 2023-11-06T01:48:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b55afc700 1 -- 172.21.15.53:0/764981020 learned_addr learned my addr 172.21.15.53:0/764981020 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:48:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b552fb700 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f09cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b562fd700 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 0x55d7f09d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b55afc700 1 -- 172.21.15.53:0/764981020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 msgr2=0x55d7f09d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b55afc700 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 0x55d7f09d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b55afc700 1 -- 172.21.15.53:0/764981020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 msgr2=0x55d7f09cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:48:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b55afc700 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f09cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.582+0000 7f7b55afc700 1 -- 172.21.15.53:0/764981020 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b40010c70 con 0x55d7f096b680 2023-11-06T01:48:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b55afc700 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096b680 0x55d7f09cad80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7b48097bb0 tx=0x7f7b480a3290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb1bd2b9f315387 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:48:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b397fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:48:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b397fa700 1 -- 172.21.15.53:0/764981020 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4809ef70 con 0x55d7f096b680 2023-11-06T01:48:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b397fa700 1 -- 172.21.15.53:0/764981020 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7b480b3040 con 0x55d7f096b680 2023-11-06T01:48:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/764981020 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b480a3120 con 0x55d7f096b680 2023-11-06T01:48:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b397fa700 1 -- 172.21.15.53:0/764981020 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b4809a630 con 0x55d7f096b680 2023-11-06T01:48:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b63ef0b80 10 client.26623 fetch_fsmap learned FSMap version 49 2023-11-06T01:48:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/764981020 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d7f06d3c00 con 0x55d7f096b680 2023-11-06T01:48:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b397fa700 1 -- 172.21.15.53:0/764981020 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b480a3120 con 0x55d7f096b680 2023-11-06T01:48:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b397fa700 1 -- 172.21.15.53:0/764981020 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7b480a2c90 con 0x55d7f096b680 2023-11-06T01:48:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b63ef0b80 10 client.26623 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:48:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b63ef0b80 10 client.26623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:48:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b63ef0b80 20 client.26623 populate_metadata read hostname 'smithi053' 2023-11-06T01:48:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7f09d3b20 0x55d7f09d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:48:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b63ef0b80 4 client.26623 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:48:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.583+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/764981020 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7f08b5ce0 con 0x55d7f09d3b20 2023-11-06T01:48:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.584+0000 7f7b552fb700 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7f09d3b20 0x55d7f09d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:48:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.584+0000 7f7b397fa700 10 client.26623 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.584+0000 7f7b552fb700 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7f09d3b20 0x55d7f09d5f40 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b868868ccbc8e2 server_cookie=2cc936de693e690e in_seq=0 out_seq=0 2023-11-06T01:48:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b397fa700 1 -- 172.21.15.53:0/764981020 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7f08b5ce0 con 0x55d7f09d3b20 2023-11-06T01:48:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b397fa700 10 client.26623 handle_command_reply: tid=0 2023-11-06T01:48:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 1 client.26623 shutdown 2023-11-06T01:48:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/764981020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096b680 msgr2=0x55d7f09cad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096b680 0x55d7f09cad80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7b48097bb0 tx=0x7f7b480a3290 comp rx=0 tx=0).stop 2023-11-06T01:48:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/764981020 shutdown_connections 2023-11-06T01:48:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d7f096ced0 0x55d7f09d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d7f09d3b20 0x55d7f09d5f40 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d7f096b680 0x55d7f09cad80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b397fa700 0 client.26623 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:48:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 1 --2- 172.21.15.53:0/764981020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7f096bfe0 0x55d7f09cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/764981020 >> 172.21.15.53:0/764981020 conn(0x55d7f08cb070 msgr2=0x55d7f09700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:47.599 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:47 smithi053 ceph-mon[99472]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 832 op/s 2023-11-06T01:48:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/764981020 shutdown_connections 2023-11-06T01:48:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 1 -- 172.21.15.53:0/764981020 wait complete. 2023-11-06T01:48:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.592+0000 7f7b63ef0b80 20 client.26623 trim_cache size 0 max 16384 2023-11-06T01:48:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:48:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:48:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:48:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:48:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 -- 172.21.15.53:0/965121800 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b3c0a05a0 msgr2=0x7f7b3c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b3c0a05a0 0x7f7b3c0a2a60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7b40005d90 tx=0x7f7b4000a5b0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 -- 172.21.15.53:0/965121800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b500ff600 msgr2=0x7f7b501a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:48:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b500ff600 0x7f7b501a7110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7b48007b60 tx=0x7f7b48005b10 comp rx=0 tx=0).stop 2023-11-06T01:48:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 -- 172.21.15.53:0/965121800 shutdown_connections 2023-11-06T01:48:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f7b3c0a05a0 0x7f7b3c0a2a60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b500ff600 0x7f7b501a7110 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b50104940 0x7f7b500717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 --2- 172.21.15.53:0/965121800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b50101f70 0x7f7b501a7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:48:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 -- 172.21.15.53:0/965121800 >> 172.21.15.53:0/965121800 conn(0x7f7b500fb1e0 msgr2=0x7f7b50102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:48:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 -- 172.21.15.53:0/965121800 shutdown_connections 2023-11-06T01:48:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:48:47.593+0000 7f7b57d60700 1 -- 172.21.15.53:0/965121800 wait complete. 2023-11-06T01:48:49.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:49 smithi062 ceph-mon[103924]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 15 MiB/s wr, 787 op/s 2023-11-06T01:48:49.495 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:49 smithi163 ceph-mon[104416]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 15 MiB/s wr, 787 op/s 2023-11-06T01:48:49.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:49 smithi053 ceph-mon[99472]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 15 MiB/s wr, 787 op/s 2023-11-06T01:48:51.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:51 smithi062 ceph-mon[103924]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 788 op/s 2023-11-06T01:48:51.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:51 smithi163 ceph-mon[104416]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 788 op/s 2023-11-06T01:48:51.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:51 smithi053 ceph-mon[99472]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 788 op/s 2023-11-06T01:48:52.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 875.0 s, 1330.4 tps, lat 74.529 ms stddev 86.434 2023-11-06T01:48:52.841 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:52 smithi053 ceph-mon[99472]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 693 op/s 2023-11-06T01:48:52.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:52 smithi062 ceph-mon[103924]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 693 op/s 2023-11-06T01:48:52.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:52 smithi163 ceph-mon[104416]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 693 op/s 2023-11-06T01:48:55.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:55 smithi163 ceph-mon[104416]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 711 op/s 2023-11-06T01:48:55.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:55 smithi053 ceph-mon[99472]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 711 op/s 2023-11-06T01:48:55.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:55 smithi062 ceph-mon[103924]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 711 op/s 2023-11-06T01:48:57.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 880.0 s, 1209.0 tps, lat 83.575 ms stddev 158.789 2023-11-06T01:48:57.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:57 smithi163 ceph-mon[104416]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 910 op/s 2023-11-06T01:48:57.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:57 smithi053 ceph-mon[99472]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 910 op/s 2023-11-06T01:48:57.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:57 smithi062 ceph-mon[103924]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 910 op/s 2023-11-06T01:48:58.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:48:58 smithi053 ceph-mon[99472]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.30k op/s 2023-11-06T01:48:58.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:48:58 smithi062 ceph-mon[103924]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.30k op/s 2023-11-06T01:48:58.737 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:48:58 smithi163 ceph-mon[104416]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.30k op/s 2023-11-06T01:49:00.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:00 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:49:00.405 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:00 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:49:00.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:00 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:49:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.765+0000 7f4053aa5700 1 -- 172.21.15.53:0/3670425425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f404c0784e0 msgr2=0x7f404c07a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.765+0000 7f4053aa5700 1 --2- 172.21.15.53:0/3670425425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f404c0784e0 0x7f404c07a970 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4044009f90 tx=0x7f404400b330 comp rx=0 tx=0).stop 2023-11-06T01:49:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.765+0000 7f4053aa5700 1 -- 172.21.15.53:0/3670425425 shutdown_connections 2023-11-06T01:49:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.765+0000 7f4053aa5700 1 --2- 172.21.15.53:0/3670425425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f404c0784e0 0x7f404c07a970 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.765+0000 7f4053aa5700 1 --2- 172.21.15.53:0/3670425425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f404c072c30 0x7f404c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.765+0000 7f4053aa5700 1 --2- 172.21.15.53:0/3670425425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f404c075b10 0x7f404c077fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.765+0000 7f4053aa5700 1 -- 172.21.15.53:0/3670425425 >> 172.21.15.53:0/3670425425 conn(0x7f404c06dbc0 msgr2=0x7f404c070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.765+0000 7f4053aa5700 1 -- 172.21.15.53:0/3670425425 shutdown_connections 2023-11-06T01:49:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.765+0000 7f4053aa5700 1 -- 172.21.15.53:0/3670425425 wait complete. 2023-11-06T01:49:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.765+0000 7f4053aa5700 1 Processor -- start 2023-11-06T01:49:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.766+0000 7f4053aa5700 1 -- start start 2023-11-06T01:49:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.766+0000 7f4053aa5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f404c072c30 0x7f404c1b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.766+0000 7f4053aa5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f404c075b10 0x7f404c1b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.766+0000 7f4053aa5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f404c0784e0 0x7f404c1b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.766+0000 7f4053aa5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c1b8050 con 0x7f404c072c30 2023-11-06T01:49:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.766+0000 7f4053aa5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c1b81c0 con 0x7f404c0784e0 2023-11-06T01:49:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.766+0000 7f4053aa5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c1b8380 con 0x7f404c075b10 2023-11-06T01:49:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.766+0000 7f4051040700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f404c075b10 0x7f404c1b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.766+0000 7f4051040700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f404c075b10 0x7f404c1b5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58262/0 (socket says 172.21.15.53:58262) 2023-11-06T01:49:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.766+0000 7f4051040700 1 -- 172.21.15.53:0/2778658179 learned_addr learned my addr 172.21.15.53:0/2778658179 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.767+0000 7f4051040700 1 -- 172.21.15.53:0/2778658179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f404c0784e0 msgr2=0x7f404c1b78b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-06T01:49:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.767+0000 7f4051040700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f404c0784e0 0x7f404c1b78b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.767+0000 7f4051040700 1 -- 172.21.15.53:0/2778658179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f404c072c30 msgr2=0x7f404c1b2cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T01:49:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.767+0000 7f4051040700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f404c072c30 0x7f404c1b2cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.767+0000 7f4051040700 1 -- 172.21.15.53:0/2778658179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40480095f0 con 0x7f404c075b10 2023-11-06T01:49:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.767+0000 7f4051040700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f404c075b10 0x7f404c1b5220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f403c009800 tx=0x7f403c009bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.767+0000 7f4042ffd700 1 -- 172.21.15.53:0/2778658179 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c004830 con 0x7f404c075b10 2023-11-06T01:49:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.767+0000 7f4042ffd700 1 -- 172.21.15.53:0/2778658179 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f403c010040 con 0x7f404c075b10 2023-11-06T01:49:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.767+0000 7f4053aa5700 1 -- 172.21.15.53:0/2778658179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4044009c40 con 0x7f404c075b10 2023-11-06T01:49:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.767+0000 7f4042ffd700 1 -- 172.21.15.53:0/2778658179 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c014880 con 0x7f404c075b10 2023-11-06T01:49:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.767+0000 7f4053aa5700 1 -- 172.21.15.53:0/2778658179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c1b89e0 con 0x7f404c075b10 2023-11-06T01:49:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.770+0000 7f4042ffd700 1 -- 172.21.15.53:0/2778658179 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f403c0050b0 con 0x7f404c075b10 2023-11-06T01:49:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.770+0000 7f4042ffd700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a0520 0x7f40380a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.771+0000 7f4042ffd700 1 -- 172.21.15.53:0/2778658179 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f403c0cb340 con 0x7f404c075b10 2023-11-06T01:49:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.771+0000 7f4051841700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a0520 0x7f40380a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.772+0000 7f405fc35b80 1 Processor -- start 2023-11-06T01:49:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.773+0000 7f405fc35b80 1 -- start start 2023-11-06T01:49:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.773+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 0x56516abf7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.773+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8110 0x56516abf88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.773+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8e80 0x56516abf9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.773+0000 7f405fc35b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516ab4f9b0 con 0x56516abf77b0 2023-11-06T01:49:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.773+0000 7f405fc35b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516ab4d370 con 0x56516abf8110 2023-11-06T01:49:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.773+0000 7f405fc35b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516ab48c60 con 0x56516abf8e80 2023-11-06T01:49:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.773+0000 7f4051841700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 0x56516abf7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.773+0000 7f4051841700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 0x56516abf7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44026/0 (socket says 172.21.15.53:44026) 2023-11-06T01:49:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.773+0000 7f4051841700 1 -- 172.21.15.53:0/502265397 learned_addr learned my addr 172.21.15.53:0/502265397 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.773+0000 7f4051040700 1 --2- 172.21.15.53:0/502265397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8110 0x56516abf88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.774+0000 7f4052042700 1 --2- 172.21.15.53:0/502265397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8e80 0x56516abf9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.774+0000 7f4051040700 1 -- 172.21.15.53:0/502265397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8e80 msgr2=0x56516abf9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.774+0000 7f4051040700 1 --2- 172.21.15.53:0/502265397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8e80 0x56516abf9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.774+0000 7f4051040700 1 -- 172.21.15.53:0/502265397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 msgr2=0x56516abf7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.774+0000 7f4051040700 1 --2- 172.21.15.53:0/502265397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 0x56516abf7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.774+0000 7f4051040700 1 -- 172.21.15.53:0/502265397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56516a976d80 con 0x56516abf8110 2023-11-06T01:49:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.774+0000 7f4051040700 1 --2- 172.21.15.53:0/502265397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8110 0x56516abf88f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f403c0c9190 tx=0x7f403c098500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7cd81808aaf9e7e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.774+0000 7f4051841700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a0520 0x7f40380a29e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f404c083d10 tx=0x7f4048009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.775+0000 7f4037fff700 1 -- 172.21.15.53:0/502265397 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0b0070 con 0x56516abf8110 2023-11-06T01:49:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.775+0000 7f4037fff700 1 -- 172.21.15.53:0/502265397 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f403c098ba0 con 0x56516abf8110 2023-11-06T01:49:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.775+0000 7f4037fff700 1 -- 172.21.15.53:0/502265397 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c098e00 con 0x56516abf8110 2023-11-06T01:49:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.776+0000 7f405fc35b80 1 -- 172.21.15.53:0/502265397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8110 msgr2=0x56516abf88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.776+0000 7f405fc35b80 1 --2- 172.21.15.53:0/502265397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8110 0x56516abf88f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f403c0c9190 tx=0x7f403c098500 comp rx=0 tx=0).stop 2023-11-06T01:49:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.776+0000 7f405fc35b80 1 -- 172.21.15.53:0/502265397 shutdown_connections 2023-11-06T01:49:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.776+0000 7f405fc35b80 1 --2- 172.21.15.53:0/502265397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8e80 0x56516abf9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.776+0000 7f405fc35b80 1 --2- 172.21.15.53:0/502265397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 0x56516abf7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.776+0000 7f405fc35b80 1 --2- 172.21.15.53:0/502265397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8110 0x56516abf88f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.776+0000 7f405fc35b80 1 -- 172.21.15.53:0/502265397 >> 172.21.15.53:0/502265397 conn(0x56516ab57070 msgr2=0x56516abf4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.776+0000 7f405fc35b80 1 -- 172.21.15.53:0/502265397 shutdown_connections 2023-11-06T01:49:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.776+0000 7f405fc35b80 1 -- 172.21.15.53:0/502265397 wait complete. 2023-11-06T01:49:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f405fc35b80 1 -- start start 2023-11-06T01:49:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f405fc35b80 1 Processor -- start 2023-11-06T01:49:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 0x56516ac01510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8110 0x56516ac01a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 0x56516ac08a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f405fc35b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516aae54b0 con 0x56516abf77b0 2023-11-06T01:49:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f405fc35b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516aae5150 con 0x56516abf8e80 2023-11-06T01:49:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f405fc35b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a9cc8c0 con 0x56516abf8110 2023-11-06T01:49:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f405fc35b80 10 client.0 _ll_register_callbacks cb 0x56516aa32a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:49:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f4051841700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 0x56516ac01510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f4051040700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8110 0x56516ac01a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f4051841700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 0x56516ac01510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44032/0 (socket says 172.21.15.53:44032) 2023-11-06T01:49:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f4051040700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8110 0x56516ac01a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58274/0 (socket says 172.21.15.53:58274) 2023-11-06T01:49:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.777+0000 7f4051841700 1 -- 172.21.15.53:0/2946338879 learned_addr learned my addr 172.21.15.53:0/2946338879 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.778+0000 7f4052042700 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 0x56516ac08a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.778+0000 7f4051040700 1 -- 172.21.15.53:0/2946338879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 msgr2=0x56516ac08a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.778+0000 7f4051040700 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 0x56516ac08a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.778+0000 7f4051040700 1 -- 172.21.15.53:0/2946338879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 msgr2=0x56516ac01510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.778+0000 7f4051040700 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 0x56516ac01510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.778+0000 7f4051040700 1 -- 172.21.15.53:0/2946338879 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c1b89e0 con 0x56516abf8110 2023-11-06T01:49:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f4051040700 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8110 0x56516ac01a50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f403c09cb90 tx=0x7f403c0993f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb3a62175f1624b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f40357fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-11-06T01:49:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f40357fa700 1 -- 172.21.15.53:0/2946338879 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0b0030 con 0x56516abf8110 2023-11-06T01:49:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f40357fa700 1 -- 172.21.15.53:0/2946338879 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f403c0b1070 con 0x56516abf8110 2023-11-06T01:49:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f405fc35b80 1 -- 172.21.15.53:0/2946338879 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56516ab73cb0 con 0x56516abf8110 2023-11-06T01:49:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f40357fa700 1 -- 172.21.15.53:0/2946338879 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f403c0a0c10 con 0x56516abf8110 2023-11-06T01:49:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f405fc35b80 10 client.26635 fetch_fsmap learned FSMap version 49 2023-11-06T01:49:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f405fc35b80 1 -- 172.21.15.53:0/2946338879 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56516a95fc00 con 0x56516abf8110 2023-11-06T01:49:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f40357fa700 1 -- 172.21.15.53:0/2946338879 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56516ab73cb0 con 0x56516abf8110 2023-11-06T01:49:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f40357fa700 1 -- 172.21.15.53:0/2946338879 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f403c099bb0 con 0x56516abf8110 2023-11-06T01:49:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f405fc35b80 10 client.26635 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:49:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f405fc35b80 10 client.26635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:49:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f405fc35b80 20 client.26635 populate_metadata read hostname 'smithi053' 2023-11-06T01:49:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56516ac0ce30 0x56516ac0f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f405fc35b80 4 client.26635 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:49:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.780+0000 7f405fc35b80 1 -- 172.21.15.53:0/2946338879 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56516abc6db0 con 0x56516ac0ce30 2023-11-06T01:49:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.781+0000 7f4051841700 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56516ac0ce30 0x56516ac0f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.781+0000 7f40357fa700 10 client.26635 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.781+0000 7f4051841700 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56516ac0ce30 0x56516ac0f220 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5f42cae8bb03a9 server_cookie=93095abd4a2595a5 in_seq=0 out_seq=0 2023-11-06T01:49:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f40357fa700 1 -- 172.21.15.53:0/2946338879 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56516abc6db0 con 0x56516ac0ce30 2023-11-06T01:49:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f40357fa700 10 client.26635 handle_command_reply: tid=0 2023-11-06T01:49:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 1 client.26635 shutdown 2023-11-06T01:49:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 1 -- 172.21.15.53:0/2946338879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8110 msgr2=0x56516ac01a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8110 0x56516ac01a50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f403c09cb90 tx=0x7f403c0993f0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 1 -- 172.21.15.53:0/2946338879 shutdown_connections 2023-11-06T01:49:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 0x56516ac08a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56516ac0ce30 0x56516ac0f220 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf77b0 0x56516ac01510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f40357fa700 0 client.26635 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2946338879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf8110 0x56516ac01a50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 1 -- 172.21.15.53:0/2946338879 >> 172.21.15.53:0/2946338879 conn(0x56516ab57070 msgr2=0x56516abf23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 1 -- 172.21.15.53:0/2946338879 shutdown_connections 2023-11-06T01:49:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 1 -- 172.21.15.53:0/2946338879 wait complete. 2023-11-06T01:49:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.782+0000 7f405fc35b80 20 client.26635 trim_cache size 0 max 16384 2023-11-06T01:49:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.804+0000 7f405fc35b80 1 Processor -- start 2023-11-06T01:49:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.804+0000 7f405fc35b80 1 -- start start 2023-11-06T01:49:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.804+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf77b0 0x56516ac00180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.804+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf8110 0x56516ac08a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.804+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 0x56516abf3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.804+0000 7f405fc35b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516ac009c0 con 0x56516abf8110 2023-11-06T01:49:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.804+0000 7f405fc35b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516abed500 con 0x56516abf8e80 2023-11-06T01:49:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.804+0000 7f405fc35b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516abea7a0 con 0x56516abf77b0 2023-11-06T01:49:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.805+0000 7f4052042700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 0x56516abf3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.805+0000 7f4052042700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 0x56516abf3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55478/0 (socket says 172.21.15.53:55478) 2023-11-06T01:49:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.805+0000 7f4052042700 1 -- 172.21.15.53:0/3068573928 learned_addr learned my addr 172.21.15.53:0/3068573928 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.805+0000 7f4052042700 1 -- 172.21.15.53:0/3068573928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf77b0 msgr2=0x56516ac00180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:49:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.805+0000 7f4052042700 1 --2- 172.21.15.53:0/3068573928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf77b0 0x56516ac00180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.805+0000 7f4052042700 1 -- 172.21.15.53:0/3068573928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf8110 msgr2=0x56516ac08a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:49:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.805+0000 7f4052042700 1 --2- 172.21.15.53:0/3068573928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf8110 0x56516ac08a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.805+0000 7f4052042700 1 -- 172.21.15.53:0/3068573928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56516a99d1f0 con 0x56516abf8e80 2023-11-06T01:49:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.805+0000 7f4052042700 1 --2- 172.21.15.53:0/3068573928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 0x56516abf3c20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4044015000 tx=0x7f4044009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad80244ab9d15db server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.819+0000 7f4040daa700 1 -- 172.21.15.53:0/3068573928 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404400e040 con 0x56516abf8e80 2023-11-06T01:49:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.819+0000 7f4040daa700 1 -- 172.21.15.53:0/3068573928 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4044021070 con 0x56516abf8e80 2023-11-06T01:49:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.819+0000 7f4040daa700 1 -- 172.21.15.53:0/3068573928 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404400bd90 con 0x56516abf8e80 2023-11-06T01:49:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.820+0000 7f405fc35b80 1 -- 172.21.15.53:0/3068573928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 msgr2=0x56516abf3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.820+0000 7f405fc35b80 1 --2- 172.21.15.53:0/3068573928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 0x56516abf3c20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4044015000 tx=0x7f4044009390 comp rx=0 tx=0).stop 2023-11-06T01:49:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.839+0000 7f405fc35b80 1 -- 172.21.15.53:0/3068573928 shutdown_connections 2023-11-06T01:49:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.839+0000 7f405fc35b80 1 --2- 172.21.15.53:0/3068573928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8e80 0x56516abf3c20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.839+0000 7f405fc35b80 1 --2- 172.21.15.53:0/3068573928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf77b0 0x56516ac00180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.839+0000 7f405fc35b80 1 --2- 172.21.15.53:0/3068573928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf8110 0x56516ac08a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.839+0000 7f405fc35b80 1 -- 172.21.15.53:0/3068573928 >> 172.21.15.53:0/3068573928 conn(0x56516ab57070 msgr2=0x56516abeeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.845+0000 7f405fc35b80 1 -- 172.21.15.53:0/3068573928 shutdown_connections 2023-11-06T01:49:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.847+0000 7f405fc35b80 1 -- 172.21.15.53:0/3068573928 wait complete. 2023-11-06T01:49:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.847+0000 7f405fc35b80 1 -- start start 2023-11-06T01:49:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.847+0000 7f405fc35b80 1 Processor -- start 2023-11-06T01:49:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.848+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf77b0 0x56516ac083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.848+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8110 0x56516ac08930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.848+0000 7f405fc35b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf8e80 0x56516ac06a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.848+0000 7f405fc35b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516ac07110 con 0x56516abf8e80 2023-11-06T01:49:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.848+0000 7f405fc35b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516ac07280 con 0x56516abf8110 2023-11-06T01:49:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.848+0000 7f405fc35b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516ac07440 con 0x56516abf77b0 2023-11-06T01:49:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.848+0000 7f405fc35b80 10 client.0 _ll_register_callbacks cb 0x56516aa32a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:49:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.848+0000 7f4052042700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf8e80 0x56516ac06a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.848+0000 7f4052042700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf8e80 0x56516ac06a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44042/0 (socket says 172.21.15.53:44042) 2023-11-06T01:49:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.848+0000 7f4052042700 1 -- 172.21.15.53:0/2746422691 learned_addr learned my addr 172.21.15.53:0/2746422691 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.849+0000 7f4051841700 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf77b0 0x56516ac083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.849+0000 7f4051040700 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8110 0x56516ac08930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.849+0000 7f4052042700 1 -- 172.21.15.53:0/2746422691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf77b0 msgr2=0x56516ac083f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.849+0000 7f4052042700 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf77b0 0x56516ac083f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.849+0000 7f4052042700 1 -- 172.21.15.53:0/2746422691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8110 msgr2=0x56516ac08930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.849+0000 7f4052042700 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8110 0x56516ac08930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.849+0000 7f4052042700 1 -- 172.21.15.53:0/2746422691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c07b3a0 con 0x56516abf8e80 2023-11-06T01:49:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.849+0000 7f4052042700 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf8e80 0x56516ac06a00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4044009960 tx=0x7f4044003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed77dbdcfa44cdd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.849+0000 7f40357fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:49:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.861+0000 7f40357fa700 1 -- 172.21.15.53:0/2746422691 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404400e040 con 0x56516abf8e80 2023-11-06T01:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.867+0000 7f40357fa700 1 -- 172.21.15.53:0/2746422691 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f404400bcd0 con 0x56516abf8e80 2023-11-06T01:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.867+0000 7f405fc35b80 1 -- 172.21.15.53:0/2746422691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f403c098ba0 con 0x56516abf8e80 2023-11-06T01:49:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.869+0000 7f40357fa700 1 -- 172.21.15.53:0/2746422691 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f404400be90 con 0x56516abf8e80 2023-11-06T01:49:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.869+0000 7f405fc35b80 10 client.17070 fetch_fsmap learned FSMap version 49 2023-11-06T01:49:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.869+0000 7f405fc35b80 1 -- 172.21.15.53:0/2746422691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56516a975250 con 0x56516abf8e80 2023-11-06T01:49:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.869+0000 7f40357fa700 1 -- 172.21.15.53:0/2746422691 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f403c098ba0 con 0x56516abf8e80 2023-11-06T01:49:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.869+0000 7f40357fa700 1 -- 172.21.15.53:0/2746422691 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f4044019070 con 0x56516abf8e80 2023-11-06T01:49:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.870+0000 7f405fc35b80 10 client.17070 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:49:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.870+0000 7f405fc35b80 10 client.17070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:49:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.870+0000 7f405fc35b80 20 client.17070 populate_metadata read hostname 'smithi053' 2023-11-06T01:49:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.870+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56516ac5f8f0 0x56516ac61d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.870+0000 7f405fc35b80 4 client.17070 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:49:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.870+0000 7f405fc35b80 1 -- 172.21.15.53:0/2746422691 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56516ab41ce0 con 0x56516ac5f8f0 2023-11-06T01:49:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.870+0000 7f4051841700 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56516ac5f8f0 0x56516ac61d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.870+0000 7f40357fa700 10 client.17070 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.870+0000 7f4051841700 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56516ac5f8f0 0x56516ac61d10 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f7b7b46a9f3a64 server_cookie=1af89b97837bdd9c in_seq=0 out_seq=0 2023-11-06T01:49:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.879+0000 7f40357fa700 1 -- 172.21.15.53:0/2746422691 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56516ab41ce0 con 0x56516ac5f8f0 2023-11-06T01:49:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.879+0000 7f40357fa700 10 client.17070 handle_command_reply: tid=0 2023-11-06T01:49:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.879+0000 7f405fc35b80 1 client.17070 shutdown 2023-11-06T01:49:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.880+0000 7f405fc35b80 1 -- 172.21.15.53:0/2746422691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf8e80 msgr2=0x56516ac06a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.880+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf8e80 0x56516ac06a00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4044009960 tx=0x7f4044003ab0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:49:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:49:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:49:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:49:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.881+0000 7f405fc35b80 1 -- 172.21.15.53:0/2746422691 shutdown_connections 2023-11-06T01:49:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.881+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x56516ac5f8f0 0x56516ac61d10 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.881+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56516abf8e80 0x56516ac06a00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.881+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56516abf77b0 0x56516ac083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.881+0000 7f405fc35b80 1 --2- 172.21.15.53:0/2746422691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56516abf8110 0x56516ac08930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.881+0000 7f405fc35b80 1 -- 172.21.15.53:0/2746422691 >> 172.21.15.53:0/2746422691 conn(0x56516ab57070 msgr2=0x56516ac00180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.881+0000 7f40357fa700 0 client.17070 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.882+0000 7f405fc35b80 1 -- 172.21.15.53:0/2746422691 shutdown_connections 2023-11-06T01:49:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.882+0000 7f405fc35b80 1 -- 172.21.15.53:0/2746422691 wait complete. 2023-11-06T01:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.883+0000 7f405fc35b80 20 client.17070 trim_cache size 0 max 16384 2023-11-06T01:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.883+0000 7f4053aa5700 1 -- 172.21.15.53:0/2778658179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a0520 msgr2=0x7f40380a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.883+0000 7f4053aa5700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a0520 0x7f40380a29e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f404c083d10 tx=0x7f4048009310 comp rx=0 tx=0).stop 2023-11-06T01:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.883+0000 7f4053aa5700 1 -- 172.21.15.53:0/2778658179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f404c075b10 msgr2=0x7f404c1b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.883+0000 7f4053aa5700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f404c075b10 0x7f404c1b5220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f403c009800 tx=0x7f403c009bc0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.883+0000 7f4053aa5700 1 -- 172.21.15.53:0/2778658179 shutdown_connections 2023-11-06T01:49:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.883+0000 7f4053aa5700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40380a0520 0x7f40380a29e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.883+0000 7f4053aa5700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f404c0784e0 0x7f404c1b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.883+0000 7f4053aa5700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f404c072c30 0x7f404c1b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.883+0000 7f4053aa5700 1 --2- 172.21.15.53:0/2778658179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f404c075b10 0x7f404c1b5220 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.884+0000 7f4053aa5700 1 -- 172.21.15.53:0/2778658179 >> 172.21.15.53:0/2778658179 conn(0x7f404c06dbc0 msgr2=0x7f404c0773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.884+0000 7f4053aa5700 1 -- 172.21.15.53:0/2778658179 shutdown_connections 2023-11-06T01:49:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:00.884+0000 7f4053aa5700 1 -- 172.21.15.53:0/2778658179 wait complete. 2023-11-06T01:49:01.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:01 smithi053 ceph-mon[99472]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 1.85k op/s 2023-11-06T01:49:01.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:49:01.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:49:01.344 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:01 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:49:01.393 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:01 smithi062 ceph-mon[103924]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 1.85k op/s 2023-11-06T01:49:01.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:49:01.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:49:01.394 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:01 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:49:01.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:01 smithi163 ceph-mon[104416]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 1.85k op/s 2023-11-06T01:49:01.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:49:01.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:49:01.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:01 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:49:02.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 885.0 s, 1380.4 tps, lat 72.769 ms stddev 80.144 2023-11-06T01:49:02.841 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:02 smithi053 ceph-mon[99472]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 1.85k op/s 2023-11-06T01:49:02.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:02 smithi062 ceph-mon[103924]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 1.85k op/s 2023-11-06T01:49:02.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:02 smithi163 ceph-mon[104416]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 1.85k op/s 2023-11-06T01:49:05.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:05 smithi062 ceph-mon[103924]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.86k op/s 2023-11-06T01:49:05.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:05 smithi163 ceph-mon[104416]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.86k op/s 2023-11-06T01:49:05.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:05 smithi053 ceph-mon[99472]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.86k op/s 2023-11-06T01:49:07.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 890.0 s, 1551.4 tps, lat 64.639 ms stddev 74.394 2023-11-06T01:49:07.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:07 smithi062 ceph-mon[103924]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 1.84k op/s 2023-11-06T01:49:07.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:07 smithi163 ceph-mon[104416]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 1.84k op/s 2023-11-06T01:49:07.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:07 smithi053 ceph-mon[99472]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 1.84k op/s 2023-11-06T01:49:09.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:09 smithi062 ceph-mon[103924]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 1.61k op/s 2023-11-06T01:49:09.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:09 smithi163 ceph-mon[104416]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 1.61k op/s 2023-11-06T01:49:09.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:09 smithi053 ceph-mon[99472]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 1.61k op/s 2023-11-06T01:49:11.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:11 smithi163 ceph-mon[104416]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 1.26k op/s 2023-11-06T01:49:11.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:11 smithi053 ceph-mon[99472]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 1.26k op/s 2023-11-06T01:49:11.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:11 smithi062 ceph-mon[103924]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 1.26k op/s 2023-11-06T01:49:12.180 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 895.0 s, 1579.0 tps, lat 62.223 ms stddev 70.415 2023-11-06T01:49:12.380 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-06T01:49:12.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.581+0000 7f19edc70700 1 -- 172.21.15.53:0/1160737204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e810d0d0 msgr2=0x7f19e810d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:12.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.581+0000 7f19edc70700 1 --2- 172.21.15.53:0/1160737204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e810d0d0 0x7f19e810d550 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f19e000b0a0 tx=0x7f19e000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.589+0000 7f19edc70700 1 -- 172.21.15.53:0/1160737204 shutdown_connections 2023-11-06T01:49:12.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.589+0000 7f19edc70700 1 --2- 172.21.15.53:0/1160737204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e810d0d0 0x7f19e810d550 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.589+0000 7f19edc70700 1 --2- 172.21.15.53:0/1160737204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19e8107d90 0x7f19e810a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.589+0000 7f19edc70700 1 --2- 172.21.15.53:0/1160737204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e810a700 0x7f19e810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.589+0000 7f19edc70700 1 -- 172.21.15.53:0/1160737204 >> 172.21.15.53:0/1160737204 conn(0x7f19e806db40 msgr2=0x7f19e806ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:12.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.590+0000 7f19edc70700 1 -- 172.21.15.53:0/1160737204 shutdown_connections 2023-11-06T01:49:12.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.590+0000 7f19edc70700 1 -- 172.21.15.53:0/1160737204 wait complete. 2023-11-06T01:49:12.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.590+0000 7f19edc70700 1 Processor -- start 2023-11-06T01:49:12.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19edc70700 1 -- start start 2023-11-06T01:49:12.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19edc70700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e8107d90 0x7f19e81a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19edc70700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e810a700 0x7f19e81a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19edc70700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19e810d0d0 0x7f19e81acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19edc70700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81ad540 con 0x7f19e810a700 2023-11-06T01:49:12.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19edc70700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81ad680 con 0x7f19e8107d90 2023-11-06T01:49:12.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19edc70700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81ad810 con 0x7f19e810d0d0 2023-11-06T01:49:12.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19dffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e810a700 0x7f19e81a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e8107d90 0x7f19e81a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19dffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e810a700 0x7f19e81a5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44058/0 (socket says 172.21.15.53:44058) 2023-11-06T01:49:12.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19dffff700 1 -- 172.21.15.53:0/1075827819 learned_addr learned my addr 172.21.15.53:0/1075827819 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:12.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19dffff700 1 -- 172.21.15.53:0/1075827819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19e810d0d0 msgr2=0x7f19e81acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:12.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19dffff700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19e810d0d0 0x7f19e81acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19dffff700 1 -- 172.21.15.53:0/1075827819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e8107d90 msgr2=0x7f19e81a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:12.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.592+0000 7f19dffff700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e8107d90 0x7f19e81a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.593+0000 7f19dffff700 1 -- 172.21.15.53:0/1075827819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0009d00 con 0x7f19e810a700 2023-11-06T01:49:12.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.593+0000 7f19e77fe700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e8107d90 0x7f19e81a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:49:12.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.593+0000 7f19dffff700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e810a700 0x7f19e81a5f00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f19d800b550 tx=0x7f19d800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:12.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.593+0000 7f19e57fa700 1 -- 172.21.15.53:0/1075827819 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8011830 con 0x7f19e810a700 2023-11-06T01:49:12.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.593+0000 7f19edc70700 1 -- 172.21.15.53:0/1075827819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e81adaf0 con 0x7f19e810a700 2023-11-06T01:49:12.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.593+0000 7f19e57fa700 1 -- 172.21.15.53:0/1075827819 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19d8011e50 con 0x7f19e810a700 2023-11-06T01:49:12.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.593+0000 7f19e57fa700 1 -- 172.21.15.53:0/1075827819 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8010e90 con 0x7f19e810a700 2023-11-06T01:49:12.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.593+0000 7f19edc70700 1 -- 172.21.15.53:0/1075827819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e81ae050 con 0x7f19e810a700 2023-11-06T01:49:12.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.596+0000 7f19e57fa700 1 -- 172.21.15.53:0/1075827819 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f19d8011990 con 0x7f19e810a700 2023-11-06T01:49:12.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.599+0000 7f19f9e00b80 1 Processor -- start 2023-11-06T01:49:12.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.599+0000 7f19e57fa700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f19d40a0570 0x7f19d40a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.599+0000 7f19e57fa700 1 -- 172.21.15.53:0/1075827819 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f19d80cb3e0 con 0x7f19e810a700 2023-11-06T01:49:12.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19f9e00b80 1 -- start start 2023-11-06T01:49:12.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 0x55d8472b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b5a70 0x55d8472b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b43f0 0x55d8472b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d84720ba10 con 0x55d8472b43f0 2023-11-06T01:49:12.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8472093d0 con 0x55d8472b5a70 2023-11-06T01:49:12.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d847204cc0 con 0x55d8472b50f0 2023-11-06T01:49:12.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19e77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 0x55d8472b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19e77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 0x55d8472b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58304/0 (socket says 172.21.15.53:58304) 2023-11-06T01:49:12.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19e77fe700 1 -- 172.21.15.53:0/4020595098 learned_addr learned my addr 172.21.15.53:0/4020595098 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19dffff700 1 --2- 172.21.15.53:0/4020595098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b5a70 0x55d8472b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19e7fff700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f19d40a0570 0x7f19d40a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19e7fff700 1 --2- 172.21.15.53:0/4020595098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b43f0 0x55d8472b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19e77fe700 1 -- 172.21.15.53:0/4020595098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b5a70 msgr2=0x55d8472b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19e77fe700 1 --2- 172.21.15.53:0/4020595098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b5a70 0x55d8472b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19e77fe700 1 -- 172.21.15.53:0/4020595098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b43f0 msgr2=0x55d8472b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19e77fe700 1 --2- 172.21.15.53:0/4020595098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b43f0 0x55d8472b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.600+0000 7f19e77fe700 1 -- 172.21.15.53:0/4020595098 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d847032de0 con 0x55d8472b50f0 2023-11-06T01:49:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.601+0000 7f19e7fff700 1 --2- 172.21.15.53:0/4020595098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b43f0 0x55d8472b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:49:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.601+0000 7f19e77fe700 1 --2- 172.21.15.53:0/4020595098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 0x55d8472b5530 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009eb0 tx=0x7f19d000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1129033af2acc35 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.601+0000 7f19ddffb700 1 -- 172.21.15.53:0/4020595098 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d000ecd0 con 0x55d8472b50f0 2023-11-06T01:49:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.601+0000 7f19ddffb700 1 -- 172.21.15.53:0/4020595098 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19d000ee30 con 0x55d8472b50f0 2023-11-06T01:49:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.601+0000 7f19ddffb700 1 -- 172.21.15.53:0/4020595098 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0012750 con 0x55d8472b50f0 2023-11-06T01:49:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.602+0000 7f19e7fff700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f19d40a0570 0x7f19d40a2a30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19e000d010 tx=0x7f19e000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.602+0000 7f19f9e00b80 1 -- 172.21.15.53:0/4020595098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 msgr2=0x55d8472b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.602+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/4020595098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 0x55d8472b5530 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009eb0 tx=0x7f19d000e4b0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.602+0000 7f19f9e00b80 1 -- 172.21.15.53:0/4020595098 shutdown_connections 2023-11-06T01:49:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.602+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/4020595098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b43f0 0x55d8472b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.602+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/4020595098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 0x55d8472b5530 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.602+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/4020595098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b5a70 0x55d8472b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.602+0000 7f19f9e00b80 1 -- 172.21.15.53:0/4020595098 >> 172.21.15.53:0/4020595098 conn(0x55d8472130d0 msgr2=0x55d8472ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:12.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.602+0000 7f19f9e00b80 1 -- 172.21.15.53:0/4020595098 shutdown_connections 2023-11-06T01:49:12.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.602+0000 7f19f9e00b80 1 -- 172.21.15.53:0/4020595098 wait complete. 2023-11-06T01:49:12.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.603+0000 7f19f9e00b80 1 -- start start 2023-11-06T01:49:12.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.603+0000 7f19f9e00b80 1 Processor -- start 2023-11-06T01:49:12.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.603+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b43f0 0x55d8472bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.603+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b50f0 0x55d8472bde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.603+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.604+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8471a1510 con 0x55d8472b5a70 2023-11-06T01:49:12.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.604+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8471a11b0 con 0x55d8472b50f0 2023-11-06T01:49:12.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.604+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d847088920 con 0x55d8472b43f0 2023-11-06T01:49:12.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.604+0000 7f19f9e00b80 10 client.0 _ll_register_callbacks cb 0x55d8470eea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:49:12.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.604+0000 7f19e7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.604+0000 7f19e7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472c4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44070/0 (socket says 172.21.15.53:44070) 2023-11-06T01:49:12.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.604+0000 7f19e7fff700 1 -- 172.21.15.53:0/752426821 learned_addr learned my addr 172.21.15.53:0/752426821 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:12.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.604+0000 7f19e77fe700 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b43f0 0x55d8472bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.605+0000 7f19e7fff700 1 -- 172.21.15.53:0/752426821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b43f0 msgr2=0x55d8472bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:12.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.605+0000 7f19e7fff700 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b43f0 0x55d8472bd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.605+0000 7f19e7fff700 1 -- 172.21.15.53:0/752426821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b50f0 msgr2=0x55d8472bde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:12.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.605+0000 7f19e7fff700 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b50f0 0x55d8472bde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.605+0000 7f19e7fff700 1 -- 172.21.15.53:0/752426821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d0009ae0 con 0x55d8472b5a70 2023-11-06T01:49:12.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.605+0000 7f19e77fe700 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b43f0 0x55d8472bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:49:12.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.605+0000 7f19e7fff700 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472c4eb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f19e0015000 tx=0x7f19e0003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bb4de735035c02 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:12.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.605+0000 7f19c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:49:12.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.605+0000 7f19c6ffd700 1 -- 172.21.15.53:0/752426821 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e000e040 con 0x55d8472b5a70 2023-11-06T01:49:12.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.605+0000 7f19c6ffd700 1 -- 172.21.15.53:0/752426821 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f19e0021030 con 0x55d8472b5a70 2023-11-06T01:49:12.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.605+0000 7f19f9e00b80 1 -- 172.21.15.53:0/752426821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d84722fd10 con 0x55d8472b5a70 2023-11-06T01:49:12.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19c6ffd700 1 -- 172.21.15.53:0/752426821 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19e001c850 con 0x55d8472b5a70 2023-11-06T01:49:12.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19f9e00b80 10 client.17079 fetch_fsmap learned FSMap version 49 2023-11-06T01:49:12.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19f9e00b80 1 -- 172.21.15.53:0/752426821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d84701bc60 con 0x55d8472b5a70 2023-11-06T01:49:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19c6ffd700 1 -- 172.21.15.53:0/752426821 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e0037020 con 0x55d8472b5a70 2023-11-06T01:49:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19c6ffd700 1 -- 172.21.15.53:0/752426821 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f19e00289c0 con 0x55d8472b5a70 2023-11-06T01:49:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19f9e00b80 10 client.17079 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:49:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19f9e00b80 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:49:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19f9e00b80 20 client.17079 populate_metadata read hostname 'smithi053' 2023-11-06T01:49:12.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d8472c9720 0x55d8472cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19f9e00b80 4 client.17079 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:49:12.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19f9e00b80 1 -- 172.21.15.53:0/752426821 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d847282e10 con 0x55d8472c9720 2023-11-06T01:49:12.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.606+0000 7f19e77fe700 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d8472c9720 0x55d8472cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.607+0000 7f19c6ffd700 10 client.17079 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:12.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.607+0000 7f19e77fe700 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d8472c9720 0x55d8472cbb40 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff627967b69b2829 server_cookie=334f2c62d89a358c in_seq=0 out_seq=0 2023-11-06T01:49:12.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.607+0000 7f19c6ffd700 1 -- 172.21.15.53:0/752426821 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d847282e10 con 0x55d8472c9720 2023-11-06T01:49:12.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.607+0000 7f19c6ffd700 10 client.17079 handle_command_reply: tid=0 2023-11-06T01:49:12.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.607+0000 7f19f9e00b80 1 client.17079 shutdown 2023-11-06T01:49:12.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19f9e00b80 1 -- 172.21.15.53:0/752426821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 msgr2=0x55d8472c4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:12.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472c4eb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f19e0015000 tx=0x7f19e0003e40 comp rx=0 tx=0).stop 2023-11-06T01:49:12.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19f9e00b80 1 -- 172.21.15.53:0/752426821 shutdown_connections 2023-11-06T01:49:12.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d8472c9720 0x55d8472cbb40 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472c4eb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b43f0 0x55d8472bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/752426821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b50f0 0x55d8472bde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19f9e00b80 1 -- 172.21.15.53:0/752426821 >> 172.21.15.53:0/752426821 conn(0x55d8472130d0 msgr2=0x55d8472ae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:12.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19c6ffd700 0 client.17079 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:12.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19f9e00b80 1 -- 172.21.15.53:0/752426821 shutdown_connections 2023-11-06T01:49:12.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19f9e00b80 1 -- 172.21.15.53:0/752426821 wait complete. 2023-11-06T01:49:12.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.608+0000 7f19f9e00b80 20 client.17079 trim_cache size 0 max 16384 2023-11-06T01:49:12.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19f9e00b80 1 Processor -- start 2023-11-06T01:49:12.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19f9e00b80 1 -- start start 2023-11-06T01:49:12.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b43f0 0x55d8472b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b50f0 0x55d8472c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8472bb560 con 0x55d8472b5a70 2023-11-06T01:49:12.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8472c9420 con 0x55d8472b50f0 2023-11-06T01:49:12.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8472aed20 con 0x55d8472b43f0 2023-11-06T01:49:12.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19e7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19dffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b50f0 0x55d8472c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19e77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b43f0 0x55d8472b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19e7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44082/0 (socket says 172.21.15.53:44082) 2023-11-06T01:49:12.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19dffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b50f0 0x55d8472c2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55516/0 (socket says 172.21.15.53:55516) 2023-11-06T01:49:12.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19e7fff700 1 -- 172.21.15.53:0/1523514446 learned_addr learned my addr 172.21.15.53:0/1523514446 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:12.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19e7fff700 1 -- 172.21.15.53:0/1523514446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b43f0 msgr2=0x55d8472b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:12.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19e7fff700 1 --2- 172.21.15.53:0/1523514446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b43f0 0x55d8472b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19e7fff700 1 -- 172.21.15.53:0/1523514446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b50f0 msgr2=0x55d8472c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:12.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19e7fff700 1 --2- 172.21.15.53:0/1523514446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b50f0 0x55d8472c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.628+0000 7f19e7fff700 1 -- 172.21.15.53:0/1523514446 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d847059250 con 0x55d8472b5a70 2023-11-06T01:49:12.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.629+0000 7f19e7fff700 1 --2- 172.21.15.53:0/1523514446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472ae7e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f19e0003fd0 tx=0x7f19e0009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5c612f6e712d03 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:12.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.629+0000 7f19de7fc700 1 -- 172.21.15.53:0/1523514446 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e000e040 con 0x55d8472b5a70 2023-11-06T01:49:12.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.629+0000 7f19de7fc700 1 -- 172.21.15.53:0/1523514446 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e0037040 con 0x55d8472b5a70 2023-11-06T01:49:12.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.629+0000 7f19de7fc700 1 -- 172.21.15.53:0/1523514446 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e001ce00 con 0x55d8472b5a70 2023-11-06T01:49:12.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 -- 172.21.15.53:0/1523514446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 msgr2=0x55d8472ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:12.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/1523514446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472ae7e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f19e0003fd0 tx=0x7f19e0009060 comp rx=0 tx=0).stop 2023-11-06T01:49:12.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 -- 172.21.15.53:0/1523514446 shutdown_connections 2023-11-06T01:49:12.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/1523514446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472ae7e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/1523514446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b43f0 0x55d8472b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/1523514446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b50f0 0x55d8472c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 -- 172.21.15.53:0/1523514446 >> 172.21.15.53:0/1523514446 conn(0x55d8472130d0 msgr2=0x55d8472bc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:12.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 -- 172.21.15.53:0/1523514446 shutdown_connections 2023-11-06T01:49:12.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 -- 172.21.15.53:0/1523514446 wait complete. 2023-11-06T01:49:12.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 -- start start 2023-11-06T01:49:12.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 Processor -- start 2023-11-06T01:49:12.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.630+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b43f0 0x55d8472c1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 0x55d8472c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19f9e00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8472c26a0 con 0x55d8472b5a70 2023-11-06T01:49:12.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8472c2810 con 0x55d8472b43f0 2023-11-06T01:49:12.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19f9e00b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8472c29d0 con 0x55d8472b50f0 2023-11-06T01:49:12.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19f9e00b80 10 client.0 _ll_register_callbacks cb 0x7f19e0003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:49:12.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b43f0 0x55d8472c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19e7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b43f0 0x55d8472c1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55522/0 (socket says 172.21.15.53:55522) 2023-11-06T01:49:12.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19e77fe700 1 -- 172.21.15.53:0/3634641539 learned_addr learned my addr 172.21.15.53:0/3634641539 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:12.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19dffff700 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 0x55d8472c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19e77fe700 1 -- 172.21.15.53:0/3634641539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 msgr2=0x55d8472c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:12.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19e77fe700 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 0x55d8472c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19e77fe700 1 -- 172.21.15.53:0/3634641539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 msgr2=0x55d8472c2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:12.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19e77fe700 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472c2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19e77fe700 1 -- 172.21.15.53:0/3634641539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d84701bc60 con 0x55d8472b43f0 2023-11-06T01:49:12.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19e7fff700 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:49:12.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19e77fe700 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b43f0 0x55d8472c1000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009dd0 tx=0x7f19d0009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de62d9f8de20e194 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:12.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-11-06T01:49:12.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19c6ffd700 1 -- 172.21.15.53:0/3634641539 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0021040 con 0x55d8472b43f0 2023-11-06T01:49:12.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19c6ffd700 1 -- 172.21.15.53:0/3634641539 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f19d0005e30 con 0x55d8472b43f0 2023-11-06T01:49:12.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.631+0000 7f19f9e00b80 1 -- 172.21.15.53:0/3634641539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19d000ee30 con 0x55d8472b43f0 2023-11-06T01:49:12.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.632+0000 7f19c6ffd700 1 -- 172.21.15.53:0/3634641539 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19d0029420 con 0x55d8472b43f0 2023-11-06T01:49:12.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.632+0000 7f19f9e00b80 10 client.26594 fetch_fsmap learned FSMap version 49 2023-11-06T01:49:12.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.632+0000 7f19f9e00b80 1 -- 172.21.15.53:0/3634641539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d8470312b0 con 0x55d8472b43f0 2023-11-06T01:49:12.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.632+0000 7f19c6ffd700 1 -- 172.21.15.53:0/3634641539 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19d000ee30 con 0x55d8472b43f0 2023-11-06T01:49:12.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.632+0000 7f19c6ffd700 1 -- 172.21.15.53:0/3634641539 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f19d0011070 con 0x55d8472b43f0 2023-11-06T01:49:12.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.632+0000 7f19f9e00b80 10 client.26594 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:49:12.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.632+0000 7f19f9e00b80 10 client.26594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-11-06T01:49:12.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.632+0000 7f19f9e00b80 20 client.26594 populate_metadata read hostname 'smithi053' 2023-11-06T01:49:12.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.632+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d84731c2b0 0x55d84731e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:12.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.632+0000 7f19f9e00b80 4 client.26594 mds_command: new command op to 24487 tid=0[{"prefix": "scrub status"}] 2023-11-06T01:49:12.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.632+0000 7f19f9e00b80 1 -- 172.21.15.53:0/3634641539 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d8471fdd40 con 0x55d84731c2b0 2023-11-06T01:49:12.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.634+0000 7f19dffff700 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d84731c2b0 0x55d84731e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:12.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19dffff700 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d84731c2b0 0x55d84731e6d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bd59b279a2618d5 server_cookie=9230d2853dd98c97 in_seq=0 out_seq=0 2023-11-06T01:49:12.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19c6ffd700 10 client.26594 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:12.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19c6ffd700 1 -- 172.21.15.53:0/3634641539 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d8471fdd40 con 0x55d84731c2b0 2023-11-06T01:49:12.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19c6ffd700 10 client.26594 handle_command_reply: tid=0 2023-11-06T01:49:12.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19f9e00b80 1 client.26594 shutdown 2023-11-06T01:49:12.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19f9e00b80 1 -- 172.21.15.53:0/3634641539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b43f0 msgr2=0x55d8472c1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:12.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b43f0 0x55d8472c1000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009dd0 tx=0x7f19d0009340 comp rx=0 tx=0).stop 2023-11-06T01:49:12.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19f9e00b80 1 -- 172.21.15.53:0/3634641539 shutdown_connections 2023-11-06T01:49:12.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8472b43f0 0x55d8472c1000 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55d84731c2b0 0x55d84731e6d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8472b5a70 0x55d8472c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19f9e00b80 1 --2- 172.21.15.53:0/3634641539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8472b50f0 0x55d8472c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19c6ffd700 0 client.26594 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:12.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19f9e00b80 1 -- 172.21.15.53:0/3634641539 >> 172.21.15.53:0/3634641539 conn(0x55d8472130d0 msgr2=0x55d8472b8330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:12.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19f9e00b80 1 -- 172.21.15.53:0/3634641539 shutdown_connections 2023-11-06T01:49:12.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.635+0000 7f19f9e00b80 1 -- 172.21.15.53:0/3634641539 wait complete. 2023-11-06T01:49:12.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19f9e00b80 20 client.26594 trim_cache size 0 max 16384 2023-11-06T01:49:12.650 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:49:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T01:49:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2023-11-06T01:49:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: "320435aa-9dc7-4148-9914-765c10b30e19": { 2023-11-06T01:49:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2023-11-06T01:49:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "320435aa-9dc7-4148-9914-765c10b30e19", 2023-11-06T01:49:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,force" 2023-11-06T01:49:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:12.652 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:49:12.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 -- 172.21.15.53:0/1075827819 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f19d40a0570 msgr2=0x7f19d40a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:12.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f19d40a0570 0x7f19d40a2a30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19e000d010 tx=0x7f19e000b7b0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 -- 172.21.15.53:0/1075827819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e810a700 msgr2=0x7f19e81a5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:12.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e810a700 0x7f19e81a5f00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f19d800b550 tx=0x7f19d800b860 comp rx=0 tx=0).stop 2023-11-06T01:49:12.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 -- 172.21.15.53:0/1075827819 shutdown_connections 2023-11-06T01:49:12.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f19d40a0570 0x7f19d40a2a30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e8107d90 0x7f19e81a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19e810d0d0 0x7f19e81acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 --2- 172.21.15.53:0/1075827819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19e810a700 0x7f19e81a5f00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:12.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 -- 172.21.15.53:0/1075827819 >> 172.21.15.53:0/1075827819 conn(0x7f19e806db40 msgr2=0x7f19e8108a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:12.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 -- 172.21.15.53:0/1075827819 shutdown_connections 2023-11-06T01:49:12.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:12.636+0000 7f19edc70700 1 -- 172.21.15.53:0/1075827819 wait complete. 2023-11-06T01:49:12.656 INFO:tasks.cephfs.filesystem:scrub status for tag:320435aa-9dc7-4148-9914-765c10b30e19 - {'path': '/', 'tag': '320435aa-9dc7-4148-9914-765c10b30e19', 'options': 'recursive,force'} 2023-11-06T01:49:12.656 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa/tasks/cephfs/filesystem.py", line 1681, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-11-06T01:49:12.841 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:12 smithi053 ceph-mon[99472]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 737 op/s 2023-11-06T01:49:12.895 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:12 smithi062 ceph-mon[103924]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 737 op/s 2023-11-06T01:49:12.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:12 smithi163 ceph-mon[104416]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 737 op/s 2023-11-06T01:49:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.047+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/515698086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ac1008e0 msgr2=0x7fa0ac108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.047+0000 7fa0b1bf6700 1 --2- 172.21.15.53:0/515698086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ac1008e0 0x7fa0ac108180 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400b0a0 tx=0x7fa0a400b3b0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.048+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/515698086 shutdown_connections 2023-11-06T01:49:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.048+0000 7fa0b1bf6700 1 --2- 172.21.15.53:0/515698086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0ac101360 0x7fa0ac101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.048+0000 7fa0b1bf6700 1 --2- 172.21.15.53:0/515698086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ac1008e0 0x7fa0ac108180 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.048+0000 7fa0b1bf6700 1 --2- 172.21.15.53:0/515698086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ac0fff20 0x7fa0ac1003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.048+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/515698086 >> 172.21.15.53:0/515698086 conn(0x7fa0ac0fa9f0 msgr2=0x7fa0ac0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.048+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/515698086 shutdown_connections 2023-11-06T01:49:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.048+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/515698086 wait complete. 2023-11-06T01:49:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.048+0000 7fa0b1bf6700 1 Processor -- start 2023-11-06T01:49:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.048+0000 7fa0b1bf6700 1 -- start start 2023-11-06T01:49:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.049+0000 7fa0b1bf6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0ac0fff20 0x7fa0ac196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.049+0000 7fa0b1bf6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ac1008e0 0x7fa0ac197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.049+0000 7fa0b1bf6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ac101360 0x7fa0ac197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.049+0000 7fa0b1bf6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac198670 con 0x7fa0ac101360 2023-11-06T01:49:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.049+0000 7fa0b1bf6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac1987e0 con 0x7fa0ac0fff20 2023-11-06T01:49:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.049+0000 7fa0b1bf6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac1989a0 con 0x7fa0ac1008e0 2023-11-06T01:49:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.049+0000 7fa0abfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ac101360 0x7fa0ac197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.049+0000 7fa0ab7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0ac0fff20 0x7fa0ac196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.049+0000 7fa0abfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ac101360 0x7fa0ac197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44098/0 (socket says 172.21.15.53:44098) 2023-11-06T01:49:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.049+0000 7fa0abfff700 1 -- 172.21.15.53:0/3445686407 learned_addr learned my addr 172.21.15.53:0/3445686407 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.049+0000 7fa0aaffd700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ac1008e0 0x7fa0ac197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.050+0000 7fa0ab7fe700 1 -- 172.21.15.53:0/3445686407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ac1008e0 msgr2=0x7fa0ac197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.050+0000 7fa0ab7fe700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ac1008e0 0x7fa0ac197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.050+0000 7fa0ab7fe700 1 -- 172.21.15.53:0/3445686407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ac101360 msgr2=0x7fa0ac197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.050+0000 7fa0ab7fe700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ac101360 0x7fa0ac197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.050+0000 7fa0ab7fe700 1 -- 172.21.15.53:0/3445686407 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a4009d00 con 0x7fa0ac0fff20 2023-11-06T01:49:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.050+0000 7fa0ab7fe700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0ac0fff20 0x7fa0ac196cf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00e9f0 tx=0x7fa09c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.050+0000 7fa0a8ff9700 1 -- 172.21.15.53:0/3445686407 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00c9c0 con 0x7fa0ac0fff20 2023-11-06T01:49:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.051+0000 7fa0a8ff9700 1 -- 172.21.15.53:0/3445686407 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa09c010430 con 0x7fa0ac0fff20 2023-11-06T01:49:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.051+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/3445686407 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac198c80 con 0x7fa0ac0fff20 2023-11-06T01:49:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.051+0000 7fa0a8ff9700 1 -- 172.21.15.53:0/3445686407 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c018680 con 0x7fa0ac0fff20 2023-11-06T01:49:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.051+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/3445686407 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac1025f0 con 0x7fa0ac0fff20 2023-11-06T01:49:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.053+0000 7fa0a8ff9700 1 -- 172.21.15.53:0/3445686407 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa09c010860 con 0x7fa0ac0fff20 2023-11-06T01:49:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.054+0000 7fa0a8ff9700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0940a0670 0x7fa0940a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.055+0000 7fa0a8ff9700 1 -- 172.21.15.53:0/3445686407 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa09c014070 con 0x7fa0ac0fff20 2023-11-06T01:49:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.055+0000 7fa0aaffd700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0940a0670 0x7fa0940a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.055+0000 7fa0aaffd700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0940a0670 0x7fa0940a2b30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009c60 tx=0x7fa0a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0bdd86b80 1 Processor -- start 2023-11-06T01:49:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0bdd86b80 1 -- start start 2023-11-06T01:49:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f5edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f5ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5eeed0 0x55644f5ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f5459b0 con 0x55644f5edfe0 2023-11-06T01:49:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f543370 con 0x55644f5eeed0 2023-11-06T01:49:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f53ec60 con 0x55644f5ed680 2023-11-06T01:49:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0aaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f5ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0ab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f5edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0aaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f5ee8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44108/0 (socket says 172.21.15.53:44108) 2023-11-06T01:49:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0aaffd700 1 -- 172.21.15.53:0/914411283 learned_addr learned my addr 172.21.15.53:0/914411283 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0abfff700 1 --2- 172.21.15.53:0/914411283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5eeed0 0x55644f5ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0aaffd700 1 -- 172.21.15.53:0/914411283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 msgr2=0x55644f5edaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0aaffd700 1 --2- 172.21.15.53:0/914411283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f5edaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0aaffd700 1 -- 172.21.15.53:0/914411283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5eeed0 msgr2=0x55644f5ef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0aaffd700 1 --2- 172.21.15.53:0/914411283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5eeed0 0x55644f5ef350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0aaffd700 1 -- 172.21.15.53:0/914411283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55644f36cd80 con 0x55644f5edfe0 2023-11-06T01:49:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.056+0000 7fa0aaffd700 1 --2- 172.21.15.53:0/914411283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f5ee8f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000e4e0 tx=0x7fa0a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f83e9aabcb5856a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.057+0000 7fa091ffb700 1 -- 172.21.15.53:0/914411283 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000ee00 con 0x55644f5edfe0 2023-11-06T01:49:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.057+0000 7fa091ffb700 1 -- 172.21.15.53:0/914411283 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0a0021070 con 0x55644f5edfe0 2023-11-06T01:49:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.057+0000 7fa091ffb700 1 -- 172.21.15.53:0/914411283 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a001d7b0 con 0x55644f5edfe0 2023-11-06T01:49:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.057+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/914411283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 msgr2=0x55644f5ee8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.057+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/914411283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f5ee8f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000e4e0 tx=0x7fa0a0014000 comp rx=0 tx=0).stop 2023-11-06T01:49:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.057+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/914411283 shutdown_connections 2023-11-06T01:49:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.057+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/914411283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5eeed0 0x55644f5ef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.057+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/914411283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f5edaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.057+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/914411283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f5ee8f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.057+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/914411283 >> 172.21.15.53:0/914411283 conn(0x55644f54d070 msgr2=0x55644f5ea040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/914411283 shutdown_connections 2023-11-06T01:49:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/914411283 wait complete. 2023-11-06T01:49:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0bdd86b80 1 -- start start 2023-11-06T01:49:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0bdd86b80 1 Processor -- start 2023-11-06T01:49:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5ed680 0x55644f5f7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5edfe0 0x55644f5f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5eeed0 0x55644f5fe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f4db4b0 con 0x55644f5eeed0 2023-11-06T01:49:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f4db150 con 0x55644f5ed680 2023-11-06T01:49:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f3c28c0 con 0x55644f5edfe0 2023-11-06T01:49:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0bdd86b80 10 client.0 _ll_register_callbacks cb 0x55644f428a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:49:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.058+0000 7fa0abfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5eeed0 0x55644f5fe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.059+0000 7fa0ab7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5ed680 0x55644f5f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.059+0000 7fa0abfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5eeed0 0x55644f5fe9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44116/0 (socket says 172.21.15.53:44116) 2023-11-06T01:49:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.059+0000 7fa0abfff700 1 -- 172.21.15.53:0/2860366839 learned_addr learned my addr 172.21.15.53:0/2860366839 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.059+0000 7fa0aaffd700 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5edfe0 0x55644f5f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.059+0000 7fa0abfff700 1 -- 172.21.15.53:0/2860366839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5edfe0 msgr2=0x55644f5f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.059+0000 7fa0abfff700 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5edfe0 0x55644f5f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.059+0000 7fa0abfff700 1 -- 172.21.15.53:0/2860366839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5ed680 msgr2=0x55644f5f7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.059+0000 7fa0abfff700 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5ed680 0x55644f5f7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.059+0000 7fa0abfff700 1 -- 172.21.15.53:0/2860366839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a0017070 con 0x55644f5eeed0 2023-11-06T01:49:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.059+0000 7fa0abfff700 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5eeed0 0x55644f5fe9f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400b070 tx=0x7fa0a40095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8e31f04ea9e795 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.059+0000 7fa08affd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:49:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.061+0000 7fa08affd700 1 -- 172.21.15.53:0/2860366839 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a400e040 con 0x55644f5eeed0 2023-11-06T01:49:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.061+0000 7fa08affd700 1 -- 172.21.15.53:0/2860366839 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa0a4003c00 con 0x55644f5eeed0 2023-11-06T01:49:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.061+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/2860366839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55644f569cb0 con 0x55644f5eeed0 2023-11-06T01:49:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.061+0000 7fa08affd700 1 -- 172.21.15.53:0/2860366839 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0a4003dc0 con 0x55644f5eeed0 2023-11-06T01:49:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.061+0000 7fa0bdd86b80 10 client.17097 fetch_fsmap learned FSMap version 49 2023-11-06T01:49:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.061+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/2860366839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55644f355c00 con 0x55644f5eeed0 2023-11-06T01:49:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.062+0000 7fa08affd700 1 -- 172.21.15.53:0/2860366839 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55644f569cb0 con 0x55644f5eeed0 2023-11-06T01:49:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.062+0000 7fa08affd700 1 -- 172.21.15.53:0/2860366839 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa0a4034020 con 0x55644f5eeed0 2023-11-06T01:49:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.062+0000 7fa0bdd86b80 10 client.17097 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:49:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.062+0000 7fa0bdd86b80 10 client.17097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:49:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.062+0000 7fa0bdd86b80 20 client.17097 populate_metadata read hostname 'smithi053' 2023-11-06T01:49:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.062+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55644f603290 0x55644f6056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.062+0000 7fa0bdd86b80 4 client.17097 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T01:49:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.062+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/2860366839 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55644f5bcdb0 con 0x55644f603290 2023-11-06T01:49:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.063+0000 7fa0ab7fe700 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55644f603290 0x55644f6056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.063+0000 7fa08affd700 10 client.17097 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.063+0000 7fa0ab7fe700 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55644f603290 0x55644f6056b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dab180f00c6fe32 server_cookie=473be73cbea61ca7 in_seq=0 out_seq=0 2023-11-06T01:49:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa08affd700 1 -- 172.21.15.53:0/2860366839 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55644f5bcdb0 con 0x55644f603290 2023-11-06T01:49:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa08affd700 10 client.17097 handle_command_reply: tid=0 2023-11-06T01:49:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa0bdd86b80 1 client.17097 shutdown 2023-11-06T01:49:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/2860366839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5eeed0 msgr2=0x55644f5fe9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5eeed0 0x55644f5fe9f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400b070 tx=0x7fa0a40095d0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/2860366839 shutdown_connections 2023-11-06T01:49:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5ed680 0x55644f5f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55644f603290 0x55644f6056b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5eeed0 0x55644f5fe9f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/2860366839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5edfe0 0x55644f5f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa08affd700 0 client.17097 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/2860366839 >> 172.21.15.53:0/2860366839 conn(0x55644f54d070 msgr2=0x55644f5e83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/2860366839 shutdown_connections 2023-11-06T01:49:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.064+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/2860366839 wait complete. 2023-11-06T01:49:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.065+0000 7fa0bdd86b80 20 client.17097 trim_cache size 0 max 16384 2023-11-06T01:49:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.083+0000 7fa0bdd86b80 1 Processor -- start 2023-11-06T01:49:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.084+0000 7fa0bdd86b80 1 -- start start 2023-11-06T01:49:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.085+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f5fc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.085+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5edfe0 0x55644f5f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.085+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5eeed0 0x55644f5f7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.085+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f5e07a0 con 0x55644f5eeed0 2023-11-06T01:49:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.085+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f5f6920 con 0x55644f5edfe0 2023-11-06T01:49:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.085+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f5e3500 con 0x55644f5ed680 2023-11-06T01:49:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.085+0000 7fa0ab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f5fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.085+0000 7fa0ab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f5fc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58360/0 (socket says 172.21.15.53:58360) 2023-11-06T01:49:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.085+0000 7fa0ab7fe700 1 -- 172.21.15.53:0/731578303 learned_addr learned my addr 172.21.15.53:0/731578303 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.086+0000 7fa0ab7fe700 1 -- 172.21.15.53:0/731578303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5edfe0 msgr2=0x55644f5f20a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T01:49:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.086+0000 7fa0ab7fe700 1 --2- 172.21.15.53:0/731578303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5edfe0 0x55644f5f20a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.086+0000 7fa0ab7fe700 1 -- 172.21.15.53:0/731578303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5eeed0 msgr2=0x55644f5f7470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T01:49:14.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.086+0000 7fa0ab7fe700 1 --2- 172.21.15.53:0/731578303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5eeed0 0x55644f5f7470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.087 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.086+0000 7fa0ab7fe700 1 -- 172.21.15.53:0/731578303 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55644f3931f0 con 0x55644f5ed680 2023-11-06T01:49:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.095+0000 7fa0ab7fe700 1 --2- 172.21.15.53:0/731578303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f5fc970 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa09c095770 tx=0x7fa09c0cb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d29b1fe5422d115c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.095+0000 7fa0927fc700 1 -- 172.21.15.53:0/731578303 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c0b7260 con 0x55644f5ed680 2023-11-06T01:49:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.095+0000 7fa0927fc700 1 -- 172.21.15.53:0/731578303 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa09c09d0a0 con 0x55644f5ed680 2023-11-06T01:49:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.095+0000 7fa0927fc700 1 -- 172.21.15.53:0/731578303 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c0989c0 con 0x55644f5ed680 2023-11-06T01:49:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.096+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/731578303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 msgr2=0x55644f5fc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.096+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/731578303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f5fc970 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa09c095770 tx=0x7fa09c0cb040 comp rx=0 tx=0).stop 2023-11-06T01:49:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.099+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/731578303 shutdown_connections 2023-11-06T01:49:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.099+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/731578303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5eeed0 0x55644f5f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.099+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/731578303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f5fc970 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.099+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/731578303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5edfe0 0x55644f5f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.099+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/731578303 >> 172.21.15.53:0/731578303 conn(0x55644f54d070 msgr2=0x55644f5f6140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.120+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/731578303 shutdown_connections 2023-11-06T01:49:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.125+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/731578303 wait complete. 2023-11-06T01:49:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.125+0000 7fa0bdd86b80 1 -- start start 2023-11-06T01:49:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.126+0000 7fa0bdd86b80 1 Processor -- start 2023-11-06T01:49:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.126+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f64cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.126+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f64d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.126+0000 7fa0bdd86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5eeed0 0x55644f6540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.126+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f654600 con 0x55644f5edfe0 2023-11-06T01:49:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.126+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f654740 con 0x55644f5eeed0 2023-11-06T01:49:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.126+0000 7fa0bdd86b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644f654900 con 0x55644f5ed680 2023-11-06T01:49:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.126+0000 7fa0bdd86b80 10 client.0 _ll_register_callbacks cb 0x7fa0a4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T01:49:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.127+0000 7fa0aaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f64d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.127+0000 7fa0aaffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f64d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44126/0 (socket says 172.21.15.53:44126) 2023-11-06T01:49:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.127+0000 7fa0aaffd700 1 -- 172.21.15.53:0/3850776242 learned_addr learned my addr 172.21.15.53:0/3850776242 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.127+0000 7fa0abfff700 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5eeed0 0x55644f6540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.127+0000 7fa0aaffd700 1 -- 172.21.15.53:0/3850776242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 msgr2=0x55644f64cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.127+0000 7fa0aaffd700 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f64cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.127+0000 7fa0aaffd700 1 -- 172.21.15.53:0/3850776242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5eeed0 msgr2=0x55644f6540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T01:49:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.127+0000 7fa0aaffd700 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5eeed0 0x55644f6540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.127+0000 7fa0aaffd700 1 -- 172.21.15.53:0/3850776242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac1025f0 con 0x55644f5edfe0 2023-11-06T01:49:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.127+0000 7fa0aaffd700 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f64d4b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000e480 tx=0x7fa0a000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b3a5c4d3c42f75 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.127+0000 7fa08affd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-06T01:49:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa08affd700 1 -- 172.21.15.53:0/3850776242 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0005250 con 0x55644f5edfe0 2023-11-06T01:49:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa08affd700 1 -- 172.21.15.53:0/3850776242 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa0a00053b0 con 0x55644f5edfe0 2023-11-06T01:49:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/3850776242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0a0021070 con 0x55644f5edfe0 2023-11-06T01:49:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa08affd700 1 -- 172.21.15.53:0/3850776242 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0a0005570 con 0x55644f5edfe0 2023-11-06T01:49:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa0bdd86b80 10 client.17100 fetch_fsmap learned FSMap version 49 2023-11-06T01:49:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/3850776242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55644f36b250 con 0x55644f5edfe0 2023-11-06T01:49:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa08affd700 1 -- 172.21.15.53:0/3850776242 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0a003e080 con 0x55644f5edfe0 2023-11-06T01:49:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa08affd700 1 -- 172.21.15.53:0/3850776242 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 49) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa0a0038030 con 0x55644f5edfe0 2023-11-06T01:49:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa0bdd86b80 10 client.17100 fetch_fsmap finished waiting for FSMap version 49 2023-11-06T01:49:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa0bdd86b80 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-11-06T01:49:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa0bdd86b80 20 client.17100 populate_metadata read hostname 'smithi053' 2023-11-06T01:49:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55644f6782d0 0x55644f67a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa0bdd86b80 4 client.17100 mds_command: new command op to 24487 tid=0[{"prefix": "flush journal"}] 2023-11-06T01:49:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.134+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/3850776242 --> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55644f537ce0 con 0x55644f6782d0 2023-11-06T01:49:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.135+0000 7fa0ab7fe700 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55644f6782d0 0x55644f67a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.137+0000 7fa0ab7fe700 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55644f6782d0 0x55644f67a6f0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb3a9dfcfdf2915 server_cookie=94a5a73abc77ec13 in_seq=0 out_seq=0 2023-11-06T01:49:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.137+0000 7fa08affd700 10 client.17100 ms_handle_connect on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa08affd700 1 -- 172.21.15.53:0/3850776242 <== mds.0 v2:172.21.15.163:6838/3554773881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55644f537ce0 con 0x55644f6782d0 2023-11-06T01:49:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa08affd700 10 client.17100 handle_command_reply: tid=0 2023-11-06T01:49:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 1 client.17100 shutdown 2023-11-06T01:49:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/3850776242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 msgr2=0x55644f64d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f64d4b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000e480 tx=0x7fa0a000ec90 comp rx=0 tx=0).stop 2023-11-06T01:49:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/3850776242 shutdown_connections 2023-11-06T01:49:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55644f5eeed0 0x55644f6540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.163:6838/3554773881,v1:172.21.15.163:6839/3554773881] conn(0x55644f6782d0 0x55644f67a6f0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55644f5ed680 0x55644f64cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa08affd700 0 client.17100 ms_handle_reset on v2:172.21.15.163:6838/3554773881 2023-11-06T01:49:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 1 --2- 172.21.15.53:0/3850776242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55644f5edfe0 0x55644f64d4b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/3850776242 >> 172.21.15.53:0/3850776242 conn(0x55644f54d070 msgr2=0x55644f5f20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/3850776242 shutdown_connections 2023-11-06T01:49:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 1 -- 172.21.15.53:0/3850776242 wait complete. 2023-11-06T01:49:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.149+0000 7fa0bdd86b80 20 client.17100 trim_cache size 0 max 16384 2023-11-06T01:49:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2023-11-06T01:49:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2023-11-06T01:49:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2023-11-06T01:49:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2023-11-06T01:49:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/3445686407 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0940a0670 msgr2=0x7fa0940a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0940a0670 0x7fa0940a2b30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009c60 tx=0x7fa0a0009260 comp rx=0 tx=0).stop 2023-11-06T01:49:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/3445686407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0ac0fff20 msgr2=0x7fa0ac196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0ac0fff20 0x7fa0ac196cf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00e9f0 tx=0x7fa09c00ed00 comp rx=0 tx=0).stop 2023-11-06T01:49:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/3445686407 shutdown_connections 2023-11-06T01:49:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fa0940a0670 0x7fa0940a2b30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0ac0fff20 0x7fa0ac196cf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ac101360 0x7fa0ac197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 --2- 172.21.15.53:0/3445686407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ac1008e0 0x7fa0ac197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/3445686407 >> 172.21.15.53:0/3445686407 conn(0x7fa0ac0fa9f0 msgr2=0x7fa0ac0fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/3445686407 shutdown_connections 2023-11-06T01:49:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2023-11-06T01:49:14.150+0000 7fa0b1bf6700 1 -- 172.21.15.53:0/3445686407 wait complete. 2023-11-06T01:49:15.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:15 smithi062 ceph-mon[103924]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 749 op/s 2023-11-06T01:49:15.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:15 smithi163 ceph-mon[104416]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 749 op/s 2023-11-06T01:49:15.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:15 smithi053 ceph-mon[99472]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 749 op/s 2023-11-06T01:49:17.183 INFO:teuthology.orchestra.run.smithi053.stderr:progress: 900.0 s, 1743.8 tps, lat 57.692 ms stddev 69.205 2023-11-06T01:49:17.392 INFO:teuthology.orchestra.run.smithi053.stdout:transaction type: 2023-11-06T01:49:17.392 INFO:teuthology.orchestra.run.smithi053.stdout:scaling factor: 32 2023-11-06T01:49:17.392 INFO:teuthology.orchestra.run.smithi053.stdout:query mode: simple 2023-11-06T01:49:17.392 INFO:teuthology.orchestra.run.smithi053.stdout:number of clients: 100 2023-11-06T01:49:17.393 INFO:teuthology.orchestra.run.smithi053.stdout:number of threads: 4 2023-11-06T01:49:17.393 INFO:teuthology.orchestra.run.smithi053.stdout:duration: 900 s 2023-11-06T01:49:17.393 INFO:teuthology.orchestra.run.smithi053.stdout:number of transactions actually processed: 1914319 2023-11-06T01:49:17.393 INFO:teuthology.orchestra.run.smithi053.stdout:latency average = 47.015 ms 2023-11-06T01:49:17.393 INFO:teuthology.orchestra.run.smithi053.stdout:latency stddev = 65.614 ms 2023-11-06T01:49:17.394 INFO:teuthology.orchestra.run.smithi053.stdout:tps = 2126.519709 (including connections establishing) 2023-11-06T01:49:17.394 INFO:teuthology.orchestra.run.smithi053.stdout:tps = 2126.524858 (excluding connections establishing) 2023-11-06T01:49:17.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-11-06T01:49:17.430 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:17 smithi053 ceph-mon[99472]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 787 op/s 2023-11-06T01:49:17.493 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:17 smithi163 ceph-mon[104416]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 787 op/s 2023-11-06T01:49:17.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:17 smithi062 ceph-mon[103924]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 787 op/s 2023-11-06T01:49:18.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:18 smithi053 ceph-mon[99472]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 769 op/s 2023-11-06T01:49:18.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:18 smithi062 ceph-mon[103924]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 769 op/s 2023-11-06T01:49:18.738 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:18 smithi163 ceph-mon[104416]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 769 op/s 2023-11-06T01:49:21.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:21 smithi163 ceph-mon[104416]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 745 op/s 2023-11-06T01:49:21.594 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:21 smithi053 ceph-mon[99472]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 745 op/s 2023-11-06T01:49:21.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:21 smithi062 ceph-mon[103924]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 745 op/s 2023-11-06T01:49:22.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:22 smithi163 ceph-mon[104416]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 810 op/s 2023-11-06T01:49:23.097 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:22 smithi053 ceph-mon[99472]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 810 op/s 2023-11-06T01:49:23.152 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:22 smithi062 ceph-mon[103924]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 810 op/s 2023-11-06T01:49:25.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:25 smithi163 ceph-mon[104416]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 920 op/s 2023-11-06T01:49:25.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:25 smithi053 ceph-mon[99472]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 920 op/s 2023-11-06T01:49:25.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:25 smithi062 ceph-mon[103924]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 920 op/s 2023-11-06T01:49:26.141 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-11-06T01:49:26.276 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/: 2023-11-06T01:49:26.276 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.276 INFO:teuthology.orchestra.run.smithi053.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:33 . 2023-11-06T01:49:26.276 INFO:teuthology.orchestra.run.smithi053.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:33 .. 2023-11-06T01:49:26.277 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 data 2023-11-06T01:49:26.277 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.277 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data: 2023-11-06T01:49:26.277 INFO:teuthology.orchestra.run.smithi053.stdout:total 31 2023-11-06T01:49:26.277 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 . 2023-11-06T01:49:26.277 INFO:teuthology.orchestra.run.smithi053.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:33 .. 2023-11-06T01:49:26.278 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Nov 6 01:34 base 2023-11-06T01:49:26.278 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Nov 6 01:34 current_logfiles 2023-11-06T01:49:26.278 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Nov 6 01:34 global 2023-11-06T01:49:26.278 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 log 2023-11-06T01:49:26.279 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_commit_ts 2023-11-06T01:49:26.279 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_dynshmem 2023-11-06T01:49:26.279 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Nov 6 01:34 pg_hba.conf 2023-11-06T01:49:26.279 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Nov 6 01:34 pg_ident.conf 2023-11-06T01:49:26.279 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:49 pg_logical 2023-11-06T01:49:26.280 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:34 pg_multixact 2023-11-06T01:49:26.280 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 pg_notify 2023-11-06T01:49:26.280 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_replslot 2023-11-06T01:49:26.280 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_serial 2023-11-06T01:49:26.280 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_snapshots 2023-11-06T01:49:26.280 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:49 pg_stat 2023-11-06T01:49:26.281 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:49 pg_stat_tmp 2023-11-06T01:49:26.281 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:49 pg_subtrans 2023-11-06T01:49:26.281 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_tblspc 2023-11-06T01:49:26.281 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pg_twophase 2023-11-06T01:49:26.281 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 PG_VERSION 2023-11-06T01:49:26.282 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Nov 6 01:49 pg_wal 2023-11-06T01:49:26.282 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:40 pg_xact 2023-11-06T01:49:26.282 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Nov 6 01:34 postgresql.auto.conf 2023-11-06T01:49:26.282 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Nov 6 01:34 postgresql.conf 2023-11-06T01:49:26.282 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Nov 6 01:34 postmaster.opts 2023-11-06T01:49:26.283 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.283 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/base: 2023-11-06T01:49:26.283 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.283 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Nov 6 01:34 . 2023-11-06T01:49:26.283 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.284 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Nov 6 01:34 1 2023-11-06T01:49:26.284 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Nov 6 01:34 12346 2023-11-06T01:49:26.284 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Nov 6 01:34 12347 2023-11-06T01:49:26.284 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 pgsql_tmp 2023-11-06T01:49:26.284 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.285 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/base/1: 2023-11-06T01:49:26.285 INFO:teuthology.orchestra.run.smithi053.stdout:total 7193 2023-11-06T01:49:26.285 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Nov 6 01:34 . 2023-11-06T01:49:26.285 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Nov 6 01:34 .. 2023-11-06T01:49:26.285 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 112 2023-11-06T01:49:26.285 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 113 2023-11-06T01:49:26.286 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Nov 6 01:34 12182 2023-11-06T01:49:26.286 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12182_fsm 2023-11-06T01:49:26.286 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12182_vm 2023-11-06T01:49:26.286 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12184 2023-11-06T01:49:26.286 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12186 2023-11-06T01:49:26.287 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187 2023-11-06T01:49:26.287 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12187_fsm 2023-11-06T01:49:26.287 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187_vm 2023-11-06T01:49:26.287 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12189 2023-11-06T01:49:26.287 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12191 2023-11-06T01:49:26.288 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192 2023-11-06T01:49:26.288 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12192_fsm 2023-11-06T01:49:26.288 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192_vm 2023-11-06T01:49:26.288 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12194 2023-11-06T01:49:26.288 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12196 2023-11-06T01:49:26.289 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197 2023-11-06T01:49:26.289 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12197_fsm 2023-11-06T01:49:26.289 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197_vm 2023-11-06T01:49:26.289 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12199 2023-11-06T01:49:26.289 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12201 2023-11-06T01:49:26.290 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202 2023-11-06T01:49:26.290 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12202_fsm 2023-11-06T01:49:26.290 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202_vm 2023-11-06T01:49:26.290 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12204 2023-11-06T01:49:26.290 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12206 2023-11-06T01:49:26.290 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207 2023-11-06T01:49:26.291 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12207_fsm 2023-11-06T01:49:26.291 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207_vm 2023-11-06T01:49:26.291 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12209 2023-11-06T01:49:26.291 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12211 2023-11-06T01:49:26.291 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12212 2023-11-06T01:49:26.292 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12214 2023-11-06T01:49:26.292 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12216 2023-11-06T01:49:26.292 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Nov 6 01:34 1247 2023-11-06T01:49:26.292 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1247_fsm 2023-11-06T01:49:26.292 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1247_vm 2023-11-06T01:49:26.293 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Nov 6 01:34 1249 2023-11-06T01:49:26.293 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1249_fsm 2023-11-06T01:49:26.293 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1249_vm 2023-11-06T01:49:26.293 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Nov 6 01:34 1255 2023-11-06T01:49:26.293 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1255_fsm 2023-11-06T01:49:26.293 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1255_vm 2023-11-06T01:49:26.294 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Nov 6 01:34 1259 2023-11-06T01:49:26.294 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1259_fsm 2023-11-06T01:49:26.294 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1259_vm 2023-11-06T01:49:26.294 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417 2023-11-06T01:49:26.294 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417_vm 2023-11-06T01:49:26.295 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418 2023-11-06T01:49:26.295 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418_vm 2023-11-06T01:49:26.295 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 174 2023-11-06T01:49:26.295 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 175 2023-11-06T01:49:26.295 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2187 2023-11-06T01:49:26.296 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224 2023-11-06T01:49:26.296 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224_vm 2023-11-06T01:49:26.296 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328 2023-11-06T01:49:26.296 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328_vm 2023-11-06T01:49:26.297 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336 2023-11-06T01:49:26.297 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336_vm 2023-11-06T01:49:26.297 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2337 2023-11-06T01:49:26.297 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2600 2023-11-06T01:49:26.297 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2600_fsm 2023-11-06T01:49:26.298 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2600_vm 2023-11-06T01:49:26.298 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601 2023-11-06T01:49:26.298 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2601_fsm 2023-11-06T01:49:26.298 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601_vm 2023-11-06T01:49:26.298 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2602 2023-11-06T01:49:26.298 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2602_fsm 2023-11-06T01:49:26.299 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2602_vm 2023-11-06T01:49:26.299 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2603 2023-11-06T01:49:26.299 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2603_fsm 2023-11-06T01:49:26.299 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2603_vm 2023-11-06T01:49:26.299 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604 2023-11-06T01:49:26.300 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604_vm 2023-11-06T01:49:26.300 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2605 2023-11-06T01:49:26.300 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2605_fsm 2023-11-06T01:49:26.300 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2605_vm 2023-11-06T01:49:26.300 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606 2023-11-06T01:49:26.301 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2606_fsm 2023-11-06T01:49:26.301 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606_vm 2023-11-06T01:49:26.301 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607 2023-11-06T01:49:26.301 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607_fsm 2023-11-06T01:49:26.301 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2607_vm 2023-11-06T01:49:26.301 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Nov 6 01:34 2608 2023-11-06T01:49:26.302 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2608_fsm 2023-11-06T01:49:26.302 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2608_vm 2023-11-06T01:49:26.302 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Nov 6 01:34 2609 2023-11-06T01:49:26.302 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2609_fsm 2023-11-06T01:49:26.302 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2609_vm 2023-11-06T01:49:26.303 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2610 2023-11-06T01:49:26.303 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2610_fsm 2023-11-06T01:49:26.303 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2610_vm 2023-11-06T01:49:26.303 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611 2023-11-06T01:49:26.303 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611_vm 2023-11-06T01:49:26.304 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612 2023-11-06T01:49:26.304 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2612_fsm 2023-11-06T01:49:26.304 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612_vm 2023-11-06T01:49:26.304 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613 2023-11-06T01:49:26.304 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613_vm 2023-11-06T01:49:26.305 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615 2023-11-06T01:49:26.305 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2615_fsm 2023-11-06T01:49:26.305 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615_vm 2023-11-06T01:49:26.305 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616 2023-11-06T01:49:26.305 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616_fsm 2023-11-06T01:49:26.306 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2616_vm 2023-11-06T01:49:26.306 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Nov 6 01:34 2617 2023-11-06T01:49:26.306 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2617_fsm 2023-11-06T01:49:26.306 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2617_vm 2023-11-06T01:49:26.306 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Nov 6 01:34 2618 2023-11-06T01:49:26.306 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2618_fsm 2023-11-06T01:49:26.307 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2618_vm 2023-11-06T01:49:26.307 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Nov 6 01:34 2619 2023-11-06T01:49:26.307 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2619_fsm 2023-11-06T01:49:26.307 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2619_vm 2023-11-06T01:49:26.307 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620 2023-11-06T01:49:26.308 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620_vm 2023-11-06T01:49:26.308 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2650 2023-11-06T01:49:26.308 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2651 2023-11-06T01:49:26.308 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2652 2023-11-06T01:49:26.309 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2653 2023-11-06T01:49:26.309 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2654 2023-11-06T01:49:26.309 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2655 2023-11-06T01:49:26.309 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2656 2023-11-06T01:49:26.309 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2657 2023-11-06T01:49:26.310 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Nov 6 01:34 2658 2023-11-06T01:49:26.310 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2659 2023-11-06T01:49:26.310 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2660 2023-11-06T01:49:26.310 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2661 2023-11-06T01:49:26.310 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2662 2023-11-06T01:49:26.311 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2663 2023-11-06T01:49:26.311 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2664 2023-11-06T01:49:26.311 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2665 2023-11-06T01:49:26.311 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2666 2023-11-06T01:49:26.311 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2667 2023-11-06T01:49:26.311 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2668 2023-11-06T01:49:26.312 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2669 2023-11-06T01:49:26.312 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2670 2023-11-06T01:49:26.312 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Nov 6 01:34 2673 2023-11-06T01:49:26.312 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Nov 6 01:34 2674 2023-11-06T01:49:26.312 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Nov 6 01:34 2675 2023-11-06T01:49:26.313 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2678 2023-11-06T01:49:26.313 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2679 2023-11-06T01:49:26.313 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2680 2023-11-06T01:49:26.313 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2681 2023-11-06T01:49:26.313 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2682 2023-11-06T01:49:26.314 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2683 2023-11-06T01:49:26.314 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2684 2023-11-06T01:49:26.314 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2685 2023-11-06T01:49:26.314 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2686 2023-11-06T01:49:26.314 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2687 2023-11-06T01:49:26.315 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2688 2023-11-06T01:49:26.315 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2689 2023-11-06T01:49:26.315 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2690 2023-11-06T01:49:26.315 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Nov 6 01:34 2691 2023-11-06T01:49:26.315 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2692 2023-11-06T01:49:26.315 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2693 2023-11-06T01:49:26.316 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2696 2023-11-06T01:49:26.316 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2699 2023-11-06T01:49:26.316 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2701 2023-11-06T01:49:26.316 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2702 2023-11-06T01:49:26.316 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2703 2023-11-06T01:49:26.317 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2704 2023-11-06T01:49:26.317 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2753 2023-11-06T01:49:26.317 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2753_fsm 2023-11-06T01:49:26.317 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2753_vm 2023-11-06T01:49:26.317 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2754 2023-11-06T01:49:26.318 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2755 2023-11-06T01:49:26.318 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2756 2023-11-06T01:49:26.318 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2757 2023-11-06T01:49:26.318 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830 2023-11-06T01:49:26.318 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830_vm 2023-11-06T01:49:26.319 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2831 2023-11-06T01:49:26.319 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832 2023-11-06T01:49:26.319 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832_vm 2023-11-06T01:49:26.319 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2833 2023-11-06T01:49:26.319 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834 2023-11-06T01:49:26.319 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834_vm 2023-11-06T01:49:26.320 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2835 2023-11-06T01:49:26.320 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836 2023-11-06T01:49:26.320 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836_vm 2023-11-06T01:49:26.320 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2837 2023-11-06T01:49:26.320 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Nov 6 01:34 2838 2023-11-06T01:49:26.321 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2838_fsm 2023-11-06T01:49:26.321 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2838_vm 2023-11-06T01:49:26.321 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2839 2023-11-06T01:49:26.321 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840 2023-11-06T01:49:26.321 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840_fsm 2023-11-06T01:49:26.322 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2840_vm 2023-11-06T01:49:26.322 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2841 2023-11-06T01:49:26.322 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995 2023-11-06T01:49:26.322 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995_vm 2023-11-06T01:49:26.322 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2996 2023-11-06T01:49:26.322 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079 2023-11-06T01:49:26.323 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3079_fsm 2023-11-06T01:49:26.323 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079_vm 2023-11-06T01:49:26.323 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3080 2023-11-06T01:49:26.323 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3081 2023-11-06T01:49:26.323 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3085 2023-11-06T01:49:26.324 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118 2023-11-06T01:49:26.324 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118_vm 2023-11-06T01:49:26.324 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3119 2023-11-06T01:49:26.324 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3164 2023-11-06T01:49:26.324 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256 2023-11-06T01:49:26.325 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256_vm 2023-11-06T01:49:26.325 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3257 2023-11-06T01:49:26.325 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3258 2023-11-06T01:49:26.325 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350 2023-11-06T01:49:26.325 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350_vm 2023-11-06T01:49:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3351 2023-11-06T01:49:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3379 2023-11-06T01:49:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3380 2023-11-06T01:49:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381 2023-11-06T01:49:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381_vm 2023-11-06T01:49:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3394 2023-11-06T01:49:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3394_fsm 2023-11-06T01:49:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3394_vm 2023-11-06T01:49:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3395 2023-11-06T01:49:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439 2023-11-06T01:49:26.328 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439_vm 2023-11-06T01:49:26.328 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3440 2023-11-06T01:49:26.328 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 3455 2023-11-06T01:49:26.328 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3456 2023-11-06T01:49:26.328 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3456_fsm 2023-11-06T01:49:26.329 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3456_vm 2023-11-06T01:49:26.329 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466 2023-11-06T01:49:26.329 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466_vm 2023-11-06T01:49:26.329 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3467 2023-11-06T01:49:26.329 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3468 2023-11-06T01:49:26.330 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501 2023-11-06T01:49:26.330 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501_vm 2023-11-06T01:49:26.330 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3502 2023-11-06T01:49:26.330 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3503 2023-11-06T01:49:26.330 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3534 2023-11-06T01:49:26.331 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541 2023-11-06T01:49:26.331 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3541_fsm 2023-11-06T01:49:26.331 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541_vm 2023-11-06T01:49:26.331 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3542 2023-11-06T01:49:26.331 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3574 2023-11-06T01:49:26.332 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3575 2023-11-06T01:49:26.332 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576 2023-11-06T01:49:26.332 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576_vm 2023-11-06T01:49:26.332 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596 2023-11-06T01:49:26.332 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596_vm 2023-11-06T01:49:26.333 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3597 2023-11-06T01:49:26.333 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598 2023-11-06T01:49:26.333 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598_vm 2023-11-06T01:49:26.333 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3599 2023-11-06T01:49:26.333 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600 2023-11-06T01:49:26.333 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3600_fsm 2023-11-06T01:49:26.334 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600_vm 2023-11-06T01:49:26.334 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601 2023-11-06T01:49:26.334 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3601_fsm 2023-11-06T01:49:26.334 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601_vm 2023-11-06T01:49:26.334 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602 2023-11-06T01:49:26.335 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3602_fsm 2023-11-06T01:49:26.335 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602_vm 2023-11-06T01:49:26.335 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3603 2023-11-06T01:49:26.335 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3603_fsm 2023-11-06T01:49:26.336 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3603_vm 2023-11-06T01:49:26.338 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3604 2023-11-06T01:49:26.339 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3605 2023-11-06T01:49:26.339 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3606 2023-11-06T01:49:26.339 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3607 2023-11-06T01:49:26.339 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3608 2023-11-06T01:49:26.340 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 3609 2023-11-06T01:49:26.340 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3712 2023-11-06T01:49:26.340 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764 2023-11-06T01:49:26.340 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3764_fsm 2023-11-06T01:49:26.340 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764_vm 2023-11-06T01:49:26.341 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3766 2023-11-06T01:49:26.341 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3767 2023-11-06T01:49:26.341 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3997 2023-11-06T01:49:26.341 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 5002 2023-11-06T01:49:26.341 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 548 2023-11-06T01:49:26.342 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 549 2023-11-06T01:49:26.342 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102 2023-11-06T01:49:26.342 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102_vm 2023-11-06T01:49:26.342 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104 2023-11-06T01:49:26.343 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104_vm 2023-11-06T01:49:26.343 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106 2023-11-06T01:49:26.343 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106_vm 2023-11-06T01:49:26.343 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6110 2023-11-06T01:49:26.343 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6111 2023-11-06T01:49:26.343 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6112 2023-11-06T01:49:26.344 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6113 2023-11-06T01:49:26.344 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6117 2023-11-06T01:49:26.344 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826 2023-11-06T01:49:26.344 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826_vm 2023-11-06T01:49:26.344 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 827 2023-11-06T01:49:26.345 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 828 2023-11-06T01:49:26.345 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Nov 6 01:34 pg_filenode.map 2023-11-06T01:49:26.345 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 PG_VERSION 2023-11-06T01:49:26.345 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.345 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-11-06T01:49:26.346 INFO:teuthology.orchestra.run.smithi053.stdout:total 7193 2023-11-06T01:49:26.346 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Nov 6 01:34 . 2023-11-06T01:49:26.346 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Nov 6 01:34 .. 2023-11-06T01:49:26.346 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 112 2023-11-06T01:49:26.346 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 113 2023-11-06T01:49:26.347 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Nov 6 01:34 12182 2023-11-06T01:49:26.347 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12182_fsm 2023-11-06T01:49:26.347 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12182_vm 2023-11-06T01:49:26.347 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12184 2023-11-06T01:49:26.347 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12186 2023-11-06T01:49:26.348 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187 2023-11-06T01:49:26.348 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12187_fsm 2023-11-06T01:49:26.348 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187_vm 2023-11-06T01:49:26.348 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12189 2023-11-06T01:49:26.348 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12191 2023-11-06T01:49:26.348 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192 2023-11-06T01:49:26.349 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12192_fsm 2023-11-06T01:49:26.349 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192_vm 2023-11-06T01:49:26.349 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12194 2023-11-06T01:49:26.349 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12196 2023-11-06T01:49:26.349 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197 2023-11-06T01:49:26.350 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12197_fsm 2023-11-06T01:49:26.350 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197_vm 2023-11-06T01:49:26.350 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12199 2023-11-06T01:49:26.350 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12201 2023-11-06T01:49:26.350 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202 2023-11-06T01:49:26.351 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12202_fsm 2023-11-06T01:49:26.351 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202_vm 2023-11-06T01:49:26.351 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12204 2023-11-06T01:49:26.351 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12206 2023-11-06T01:49:26.351 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207 2023-11-06T01:49:26.352 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12207_fsm 2023-11-06T01:49:26.352 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207_vm 2023-11-06T01:49:26.352 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12209 2023-11-06T01:49:26.352 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12211 2023-11-06T01:49:26.352 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12212 2023-11-06T01:49:26.352 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12214 2023-11-06T01:49:26.353 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12216 2023-11-06T01:49:26.353 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Nov 6 01:34 1247 2023-11-06T01:49:26.353 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1247_fsm 2023-11-06T01:49:26.353 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1247_vm 2023-11-06T01:49:26.353 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Nov 6 01:34 1249 2023-11-06T01:49:26.354 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1249_fsm 2023-11-06T01:49:26.354 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1249_vm 2023-11-06T01:49:26.354 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Nov 6 01:34 1255 2023-11-06T01:49:26.354 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1255_fsm 2023-11-06T01:49:26.354 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1255_vm 2023-11-06T01:49:26.355 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Nov 6 01:34 1259 2023-11-06T01:49:26.355 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1259_fsm 2023-11-06T01:49:26.355 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1259_vm 2023-11-06T01:49:26.357 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417 2023-11-06T01:49:26.357 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417_vm 2023-11-06T01:49:26.357 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418 2023-11-06T01:49:26.358 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418_vm 2023-11-06T01:49:26.358 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 174 2023-11-06T01:49:26.358 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 175 2023-11-06T01:49:26.358 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2187 2023-11-06T01:49:26.358 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224 2023-11-06T01:49:26.359 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224_vm 2023-11-06T01:49:26.359 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328 2023-11-06T01:49:26.359 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328_vm 2023-11-06T01:49:26.359 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336 2023-11-06T01:49:26.359 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336_vm 2023-11-06T01:49:26.360 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2337 2023-11-06T01:49:26.360 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2600 2023-11-06T01:49:26.360 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2600_fsm 2023-11-06T01:49:26.360 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2600_vm 2023-11-06T01:49:26.360 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601 2023-11-06T01:49:26.361 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2601_fsm 2023-11-06T01:49:26.361 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601_vm 2023-11-06T01:49:26.361 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2602 2023-11-06T01:49:26.361 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2602_fsm 2023-11-06T01:49:26.361 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2602_vm 2023-11-06T01:49:26.362 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2603 2023-11-06T01:49:26.362 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2603_fsm 2023-11-06T01:49:26.362 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2603_vm 2023-11-06T01:49:26.362 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604 2023-11-06T01:49:26.362 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604_vm 2023-11-06T01:49:26.363 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2605 2023-11-06T01:49:26.363 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2605_fsm 2023-11-06T01:49:26.363 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2605_vm 2023-11-06T01:49:26.363 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606 2023-11-06T01:49:26.363 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2606_fsm 2023-11-06T01:49:26.364 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606_vm 2023-11-06T01:49:26.364 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607 2023-11-06T01:49:26.364 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607_fsm 2023-11-06T01:49:26.364 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2607_vm 2023-11-06T01:49:26.364 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Nov 6 01:34 2608 2023-11-06T01:49:26.365 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2608_fsm 2023-11-06T01:49:26.365 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2608_vm 2023-11-06T01:49:26.365 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Nov 6 01:34 2609 2023-11-06T01:49:26.365 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2609_fsm 2023-11-06T01:49:26.365 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2609_vm 2023-11-06T01:49:26.365 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2610 2023-11-06T01:49:26.366 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2610_fsm 2023-11-06T01:49:26.366 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2610_vm 2023-11-06T01:49:26.366 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611 2023-11-06T01:49:26.366 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611_vm 2023-11-06T01:49:26.366 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612 2023-11-06T01:49:26.367 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2612_fsm 2023-11-06T01:49:26.367 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612_vm 2023-11-06T01:49:26.367 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613 2023-11-06T01:49:26.367 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613_vm 2023-11-06T01:49:26.367 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615 2023-11-06T01:49:26.368 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2615_fsm 2023-11-06T01:49:26.368 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615_vm 2023-11-06T01:49:26.368 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616 2023-11-06T01:49:26.368 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616_fsm 2023-11-06T01:49:26.368 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2616_vm 2023-11-06T01:49:26.368 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Nov 6 01:34 2617 2023-11-06T01:49:26.369 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2617_fsm 2023-11-06T01:49:26.369 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2617_vm 2023-11-06T01:49:26.369 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Nov 6 01:34 2618 2023-11-06T01:49:26.369 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2618_fsm 2023-11-06T01:49:26.369 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2618_vm 2023-11-06T01:49:26.370 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Nov 6 01:34 2619 2023-11-06T01:49:26.370 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2619_fsm 2023-11-06T01:49:26.370 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2619_vm 2023-11-06T01:49:26.370 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620 2023-11-06T01:49:26.370 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620_vm 2023-11-06T01:49:26.371 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2650 2023-11-06T01:49:26.371 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2651 2023-11-06T01:49:26.371 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2652 2023-11-06T01:49:26.371 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2653 2023-11-06T01:49:26.371 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2654 2023-11-06T01:49:26.372 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2655 2023-11-06T01:49:26.372 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2656 2023-11-06T01:49:26.372 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2657 2023-11-06T01:49:26.372 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Nov 6 01:34 2658 2023-11-06T01:49:26.372 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2659 2023-11-06T01:49:26.373 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2660 2023-11-06T01:49:26.373 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2661 2023-11-06T01:49:26.373 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2662 2023-11-06T01:49:26.373 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2663 2023-11-06T01:49:26.373 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2664 2023-11-06T01:49:26.373 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2665 2023-11-06T01:49:26.374 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2666 2023-11-06T01:49:26.374 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2667 2023-11-06T01:49:26.374 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2668 2023-11-06T01:49:26.374 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2669 2023-11-06T01:49:26.374 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2670 2023-11-06T01:49:26.375 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Nov 6 01:34 2673 2023-11-06T01:49:26.375 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Nov 6 01:34 2674 2023-11-06T01:49:26.375 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Nov 6 01:34 2675 2023-11-06T01:49:26.375 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2678 2023-11-06T01:49:26.375 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2679 2023-11-06T01:49:26.376 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2680 2023-11-06T01:49:26.376 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2681 2023-11-06T01:49:26.376 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2682 2023-11-06T01:49:26.376 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2683 2023-11-06T01:49:26.376 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2684 2023-11-06T01:49:26.377 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2685 2023-11-06T01:49:26.377 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2686 2023-11-06T01:49:26.377 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2687 2023-11-06T01:49:26.377 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2688 2023-11-06T01:49:26.377 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2689 2023-11-06T01:49:26.377 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2690 2023-11-06T01:49:26.378 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Nov 6 01:34 2691 2023-11-06T01:49:26.378 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2692 2023-11-06T01:49:26.378 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2693 2023-11-06T01:49:26.378 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2696 2023-11-06T01:49:26.378 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2699 2023-11-06T01:49:26.379 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2701 2023-11-06T01:49:26.379 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2702 2023-11-06T01:49:26.379 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2703 2023-11-06T01:49:26.379 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2704 2023-11-06T01:49:26.379 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2753 2023-11-06T01:49:26.380 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2753_fsm 2023-11-06T01:49:26.380 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2753_vm 2023-11-06T01:49:26.380 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2754 2023-11-06T01:49:26.380 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2755 2023-11-06T01:49:26.380 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2756 2023-11-06T01:49:26.380 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2757 2023-11-06T01:49:26.381 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830 2023-11-06T01:49:26.381 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830_vm 2023-11-06T01:49:26.381 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2831 2023-11-06T01:49:26.381 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832 2023-11-06T01:49:26.381 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832_vm 2023-11-06T01:49:26.382 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2833 2023-11-06T01:49:26.382 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834 2023-11-06T01:49:26.382 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834_vm 2023-11-06T01:49:26.382 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2835 2023-11-06T01:49:26.382 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836 2023-11-06T01:49:26.383 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836_vm 2023-11-06T01:49:26.383 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2837 2023-11-06T01:49:26.383 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Nov 6 01:34 2838 2023-11-06T01:49:26.383 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2838_fsm 2023-11-06T01:49:26.383 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2838_vm 2023-11-06T01:49:26.383 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2839 2023-11-06T01:49:26.384 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840 2023-11-06T01:49:26.384 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840_fsm 2023-11-06T01:49:26.384 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2840_vm 2023-11-06T01:49:26.384 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2841 2023-11-06T01:49:26.384 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995 2023-11-06T01:49:26.385 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995_vm 2023-11-06T01:49:26.385 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2996 2023-11-06T01:49:26.385 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079 2023-11-06T01:49:26.385 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3079_fsm 2023-11-06T01:49:26.385 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079_vm 2023-11-06T01:49:26.385 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3080 2023-11-06T01:49:26.386 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3081 2023-11-06T01:49:26.386 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3085 2023-11-06T01:49:26.386 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118 2023-11-06T01:49:26.386 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118_vm 2023-11-06T01:49:26.386 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3119 2023-11-06T01:49:26.387 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3164 2023-11-06T01:49:26.387 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256 2023-11-06T01:49:26.387 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256_vm 2023-11-06T01:49:26.387 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3257 2023-11-06T01:49:26.387 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3258 2023-11-06T01:49:26.388 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350 2023-11-06T01:49:26.388 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350_vm 2023-11-06T01:49:26.388 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3351 2023-11-06T01:49:26.388 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3379 2023-11-06T01:49:26.388 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3380 2023-11-06T01:49:26.388 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381 2023-11-06T01:49:26.389 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381_vm 2023-11-06T01:49:26.389 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3394 2023-11-06T01:49:26.389 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3394_fsm 2023-11-06T01:49:26.389 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3394_vm 2023-11-06T01:49:26.389 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3395 2023-11-06T01:49:26.390 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439 2023-11-06T01:49:26.390 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439_vm 2023-11-06T01:49:26.390 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3440 2023-11-06T01:49:26.390 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 3455 2023-11-06T01:49:26.390 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3456 2023-11-06T01:49:26.391 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3456_fsm 2023-11-06T01:49:26.391 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3456_vm 2023-11-06T01:49:26.391 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466 2023-11-06T01:49:26.391 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466_vm 2023-11-06T01:49:26.391 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3467 2023-11-06T01:49:26.392 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3468 2023-11-06T01:49:26.392 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501 2023-11-06T01:49:26.392 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501_vm 2023-11-06T01:49:26.392 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3502 2023-11-06T01:49:26.392 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3503 2023-11-06T01:49:26.393 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3534 2023-11-06T01:49:26.393 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541 2023-11-06T01:49:26.393 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3541_fsm 2023-11-06T01:49:26.393 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541_vm 2023-11-06T01:49:26.393 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3542 2023-11-06T01:49:26.393 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3574 2023-11-06T01:49:26.394 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3575 2023-11-06T01:49:26.394 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576 2023-11-06T01:49:26.394 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576_vm 2023-11-06T01:49:26.394 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596 2023-11-06T01:49:26.394 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596_vm 2023-11-06T01:49:26.395 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3597 2023-11-06T01:49:26.395 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598 2023-11-06T01:49:26.395 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598_vm 2023-11-06T01:49:26.395 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3599 2023-11-06T01:49:26.395 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600 2023-11-06T01:49:26.396 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3600_fsm 2023-11-06T01:49:26.396 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600_vm 2023-11-06T01:49:26.396 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601 2023-11-06T01:49:26.396 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3601_fsm 2023-11-06T01:49:26.396 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601_vm 2023-11-06T01:49:26.396 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602 2023-11-06T01:49:26.397 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3602_fsm 2023-11-06T01:49:26.397 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602_vm 2023-11-06T01:49:26.397 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3603 2023-11-06T01:49:26.397 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3603_fsm 2023-11-06T01:49:26.397 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3603_vm 2023-11-06T01:49:26.398 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3604 2023-11-06T01:49:26.398 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3605 2023-11-06T01:49:26.398 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3606 2023-11-06T01:49:26.398 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3607 2023-11-06T01:49:26.398 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3608 2023-11-06T01:49:26.399 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 3609 2023-11-06T01:49:26.399 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3712 2023-11-06T01:49:26.399 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764 2023-11-06T01:49:26.399 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3764_fsm 2023-11-06T01:49:26.399 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764_vm 2023-11-06T01:49:26.399 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3766 2023-11-06T01:49:26.400 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3767 2023-11-06T01:49:26.400 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3997 2023-11-06T01:49:26.400 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 5002 2023-11-06T01:49:26.400 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 548 2023-11-06T01:49:26.400 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 549 2023-11-06T01:49:26.401 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102 2023-11-06T01:49:26.401 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102_vm 2023-11-06T01:49:26.401 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104 2023-11-06T01:49:26.401 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104_vm 2023-11-06T01:49:26.401 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106 2023-11-06T01:49:26.402 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106_vm 2023-11-06T01:49:26.402 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6110 2023-11-06T01:49:26.402 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6111 2023-11-06T01:49:26.402 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6112 2023-11-06T01:49:26.402 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6113 2023-11-06T01:49:26.402 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6117 2023-11-06T01:49:26.403 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826 2023-11-06T01:49:26.403 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826_vm 2023-11-06T01:49:26.403 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 827 2023-11-06T01:49:26.403 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 828 2023-11-06T01:49:26.403 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Nov 6 01:34 pg_filenode.map 2023-11-06T01:49:26.404 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 PG_VERSION 2023-11-06T01:49:26.404 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.404 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-11-06T01:49:26.404 INFO:teuthology.orchestra.run.smithi053.stdout:total 658392 2023-11-06T01:49:26.404 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Nov 6 01:34 . 2023-11-06T01:49:26.405 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Nov 6 01:34 .. 2023-11-06T01:49:26.405 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 112 2023-11-06T01:49:26.405 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 113 2023-11-06T01:49:26.405 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Nov 6 01:34 12182 2023-11-06T01:49:26.405 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12182_fsm 2023-11-06T01:49:26.405 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12182_vm 2023-11-06T01:49:26.406 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12184 2023-11-06T01:49:26.406 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12186 2023-11-06T01:49:26.406 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187 2023-11-06T01:49:26.406 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12187_fsm 2023-11-06T01:49:26.406 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12187_vm 2023-11-06T01:49:26.407 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12189 2023-11-06T01:49:26.407 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12191 2023-11-06T01:49:26.407 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192 2023-11-06T01:49:26.407 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12192_fsm 2023-11-06T01:49:26.407 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12192_vm 2023-11-06T01:49:26.408 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12194 2023-11-06T01:49:26.408 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12196 2023-11-06T01:49:26.408 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197 2023-11-06T01:49:26.408 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12197_fsm 2023-11-06T01:49:26.408 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12197_vm 2023-11-06T01:49:26.409 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12199 2023-11-06T01:49:26.409 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12201 2023-11-06T01:49:26.409 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202 2023-11-06T01:49:26.409 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12202_fsm 2023-11-06T01:49:26.409 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12202_vm 2023-11-06T01:49:26.409 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12204 2023-11-06T01:49:26.410 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12206 2023-11-06T01:49:26.410 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207 2023-11-06T01:49:26.410 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 12207_fsm 2023-11-06T01:49:26.410 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12207_vm 2023-11-06T01:49:26.410 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12209 2023-11-06T01:49:26.411 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12211 2023-11-06T01:49:26.411 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12212 2023-11-06T01:49:26.411 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 12214 2023-11-06T01:49:26.411 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 12216 2023-11-06T01:49:26.411 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Nov 6 01:34 1247 2023-11-06T01:49:26.411 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1247_fsm 2023-11-06T01:49:26.412 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1247_vm 2023-11-06T01:49:26.412 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Nov 6 01:34 1249 2023-11-06T01:49:26.412 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1249_fsm 2023-11-06T01:49:26.412 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1249_vm 2023-11-06T01:49:26.412 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Nov 6 01:34 1255 2023-11-06T01:49:26.413 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1255_fsm 2023-11-06T01:49:26.413 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1255_vm 2023-11-06T01:49:26.413 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Nov 6 01:49 1259 2023-11-06T01:49:26.413 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1259_fsm 2023-11-06T01:49:26.413 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1259_vm 2023-11-06T01:49:26.413 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417 2023-11-06T01:49:26.414 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1417_vm 2023-11-06T01:49:26.414 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418 2023-11-06T01:49:26.414 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 1418_vm 2023-11-06T01:49:26.414 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 12943360 Nov 6 01:49 16387 2023-11-06T01:49:26.414 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:49 16387_fsm 2023-11-06T01:49:26.415 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:48 16387_vm 2023-11-06T01:49:26.415 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8912896 Nov 6 01:49 16393 2023-11-06T01:49:26.415 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:49 16393_fsm 2023-11-06T01:49:26.415 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:49 16393_vm 2023-11-06T01:49:26.415 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 470532096 Nov 6 01:49 16396 2023-11-06T01:49:26.416 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 139264 Nov 6 01:48 16396_fsm 2023-11-06T01:49:26.416 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:37 16396_vm 2023-11-06T01:49:26.416 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Nov 6 01:49 16397 2023-11-06T01:49:26.416 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 622592 Nov 6 01:49 16399 2023-11-06T01:49:26.416 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 72646656 Nov 6 01:49 16401 2023-11-06T01:49:26.417 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 100605952 Nov 6 01:49 16403 2023-11-06T01:49:26.417 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:49 16403_fsm 2023-11-06T01:49:26.417 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 174 2023-11-06T01:49:26.417 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 175 2023-11-06T01:49:26.417 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2187 2023-11-06T01:49:26.417 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224 2023-11-06T01:49:26.418 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2224_vm 2023-11-06T01:49:26.418 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328 2023-11-06T01:49:26.418 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2328_vm 2023-11-06T01:49:26.418 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336 2023-11-06T01:49:26.418 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2336_vm 2023-11-06T01:49:26.419 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2337 2023-11-06T01:49:26.419 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2600 2023-11-06T01:49:26.419 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2600_fsm 2023-11-06T01:49:26.419 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2600_vm 2023-11-06T01:49:26.419 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601 2023-11-06T01:49:26.420 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2601_fsm 2023-11-06T01:49:26.420 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2601_vm 2023-11-06T01:49:26.420 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2602 2023-11-06T01:49:26.420 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2602_fsm 2023-11-06T01:49:26.420 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2602_vm 2023-11-06T01:49:26.421 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2603 2023-11-06T01:49:26.421 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2603_fsm 2023-11-06T01:49:26.421 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2603_vm 2023-11-06T01:49:26.421 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604 2023-11-06T01:49:26.421 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2604_vm 2023-11-06T01:49:26.421 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2605 2023-11-06T01:49:26.422 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2605_fsm 2023-11-06T01:49:26.422 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2605_vm 2023-11-06T01:49:26.422 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606 2023-11-06T01:49:26.422 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2606_fsm 2023-11-06T01:49:26.422 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2606_vm 2023-11-06T01:49:26.423 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607 2023-11-06T01:49:26.423 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2607_fsm 2023-11-06T01:49:26.423 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2607_vm 2023-11-06T01:49:26.423 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Nov 6 01:34 2608 2023-11-06T01:49:26.423 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2608_fsm 2023-11-06T01:49:26.424 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2608_vm 2023-11-06T01:49:26.424 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Nov 6 01:34 2609 2023-11-06T01:49:26.424 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2609_fsm 2023-11-06T01:49:26.424 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2609_vm 2023-11-06T01:49:26.424 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2610 2023-11-06T01:49:26.425 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2610_fsm 2023-11-06T01:49:26.425 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2610_vm 2023-11-06T01:49:26.425 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611 2023-11-06T01:49:26.425 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2611_vm 2023-11-06T01:49:26.425 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612 2023-11-06T01:49:26.425 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2612_fsm 2023-11-06T01:49:26.426 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2612_vm 2023-11-06T01:49:26.426 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613 2023-11-06T01:49:26.426 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2613_vm 2023-11-06T01:49:26.426 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615 2023-11-06T01:49:26.426 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2615_fsm 2023-11-06T01:49:26.427 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2615_vm 2023-11-06T01:49:26.427 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616 2023-11-06T01:49:26.427 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2616_fsm 2023-11-06T01:49:26.427 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2616_vm 2023-11-06T01:49:26.427 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Nov 6 01:34 2617 2023-11-06T01:49:26.427 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2617_fsm 2023-11-06T01:49:26.428 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2617_vm 2023-11-06T01:49:26.428 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Nov 6 01:34 2618 2023-11-06T01:49:26.428 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2618_fsm 2023-11-06T01:49:26.428 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2618_vm 2023-11-06T01:49:26.428 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Nov 6 01:48 2619 2023-11-06T01:49:26.429 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:38 2619_fsm 2023-11-06T01:49:26.429 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2619_vm 2023-11-06T01:49:26.429 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620 2023-11-06T01:49:26.429 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2620_vm 2023-11-06T01:49:26.429 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2650 2023-11-06T01:49:26.430 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2651 2023-11-06T01:49:26.430 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2652 2023-11-06T01:49:26.430 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2653 2023-11-06T01:49:26.430 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2654 2023-11-06T01:49:26.430 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2655 2023-11-06T01:49:26.430 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2656 2023-11-06T01:49:26.431 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2657 2023-11-06T01:49:26.431 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Nov 6 01:34 2658 2023-11-06T01:49:26.431 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2659 2023-11-06T01:49:26.431 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2660 2023-11-06T01:49:26.431 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2661 2023-11-06T01:49:26.432 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2662 2023-11-06T01:49:26.432 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2663 2023-11-06T01:49:26.432 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2664 2023-11-06T01:49:26.432 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2665 2023-11-06T01:49:26.432 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2666 2023-11-06T01:49:26.433 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2667 2023-11-06T01:49:26.433 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2668 2023-11-06T01:49:26.433 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2669 2023-11-06T01:49:26.433 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2670 2023-11-06T01:49:26.433 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Nov 6 01:34 2673 2023-11-06T01:49:26.434 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Nov 6 01:34 2674 2023-11-06T01:49:26.434 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Nov 6 01:34 2675 2023-11-06T01:49:26.434 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2678 2023-11-06T01:49:26.434 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2679 2023-11-06T01:49:26.434 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2680 2023-11-06T01:49:26.435 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2681 2023-11-06T01:49:26.435 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2682 2023-11-06T01:49:26.435 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2683 2023-11-06T01:49:26.435 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2684 2023-11-06T01:49:26.435 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2685 2023-11-06T01:49:26.435 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2686 2023-11-06T01:49:26.436 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2687 2023-11-06T01:49:26.436 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2688 2023-11-06T01:49:26.436 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Nov 6 01:34 2689 2023-11-06T01:49:26.436 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Nov 6 01:34 2690 2023-11-06T01:49:26.436 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Nov 6 01:34 2691 2023-11-06T01:49:26.437 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2692 2023-11-06T01:49:26.437 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2693 2023-11-06T01:49:26.437 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:38 2696 2023-11-06T01:49:26.437 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2699 2023-11-06T01:49:26.437 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2701 2023-11-06T01:49:26.438 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2702 2023-11-06T01:49:26.438 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2703 2023-11-06T01:49:26.438 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 2704 2023-11-06T01:49:26.438 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2753 2023-11-06T01:49:26.438 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2753_fsm 2023-11-06T01:49:26.438 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2753_vm 2023-11-06T01:49:26.439 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2754 2023-11-06T01:49:26.439 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2755 2023-11-06T01:49:26.439 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2756 2023-11-06T01:49:26.439 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 2757 2023-11-06T01:49:26.439 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830 2023-11-06T01:49:26.440 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2830_vm 2023-11-06T01:49:26.440 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2831 2023-11-06T01:49:26.440 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832 2023-11-06T01:49:26.440 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2832_vm 2023-11-06T01:49:26.440 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2833 2023-11-06T01:49:26.440 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834 2023-11-06T01:49:26.441 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2834_vm 2023-11-06T01:49:26.441 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2835 2023-11-06T01:49:26.441 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836 2023-11-06T01:49:26.441 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2836_vm 2023-11-06T01:49:26.441 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2837 2023-11-06T01:49:26.442 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Nov 6 01:34 2838 2023-11-06T01:49:26.442 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2838_fsm 2023-11-06T01:49:26.442 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2838_vm 2023-11-06T01:49:26.442 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2839 2023-11-06T01:49:26.442 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840 2023-11-06T01:49:26.443 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2840_fsm 2023-11-06T01:49:26.443 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2840_vm 2023-11-06T01:49:26.443 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2841 2023-11-06T01:49:26.443 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995 2023-11-06T01:49:26.443 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2995_vm 2023-11-06T01:49:26.443 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2996 2023-11-06T01:49:26.444 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079 2023-11-06T01:49:26.444 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3079_fsm 2023-11-06T01:49:26.444 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3079_vm 2023-11-06T01:49:26.444 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3080 2023-11-06T01:49:26.444 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3081 2023-11-06T01:49:26.445 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3085 2023-11-06T01:49:26.445 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118 2023-11-06T01:49:26.445 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3118_vm 2023-11-06T01:49:26.445 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3119 2023-11-06T01:49:26.445 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3164 2023-11-06T01:49:26.445 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256 2023-11-06T01:49:26.446 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3256_vm 2023-11-06T01:49:26.446 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3257 2023-11-06T01:49:26.446 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3258 2023-11-06T01:49:26.446 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350 2023-11-06T01:49:26.446 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3350_vm 2023-11-06T01:49:26.447 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3351 2023-11-06T01:49:26.447 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3379 2023-11-06T01:49:26.447 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3380 2023-11-06T01:49:26.447 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381 2023-11-06T01:49:26.447 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3381_vm 2023-11-06T01:49:26.448 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3394 2023-11-06T01:49:26.448 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3394_fsm 2023-11-06T01:49:26.448 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3394_vm 2023-11-06T01:49:26.448 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3395 2023-11-06T01:49:26.448 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439 2023-11-06T01:49:26.449 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3439_vm 2023-11-06T01:49:26.449 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3440 2023-11-06T01:49:26.449 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Nov 6 01:34 3455 2023-11-06T01:49:26.449 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3456 2023-11-06T01:49:26.449 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3456_fsm 2023-11-06T01:49:26.449 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3456_vm 2023-11-06T01:49:26.450 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466 2023-11-06T01:49:26.450 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3466_vm 2023-11-06T01:49:26.450 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3467 2023-11-06T01:49:26.450 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3468 2023-11-06T01:49:26.451 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501 2023-11-06T01:49:26.451 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3501_vm 2023-11-06T01:49:26.451 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3502 2023-11-06T01:49:26.451 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3503 2023-11-06T01:49:26.451 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3534 2023-11-06T01:49:26.451 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541 2023-11-06T01:49:26.452 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3541_fsm 2023-11-06T01:49:26.452 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3541_vm 2023-11-06T01:49:26.452 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3542 2023-11-06T01:49:26.452 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3574 2023-11-06T01:49:26.452 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3575 2023-11-06T01:49:26.453 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576 2023-11-06T01:49:26.453 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3576_vm 2023-11-06T01:49:26.453 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596 2023-11-06T01:49:26.453 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3596_vm 2023-11-06T01:49:26.453 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3597 2023-11-06T01:49:26.454 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598 2023-11-06T01:49:26.454 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3598_vm 2023-11-06T01:49:26.454 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3599 2023-11-06T01:49:26.454 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600 2023-11-06T01:49:26.454 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3600_fsm 2023-11-06T01:49:26.455 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3600_vm 2023-11-06T01:49:26.455 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601 2023-11-06T01:49:26.455 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3601_fsm 2023-11-06T01:49:26.455 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3601_vm 2023-11-06T01:49:26.455 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602 2023-11-06T01:49:26.456 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3602_fsm 2023-11-06T01:49:26.456 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3602_vm 2023-11-06T01:49:26.456 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3603 2023-11-06T01:49:26.456 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3603_fsm 2023-11-06T01:49:26.456 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3603_vm 2023-11-06T01:49:26.457 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3604 2023-11-06T01:49:26.457 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3605 2023-11-06T01:49:26.457 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3606 2023-11-06T01:49:26.457 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3607 2023-11-06T01:49:26.457 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3608 2023-11-06T01:49:26.458 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Nov 6 01:34 3609 2023-11-06T01:49:26.458 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3712 2023-11-06T01:49:26.458 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764 2023-11-06T01:49:26.458 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 3764_fsm 2023-11-06T01:49:26.458 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3764_vm 2023-11-06T01:49:26.459 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3766 2023-11-06T01:49:26.459 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 3767 2023-11-06T01:49:26.459 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3997 2023-11-06T01:49:26.459 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 5002 2023-11-06T01:49:26.459 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 548 2023-11-06T01:49:26.459 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 549 2023-11-06T01:49:26.460 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102 2023-11-06T01:49:26.460 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6102_vm 2023-11-06T01:49:26.460 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104 2023-11-06T01:49:26.460 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6104_vm 2023-11-06T01:49:26.460 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106 2023-11-06T01:49:26.461 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6106_vm 2023-11-06T01:49:26.461 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6110 2023-11-06T01:49:26.461 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6111 2023-11-06T01:49:26.461 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6112 2023-11-06T01:49:26.461 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6113 2023-11-06T01:49:26.462 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6117 2023-11-06T01:49:26.462 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826 2023-11-06T01:49:26.462 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 826_vm 2023-11-06T01:49:26.462 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 827 2023-11-06T01:49:26.462 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 828 2023-11-06T01:49:26.462 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Nov 6 01:34 pg_filenode.map 2023-11-06T01:49:26.463 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Nov 6 01:34 pg_internal.init 2023-11-06T01:49:26.463 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:34 PG_VERSION 2023-11-06T01:49:26.463 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.463 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-11-06T01:49:26.463 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.464 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:49:26.464 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Nov 6 01:34 .. 2023-11-06T01:49:26.464 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.464 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/global: 2023-11-06T01:49:26.464 INFO:teuthology.orchestra.run.smithi053.stdout:total 574 2023-11-06T01:49:26.465 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Nov 6 01:34 . 2023-11-06T01:49:26.465 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.465 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1136 2023-11-06T01:49:26.465 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1136_fsm 2023-11-06T01:49:26.465 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1136_vm 2023-11-06T01:49:26.466 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 1137 2023-11-06T01:49:26.466 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1213 2023-11-06T01:49:26.466 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1213_fsm 2023-11-06T01:49:26.466 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1213_vm 2023-11-06T01:49:26.466 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1214 2023-11-06T01:49:26.467 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1214_fsm 2023-11-06T01:49:26.467 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1214_vm 2023-11-06T01:49:26.467 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 1232 2023-11-06T01:49:26.467 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 1233 2023-11-06T01:49:26.467 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1260 2023-11-06T01:49:26.467 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1260_fsm 2023-11-06T01:49:26.468 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1260_vm 2023-11-06T01:49:26.468 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1261 2023-11-06T01:49:26.468 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1261_fsm 2023-11-06T01:49:26.468 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1261_vm 2023-11-06T01:49:26.468 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1262 2023-11-06T01:49:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 1262_fsm 2023-11-06T01:49:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 1262_vm 2023-11-06T01:49:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2396 2023-11-06T01:49:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Nov 6 01:34 2396_fsm 2023-11-06T01:49:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2396_vm 2023-11-06T01:49:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2397 2023-11-06T01:49:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2671 2023-11-06T01:49:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2672 2023-11-06T01:49:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2676 2023-11-06T01:49:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2677 2023-11-06T01:49:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2694 2023-11-06T01:49:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2695 2023-11-06T01:49:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2697 2023-11-06T01:49:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Nov 6 01:34 2698 2023-11-06T01:49:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2846 2023-11-06T01:49:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2846_vm 2023-11-06T01:49:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2847 2023-11-06T01:49:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2964 2023-11-06T01:49:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2964_vm 2023-11-06T01:49:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2965 2023-11-06T01:49:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2966 2023-11-06T01:49:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 2966_vm 2023-11-06T01:49:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 2967 2023-11-06T01:49:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3592 2023-11-06T01:49:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 3592_vm 2023-11-06T01:49:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 3593 2023-11-06T01:49:26.473 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 4060 2023-11-06T01:49:26.474 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 4060_vm 2023-11-06T01:49:26.474 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 4061 2023-11-06T01:49:26.474 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6000 2023-11-06T01:49:26.474 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6000_vm 2023-11-06T01:49:26.474 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6001 2023-11-06T01:49:26.475 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6002 2023-11-06T01:49:26.475 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6100 2023-11-06T01:49:26.475 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 6100_vm 2023-11-06T01:49:26.475 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6114 2023-11-06T01:49:26.475 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 6115 2023-11-06T01:49:26.476 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:49 pg_control 2023-11-06T01:49:26.476 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Nov 6 01:34 pg_filenode.map 2023-11-06T01:49:26.476 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Nov 6 01:34 pg_internal.init 2023-11-06T01:49:26.476 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.476 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/log: 2023-11-06T01:49:26.476 INFO:teuthology.orchestra.run.smithi053.stdout:total 3 2023-11-06T01:49:26.477 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 . 2023-11-06T01:49:26.477 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.477 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2808 Nov 6 01:49 postgresql-Mon.log 2023-11-06T01:49:26.477 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.477 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-11-06T01:49:26.478 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.478 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:49:26.478 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.478 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.478 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-11-06T01:49:26.479 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.479 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:49:26.479 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.479 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.479 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-11-06T01:49:26.479 INFO:teuthology.orchestra.run.smithi053.stdout:total 1 2023-11-06T01:49:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:49 . 2023-11-06T01:49:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 mappings 2023-11-06T01:49:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Nov 6 01:49 replorigin_checkpoint 2023-11-06T01:49:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 snapshots 2023-11-06T01:49:26.481 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.481 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-11-06T01:49:26.481 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.481 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:49:26.481 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:49 .. 2023-11-06T01:49:26.482 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-11-06T01:49:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:49:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:49 .. 2023-11-06T01:49:26.482 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-11-06T01:49:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:34 . 2023-11-06T01:49:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 members 2023-11-06T01:49:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 offsets 2023-11-06T01:49:26.484 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-11-06T01:49:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:total 8 2023-11-06T01:49:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 . 2023-11-06T01:49:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:34 .. 2023-11-06T01:49:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 0000 2023-11-06T01:49:26.485 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-11-06T01:49:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:total 8 2023-11-06T01:49:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 . 2023-11-06T01:49:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:34 .. 2023-11-06T01:49:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 0000 2023-11-06T01:49:26.486 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-11-06T01:49:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:total 8 2023-11-06T01:49:26.487 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:34 . 2023-11-06T01:49:26.487 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.487 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Nov 6 01:34 0000 2023-11-06T01:49:26.487 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.487 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-11-06T01:49:26.488 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.488 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:49:26.488 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.488 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.488 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-11-06T01:49:26.488 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.489 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:49:26.489 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.489 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.489 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-11-06T01:49:26.489 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.490 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:49:26.490 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.490 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.490 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-11-06T01:49:26.490 INFO:teuthology.orchestra.run.smithi053.stdout:total 13 2023-11-06T01:49:26.491 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Nov 6 01:49 . 2023-11-06T01:49:26.491 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.491 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Nov 6 01:49 db_0.stat 2023-11-06T01:49:26.491 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Nov 6 01:49 db_12347.stat 2023-11-06T01:49:26.491 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Nov 6 01:49 global.stat 2023-11-06T01:49:26.492 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.492 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-11-06T01:49:26.492 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.492 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:49 . 2023-11-06T01:49:26.492 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.492 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.493 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-11-06T01:49:26.493 INFO:teuthology.orchestra.run.smithi053.stdout:total 64 2023-11-06T01:49:26.493 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Nov 6 01:49 . 2023-11-06T01:49:26.493 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.493 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Nov 6 01:49 001D 2023-11-06T01:49:26.494 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.494 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-11-06T01:49:26.494 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.494 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:49:26.494 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.495 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.495 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-11-06T01:49:26.495 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.495 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:49:26.495 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.496 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.496 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-11-06T01:49:26.496 INFO:teuthology.orchestra.run.smithi053.stdout:total 1032192 2023-11-06T01:49:26.496 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Nov 6 01:49 . 2023-11-06T01:49:26.496 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.496 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 000000010000000200000027 2023-11-06T01:49:26.497 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 000000010000000200000028 2023-11-06T01:49:26.497 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 000000010000000200000029 2023-11-06T01:49:26.497 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 00000001000000020000002A 2023-11-06T01:49:26.497 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 00000001000000020000002B 2023-11-06T01:49:26.497 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 00000001000000020000002C 2023-11-06T01:49:26.498 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 00000001000000020000002D 2023-11-06T01:49:26.498 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 00000001000000020000002E 2023-11-06T01:49:26.498 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 00000001000000020000002F 2023-11-06T01:49:26.498 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 000000010000000200000030 2023-11-06T01:49:26.498 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 000000010000000200000031 2023-11-06T01:49:26.499 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 000000010000000200000032 2023-11-06T01:49:26.499 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 000000010000000200000033 2023-11-06T01:49:26.499 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 000000010000000200000034 2023-11-06T01:49:26.499 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:49 000000010000000200000035 2023-11-06T01:49:26.499 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:49 000000010000000200000036 2023-11-06T01:49:26.499 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:49 000000010000000200000037 2023-11-06T01:49:26.500 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:49 000000010000000200000038 2023-11-06T01:49:26.500 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:49 000000010000000200000039 2023-11-06T01:49:26.500 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:49 00000001000000020000003A 2023-11-06T01:49:26.500 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:49 00000001000000020000003B 2023-11-06T01:49:26.500 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 00000001000000020000003C 2023-11-06T01:49:26.501 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 00000001000000020000003D 2023-11-06T01:49:26.501 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 00000001000000020000003E 2023-11-06T01:49:26.501 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 00000001000000020000003F 2023-11-06T01:49:26.501 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 000000010000000200000040 2023-11-06T01:49:26.501 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 000000010000000200000041 2023-11-06T01:49:26.502 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 000000010000000200000042 2023-11-06T01:49:26.502 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 000000010000000200000043 2023-11-06T01:49:26.502 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 000000010000000200000044 2023-11-06T01:49:26.502 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 000000010000000200000045 2023-11-06T01:49:26.502 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000046 2023-11-06T01:49:26.502 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 000000010000000200000047 2023-11-06T01:49:26.503 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000048 2023-11-06T01:49:26.503 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 000000010000000200000049 2023-11-06T01:49:26.503 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 00000001000000020000004A 2023-11-06T01:49:26.504 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 00000001000000020000004B 2023-11-06T01:49:26.505 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:46 00000001000000020000004C 2023-11-06T01:49:26.505 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 00000001000000020000004D 2023-11-06T01:49:26.505 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 00000001000000020000004E 2023-11-06T01:49:26.505 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 00000001000000020000004F 2023-11-06T01:49:26.505 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000050 2023-11-06T01:49:26.506 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000051 2023-11-06T01:49:26.506 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000052 2023-11-06T01:49:26.506 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 000000010000000200000053 2023-11-06T01:49:26.506 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000054 2023-11-06T01:49:26.506 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000055 2023-11-06T01:49:26.507 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000056 2023-11-06T01:49:26.507 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000057 2023-11-06T01:49:26.507 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 000000010000000200000058 2023-11-06T01:49:26.507 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000059 2023-11-06T01:49:26.507 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 00000001000000020000005A 2023-11-06T01:49:26.507 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 00000001000000020000005B 2023-11-06T01:49:26.508 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 00000001000000020000005C 2023-11-06T01:49:26.508 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 00000001000000020000005D 2023-11-06T01:49:26.508 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 00000001000000020000005E 2023-11-06T01:49:26.508 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 00000001000000020000005F 2023-11-06T01:49:26.508 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000060 2023-11-06T01:49:26.509 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000061 2023-11-06T01:49:26.509 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000062 2023-11-06T01:49:26.509 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000063 2023-11-06T01:49:26.509 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:48 000000010000000200000064 2023-11-06T01:49:26.509 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Nov 6 01:47 000000010000000200000065 2023-11-06T01:49:26.510 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 archive_status 2023-11-06T01:49:26.510 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.510 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-11-06T01:49:26.510 INFO:teuthology.orchestra.run.smithi053.stdout:total 0 2023-11-06T01:49:26.510 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Nov 6 01:34 . 2023-11-06T01:49:26.511 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Nov 6 01:49 .. 2023-11-06T01:49:26.511 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.511 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-11-06T01:49:26.511 INFO:teuthology.orchestra.run.smithi053.stdout:total 472 2023-11-06T01:49:26.511 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Nov 6 01:40 . 2023-11-06T01:49:26.511 INFO:teuthology.orchestra.run.smithi053.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Nov 6 01:49 .. 2023-11-06T01:49:26.512 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Nov 6 01:40 0000 2023-11-06T01:49:26.512 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Nov 6 01:49 0001 2023-11-06T01:49:26.512 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-11-06T01:49:26.523 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi053.front.sepia.ceph.com 2023-11-06T01:49:26.531 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-11-06T01:49:26.570 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-11-06T01:49:26.571 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-11-06T01:49:26.571 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-11-06T01:49:26.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:49:26.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.754+0000 7f107abcb700 1 -- 172.21.15.53:0/61457983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10741008a0 msgr2=0x7f1074108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:26.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.754+0000 7f107abcb700 1 --2- 172.21.15.53:0/61457983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10741008a0 0x7f1074108140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f106c009f90 tx=0x7f106c00b330 comp rx=0 tx=0).stop 2023-11-06T01:49:26.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.754+0000 7f107abcb700 1 -- 172.21.15.53:0/61457983 shutdown_connections 2023-11-06T01:49:26.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.754+0000 7f107abcb700 1 --2- 172.21.15.53:0/61457983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1074101290 0x7f10741016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:26.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.754+0000 7f107abcb700 1 --2- 172.21.15.53:0/61457983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10741008a0 0x7f1074108140 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:26.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.754+0000 7f107abcb700 1 --2- 172.21.15.53:0/61457983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10740ffee0 0x7f1074100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:26.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.754+0000 7f107abcb700 1 -- 172.21.15.53:0/61457983 >> 172.21.15.53:0/61457983 conn(0x7f10740fa950 msgr2=0x7f10740fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:26.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.754+0000 7f107abcb700 1 -- 172.21.15.53:0/61457983 shutdown_connections 2023-11-06T01:49:26.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.754+0000 7f107abcb700 1 -- 172.21.15.53:0/61457983 wait complete. 2023-11-06T01:49:26.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f107abcb700 1 Processor -- start 2023-11-06T01:49:26.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f107abcb700 1 -- start start 2023-11-06T01:49:26.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f107abcb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10740ffee0 0x7f10740ff960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:26.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f107abcb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10741008a0 0x7f107419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:26.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f107abcb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1074101290 0x7f107419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:26.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f107abcb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107419e680 con 0x7f10740ffee0 2023-11-06T01:49:26.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f107abcb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107419e7f0 con 0x7f1074101290 2023-11-06T01:49:26.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f107abcb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107419e9b0 con 0x7f10741008a0 2023-11-06T01:49:26.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f1078967700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10740ffee0 0x7f10740ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:26.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f1078967700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10740ffee0 0x7f10740ff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44138/0 (socket says 172.21.15.53:44138) 2023-11-06T01:49:26.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f1078967700 1 -- 172.21.15.53:0/2723321227 learned_addr learned my addr 172.21.15.53:0/2723321227 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:26.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f1073fff700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10741008a0 0x7f107419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:26.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f1079168700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1074101290 0x7f107419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:26.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f1078967700 1 -- 172.21.15.53:0/2723321227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10741008a0 msgr2=0x7f107419b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:26.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f1078967700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10741008a0 0x7f107419b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:26.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f1078967700 1 -- 172.21.15.53:0/2723321227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1074101290 msgr2=0x7f107419deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:26.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f1078967700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1074101290 0x7f107419deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:26.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f1078967700 1 -- 172.21.15.53:0/2723321227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068009520 con 0x7f10740ffee0 2023-11-06T01:49:26.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.755+0000 7f1073fff700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10741008a0 0x7f107419b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:49:26.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.756+0000 7f1078967700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10740ffee0 0x7f10740ff960 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f106800ea50 tx=0x7f106800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:26.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.756+0000 7f1071ffb700 1 -- 172.21.15.53:0/2723321227 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106800cbd0 con 0x7f10740ffee0 2023-11-06T01:49:26.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.756+0000 7f107abcb700 1 -- 172.21.15.53:0/2723321227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c009c40 con 0x7f10740ffee0 2023-11-06T01:49:26.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.756+0000 7f1071ffb700 1 -- 172.21.15.53:0/2723321227 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f106800cd30 con 0x7f10740ffee0 2023-11-06T01:49:26.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.756+0000 7f1071ffb700 1 -- 172.21.15.53:0/2723321227 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068018680 con 0x7f10740ffee0 2023-11-06T01:49:26.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.756+0000 7f107abcb700 1 -- 172.21.15.53:0/2723321227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107419f010 con 0x7f10740ffee0 2023-11-06T01:49:26.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.757+0000 7f107abcb700 1 -- 172.21.15.53:0/2723321227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1074106be0 con 0x7f10740ffee0 2023-11-06T01:49:26.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.758+0000 7f1071ffb700 1 -- 172.21.15.53:0/2723321227 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f10680188a0 con 0x7f10740ffee0 2023-11-06T01:49:26.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.758+0000 7f1071ffb700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10640a0570 0x7f10640a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:26.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.759+0000 7f1071ffb700 1 -- 172.21.15.53:0/2723321227 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1068014070 con 0x7f10740ffee0 2023-11-06T01:49:26.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.759+0000 7f1073fff700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10640a0570 0x7f10640a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:26.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.759+0000 7f1073fff700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10640a0570 0x7f10640a2a30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f10740ff6d0 tx=0x7f1060009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:26.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.763+0000 7f1071ffb700 1 -- 172.21.15.53:0/2723321227 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1068095a70 con 0x7f10740ffee0 2023-11-06T01:49:26.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.969+0000 7f107abcb700 1 -- 172.21.15.53:0/2723321227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f107402cc70 con 0x7f10740ffee0 2023-11-06T01:49:26.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.971+0000 7f1071ffb700 1 -- 172.21.15.53:0/2723321227 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22543 (secure 0 0 0) 0x7f1068099090 con 0x7f10740ffee0 2023-11-06T01:49:26.973 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:26.974 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:33:27.113678+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:49:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.973+0000 7f107abcb700 1 -- 172.21.15.53:0/2723321227 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10640a0570 msgr2=0x7f10640a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.973+0000 7f107abcb700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10640a0570 0x7f10640a2a30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f10740ff6d0 tx=0x7f1060009310 comp rx=0 tx=0).stop 2023-11-06T01:49:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.973+0000 7f107abcb700 1 -- 172.21.15.53:0/2723321227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10740ffee0 msgr2=0x7f10740ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.973+0000 7f107abcb700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10740ffee0 0x7f10740ff960 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f106800ea50 tx=0x7f106800ed60 comp rx=0 tx=0).stop 2023-11-06T01:49:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.973+0000 7f107abcb700 1 -- 172.21.15.53:0/2723321227 shutdown_connections 2023-11-06T01:49:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.973+0000 7f107abcb700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f10640a0570 0x7f10640a2a30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.973+0000 7f107abcb700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1074101290 0x7f107419deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.973+0000 7f107abcb700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10740ffee0 0x7f10740ff960 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.973+0000 7f107abcb700 1 --2- 172.21.15.53:0/2723321227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10741008a0 0x7f107419b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:26.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.973+0000 7f107abcb700 1 -- 172.21.15.53:0/2723321227 >> 172.21.15.53:0/2723321227 conn(0x7f10740fa950 msgr2=0x7f10740fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:26.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.974+0000 7f107abcb700 1 -- 172.21.15.53:0/2723321227 shutdown_connections 2023-11-06T01:49:26.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:26.974+0000 7f107abcb700 1 -- 172.21.15.53:0/2723321227 wait complete. 2023-11-06T01:49:26.986 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T01:32:24.604601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T01:32:25.457741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:49:26.987 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T01:49:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.117+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/3963150712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b540ffcb0 msgr2=0x7f5b541000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.117+0000 7f5b5b9f0700 1 --2- 172.21.15.53:0/3963150712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b540ffcb0 0x7f5b541000d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5b500099c0 tx=0x7f5b50009cd0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.117+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/3963150712 shutdown_connections 2023-11-06T01:49:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.117+0000 7f5b5b9f0700 1 --2- 172.21.15.53:0/3963150712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b54101840 0x7f5b541080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.117+0000 7f5b5b9f0700 1 --2- 172.21.15.53:0/3963150712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b540ffcb0 0x7f5b541000d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.117+0000 7f5b5b9f0700 1 --2- 172.21.15.53:0/3963150712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b54100ea0 0x7f5b54101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.117+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/3963150712 >> 172.21.15.53:0/3963150712 conn(0x7f5b540fb220 msgr2=0x7f5b540fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.117+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/3963150712 shutdown_connections 2023-11-06T01:49:27.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.117+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/3963150712 wait complete. 2023-11-06T01:49:27.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.118+0000 7f5b5b9f0700 1 Processor -- start 2023-11-06T01:49:27.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.118+0000 7f5b5b9f0700 1 -- start start 2023-11-06T01:49:27.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.118+0000 7f5b5b9f0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b540ffcb0 0x7f5b541959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.118+0000 7f5b5b9f0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b54100ea0 0x7f5b54195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.118+0000 7f5b5b9f0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b54101840 0x7f5b5419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5b9f0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419d520 con 0x7f5b540ffcb0 2023-11-06T01:49:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5b9f0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419d660 con 0x7f5b54101840 2023-11-06T01:49:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5b9f0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419d7f0 con 0x7f5b54100ea0 2023-11-06T01:49:27.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5978c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b540ffcb0 0x7f5b541959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5978c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b540ffcb0 0x7f5b541959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44152/0 (socket says 172.21.15.53:44152) 2023-11-06T01:49:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5978c700 1 -- 172.21.15.53:0/4015473385 learned_addr learned my addr 172.21.15.53:0/4015473385 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b58f8b700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b54100ea0 0x7f5b54195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b59f8d700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b54101840 0x7f5b5419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5978c700 1 -- 172.21.15.53:0/4015473385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b54100ea0 msgr2=0x7f5b54195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5978c700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b54100ea0 0x7f5b54195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5978c700 1 -- 172.21.15.53:0/4015473385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b54101840 msgr2=0x7f5b5419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5978c700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b54101840 0x7f5b5419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5978c700 1 -- 172.21.15.53:0/4015473385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b500095f0 con 0x7f5b540ffcb0 2023-11-06T01:49:27.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5978c700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b540ffcb0 0x7f5b541959a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5b500058e0 tx=0x7f5b5000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:27.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b4a7fc700 1 -- 172.21.15.53:0/4015473385 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5001d070 con 0x7f5b540ffcb0 2023-11-06T01:49:27.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b4a7fc700 1 -- 172.21.15.53:0/4015473385 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b50005050 con 0x7f5b540ffcb0 2023-11-06T01:49:27.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/4015473385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5419da70 con 0x7f5b540ffcb0 2023-11-06T01:49:27.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b4a7fc700 1 -- 172.21.15.53:0/4015473385 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50004b30 con 0x7f5b540ffcb0 2023-11-06T01:49:27.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.119+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/4015473385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5419de90 con 0x7f5b540ffcb0 2023-11-06T01:49:27.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.120+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/4015473385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b54106c30 con 0x7f5b540ffcb0 2023-11-06T01:49:27.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.121+0000 7f5b4a7fc700 1 -- 172.21.15.53:0/4015473385 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5b500051c0 con 0x7f5b540ffcb0 2023-11-06T01:49:27.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.122+0000 7f5b4a7fc700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5b400a0520 0x7f5b400a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.122+0000 7f5b4a7fc700 1 -- 172.21.15.53:0/4015473385 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5b500cc780 con 0x7f5b540ffcb0 2023-11-06T01:49:27.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.122+0000 7f5b58f8b700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5b400a0520 0x7f5b400a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.122+0000 7f5b58f8b700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5b400a0520 0x7f5b400a29e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009c60 tx=0x7f5b44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:27.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.126+0000 7f5b4a7fc700 1 -- 172.21.15.53:0/4015473385 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5b50096390 con 0x7f5b540ffcb0 2023-11-06T01:49:27.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.329+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/4015473385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b54061a40 con 0x7f5b540ffcb0 2023-11-06T01:49:27.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.329+0000 7f5b4a7fc700 1 -- 172.21.15.53:0/4015473385 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b500999b0 con 0x7f5b540ffcb0 2023-11-06T01:49:27.329 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:49:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.331+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/4015473385 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5b400a0520 msgr2=0x7f5b400a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.331+0000 7f5b5b9f0700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5b400a0520 0x7f5b400a29e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009c60 tx=0x7f5b44009260 comp rx=0 tx=0).stop 2023-11-06T01:49:27.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.332+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/4015473385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b540ffcb0 msgr2=0x7f5b541959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.332+0000 7f5b5b9f0700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b540ffcb0 0x7f5b541959a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5b500058e0 tx=0x7f5b5000f690 comp rx=0 tx=0).stop 2023-11-06T01:49:27.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.332+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/4015473385 shutdown_connections 2023-11-06T01:49:27.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.332+0000 7f5b5b9f0700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f5b400a0520 0x7f5b400a29e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.332+0000 7f5b5b9f0700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b54101840 0x7f5b5419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.332+0000 7f5b5b9f0700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b540ffcb0 0x7f5b541959a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.332+0000 7f5b5b9f0700 1 --2- 172.21.15.53:0/4015473385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b54100ea0 0x7f5b54195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.332+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/4015473385 >> 172.21.15.53:0/4015473385 conn(0x7f5b540fb220 msgr2=0x7f5b540fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:27.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.332+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/4015473385 shutdown_connections 2023-11-06T01:49:27.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.332+0000 7f5b5b9f0700 1 -- 172.21.15.53:0/4015473385 wait complete. 2023-11-06T01:49:27.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:27 smithi053 ceph-mon[99472]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.26k op/s 2023-11-06T01:49:27.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:27 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2723321227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:49:27.343 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T01:49:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.474+0000 7f498b785700 1 -- 172.21.15.53:0/2855944451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984101810 msgr2=0x7f49841080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.474+0000 7f498b785700 1 --2- 172.21.15.53:0/2855944451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984101810 0x7f49841080c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f497c00b0d0 tx=0x7f497c00b3e0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.474+0000 7f498b785700 1 -- 172.21.15.53:0/2855944451 shutdown_connections 2023-11-06T01:49:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.474+0000 7f498b785700 1 --2- 172.21.15.53:0/2855944451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49840ffc80 0x7f49841000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.474+0000 7f498b785700 1 --2- 172.21.15.53:0/2855944451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984101810 0x7f49841080c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.474+0000 7f498b785700 1 --2- 172.21.15.53:0/2855944451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4984100e70 0x7f49841012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.474+0000 7f498b785700 1 -- 172.21.15.53:0/2855944451 >> 172.21.15.53:0/2855944451 conn(0x7f49840fb1f0 msgr2=0x7f49840fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.474+0000 7f498b785700 1 -- 172.21.15.53:0/2855944451 shutdown_connections 2023-11-06T01:49:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.474+0000 7f498b785700 1 -- 172.21.15.53:0/2855944451 wait complete. 2023-11-06T01:49:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f498b785700 1 Processor -- start 2023-11-06T01:49:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f498b785700 1 -- start start 2023-11-06T01:49:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f498b785700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49840ffc80 0x7f4984195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f498b785700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100e70 0x7f4984195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f498b785700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984101810 0x7f498419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f498b785700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498419d5f0 con 0x7f4984100e70 2023-11-06T01:49:27.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f498b785700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498419d730 con 0x7f4984101810 2023-11-06T01:49:27.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f498b785700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498419d8c0 con 0x7f49840ffc80 2023-11-06T01:49:27.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4988d20700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100e70 0x7f4984195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4989521700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49840ffc80 0x7f4984195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4989d22700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984101810 0x7f498419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4988d20700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100e70 0x7f4984195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44166/0 (socket says 172.21.15.53:44166) 2023-11-06T01:49:27.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4988d20700 1 -- 172.21.15.53:0/3821951875 learned_addr learned my addr 172.21.15.53:0/3821951875 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:27.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4989521700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49840ffc80 0x7f4984195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58404/0 (socket says 172.21.15.53:58404) 2023-11-06T01:49:27.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4989d22700 1 -- 172.21.15.53:0/3821951875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49840ffc80 msgr2=0x7f4984195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4989d22700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49840ffc80 0x7f4984195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4989d22700 1 -- 172.21.15.53:0/3821951875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100e70 msgr2=0x7f4984195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4989d22700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100e70 0x7f4984195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4989d22700 1 -- 172.21.15.53:0/3821951875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c009d00 con 0x7f4984101810 2023-11-06T01:49:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4988d20700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100e70 0x7f4984195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:49:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.475+0000 7f4989d22700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984101810 0x7f498419d020 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f497c00b770 tx=0x7f497c008ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.476+0000 7f49767fc700 1 -- 172.21.15.53:0/3821951875 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c00e040 con 0x7f4984101810 2023-11-06T01:49:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.476+0000 7f498b785700 1 -- 172.21.15.53:0/3821951875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4984067ef0 con 0x7f4984101810 2023-11-06T01:49:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.476+0000 7f49767fc700 1 -- 172.21.15.53:0/3821951875 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f497c01d930 con 0x7f4984101810 2023-11-06T01:49:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.476+0000 7f49767fc700 1 -- 172.21.15.53:0/3821951875 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c01b8f0 con 0x7f4984101810 2023-11-06T01:49:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.476+0000 7f498b785700 1 -- 172.21.15.53:0/3821951875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4984068450 con 0x7f4984101810 2023-11-06T01:49:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.477+0000 7f498b785700 1 -- 172.21.15.53:0/3821951875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498410b4a0 con 0x7f4984101810 2023-11-06T01:49:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.478+0000 7f49767fc700 1 -- 172.21.15.53:0/3821951875 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f497c01daa0 con 0x7f4984101810 2023-11-06T01:49:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.479+0000 7f49767fc700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49700a0520 0x7f49700a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.479+0000 7f4989521700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49700a0520 0x7f49700a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.479+0000 7f49767fc700 1 -- 172.21.15.53:0/3821951875 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f497c0cd0a0 con 0x7f4984101810 2023-11-06T01:49:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.479+0000 7f4989521700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49700a0520 0x7f49700a29e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4984100cd0 tx=0x7f4980009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.483+0000 7f49767fc700 1 -- 172.21.15.53:0/3821951875 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f497c096cb0 con 0x7f4984101810 2023-11-06T01:49:27.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:27 smithi163 ceph-mon[104416]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.26k op/s 2023-11-06T01:49:27.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:27 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2723321227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:49:27.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:27 smithi062 ceph-mon[103924]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.26k op/s 2023-11-06T01:49:27.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:27 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2723321227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:49:27.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.688+0000 7f498b785700 1 -- 172.21.15.53:0/3821951875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f498410b660 con 0x7f4984101810 2023-11-06T01:49:27.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.688+0000 7f49767fc700 1 -- 172.21.15.53:0/3821951875 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f497c09a2d0 con 0x7f4984101810 2023-11-06T01:49:27.688 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-06T01:49:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.690+0000 7f498b785700 1 -- 172.21.15.53:0/3821951875 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49700a0520 msgr2=0x7f49700a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.690+0000 7f498b785700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49700a0520 0x7f49700a29e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4984100cd0 tx=0x7f4980009260 comp rx=0 tx=0).stop 2023-11-06T01:49:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.690+0000 7f498b785700 1 -- 172.21.15.53:0/3821951875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984101810 msgr2=0x7f498419d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.690+0000 7f498b785700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984101810 0x7f498419d020 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f497c00b770 tx=0x7f497c008ec0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.690+0000 7f498b785700 1 -- 172.21.15.53:0/3821951875 shutdown_connections 2023-11-06T01:49:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.690+0000 7f498b785700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f49700a0520 0x7f49700a29e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.690+0000 7f498b785700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984101810 0x7f498419d020 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.690+0000 7f498b785700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49840ffc80 0x7f4984195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.690+0000 7f498b785700 1 --2- 172.21.15.53:0/3821951875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100e70 0x7f4984195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.690+0000 7f498b785700 1 -- 172.21.15.53:0/3821951875 >> 172.21.15.53:0/3821951875 conn(0x7f49840fb1f0 msgr2=0x7f49840fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.691+0000 7f498b785700 1 -- 172.21.15.53:0/3821951875 shutdown_connections 2023-11-06T01:49:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.691+0000 7f498b785700 1 -- 172.21.15.53:0/3821951875 wait complete. 2023-11-06T01:49:27.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T01:49:27.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.833+0000 7fe738dd7700 1 -- 172.21.15.53:0/1714530387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe734101880 msgr2=0x7fe734103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.833+0000 7fe738dd7700 1 --2- 172.21.15.53:0/1714530387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe734101880 0x7fe734103c70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe724009990 tx=0x7fe724009ca0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.833+0000 7fe738dd7700 1 -- 172.21.15.53:0/1714530387 shutdown_connections 2023-11-06T01:49:27.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.833+0000 7fe738dd7700 1 --2- 172.21.15.53:0/1714530387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe734101880 0x7fe734103c70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.833+0000 7fe738dd7700 1 --2- 172.21.15.53:0/1714530387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734106ae0 0x7fe734108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.833+0000 7fe738dd7700 1 --2- 172.21.15.53:0/1714530387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7341041b0 0x7fe7341065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.833+0000 7fe738dd7700 1 -- 172.21.15.53:0/1714530387 >> 172.21.15.53:0/1714530387 conn(0x7fe7340fb1f0 msgr2=0x7fe7340fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:27.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.833+0000 7fe738dd7700 1 -- 172.21.15.53:0/1714530387 shutdown_connections 2023-11-06T01:49:27.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.833+0000 7fe738dd7700 1 -- 172.21.15.53:0/1714530387 wait complete. 2023-11-06T01:49:27.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe738dd7700 1 Processor -- start 2023-11-06T01:49:27.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe738dd7700 1 -- start start 2023-11-06T01:49:27.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe738dd7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734101880 0x7fe734196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe738dd7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7341041b0 0x7fe734196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe738dd7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe734106ae0 0x7fe73419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe738dd7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73419e5d0 con 0x7fe734106ae0 2023-11-06T01:49:27.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe738dd7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73419e710 con 0x7fe7341041b0 2023-11-06T01:49:27.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe738dd7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73419e8a0 con 0x7fe734101880 2023-11-06T01:49:27.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe73259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734101880 0x7fe734196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe732d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe734106ae0 0x7fe73419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe731d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7341041b0 0x7fe734196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe73259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734101880 0x7fe734196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58418/0 (socket says 172.21.15.53:58418) 2023-11-06T01:49:27.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe73259c700 1 -- 172.21.15.53:0/1989121313 learned_addr learned my addr 172.21.15.53:0/1989121313 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:27.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe73259c700 1 -- 172.21.15.53:0/1989121313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7341041b0 msgr2=0x7fe734196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe73259c700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7341041b0 0x7fe734196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe73259c700 1 -- 172.21.15.53:0/1989121313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe734106ae0 msgr2=0x7fe73419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:27.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe73259c700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe734106ae0 0x7fe73419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:27.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe73259c700 1 -- 172.21.15.53:0/1989121313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c009520 con 0x7fe734101880 2023-11-06T01:49:27.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe731d9b700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7341041b0 0x7fe734196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:49:27.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.834+0000 7fe732d9d700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe734106ae0 0x7fe73419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:49:27.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.835+0000 7fe73259c700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734101880 0x7fe734196a50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe7240059d0 tx=0x7fe724004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:27.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.835+0000 7fe72b7fe700 1 -- 172.21.15.53:0/1989121313 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72401d070 con 0x7fe734101880 2023-11-06T01:49:27.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.835+0000 7fe72b7fe700 1 -- 172.21.15.53:0/1989121313 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe72400fb90 con 0x7fe734101880 2023-11-06T01:49:27.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.835+0000 7fe738dd7700 1 -- 172.21.15.53:0/1989121313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7240095f0 con 0x7fe734101880 2023-11-06T01:49:27.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.835+0000 7fe72b7fe700 1 -- 172.21.15.53:0/1989121313 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7240173f0 con 0x7fe734101880 2023-11-06T01:49:27.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.835+0000 7fe738dd7700 1 -- 172.21.15.53:0/1989121313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73419edb0 con 0x7fe734101880 2023-11-06T01:49:27.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.836+0000 7fe738dd7700 1 -- 172.21.15.53:0/1989121313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73410c560 con 0x7fe734101880 2023-11-06T01:49:27.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.837+0000 7fe72b7fe700 1 -- 172.21.15.53:0/1989121313 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe72400fd00 con 0x7fe734101880 2023-11-06T01:49:27.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.837+0000 7fe72b7fe700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7200a0520 0x7fe7200a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:27.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.838+0000 7fe72b7fe700 1 -- 172.21.15.53:0/1989121313 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe7240ccf70 con 0x7fe734101880 2023-11-06T01:49:27.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.838+0000 7fe731d9b700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7200a0520 0x7fe7200a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:27.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.838+0000 7fe731d9b700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7200a0520 0x7fe7200a29e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009fd0 tx=0x7fe71c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:27.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:27.842+0000 7fe72b7fe700 1 -- 172.21.15.53:0/1989121313 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe724096b80 con 0x7fe734101880 2023-11-06T01:49:28.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.041+0000 7fe738dd7700 1 -- 172.21.15.53:0/1989121313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe73419f3f0 con 0x7fe734101880 2023-11-06T01:49:28.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.042+0000 7fe72b7fe700 1 -- 172.21.15.53:0/1989121313 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe72409a1a0 con 0x7fe734101880 2023-11-06T01:49:28.041 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-06T01:49:28.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 -- 172.21.15.53:0/1989121313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7200a0520 msgr2=0x7fe7200a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:28.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7200a0520 0x7fe7200a29e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009fd0 tx=0x7fe71c009260 comp rx=0 tx=0).stop 2023-11-06T01:49:28.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 -- 172.21.15.53:0/1989121313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734101880 msgr2=0x7fe734196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:28.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734101880 0x7fe734196a50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe7240059d0 tx=0x7fe724004730 comp rx=0 tx=0).stop 2023-11-06T01:49:28.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 -- 172.21.15.53:0/1989121313 shutdown_connections 2023-11-06T01:49:28.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fe7200a0520 0x7fe7200a29e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:28.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe734106ae0 0x7fe73419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:28.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734101880 0x7fe734196a50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:28.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 --2- 172.21.15.53:0/1989121313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7341041b0 0x7fe734196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:28.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 -- 172.21.15.53:0/1989121313 >> 172.21.15.53:0/1989121313 conn(0x7fe7340fb1f0 msgr2=0x7fe734102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:28.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 -- 172.21.15.53:0/1989121313 shutdown_connections 2023-11-06T01:49:28.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.044+0000 7fe738dd7700 1 -- 172.21.15.53:0/1989121313 wait complete. 2023-11-06T01:49:28.055 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-06T01:49:28.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.183+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/3777315825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54100e60 msgr2=0x7f6c541012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:28.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.183+0000 7f6c5b8b3700 1 --2- 172.21.15.53:0/3777315825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54100e60 0x7f6c541012c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6c440099c0 tx=0x7f6c44009cd0 comp rx=0 tx=0).stop 2023-11-06T01:49:28.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.183+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/3777315825 shutdown_connections 2023-11-06T01:49:28.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.183+0000 7f6c5b8b3700 1 --2- 172.21.15.53:0/3777315825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54101800 0x7f6c541080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:28.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.183+0000 7f6c5b8b3700 1 --2- 172.21.15.53:0/3777315825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c540ffc70 0x7f6c54100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:28.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.183+0000 7f6c5b8b3700 1 --2- 172.21.15.53:0/3777315825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54100e60 0x7f6c541012c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:28.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.183+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/3777315825 >> 172.21.15.53:0/3777315825 conn(0x7f6c540fb1c0 msgr2=0x7f6c540fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:28.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.183+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/3777315825 shutdown_connections 2023-11-06T01:49:28.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.183+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/3777315825 wait complete. 2023-11-06T01:49:28.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5b8b3700 1 Processor -- start 2023-11-06T01:49:28.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5b8b3700 1 -- start start 2023-11-06T01:49:28.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5b8b3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c540ffc70 0x7f6c54195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5b8b3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54100e60 0x7f6c54195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5b8b3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54101800 0x7f6c5419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5b8b3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419d490 con 0x7f6c54101800 2023-11-06T01:49:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5b8b3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419d5d0 con 0x7f6c54100e60 2023-11-06T01:49:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5b8b3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419d760 con 0x7f6c540ffc70 2023-11-06T01:49:28.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c59e50700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54101800 0x7f6c5419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:28.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5964f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c540ffc70 0x7f6c54195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:28.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5964f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c540ffc70 0x7f6c54195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58432/0 (socket says 172.21.15.53:58432) 2023-11-06T01:49:28.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5964f700 1 -- 172.21.15.53:0/1915206644 learned_addr learned my addr 172.21.15.53:0/1915206644 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:49:28.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c58e4e700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54100e60 0x7f6c54195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:28.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5964f700 1 -- 172.21.15.53:0/1915206644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54100e60 msgr2=0x7f6c54195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:28.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5964f700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54100e60 0x7f6c54195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:28.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5964f700 1 -- 172.21.15.53:0/1915206644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54101800 msgr2=0x7f6c5419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:28.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5964f700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54101800 0x7f6c5419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:28.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c5964f700 1 -- 172.21.15.53:0/1915206644 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440095f0 con 0x7f6c540ffc70 2023-11-06T01:49:28.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.184+0000 7f6c59e50700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54101800 0x7f6c5419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:49:28.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.185+0000 7f6c5964f700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c540ffc70 0x7f6c54195910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000e970 tx=0x7f6c5000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:28.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.185+0000 7f6c4a7fc700 1 -- 172.21.15.53:0/1915206644 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50009790 con 0x7f6c540ffc70 2023-11-06T01:49:28.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.185+0000 7f6c4a7fc700 1 -- 172.21.15.53:0/1915206644 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c5000c7e0 con 0x7f6c540ffc70 2023-11-06T01:49:28.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.185+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/1915206644 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5419da40 con 0x7f6c540ffc70 2023-11-06T01:49:28.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.185+0000 7f6c4a7fc700 1 -- 172.21.15.53:0/1915206644 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50003c70 con 0x7f6c540ffc70 2023-11-06T01:49:28.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.185+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/1915206644 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5419df50 con 0x7f6c540ffc70 2023-11-06T01:49:28.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.186+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/1915206644 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5410b480 con 0x7f6c540ffc70 2023-11-06T01:49:28.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.187+0000 7f6c4a7fc700 1 -- 172.21.15.53:0/1915206644 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f6c5000cc50 con 0x7f6c540ffc70 2023-11-06T01:49:28.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.187+0000 7f6c4a7fc700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6c400a0570 0x7f6c400a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:49:28.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.187+0000 7f6c58e4e700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6c400a0570 0x7f6c400a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:49:28.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.187+0000 7f6c4a7fc700 1 -- 172.21.15.53:0/1915206644 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6c50014070 con 0x7f6c540ffc70 2023-11-06T01:49:28.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.188+0000 7f6c58e4e700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6c400a0570 0x7f6c400a2a30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400b580 tx=0x7f6c44005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:49:28.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.191+0000 7f6c4a7fc700 1 -- 172.21.15.53:0/1915206644 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6c500957c0 con 0x7f6c540ffc70 2023-11-06T01:49:28.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:28 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/4015473385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:49:28.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:28 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3821951875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:49:28.342 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:28 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1989121313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:49:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.456+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/1915206644 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f6c54106b90 con 0x7f6c540ffc70 2023-11-06T01:49:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.466+0000 7f6c4a7fc700 1 -- 172.21.15.53:0/1915206644 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+729705 (secure 0 0 0) 0x7f6c50003dd0 con 0x7f6c540ffc70 2023-11-06T01:49:28.473 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:49:28.474 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:49:28.474 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 18, 2023-11-06T01:49:28.474 INFO:teuthology.orchestra.run.smithi053.stdout: "active_gid": 14152, 2023-11-06T01:49:28.474 INFO:teuthology.orchestra.run.smithi053.stdout: "active_name": "x", 2023-11-06T01:49:28.475 INFO:teuthology.orchestra.run.smithi053.stdout: "active_addrs": { 2023-11-06T01:49:28.475 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2023-11-06T01:49:28.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:28.475 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2023-11-06T01:49:28.476 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6800", 2023-11-06T01:49:28.476 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1491433716 2023-11-06T01:49:28.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:28.477 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2023-11-06T01:49:28.477 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2023-11-06T01:49:28.477 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1491433716 2023-11-06T01:49:28.477 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:28.477 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-06T01:49:28.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.478 INFO:teuthology.orchestra.run.smithi053.stdout: "active_addr": "172.21.15.53:6801/1491433716", 2023-11-06T01:49:28.478 INFO:teuthology.orchestra.run.smithi053.stdout: "active_change": "2023-11-06T01:26:25.398279+0000", 2023-11-06T01:49:28.478 INFO:teuthology.orchestra.run.smithi053.stdout: "active_mgr_features": 4540138322906710015, 2023-11-06T01:49:28.478 INFO:teuthology.orchestra.run.smithi053.stdout: "available": true, 2023-11-06T01:49:28.479 INFO:teuthology.orchestra.run.smithi053.stdout: "standbys": [ 2023-11-06T01:49:28.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:28.479 INFO:teuthology.orchestra.run.smithi053.stdout: "gid": 24107, 2023-11-06T01:49:28.479 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "y", 2023-11-06T01:49:28.479 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_features": 4540138322906710015, 2023-11-06T01:49:28.480 INFO:teuthology.orchestra.run.smithi053.stdout: "available_modules": [ 2023-11-06T01:49:28.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:28.480 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "alerts", 2023-11-06T01:49:28.480 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:28.480 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:28.481 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:28.481 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:28.481 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:28.481 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.481 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.482 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.482 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:28.482 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.482 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How frequently to reexamine health status", 2023-11-06T01:49:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.484 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:28.484 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:28.484 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.485 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.485 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.485 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.485 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.485 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.486 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.486 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.486 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.486 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.487 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.487 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.487 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.487 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.488 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.488 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.488 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.488 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.489 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:28.489 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:28.489 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.490 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.490 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.490 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.490 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.490 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.490 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.491 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.491 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.491 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.491 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.492 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:28.492 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:28.492 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.492 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.493 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.493 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:28.493 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.493 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.493 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.494 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.494 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.494 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.494 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.494 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.496 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:28.496 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:28.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.496 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.497 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.497 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.497 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.497 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.497 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.498 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.498 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.498 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.498 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_destination": { 2023-11-06T01:49:28.499 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_destination", 2023-11-06T01:49:28.499 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.499 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.499 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.499 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.500 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.500 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.500 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.500 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email address to send alerts to", 2023-11-06T01:49:28.500 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.500 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.501 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.501 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_from_name": { 2023-11-06T01:49:28.501 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_from_name", 2023-11-06T01:49:28.501 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.502 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.502 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.502 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "Ceph", 2023-11-06T01:49:28.502 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.502 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.503 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.503 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email From: name", 2023-11-06T01:49:28.503 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.503 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.503 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.504 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_host": { 2023-11-06T01:49:28.504 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_host", 2023-11-06T01:49:28.504 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.505 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.505 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.505 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.505 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.505 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.505 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.506 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP server", 2023-11-06T01:49:28.506 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.506 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.506 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_password": { 2023-11-06T01:49:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_password", 2023-11-06T01:49:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.507 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.508 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Password to authenticate with", 2023-11-06T01:49:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_port": { 2023-11-06T01:49:28.509 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_port", 2023-11-06T01:49:28.510 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.510 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.510 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.510 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "465", 2023-11-06T01:49:28.510 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.511 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.511 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.511 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP port", 2023-11-06T01:49:28.511 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.512 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.512 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.512 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_sender": { 2023-11-06T01:49:28.512 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_sender", 2023-11-06T01:49:28.513 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.513 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.513 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.513 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.514 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.514 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.514 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.514 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP envelope sender", 2023-11-06T01:49:28.515 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.515 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.515 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.516 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_ssl": { 2023-11-06T01:49:28.516 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_ssl", 2023-11-06T01:49:28.516 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.516 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.517 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.517 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.517 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.517 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.518 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.518 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-06T01:49:28.518 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.518 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.519 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.519 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_user": { 2023-11-06T01:49:28.519 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_user", 2023-11-06T01:49:28.520 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.520 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.520 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.520 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.521 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.521 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.521 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.521 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "User to authenticate as", 2023-11-06T01:49:28.522 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.522 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.522 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.522 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:28.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:28.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:28.523 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "balancer", 2023-11-06T01:49:28.524 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:28.524 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:28.524 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:28.524 INFO:teuthology.orchestra.run.smithi053.stdout: "active": { 2023-11-06T01:49:28.524 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "active", 2023-11-06T01:49:28.525 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.525 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.525 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.525 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.526 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.526 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.526 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.526 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically balance PGs across cluster", 2023-11-06T01:49:28.526 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.527 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.527 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.527 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_time": { 2023-11-06T01:49:28.527 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_time", 2023-11-06T01:49:28.528 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.528 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.528 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.528 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0000", 2023-11-06T01:49:28.528 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.529 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.529 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.529 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "beginning time of day to automatically balance", 2023-11-06T01:49:28.529 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-06T01:49:28.530 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.530 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.530 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_weekday": { 2023-11-06T01:49:28.530 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_weekday", 2023-11-06T01:49:28.531 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:28.531 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.531 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.531 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:28.531 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2023-11-06T01:49:28.532 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2023-11-06T01:49:28.532 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.532 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-06T01:49:28.532 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-06T01:49:28.532 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.533 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.533 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_max_iterations": { 2023-11-06T01:49:28.533 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_max_iterations", 2023-11-06T01:49:28.533 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:28.533 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.534 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.534 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "25", 2023-11-06T01:49:28.534 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:28.534 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "250", 2023-11-06T01:49:28.534 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.535 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-06T01:49:28.535 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.535 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.535 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.536 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_metrics": { 2023-11-06T01:49:28.536 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_metrics", 2023-11-06T01:49:28.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.536 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.536 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "pgs,objects,bytes", 2023-11-06T01:49:28.537 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.537 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.537 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.537 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-06T01:49:28.537 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-06T01:49:28.538 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.538 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.538 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_step": { 2023-11-06T01:49:28.539 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_step", 2023-11-06T01:49:28.539 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:28.539 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.539 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.539 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.5", 2023-11-06T01:49:28.539 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0.001", 2023-11-06T01:49:28.540 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "0.999", 2023-11-06T01:49:28.540 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.540 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "aggressiveness of optimization", 2023-11-06T01:49:28.540 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-06T01:49:28.540 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.541 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.541 INFO:teuthology.orchestra.run.smithi053.stdout: "end_time": { 2023-11-06T01:49:28.541 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_time", 2023-11-06T01:49:28.541 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.542 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.542 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.542 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2359", 2023-11-06T01:49:28.542 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.542 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.543 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.543 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ending time of day to automatically balance", 2023-11-06T01:49:28.543 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-06T01:49:28.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.543 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.544 INFO:teuthology.orchestra.run.smithi053.stdout: "end_weekday": { 2023-11-06T01:49:28.544 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_weekday", 2023-11-06T01:49:28.544 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:28.544 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.544 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.545 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:28.545 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2023-11-06T01:49:28.545 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2023-11-06T01:49:28.546 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.546 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-06T01:49:28.546 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-06T01:49:28.546 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.546 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.547 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:28.547 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:28.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.548 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.548 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.548 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.548 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.548 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.549 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.549 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.549 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.549 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.550 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.550 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.550 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.550 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.551 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.551 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.551 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.551 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.552 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:28.552 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:28.552 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.552 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.552 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.553 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.553 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.553 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.553 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.553 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.554 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.554 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.554 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.555 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:28.555 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:28.555 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.556 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.556 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.556 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:28.556 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.557 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.557 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.557 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.557 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.558 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.558 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.558 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.559 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.559 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.559 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.559 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.560 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.560 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.560 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:28.561 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:28.561 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.561 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.561 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.561 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.562 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.562 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.562 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.562 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.562 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.563 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.563 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.563 INFO:teuthology.orchestra.run.smithi053.stdout: "min_score": { 2023-11-06T01:49:28.563 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_score", 2023-11-06T01:49:28.564 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:28.564 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.564 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.564 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:28.564 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.565 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.565 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.565 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-06T01:49:28.565 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.565 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.566 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.566 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2023-11-06T01:49:28.566 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2023-11-06T01:49:28.566 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.567 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.567 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.567 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "upmap", 2023-11-06T01:49:28.567 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.567 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.567 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.568 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-compat", 2023-11-06T01:49:28.568 INFO:teuthology.orchestra.run.smithi053.stdout: "none", 2023-11-06T01:49:28.568 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap" 2023-11-06T01:49:28.568 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.568 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Balancer mode", 2023-11-06T01:49:28.569 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.569 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.569 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.569 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_ids": { 2023-11-06T01:49:28.569 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_ids", 2023-11-06T01:49:28.570 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.570 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.570 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.570 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.570 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.571 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.571 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.571 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-06T01:49:28.571 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.571 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.571 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.572 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:28.572 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:28.572 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.573 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.573 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.573 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:28.573 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.573 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.574 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.574 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-06T01:49:28.574 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.574 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.575 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_deviation": { 2023-11-06T01:49:28.575 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_deviation", 2023-11-06T01:49:28.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.575 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.576 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.576 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:28.576 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:28.576 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.576 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.577 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-06T01:49:28.577 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-06T01:49:28.577 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.577 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.578 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_optimizations": { 2023-11-06T01:49:28.578 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_optimizations", 2023-11-06T01:49:28.578 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:28.578 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.578 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.579 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:28.579 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.579 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.579 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.579 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-06T01:49:28.580 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.580 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.580 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:28.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:28.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:28.581 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephadm", 2023-11-06T01:49:28.581 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:28.581 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:28.582 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:28.582 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_down_multiplier": { 2023-11-06T01:49:28.582 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_down_multiplier", 2023-11-06T01:49:28.582 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:28.582 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.583 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2023-11-06T01:49:28.583 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.583 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.584 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.584 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-06T01:49:28.584 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.584 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.585 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.585 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_refresh_rate": { 2023-11-06T01:49:28.585 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_refresh_rate", 2023-11-06T01:49:28.586 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.586 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.586 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.586 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "20", 2023-11-06T01:49:28.586 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.587 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.587 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.587 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-06T01:49:28.587 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.588 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.588 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.588 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_starting_port": { 2023-11-06T01:49:28.588 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_starting_port", 2023-11-06T01:49:28.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.589 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.589 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4721", 2023-11-06T01:49:28.590 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.590 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.590 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.590 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-06T01:49:28.591 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.591 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.592 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_ptrace": { 2023-11-06T01:49:28.592 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_ptrace", 2023-11-06T01:49:28.592 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.592 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.593 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.593 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.593 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.593 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.594 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.594 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-06T01:49:28.594 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-06T01:49:28.595 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.595 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.595 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_interval": { 2023-11-06T01:49:28.595 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_interval", 2023-11-06T01:49:28.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.596 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.596 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.596 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:28.596 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.596 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.597 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.597 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-06T01:49:28.597 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.597 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.598 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.598 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_memory_target_ratio": { 2023-11-06T01:49:28.598 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_memory_target_ratio", 2023-11-06T01:49:28.598 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:28.599 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.599 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.599 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.7", 2023-11-06T01:49:28.599 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.599 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.599 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.600 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-06T01:49:28.600 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.600 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.600 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.601 INFO:teuthology.orchestra.run.smithi053.stdout: "cgroups_split": { 2023-11-06T01:49:28.601 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cgroups_split", 2023-11-06T01:49:28.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.601 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.601 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.602 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.602 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.602 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.602 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.602 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-06T01:49:28.603 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.603 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.603 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.603 INFO:teuthology.orchestra.run.smithi053.stdout: "config_checks_enabled": { 2023-11-06T01:49:28.603 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_checks_enabled", 2023-11-06T01:49:28.604 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.606 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.606 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.606 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.606 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.606 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.607 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.607 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-06T01:49:28.607 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.607 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.608 INFO:teuthology.orchestra.run.smithi053.stdout: "config_dashboard": { 2023-11-06T01:49:28.608 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_dashboard", 2023-11-06T01:49:28.608 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.608 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.609 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.609 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.609 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.609 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.609 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.610 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-06T01:49:28.610 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.610 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.610 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.611 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_alertmanager": { 2023-11-06T01:49:28.611 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_alertmanager", 2023-11-06T01:49:28.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.611 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.611 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.612 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-06T01:49:28.612 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.612 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.612 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.612 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:28.612 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.613 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.613 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_base": { 2023-11-06T01:49:28.613 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_base", 2023-11-06T01:49:28.614 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.614 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.614 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.614 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-06T01:49:28.614 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.614 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.615 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.615 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Container image name, without the tag", 2023-11-06T01:49:28.615 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.615 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.615 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.616 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_elasticsearch": { 2023-11-06T01:49:28.616 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_elasticsearch", 2023-11-06T01:49:28.616 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.616 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.617 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.617 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-06T01:49:28.617 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.617 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.617 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.618 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "elasticsearch container image", 2023-11-06T01:49:28.618 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.618 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.619 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_grafana": { 2023-11-06T01:49:28.619 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_grafana", 2023-11-06T01:49:28.619 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.619 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.620 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.620 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-06T01:49:28.620 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.620 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.621 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.621 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:28.621 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.621 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.621 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.622 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_haproxy": { 2023-11-06T01:49:28.622 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_haproxy", 2023-11-06T01:49:28.622 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.623 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.623 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.623 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-06T01:49:28.623 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.623 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.624 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.624 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "HAproxy container image", 2023-11-06T01:49:28.624 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.625 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.625 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_agent": { 2023-11-06T01:49:28.625 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_agent", 2023-11-06T01:49:28.626 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.626 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.626 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.626 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-06T01:49:28.627 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.627 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.627 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.627 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger agent container image", 2023-11-06T01:49:28.627 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.628 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.628 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.628 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_collector": { 2023-11-06T01:49:28.628 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_collector", 2023-11-06T01:49:28.629 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.629 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.629 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.629 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-06T01:49:28.629 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.629 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.630 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.630 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger collector container image", 2023-11-06T01:49:28.630 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.631 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.631 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.632 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_query": { 2023-11-06T01:49:28.632 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_query", 2023-11-06T01:49:28.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.632 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.633 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-06T01:49:28.633 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.633 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.633 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.634 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger query container image", 2023-11-06T01:49:28.634 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.634 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.634 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.635 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_keepalived": { 2023-11-06T01:49:28.635 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_keepalived", 2023-11-06T01:49:28.635 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.635 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.636 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.636 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-06T01:49:28.636 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.636 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.636 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.637 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Keepalived container image", 2023-11-06T01:49:28.637 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.637 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.637 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.638 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_loki": { 2023-11-06T01:49:28.638 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_loki", 2023-11-06T01:49:28.638 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.638 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.639 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-06T01:49:28.639 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.639 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.639 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.639 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Loki container image", 2023-11-06T01:49:28.640 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.640 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.640 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.640 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_node_exporter": { 2023-11-06T01:49:28.640 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_node_exporter", 2023-11-06T01:49:28.641 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.641 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.641 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.641 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-06T01:49:28.642 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.642 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.642 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.642 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:28.642 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.642 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.643 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.643 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_nvmeof": { 2023-11-06T01:49:28.643 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_nvmeof", 2023-11-06T01:49:28.643 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.644 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.644 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.644 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-11-06T01:49:28.644 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.644 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.645 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.645 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Nvme-of container image", 2023-11-06T01:49:28.645 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.645 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.645 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.646 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_prometheus": { 2023-11-06T01:49:28.646 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_prometheus", 2023-11-06T01:49:28.646 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.646 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.647 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.647 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-06T01:49:28.647 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.647 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.647 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.648 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:28.648 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.648 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.648 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.648 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_promtail": { 2023-11-06T01:49:28.649 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_promtail", 2023-11-06T01:49:28.649 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.649 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.649 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.649 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-06T01:49:28.650 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.650 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.650 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.650 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Promtail container image", 2023-11-06T01:49:28.650 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.651 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.651 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_snmp_gateway": { 2023-11-06T01:49:28.651 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_snmp_gateway", 2023-11-06T01:49:28.652 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.652 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.652 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.652 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-06T01:49:28.652 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.653 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.653 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.653 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SNMP Gateway container image", 2023-11-06T01:49:28.653 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.653 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "container_init": { 2023-11-06T01:49:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_init", 2023-11-06T01:49:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Run podman/docker with `--init`", 2023-11-06T01:49:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: "daemon_cache_timeout": { 2023-11-06T01:49:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "daemon_cache_timeout", 2023-11-06T01:49:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-06T01:49:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "default_cephadm_command_timeout": { 2023-11-06T01:49:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_cephadm_command_timeout", 2023-11-06T01:49:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "900", 2023-11-06T01:49:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-06T01:49:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "default_registry": { 2023-11-06T01:49:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_registry", 2023-11-06T01:49:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io", 2023-11-06T01:49:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-06T01:49:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "device_cache_timeout": { 2023-11-06T01:49:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_cache_timeout", 2023-11-06T01:49:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1800", 2023-11-06T01:49:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache device inventory", 2023-11-06T01:49:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "device_enhanced_scan": { 2023-11-06T01:49:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_enhanced_scan", 2023-11-06T01:49:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-06T01:49:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "facts_cache_timeout": { 2023-11-06T01:49:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "facts_cache_timeout", 2023-11-06T01:49:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache host facts data", 2023-11-06T01:49:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "host_check_interval": { 2023-11-06T01:49:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "host_check_interval", 2023-11-06T01:49:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to perform a host check", 2023-11-06T01:49:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "inventory_list_all": { 2023-11-06T01:49:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "inventory_list_all", 2023-11-06T01:49:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-06T01:49:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "log_refresh_metadata": { 2023-11-06T01:49:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_refresh_metadata", 2023-11-06T01:49:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-06T01:49:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-06T01:49:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-06T01:49:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-06T01:49:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-06T01:49:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-06T01:49:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-06T01:49:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "*", 2023-11-06T01:49:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-06T01:49:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "max_count_per_host": { 2023-11-06T01:49:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_count_per_host", 2023-11-06T01:49:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of daemons per service per host", 2023-11-06T01:49:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd_draining_count": { 2023-11-06T01:49:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_osd_draining_count", 2023-11-06T01:49:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-06T01:49:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "migration_current": { 2023-11-06T01:49:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "migration_current", 2023-11-06T01:49:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "internal - do not modify", 2023-11-06T01:49:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2023-11-06T01:49:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2023-11-06T01:49:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "root", 2023-11-06T01:49:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm-package", 2023-11-06T01:49:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "root" 2023-11-06T01:49:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "mode for remote execution of cephadm", 2023-11-06T01:49:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "prometheus_alerts_path": { 2023-11-06T01:49:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus_alerts_path", 2023-11-06T01:49:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-06T01:49:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-06T01:49:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_insecure": { 2023-11-06T01:49:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_insecure", 2023-11-06T01:49:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-06T01:49:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_password": { 2023-11-06T01:49:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_password", 2023-11-06T01:49:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-06T01:49:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_url": { 2023-11-06T01:49:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_url", 2023-11-06T01:49:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-06T01:49:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_username": { 2023-11-06T01:49:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_username", 2023-11-06T01:49:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-06T01:49:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "secure_monitoring_stack": { 2023-11-06T01:49:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "secure_monitoring_stack", 2023-11-06T01:49:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-06T01:49:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "service_discovery_port": { 2023-11-06T01:49:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "service_discovery_port", 2023-11-06T01:49:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8765", 2023-11-06T01:49:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "cephadm service discovery port", 2023-11-06T01:49:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "ssh_config_file": { 2023-11-06T01:49:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssh_config_file", 2023-11-06T01:49:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-06T01:49:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "use_agent": { 2023-11-06T01:49:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_agent", 2023-11-06T01:49:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-06T01:49:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: "use_repo_digest": { 2023-11-06T01:49:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_repo_digest", 2023-11-06T01:49:28.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.746 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.746 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.746 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.747 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.747 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.747 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-06T01:49:28.747 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.747 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.748 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_failed_host_check": { 2023-11-06T01:49:28.748 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_failed_host_check", 2023-11-06T01:49:28.748 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.749 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.749 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.749 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.749 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.749 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.749 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.750 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if the host check fails", 2023-11-06T01:49:28.750 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.750 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.750 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.751 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_daemons": { 2023-11-06T01:49:28.751 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_daemons", 2023-11-06T01:49:28.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.751 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.752 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.752 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.752 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.752 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.752 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-06T01:49:28.753 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.753 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.753 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.753 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_hosts": { 2023-11-06T01:49:28.754 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_hosts", 2023-11-06T01:49:28.754 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.754 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.754 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.754 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.755 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.755 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.755 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.755 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-06T01:49:28.756 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.756 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:28.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:28.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:28.757 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crash", 2023-11-06T01:49:28.757 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:28.757 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:28.758 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:28.758 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:28.758 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:28.758 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.758 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.759 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.759 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.759 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.759 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.760 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.760 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.760 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.760 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.761 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.761 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.761 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.761 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.761 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.762 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.762 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.762 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:28.763 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:28.763 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.763 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.763 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.763 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.763 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.764 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.764 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.764 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.764 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.765 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.765 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:28.766 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:28.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.766 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.767 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:28.767 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.767 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.767 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.768 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.768 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.768 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.768 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.768 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.769 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.769 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.769 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.769 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.770 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.770 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.771 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:28.771 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:28.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.771 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.773 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.773 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.773 INFO:teuthology.orchestra.run.smithi053.stdout: "retain_interval": { 2023-11-06T01:49:28.773 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retain_interval", 2023-11-06T01:49:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "31536000", 2023-11-06T01:49:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-06T01:49:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.776 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.776 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_recent_interval": { 2023-11-06T01:49:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_recent_interval", 2023-11-06T01:49:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1209600", 2023-11-06T01:49:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-06T01:49:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dashboard", 2023-11-06T01:49:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-06T01:49:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-06T01:49:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_HOST": { 2023-11-06T01:49:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-06T01:49:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-06T01:49:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-06T01:49:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_ENABLED": { 2023-11-06T01:49:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_ENABLED", 2023-11-06T01:49:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-06T01:49:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-06T01:49:28.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.792 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.792 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.792 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.793 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.793 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.793 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.793 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.794 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.794 INFO:teuthology.orchestra.run.smithi053.stdout: "ENABLE_BROWSABLE_API": { 2023-11-06T01:49:28.794 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-06T01:49:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.796 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.796 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.796 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.796 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.796 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.797 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.797 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-06T01:49:28.797 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-06T01:49:28.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.798 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.798 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.798 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.798 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.799 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.799 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.799 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.799 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-06T01:49:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-06T01:49:28.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.801 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.801 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.801 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.803 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.803 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.803 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-06T01:49:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-06T01:49:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.806 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.806 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.806 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-06T01:49:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-06T01:49:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.808 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.808 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.808 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.808 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.808 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-06T01:49:28.810 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-06T01:49:28.810 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.810 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.811 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.811 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.811 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-06T01:49:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-06T01:49:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-06T01:49:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-06T01:49:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-06T01:49:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-06T01:49:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_PASSWORD": { 2023-11-06T01:49:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-06T01:49:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2023-11-06T01:49:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-06T01:49:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-06T01:49:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_URL": { 2023-11-06T01:49:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_URL", 2023-11-06T01:49:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_USERNAME": { 2023-11-06T01:49:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-06T01:49:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2023-11-06T01:49:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-06T01:49:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-06T01:49:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-06T01:49:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-06T01:49:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-06T01:49:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-06T01:49:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-06T01:49:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-06T01:49:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_HOST": { 2023-11-06T01:49:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-06T01:49:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-06T01:49:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-06T01:49:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-06T01:49:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-06T01:49:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-06T01:49:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-06T01:49:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-06T01:49:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-06T01:49:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-06T01:49:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-06T01:49:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-06T01:49:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-06T01:49:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-06T01:49:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-06T01:49:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-06T01:49:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-06T01:49:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_ENABLED": { 2023-11-06T01:49:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-06T01:49:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-06T01:49:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-06T01:49:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-06T01:49:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-06T01:49:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-06T01:49:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-06T01:49:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-06T01:49:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8", 2023-11-06T01:49:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-06T01:49:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-06T01:49:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "45", 2023-11-06T01:49:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ACCESS_KEY": { 2023-11-06T01:49:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-06T01:49:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-06T01:49:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-06T01:49:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2023-11-06T01:49:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SECRET_KEY": { 2023-11-06T01:49:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-06T01:49:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SSL_VERIFY": { 2023-11-06T01:49:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-06T01:49:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "UNSAFE_TLS_v1_2": { 2023-11-06T01:49:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-06T01:49:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-06T01:49:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-06T01:49:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-06T01:49:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-06T01:49:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-06T01:49:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-06T01:49:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "cross_origin_url": { 2023-11-06T01:49:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cross_origin_url", 2023-11-06T01:49:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "crt_file": { 2023-11-06T01:49:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crt_file", 2023-11-06T01:49:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: "debug": { 2023-11-06T01:49:28.918 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "debug", 2023-11-06T01:49:28.918 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.918 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.918 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.918 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.919 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.919 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.919 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.919 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable/disable debug options", 2023-11-06T01:49:28.919 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.919 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.920 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.920 INFO:teuthology.orchestra.run.smithi053.stdout: "jwt_token_ttl": { 2023-11-06T01:49:28.920 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "jwt_token_ttl", 2023-11-06T01:49:28.920 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.921 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.921 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.921 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "28800", 2023-11-06T01:49:28.921 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.921 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.921 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.922 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.922 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.922 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.922 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.923 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2023-11-06T01:49:28.923 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2023-11-06T01:49:28.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.923 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.923 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.924 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.924 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.924 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.924 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.924 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.925 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.925 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.925 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.925 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:28.925 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:28.926 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.926 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.926 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.926 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.926 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.927 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.927 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.927 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.927 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.927 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.928 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.928 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.928 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.928 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.928 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.928 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.929 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.929 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.929 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:28.929 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:28.930 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.930 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.930 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.930 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.930 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.931 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.931 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.931 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.931 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.931 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.932 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.932 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:28.932 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:28.932 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.933 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.933 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.933 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:28.933 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.933 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.933 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.934 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.934 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.934 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.934 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.935 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.935 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.935 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.935 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.935 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.936 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.936 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.936 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:28.937 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:28.937 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.937 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.937 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.937 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.938 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.938 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.938 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.938 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.939 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.939 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.939 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.939 INFO:teuthology.orchestra.run.smithi053.stdout: "motd": { 2023-11-06T01:49:28.940 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "motd", 2023-11-06T01:49:28.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.940 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.940 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.941 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.941 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.941 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.941 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "The message of the day", 2023-11-06T01:49:28.941 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.942 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.942 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.942 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect_resolve_ip_addr": { 2023-11-06T01:49:28.942 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "redirect_resolve_ip_addr", 2023-11-06T01:49:28.943 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.943 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.943 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.943 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.944 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.944 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.944 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.944 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.945 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.945 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.945 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.946 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2023-11-06T01:49:28.946 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2023-11-06T01:49:28.946 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.946 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.947 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.947 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2023-11-06T01:49:28.947 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.947 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.948 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.948 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.948 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.948 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.949 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.949 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2023-11-06T01:49:28.949 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2023-11-06T01:49:28.950 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.950 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.950 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.950 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8080", 2023-11-06T01:49:28.951 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.951 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.951 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.951 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.951 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.952 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.952 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.952 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2023-11-06T01:49:28.952 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2023-11-06T01:49:28.953 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.953 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.953 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.953 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.953 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.954 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.954 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.954 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.954 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.955 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.955 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl_server_port": { 2023-11-06T01:49:28.955 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl_server_port", 2023-11-06T01:49:28.956 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.956 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.956 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.956 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8443", 2023-11-06T01:49:28.956 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.957 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.957 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.957 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.957 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.957 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.958 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.958 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2023-11-06T01:49:28.958 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2023-11-06T01:49:28.958 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.959 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.959 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.959 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "redirect", 2023-11-06T01:49:28.959 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.960 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.960 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.960 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.960 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect" 2023-11-06T01:49:28.960 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.961 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.961 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.961 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.961 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.962 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2023-11-06T01:49:28.962 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2023-11-06T01:49:28.962 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:28.962 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.962 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.963 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2023-11-06T01:49:28.963 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2023-11-06T01:49:28.963 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2023-11-06T01:49:28.963 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.963 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.964 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.964 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.964 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.964 INFO:teuthology.orchestra.run.smithi053.stdout: "url_prefix": { 2023-11-06T01:49:28.965 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url_prefix", 2023-11-06T01:49:28.965 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.965 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.965 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:28.965 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.965 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.966 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.966 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.966 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.966 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.966 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.967 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:28.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:28.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:28.968 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "devicehealth", 2023-11-06T01:49:28.968 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:28.968 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:28.968 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:28.969 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_monitoring": { 2023-11-06T01:49:28.969 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_monitoring", 2023-11-06T01:49:28.969 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.969 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.969 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.970 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:28.970 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.970 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.970 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.970 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "monitor device health metrics", 2023-11-06T01:49:28.971 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.971 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.971 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:28.972 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:28.972 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.972 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.972 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.973 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:28.973 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.973 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.973 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.974 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.974 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.974 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.974 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.975 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.975 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.975 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:28.977 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.978 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.979 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:28.980 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.981 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.982 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:28.983 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:28.984 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.985 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.986 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.987 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.988 INFO:teuthology.orchestra.run.smithi053.stdout: "mark_out_threshold": { 2023-11-06T01:49:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mark_out_threshold", 2023-11-06T01:49:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.989 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2419200", 2023-11-06T01:49:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-06T01:49:28.990 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": { 2023-11-06T01:49:28.991 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_name", 2023-11-06T01:49:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "device_health_metrics", 2023-11-06T01:49:28.992 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-06T01:49:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.993 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "retention_period": { 2023-11-06T01:49:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retention_period", 2023-11-06T01:49:28.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15552000", 2023-11-06T01:49:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain device health metrics", 2023-11-06T01:49:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_frequency": { 2023-11-06T01:49:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_frequency", 2023-11-06T01:49:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:28.997 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2023-11-06T01:49:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:28.998 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-06T01:49:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:28.999 INFO:teuthology.orchestra.run.smithi053.stdout: "self_heal": { 2023-11-06T01:49:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "self_heal", 2023-11-06T01:49:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.000 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-06T01:49:29.001 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:29.002 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:29.003 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.003 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.003 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:29.003 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.004 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.004 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.004 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and check device health", 2023-11-06T01:49:29.004 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.004 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.005 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.005 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_threshold": { 2023-11-06T01:49:29.005 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_threshold", 2023-11-06T01:49:29.005 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.006 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.006 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.006 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7257600", 2023-11-06T01:49:29.007 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.007 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.007 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.007 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-06T01:49:29.007 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.008 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.008 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.009 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "diskprediction_local", 2023-11-06T01:49:29.009 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.009 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.010 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.010 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.010 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.010 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.010 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.011 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.011 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.011 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.011 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.011 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.012 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.012 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.014 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.014 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.014 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.014 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.015 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.015 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.015 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.015 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.015 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.016 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.016 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.016 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.016 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.016 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.017 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.017 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.017 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.017 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.018 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.018 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.018 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.018 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.019 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.019 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.019 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.019 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.020 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.020 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.020 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.021 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.021 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.021 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.021 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.022 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.022 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.022 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.022 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.023 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.023 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.023 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.024 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.024 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.024 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.025 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.025 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.025 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.025 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.025 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "predict_interval": { 2023-11-06T01:49:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predict_interval", 2023-11-06T01:49:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2023-11-06T01:49:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "predictor_model": { 2023-11-06T01:49:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predictor_model", 2023-11-06T01:49:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "prophetstor", 2023-11-06T01:49:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "influx", 2023-11-06T01:49:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": false, 2023-11-06T01:49:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "influxdb python module not found", 2023-11-06T01:49:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "batch_size": { 2023-11-06T01:49:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "batch_size", 2023-11-06T01:49:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5000", 2023-11-06T01:49:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-06T01:49:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "database": { 2023-11-06T01:49:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "database", 2023-11-06T01:49:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "ceph", 2023-11-06T01:49:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-06T01:49:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "hostname": { 2023-11-06T01:49:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "hostname", 2023-11-06T01:49:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server hostname", 2023-11-06T01:49:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "30", 2023-11-06T01:49:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "5", 2023-11-06T01:49:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-06T01:49:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "password": { 2023-11-06T01:49:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "password", 2023-11-06T01:49:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "password of InfluxDB server user", 2023-11-06T01:49:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: "port": { 2023-11-06T01:49:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "port", 2023-11-06T01:49:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8086", 2023-11-06T01:49:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server port", 2023-11-06T01:49:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2023-11-06T01:49:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2023-11-06T01:49:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "false", 2023-11-06T01:49:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-06T01:49:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "threads": { 2023-11-06T01:49:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threads", 2023-11-06T01:49:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "32", 2023-11-06T01:49:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-06T01:49:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "username": { 2023-11-06T01:49:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "username", 2023-11-06T01:49:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "username of InfluxDB server user", 2023-11-06T01:49:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "verify_ssl": { 2023-11-06T01:49:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "verify_ssl", 2023-11-06T01:49:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "true", 2023-11-06T01:49:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-06T01:49:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "insights", 2023-11-06T01:49:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "iostat", 2023-11-06T01:49:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "k8sevents", 2023-11-06T01:49:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_event_retention_days": { 2023-11-06T01:49:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ceph_event_retention_days", 2023-11-06T01:49:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7", 2023-11-06T01:49:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-06T01:49:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "config_check_secs": { 2023-11-06T01:49:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_check_secs", 2023-11-06T01:49:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "10", 2023-11-06T01:49:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-06T01:49:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.130 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.130 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.130 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.131 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.131 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.131 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.131 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.131 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.132 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.132 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.132 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.132 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.132 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.133 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.133 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.133 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.133 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.133 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.134 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.134 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.134 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.134 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.134 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.135 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.135 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.135 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.135 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.136 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.136 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.136 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.137 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "localpool", 2023-11-06T01:49:29.137 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.137 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.137 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.137 INFO:teuthology.orchestra.run.smithi053.stdout: "failure_domain": { 2023-11-06T01:49:29.138 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "failure_domain", 2023-11-06T01:49:29.138 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.138 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.138 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.138 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "host", 2023-11-06T01:49:29.139 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.139 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.139 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.139 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "failure domain for any created local pool", 2023-11-06T01:49:29.140 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-06T01:49:29.140 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.140 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.140 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.141 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.141 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.141 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.141 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.141 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.141 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.142 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.142 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.142 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.142 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.143 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.143 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.143 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.143 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.143 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.144 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.144 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.144 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.144 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.145 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.145 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.145 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.145 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.145 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.146 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.146 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.146 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.146 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.147 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.147 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.147 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.147 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.148 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.148 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.148 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.148 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.148 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.148 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.149 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.149 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.149 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.149 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.150 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.150 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.150 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.150 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.150 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.151 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.151 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.151 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.151 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.152 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.152 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.152 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.153 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.153 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.153 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.153 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.154 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.154 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.154 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.155 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.155 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.155 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.155 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": { 2023-11-06T01:49:29.156 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_size", 2023-11-06T01:49:29.156 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.156 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.156 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.157 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.157 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.157 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.157 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.158 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default min_size for any created local pool", 2023-11-06T01:49:29.158 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-06T01:49:29.158 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.158 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.159 INFO:teuthology.orchestra.run.smithi053.stdout: "num_rep": { 2023-11-06T01:49:29.159 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "num_rep", 2023-11-06T01:49:29.159 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.159 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.160 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.160 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3", 2023-11-06T01:49:29.160 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.160 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.160 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.161 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default replica count for any created local pool", 2023-11-06T01:49:29.161 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.161 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.161 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.161 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": { 2023-11-06T01:49:29.162 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_num", 2023-11-06T01:49:29.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.162 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.162 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "128", 2023-11-06T01:49:29.163 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.163 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.163 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.164 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default pg_num for any created local pool", 2023-11-06T01:49:29.164 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.164 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.164 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.165 INFO:teuthology.orchestra.run.smithi053.stdout: "prefix": { 2023-11-06T01:49:29.165 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prefix", 2023-11-06T01:49:29.165 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.165 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.166 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.166 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.166 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.166 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.166 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.167 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name prefix for any created local pool", 2023-11-06T01:49:29.167 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.167 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.167 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.168 INFO:teuthology.orchestra.run.smithi053.stdout: "subtree": { 2023-11-06T01:49:29.168 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "subtree", 2023-11-06T01:49:29.168 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.168 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.168 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.169 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "rack", 2023-11-06T01:49:29.169 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.169 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.169 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.169 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-06T01:49:29.170 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-06T01:49:29.170 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.170 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.171 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mds_autoscaler", 2023-11-06T01:49:29.171 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.171 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.172 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.172 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.172 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.172 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.173 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.173 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.173 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.173 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.173 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.174 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.174 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.174 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.174 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.175 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.175 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.175 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.175 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.175 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.176 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.176 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.176 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.176 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.176 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.177 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.177 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.177 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.177 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.178 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.178 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.178 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.178 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.178 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.179 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.179 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.179 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.180 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.180 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.180 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.180 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.180 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.180 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.181 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.181 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.181 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.181 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.181 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.182 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.182 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.182 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.182 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.182 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.183 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.183 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.183 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.183 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.183 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.184 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.184 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.184 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.184 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.184 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.184 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.185 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.185 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.186 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirroring", 2023-11-06T01:49:29.187 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.187 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.187 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.187 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.187 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.188 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.188 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.188 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.189 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.189 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.189 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.189 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.189 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.189 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.190 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.190 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.190 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.190 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.191 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.191 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.191 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.191 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "nfs", 2023-11-06T01:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2023-11-06T01:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "fail_fs": { 2023-11-06T01:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "fail_fs", 2023-11-06T01:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-06T01:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator": { 2023-11-06T01:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2023-11-06T01:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm", 2023-11-06T01:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "rook", 2023-11-06T01:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "test_orchestrator" 2023-11-06T01:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Orchestrator backend", 2023-11-06T01:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_perf_query", 2023-11-06T01:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_support", 2023-11-06T01:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_autoscaler", 2023-11-06T01:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.288 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.288 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.288 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.288 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.288 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.288 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.289 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.289 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.289 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.289 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.289 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.290 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.290 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.290 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:29.290 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:29.290 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.291 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.291 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:29.291 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.291 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.292 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.292 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.292 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.292 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.292 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.293 INFO:teuthology.orchestra.run.smithi053.stdout: "threshold": { 2023-11-06T01:49:29.293 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threshold", 2023-11-06T01:49:29.293 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:29.294 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.294 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2023-11-06T01:49:29.294 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1.0", 2023-11-06T01:49:29.295 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.295 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.295 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "scaling threshold", 2023-11-06T01:49:29.295 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-06T01:49:29.295 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.296 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.297 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "progress", 2023-11-06T01:49:29.297 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.297 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.297 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.298 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_pg_recovery_event": { 2023-11-06T01:49:29.298 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_pg_recovery_event", 2023-11-06T01:49:29.298 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.299 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.299 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.299 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.299 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.300 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.300 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.300 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-06T01:49:29.300 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.301 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.301 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2023-11-06T01:49:29.302 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2023-11-06T01:49:29.302 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.302 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.302 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.303 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.303 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.303 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.303 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.304 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.304 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.304 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.304 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.305 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.305 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.305 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.306 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.306 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.306 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.306 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.306 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.307 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.307 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.307 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.307 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.307 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.308 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.308 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.308 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.308 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.308 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.309 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.309 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.309 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.310 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.310 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.310 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.311 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.311 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.311 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.311 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.311 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.312 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.312 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.312 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.312 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.313 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.313 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.313 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.313 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.313 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.314 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.314 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.314 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.314 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.314 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.315 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.315 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.315 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.315 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.315 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.316 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.316 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.316 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.316 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.316 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.317 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.317 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.317 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.317 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.317 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.318 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.318 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.318 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.318 INFO:teuthology.orchestra.run.smithi053.stdout: "max_completed_events": { 2023-11-06T01:49:29.319 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_completed_events", 2023-11-06T01:49:29.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.319 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.319 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "50", 2023-11-06T01:49:29.320 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.320 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.320 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.320 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "number of past completed events to remember", 2023-11-06T01:49:29.320 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.320 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.321 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.321 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:29.321 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:29.321 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.322 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.322 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.322 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:29.322 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.322 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.323 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.323 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long the module is going to sleep", 2023-11-06T01:49:29.323 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.323 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.323 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.324 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus", 2023-11-06T01:49:29.325 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.325 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.325 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.325 INFO:teuthology.orchestra.run.smithi053.stdout: "cache": { 2023-11-06T01:49:29.325 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cache", 2023-11-06T01:49:29.326 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.326 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.326 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.326 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.326 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.327 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.327 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.327 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.327 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.327 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.327 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.328 INFO:teuthology.orchestra.run.smithi053.stdout: "exclude_perf_counters": { 2023-11-06T01:49:29.328 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "exclude_perf_counters", 2023-11-06T01:49:29.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.328 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.329 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.329 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.329 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.329 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.329 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.330 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-06T01:49:29.330 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-06T01:49:29.330 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.330 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.331 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.331 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.331 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.332 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.332 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.332 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.332 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.332 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.333 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.333 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.333 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.333 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.333 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.334 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.334 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.334 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.334 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.335 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.335 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.335 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.335 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.336 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.336 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.336 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.336 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.337 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.337 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.337 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.338 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.338 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.338 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.338 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.339 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.339 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.339 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.340 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.340 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.340 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.340 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.341 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.341 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.341 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.341 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.341 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.342 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.342 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.342 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.342 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.342 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.343 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.343 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.343 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.343 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.344 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.344 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.344 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.344 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.344 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.345 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.345 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.345 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.345 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.345 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.346 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.346 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.346 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.347 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools": { 2023-11-06T01:49:29.347 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools", 2023-11-06T01:49:29.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.347 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.348 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.348 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.348 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.348 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.348 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.348 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.349 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.349 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.349 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-06T01:49:29.349 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-06T01:49:29.350 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.350 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.350 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.350 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "300", 2023-11-06T01:49:29.350 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.351 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.351 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.351 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.351 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.351 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.352 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.352 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_interval": { 2023-11-06T01:49:29.352 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_interval", 2023-11-06T01:49:29.352 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:29.352 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.353 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.353 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15.0", 2023-11-06T01:49:29.353 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.353 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.353 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.354 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.354 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.354 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.354 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.355 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2023-11-06T01:49:29.355 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2023-11-06T01:49:29.355 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.355 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.356 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.356 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2023-11-06T01:49:29.356 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.356 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.356 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.357 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-06T01:49:29.357 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.357 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.358 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2023-11-06T01:49:29.358 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2023-11-06T01:49:29.358 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.358 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.358 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.359 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "9283", 2023-11-06T01:49:29.359 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.359 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.359 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.360 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-06T01:49:29.360 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.360 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.360 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.360 INFO:teuthology.orchestra.run.smithi053.stdout: "stale_cache_strategy": { 2023-11-06T01:49:29.361 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stale_cache_strategy", 2023-11-06T01:49:29.361 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.361 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.361 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.361 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "log", 2023-11-06T01:49:29.362 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.362 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.362 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.362 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.362 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.363 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.363 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.363 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2023-11-06T01:49:29.363 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2023-11-06T01:49:29.364 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.364 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.364 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.364 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "default", 2023-11-06T01:49:29.364 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.365 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.365 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.365 INFO:teuthology.orchestra.run.smithi053.stdout: "default", 2023-11-06T01:49:29.365 INFO:teuthology.orchestra.run.smithi053.stdout: "error" 2023-11-06T01:49:29.366 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.366 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.366 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.366 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.366 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.367 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2023-11-06T01:49:29.367 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2023-11-06T01:49:29.367 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.368 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.368 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.368 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2023-11-06T01:49:29.368 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2023-11-06T01:49:29.368 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2023-11-06T01:49:29.369 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.369 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.369 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.370 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.371 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_support", 2023-11-06T01:49:29.371 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.371 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.371 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.372 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.372 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.372 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.372 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.373 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.373 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.373 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.373 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.374 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.374 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.374 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.374 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.375 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.375 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.375 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.375 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.376 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.376 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.376 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.377 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.377 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.377 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.377 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.377 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.378 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.378 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.378 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.378 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.378 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.379 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.379 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.379 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.380 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.380 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.380 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.381 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.381 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.381 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.381 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.381 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.382 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.382 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.382 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.382 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.382 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.382 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.383 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.383 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.383 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.383 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.384 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.384 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.384 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.384 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.385 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.385 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.385 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.385 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.385 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.386 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.386 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.386 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.386 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_snap_create": { 2023-11-06T01:49:29.387 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_snap_create", 2023-11-06T01:49:29.387 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.387 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.387 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.387 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:29.387 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.388 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.388 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.388 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.388 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.388 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.389 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.389 INFO:teuthology.orchestra.run.smithi053.stdout: "mirror_snapshot_schedule": { 2023-11-06T01:49:29.389 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirror_snapshot_schedule", 2023-11-06T01:49:29.389 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.389 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.390 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.390 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.390 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.390 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.391 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.391 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.391 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.391 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.392 INFO:teuthology.orchestra.run.smithi053.stdout: "trash_purge_schedule": { 2023-11-06T01:49:29.392 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "trash_purge_schedule", 2023-11-06T01:49:29.392 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.392 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.392 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.393 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.393 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.393 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.393 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.393 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.393 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.394 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.394 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.395 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "restful", 2023-11-06T01:49:29.395 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.395 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.395 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.396 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_auth": { 2023-11-06T01:49:29.396 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_auth", 2023-11-06T01:49:29.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.396 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.396 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.397 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.397 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.397 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.397 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.397 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.397 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.398 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.398 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2023-11-06T01:49:29.399 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2023-11-06T01:49:29.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.399 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.400 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.400 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.400 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.400 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.400 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.401 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.401 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.401 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.402 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.402 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.402 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.402 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.403 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.403 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.403 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.403 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.404 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.404 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.404 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.404 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.404 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.405 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.405 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.405 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.405 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.406 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.406 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.406 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.406 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.407 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.407 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.408 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.408 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.408 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.408 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.409 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.409 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.409 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.409 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.410 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.410 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.411 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.411 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.411 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.411 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.411 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.412 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.412 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.412 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.412 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.412 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.413 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.413 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.413 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.413 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.414 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.414 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.414 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.415 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.415 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.415 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.415 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.416 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.416 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.416 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.416 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.416 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.417 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.417 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.417 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.418 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2023-11-06T01:49:29.418 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2023-11-06T01:49:29.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.418 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.419 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.419 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.419 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.419 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.419 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.419 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.420 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.420 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.420 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.422 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2023-11-06T01:49:29.422 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2023-11-06T01:49:29.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.423 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.423 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.423 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.423 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.424 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.424 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.424 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.424 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.424 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.425 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.426 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rgw", 2023-11-06T01:49:29.426 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.426 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.426 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.426 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.427 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.427 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.427 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.428 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.428 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.428 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.428 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.428 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.428 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.429 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.429 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.429 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.429 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.429 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.430 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.430 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.430 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.430 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.431 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.431 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.431 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.431 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.431 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.431 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.432 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.432 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.432 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.432 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.432 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.433 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.433 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.433 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.433 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.434 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.434 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.434 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.434 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.434 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.435 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.435 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.435 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.435 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.435 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.435 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.436 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.436 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.436 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.436 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.436 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.437 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.437 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.437 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.438 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.438 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.438 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.438 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.438 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.439 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.439 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.439 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.439 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.439 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.440 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.441 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rook", 2023-11-06T01:49:29.441 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.441 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.441 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.441 INFO:teuthology.orchestra.run.smithi053.stdout: "drive_group_interval": { 2023-11-06T01:49:29.442 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "drive_group_interval", 2023-11-06T01:49:29.442 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:29.442 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.442 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.443 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "300.0", 2023-11-06T01:49:29.443 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.443 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.443 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.444 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-06T01:49:29.444 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.444 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.444 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.445 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.445 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.446 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.446 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.446 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.446 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.446 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.447 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.447 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.447 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.447 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.448 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.448 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.448 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.448 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.449 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.449 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.449 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.449 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.450 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.450 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.450 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.450 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.450 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.451 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.451 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.451 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.451 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.452 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.452 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.452 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.452 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.453 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.453 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.453 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.453 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.453 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.454 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.454 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.454 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.454 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.454 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.455 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.455 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.455 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.455 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.455 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.456 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.456 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.456 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.456 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.456 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.457 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.457 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.457 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.457 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.457 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.458 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.458 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.458 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.458 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.458 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.459 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.459 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.459 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.459 INFO:teuthology.orchestra.run.smithi053.stdout: "storage_class": { 2023-11-06T01:49:29.460 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "storage_class", 2023-11-06T01:49:29.460 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.460 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.460 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.460 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "local", 2023-11-06T01:49:29.461 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.461 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.461 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.461 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-06T01:49:29.461 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.462 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.462 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.463 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "selftest", 2023-11-06T01:49:29.463 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.463 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.463 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.464 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.464 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.464 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.464 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.464 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.464 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.465 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.465 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.465 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.465 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.465 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.466 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.466 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.466 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.466 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.466 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.467 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.467 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.467 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.467 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.468 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.468 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.468 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.468 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.468 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.469 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.469 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.469 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.469 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.469 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.470 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.470 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.470 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.470 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.471 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.471 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.471 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.471 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.471 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.472 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.472 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.472 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.472 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.473 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.473 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.473 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.473 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.473 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.474 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.474 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.474 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.474 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.475 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.475 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.475 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.475 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.476 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.476 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.476 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.476 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.477 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.477 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.477 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.477 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.477 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.478 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.478 INFO:teuthology.orchestra.run.smithi053.stdout: "roption1": { 2023-11-06T01:49:29.478 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption1", 2023-11-06T01:49:29.479 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.479 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.479 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.479 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.480 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.480 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.480 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.480 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.480 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.481 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.481 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.482 INFO:teuthology.orchestra.run.smithi053.stdout: "roption2": { 2023-11-06T01:49:29.482 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption2", 2023-11-06T01:49:29.482 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.482 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.483 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.483 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "xyz", 2023-11-06T01:49:29.483 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.483 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.483 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.484 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.484 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.484 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.484 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.485 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption1": { 2023-11-06T01:49:29.485 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption1", 2023-11-06T01:49:29.485 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.485 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.485 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.486 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.486 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.486 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.486 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.487 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.487 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.487 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.487 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.488 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption2": { 2023-11-06T01:49:29.488 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption2", 2023-11-06T01:49:29.488 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.488 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.488 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.488 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.489 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.489 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.489 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.489 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.489 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.490 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.490 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.490 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption3": { 2023-11-06T01:49:29.490 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption3", 2023-11-06T01:49:29.491 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:29.491 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.491 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.491 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.491 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.492 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.492 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.492 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.492 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.492 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.493 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.493 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption4": { 2023-11-06T01:49:29.493 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption4", 2023-11-06T01:49:29.493 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.493 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.494 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.494 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.494 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.494 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.494 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.495 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.495 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.495 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.495 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.496 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption5": { 2023-11-06T01:49:29.496 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption5", 2023-11-06T01:49:29.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.496 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.496 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.497 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.497 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.497 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.497 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.497 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.498 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.498 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.498 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption6": { 2023-11-06T01:49:29.498 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption6", 2023-11-06T01:49:29.499 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.499 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.499 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.499 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.499 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.500 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.500 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.500 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.500 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.500 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.501 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.501 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption7": { 2023-11-06T01:49:29.501 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption7", 2023-11-06T01:49:29.501 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.502 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.502 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.502 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.502 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:29.502 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "42", 2023-11-06T01:49:29.503 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.503 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.503 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.503 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.503 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.504 INFO:teuthology.orchestra.run.smithi053.stdout: "testkey": { 2023-11-06T01:49:29.504 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testkey", 2023-11-06T01:49:29.504 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.504 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.505 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.505 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.505 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.505 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.506 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.506 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.506 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.506 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.506 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.507 INFO:teuthology.orchestra.run.smithi053.stdout: "testlkey": { 2023-11-06T01:49:29.507 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testlkey", 2023-11-06T01:49:29.507 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.507 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.508 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.508 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.508 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.508 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.508 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.509 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.509 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.509 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.509 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.510 INFO:teuthology.orchestra.run.smithi053.stdout: "testnewline": { 2023-11-06T01:49:29.510 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testnewline", 2023-11-06T01:49:29.510 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.510 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.510 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.511 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.511 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.511 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.511 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.512 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.512 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.512 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.512 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.512 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.513 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snap_schedule", 2023-11-06T01:49:29.514 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.514 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.514 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.514 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_m_granularity": { 2023-11-06T01:49:29.514 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_m_granularity", 2023-11-06T01:49:29.515 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.515 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.515 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.515 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.516 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.516 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.516 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.516 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow minute scheduled snapshots", 2023-11-06T01:49:29.516 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.517 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.517 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.517 INFO:teuthology.orchestra.run.smithi053.stdout: "dump_on_update": { 2023-11-06T01:49:29.518 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dump_on_update", 2023-11-06T01:49:29.518 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.518 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.518 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.519 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.519 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.519 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.519 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.520 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "dump database to debug log on update", 2023-11-06T01:49:29.520 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.520 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.520 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.521 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.521 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.521 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.521 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.521 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.522 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.522 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.522 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.522 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.522 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.523 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.523 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.523 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.523 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.523 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.524 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.524 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.524 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.524 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.524 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.525 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.525 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.525 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.525 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.526 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.526 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.526 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.526 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.526 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.526 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stats", 2023-11-06T01:49:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "status", 2023-11-06T01:49:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telegraf", 2023-11-06T01:49:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "address": { 2023-11-06T01:49:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "address", 2023-11-06T01:49:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-06T01:49:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15", 2023-11-06T01:49:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telemetry", 2023-11-06T01:49:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_basic": { 2023-11-06T01:49:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_basic", 2023-11-06T01:49:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-06T01:49:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_crash": { 2023-11-06T01:49:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_crash", 2023-11-06T01:49:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-06T01:49:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_device": { 2023-11-06T01:49:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_device", 2023-11-06T01:49:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-06T01:49:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_ident": { 2023-11-06T01:49:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_ident", 2023-11-06T01:49:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-06T01:49:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_perf": { 2023-11-06T01:49:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_perf", 2023-11-06T01:49:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-06T01:49:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "contact": { 2023-11-06T01:49:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "contact", 2023-11-06T01:49:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: "description": { 2023-11-06T01:49:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "description", 2023-11-06T01:49:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "device_url": { 2023-11-06T01:49:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_url", 2023-11-06T01:49:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-06T01:49:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.612 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.612 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.612 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.612 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2023-11-06T01:49:29.613 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2023-11-06T01:49:29.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.613 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.613 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.614 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.614 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.614 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.614 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.614 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.615 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.615 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.615 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.615 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:29.616 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:29.616 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.616 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.616 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.617 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "24", 2023-11-06T01:49:29.617 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "8", 2023-11-06T01:49:29.617 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.617 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.617 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.618 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.618 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.619 INFO:teuthology.orchestra.run.smithi053.stdout: "last_opt_revision": { 2023-11-06T01:49:29.619 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "last_opt_revision", 2023-11-06T01:49:29.619 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.619 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.620 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.620 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1", 2023-11-06T01:49:29.620 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.621 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.621 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.621 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.621 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.621 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.622 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.622 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard": { 2023-11-06T01:49:29.622 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard", 2023-11-06T01:49:29.623 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.625 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.625 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.626 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.626 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.626 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.626 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.627 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.627 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.627 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.627 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.628 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard_description": { 2023-11-06T01:49:29.628 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard_description", 2023-11-06T01:49:29.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.628 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.629 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.629 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.629 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.629 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.629 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.630 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.630 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.630 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.630 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.631 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.631 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.631 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.631 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.632 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.632 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.632 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.632 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.633 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.633 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.633 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.633 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.633 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.633 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.634 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.634 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.634 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.634 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.635 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.635 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.635 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.635 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.636 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.636 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.636 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.636 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.636 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.637 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.637 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.637 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.637 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.637 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.638 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.638 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.638 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.638 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.639 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.639 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.639 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.639 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.639 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.640 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.640 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.640 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.640 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.640 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.641 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.641 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.641 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.641 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.642 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.642 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.642 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.642 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.642 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.643 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.643 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.643 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.643 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.643 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.644 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.644 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.644 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.644 INFO:teuthology.orchestra.run.smithi053.stdout: "organization": { 2023-11-06T01:49:29.645 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "organization", 2023-11-06T01:49:29.645 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.645 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.645 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.645 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.645 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.646 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.646 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.646 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.646 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.646 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.647 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.647 INFO:teuthology.orchestra.run.smithi053.stdout: "proxy": { 2023-11-06T01:49:29.647 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "proxy", 2023-11-06T01:49:29.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.648 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.648 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.648 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.648 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.648 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.649 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.649 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.649 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.649 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.650 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.650 INFO:teuthology.orchestra.run.smithi053.stdout: "url": { 2023-11-06T01:49:29.650 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url", 2023-11-06T01:49:29.651 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.651 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.651 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.651 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-06T01:49:29.651 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.652 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.652 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.652 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.652 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.653 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.654 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "test_orchestrator", 2023-11-06T01:49:29.654 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.654 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.654 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.655 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.655 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.655 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.655 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.656 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.656 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.656 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.656 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.657 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.657 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.657 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.657 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.657 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.658 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.658 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.658 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.658 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.658 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.659 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.659 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.659 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.660 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.660 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.660 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.660 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.661 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.661 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.661 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.661 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.661 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.661 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.662 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.662 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.663 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.663 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.663 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.663 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.663 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.664 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.664 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.664 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.665 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.665 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.665 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.666 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.666 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.666 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.666 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.667 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.667 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.667 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.668 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.668 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.668 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.668 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.669 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.669 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.669 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.669 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.670 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.670 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.670 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.670 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.670 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.671 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "volumes", 2023-11-06T01:49:29.671 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.671 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.671 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.672 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.672 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.672 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.673 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.673 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.673 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.673 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.673 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.673 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.674 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.674 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.674 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.674 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.674 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.675 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.675 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.675 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.675 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.676 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.676 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.676 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.676 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.676 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.677 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.677 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.677 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.677 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.677 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.678 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.678 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.678 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.678 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.679 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.679 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.679 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.679 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.679 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.679 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.680 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.680 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.680 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.680 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.680 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.681 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.681 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.681 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.681 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.681 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.682 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.682 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.682 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.682 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.683 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.683 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.683 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.683 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.684 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.684 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.684 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.684 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.684 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.685 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.685 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_clones": { 2023-11-06T01:49:29.685 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_clones", 2023-11-06T01:49:29.685 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.686 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.686 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.686 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4", 2023-11-06T01:49:29.686 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.686 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.687 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.687 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-06T01:49:29.687 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.687 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.687 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.688 INFO:teuthology.orchestra.run.smithi053.stdout: "snapshot_clone_delay": { 2023-11-06T01:49:29.688 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snapshot_clone_delay", 2023-11-06T01:49:29.688 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.688 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.688 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.689 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:29.689 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.689 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.689 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.689 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-06T01:49:29.690 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.690 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.691 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix", 2023-11-06T01:49:29.691 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.692 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.692 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.692 INFO:teuthology.orchestra.run.smithi053.stdout: "discovery_interval": { 2023-11-06T01:49:29.692 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "discovery_interval", 2023-11-06T01:49:29.693 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:29.693 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.693 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.693 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "100", 2023-11-06T01:49:29.693 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.694 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.694 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.694 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.694 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.695 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.695 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.695 INFO:teuthology.orchestra.run.smithi053.stdout: "identifier": { 2023-11-06T01:49:29.695 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "identifier", 2023-11-06T01:49:29.696 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.696 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.696 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.696 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.697 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.697 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.697 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.697 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.697 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.698 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.698 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.699 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:29.699 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:29.699 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.699 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.700 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:29.700 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.700 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.701 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.701 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.701 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.702 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.702 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.702 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.702 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.703 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.703 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.703 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.703 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.704 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.704 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.704 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.704 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.704 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.705 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.705 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.705 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.705 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.705 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.706 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.706 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.706 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.706 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.707 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.707 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.707 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.707 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.707 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.708 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.708 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.708 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.708 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.708 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.709 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.709 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.709 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.709 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.710 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.710 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.710 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.710 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.710 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.711 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.711 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.711 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.711 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.711 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.712 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.712 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.712 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.712 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.712 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.713 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.713 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.713 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.713 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.714 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.714 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.714 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.714 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.714 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.714 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.715 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.715 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.715 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.715 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.715 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.716 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.716 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.716 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_host": { 2023-11-06T01:49:29.716 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_host", 2023-11-06T01:49:29.717 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.717 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.717 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.717 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.717 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.718 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.718 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.718 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.718 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.718 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.719 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.719 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_port": { 2023-11-06T01:49:29.719 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_port", 2023-11-06T01:49:29.719 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.719 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.720 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.720 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10051", 2023-11-06T01:49:29.720 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.720 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.720 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.721 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.721 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.721 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.721 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.722 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_sender": { 2023-11-06T01:49:29.722 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_sender", 2023-11-06T01:49:29.722 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.722 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.722 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.723 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-06T01:49:29.723 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.723 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.723 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.724 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.724 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.724 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.724 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.725 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-06T01:49:29.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.726 INFO:teuthology.orchestra.run.smithi053.stdout: "gid": 24109, 2023-11-06T01:49:29.726 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "z", 2023-11-06T01:49:29.726 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_features": 4540138322906710015, 2023-11-06T01:49:29.727 INFO:teuthology.orchestra.run.smithi053.stdout: "available_modules": [ 2023-11-06T01:49:29.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.727 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "alerts", 2023-11-06T01:49:29.727 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.727 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.728 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.728 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:29.728 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:29.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.729 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.729 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.729 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:29.729 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.729 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.730 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.730 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How frequently to reexamine health status", 2023-11-06T01:49:29.730 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.731 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.731 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.731 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.731 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.732 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.732 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.732 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.732 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.732 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.733 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.733 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.733 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.733 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.733 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.734 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.734 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.734 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.734 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.734 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.735 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.735 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.735 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.735 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.736 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.736 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.736 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.736 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.737 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.737 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.737 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.738 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.738 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.738 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.738 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.738 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.739 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.739 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.739 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.739 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.740 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.740 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.740 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.740 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.740 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.741 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.741 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.741 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.741 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.741 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.742 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.742 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.742 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.742 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.742 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.743 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.743 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.743 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.743 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.744 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.744 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.744 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.744 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.745 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.745 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.745 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.745 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.745 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.745 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.746 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_destination": { 2023-11-06T01:49:29.746 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_destination", 2023-11-06T01:49:29.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.746 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.747 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.747 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.747 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.747 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.747 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.747 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email address to send alerts to", 2023-11-06T01:49:29.748 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.748 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.748 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.748 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_from_name": { 2023-11-06T01:49:29.749 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_from_name", 2023-11-06T01:49:29.749 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.749 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.749 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.749 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "Ceph", 2023-11-06T01:49:29.750 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.750 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.750 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.750 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email From: name", 2023-11-06T01:49:29.750 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.751 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.751 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.751 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_host": { 2023-11-06T01:49:29.751 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_host", 2023-11-06T01:49:29.752 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.752 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.752 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.752 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.753 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.753 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.753 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP server", 2023-11-06T01:49:29.753 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.753 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.753 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.754 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_password": { 2023-11-06T01:49:29.754 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_password", 2023-11-06T01:49:29.754 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.754 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.755 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.755 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.755 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.755 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.755 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.756 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Password to authenticate with", 2023-11-06T01:49:29.756 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.756 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.757 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_port": { 2023-11-06T01:49:29.757 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_port", 2023-11-06T01:49:29.757 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.757 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.757 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.758 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "465", 2023-11-06T01:49:29.758 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.758 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.758 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.758 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP port", 2023-11-06T01:49:29.759 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.759 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.759 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.759 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_sender": { 2023-11-06T01:49:29.760 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_sender", 2023-11-06T01:49:29.760 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.760 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.760 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.761 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.761 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.761 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.761 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.761 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP envelope sender", 2023-11-06T01:49:29.762 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.762 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.762 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.763 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_ssl": { 2023-11-06T01:49:29.763 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_ssl", 2023-11-06T01:49:29.763 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.763 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.763 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.764 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.764 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.764 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.764 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.765 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-06T01:49:29.765 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.765 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.766 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_user": { 2023-11-06T01:49:29.766 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_user", 2023-11-06T01:49:29.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.766 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.767 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.767 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.767 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.767 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.768 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.768 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "User to authenticate as", 2023-11-06T01:49:29.768 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.768 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.769 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.770 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "balancer", 2023-11-06T01:49:29.770 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.770 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.771 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.771 INFO:teuthology.orchestra.run.smithi053.stdout: "active": { 2023-11-06T01:49:29.771 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "active", 2023-11-06T01:49:29.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.772 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.772 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.772 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.772 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.772 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.773 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.773 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically balance PGs across cluster", 2023-11-06T01:49:29.773 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.773 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.774 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_time": { 2023-11-06T01:49:29.774 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_time", 2023-11-06T01:49:29.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.775 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.775 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.775 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0000", 2023-11-06T01:49:29.775 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.775 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.776 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.776 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "beginning time of day to automatically balance", 2023-11-06T01:49:29.776 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-06T01:49:29.776 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.776 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.777 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_weekday": { 2023-11-06T01:49:29.777 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_weekday", 2023-11-06T01:49:29.777 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:29.777 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.778 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.778 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:29.778 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2023-11-06T01:49:29.778 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2023-11-06T01:49:29.778 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.779 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-06T01:49:29.779 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-06T01:49:29.779 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.779 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.780 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_max_iterations": { 2023-11-06T01:49:29.780 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_max_iterations", 2023-11-06T01:49:29.780 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:29.780 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.781 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.781 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "25", 2023-11-06T01:49:29.781 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:29.781 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "250", 2023-11-06T01:49:29.781 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.781 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-06T01:49:29.782 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.782 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.782 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.783 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_metrics": { 2023-11-06T01:49:29.783 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_metrics", 2023-11-06T01:49:29.783 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.783 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.783 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.784 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "pgs,objects,bytes", 2023-11-06T01:49:29.784 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.784 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.784 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.784 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-06T01:49:29.785 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-06T01:49:29.785 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.785 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.785 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_step": { 2023-11-06T01:49:29.785 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_step", 2023-11-06T01:49:29.786 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:29.786 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.786 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.786 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.5", 2023-11-06T01:49:29.786 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0.001", 2023-11-06T01:49:29.787 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "0.999", 2023-11-06T01:49:29.787 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.787 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "aggressiveness of optimization", 2023-11-06T01:49:29.787 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-06T01:49:29.787 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.788 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.788 INFO:teuthology.orchestra.run.smithi053.stdout: "end_time": { 2023-11-06T01:49:29.788 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_time", 2023-11-06T01:49:29.789 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.789 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.789 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.789 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2359", 2023-11-06T01:49:29.789 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.790 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.790 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.790 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ending time of day to automatically balance", 2023-11-06T01:49:29.790 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-06T01:49:29.790 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.791 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.791 INFO:teuthology.orchestra.run.smithi053.stdout: "end_weekday": { 2023-11-06T01:49:29.791 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_weekday", 2023-11-06T01:49:29.791 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:29.792 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.792 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:29.792 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2023-11-06T01:49:29.792 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2023-11-06T01:49:29.792 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.793 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-06T01:49:29.793 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-06T01:49:29.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.793 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.794 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.794 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.795 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.795 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.795 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.795 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.795 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.795 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.796 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.796 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.796 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.796 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.797 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.797 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.797 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.797 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.797 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.798 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.798 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.798 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.799 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.799 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.800 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.801 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.801 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.801 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.801 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.801 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.802 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.802 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.802 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.803 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.803 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.804 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.804 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.804 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.804 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.804 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.805 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.805 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.805 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.805 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.805 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.805 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.806 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.806 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.806 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.806 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.807 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.807 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.807 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.808 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.808 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.808 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.808 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.808 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.809 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.809 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.809 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.809 INFO:teuthology.orchestra.run.smithi053.stdout: "min_score": { 2023-11-06T01:49:29.810 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_score", 2023-11-06T01:49:29.810 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:29.810 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.810 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.810 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:29.811 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.811 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.811 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.811 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-06T01:49:29.811 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.811 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.812 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.812 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2023-11-06T01:49:29.812 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2023-11-06T01:49:29.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.813 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.813 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "upmap", 2023-11-06T01:49:29.813 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.813 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.814 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.814 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-compat", 2023-11-06T01:49:29.814 INFO:teuthology.orchestra.run.smithi053.stdout: "none", 2023-11-06T01:49:29.814 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap" 2023-11-06T01:49:29.814 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.815 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Balancer mode", 2023-11-06T01:49:29.815 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.815 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.815 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.815 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_ids": { 2023-11-06T01:49:29.816 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_ids", 2023-11-06T01:49:29.816 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.816 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.816 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.816 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.817 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.817 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.817 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.817 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-06T01:49:29.817 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.818 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.818 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:29.818 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:29.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.819 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.819 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.819 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:29.819 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.820 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.820 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.820 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-06T01:49:29.820 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.820 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.821 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.821 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_deviation": { 2023-11-06T01:49:29.821 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_deviation", 2023-11-06T01:49:29.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.821 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.822 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:29.822 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:29.822 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.822 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.823 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-06T01:49:29.823 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-06T01:49:29.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.823 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.824 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_optimizations": { 2023-11-06T01:49:29.824 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_optimizations", 2023-11-06T01:49:29.824 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:29.824 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.825 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:29.825 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.825 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.825 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.825 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-06T01:49:29.825 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.826 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.826 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.827 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephadm", 2023-11-06T01:49:29.827 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.827 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.827 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.828 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_down_multiplier": { 2023-11-06T01:49:29.828 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_down_multiplier", 2023-11-06T01:49:29.829 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:29.829 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.830 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.830 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2023-11-06T01:49:29.830 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.830 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.830 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.831 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-06T01:49:29.831 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.831 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.831 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.832 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_refresh_rate": { 2023-11-06T01:49:29.832 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_refresh_rate", 2023-11-06T01:49:29.832 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.832 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.832 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "20", 2023-11-06T01:49:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.833 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-06T01:49:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.834 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_starting_port": { 2023-11-06T01:49:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_starting_port", 2023-11-06T01:49:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.835 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4721", 2023-11-06T01:49:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-06T01:49:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_ptrace": { 2023-11-06T01:49:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_ptrace", 2023-11-06T01:49:29.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.838 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-06T01:49:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-06T01:49:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.839 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_interval": { 2023-11-06T01:49:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_interval", 2023-11-06T01:49:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.840 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.841 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-06T01:49:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.842 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_memory_target_ratio": { 2023-11-06T01:49:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_memory_target_ratio", 2023-11-06T01:49:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.843 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.7", 2023-11-06T01:49:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-06T01:49:29.844 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "cgroups_split": { 2023-11-06T01:49:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cgroups_split", 2023-11-06T01:49:29.845 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.846 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-06T01:49:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.847 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "config_checks_enabled": { 2023-11-06T01:49:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_checks_enabled", 2023-11-06T01:49:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.848 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.849 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-06T01:49:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.850 INFO:teuthology.orchestra.run.smithi053.stdout: "config_dashboard": { 2023-11-06T01:49:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_dashboard", 2023-11-06T01:49:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.851 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-06T01:49:29.852 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_alertmanager": { 2023-11-06T01:49:29.853 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_alertmanager", 2023-11-06T01:49:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-06T01:49:29.854 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_base": { 2023-11-06T01:49:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_base", 2023-11-06T01:49:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.856 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-06T01:49:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.857 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Container image name, without the tag", 2023-11-06T01:49:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_elasticsearch": { 2023-11-06T01:49:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_elasticsearch", 2023-11-06T01:49:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-06T01:49:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "elasticsearch container image", 2023-11-06T01:49:29.860 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_grafana": { 2023-11-06T01:49:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_grafana", 2023-11-06T01:49:29.861 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-06T01:49:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.862 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_haproxy": { 2023-11-06T01:49:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_haproxy", 2023-11-06T01:49:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.864 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-06T01:49:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.865 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "HAproxy container image", 2023-11-06T01:49:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_agent": { 2023-11-06T01:49:29.866 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_agent", 2023-11-06T01:49:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-06T01:49:29.867 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger agent container image", 2023-11-06T01:49:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_collector": { 2023-11-06T01:49:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_collector", 2023-11-06T01:49:29.869 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-06T01:49:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.870 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger collector container image", 2023-11-06T01:49:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_query": { 2023-11-06T01:49:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_query", 2023-11-06T01:49:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-06T01:49:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.873 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger query container image", 2023-11-06T01:49:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.874 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_keepalived": { 2023-11-06T01:49:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_keepalived", 2023-11-06T01:49:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-06T01:49:29.875 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Keepalived container image", 2023-11-06T01:49:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.876 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_loki": { 2023-11-06T01:49:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_loki", 2023-11-06T01:49:29.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-06T01:49:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.878 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Loki container image", 2023-11-06T01:49:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_node_exporter": { 2023-11-06T01:49:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_node_exporter", 2023-11-06T01:49:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.880 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-06T01:49:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:29.881 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_nvmeof": { 2023-11-06T01:49:29.882 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_nvmeof", 2023-11-06T01:49:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-11-06T01:49:29.883 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Nvme-of container image", 2023-11-06T01:49:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.884 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_prometheus": { 2023-11-06T01:49:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_prometheus", 2023-11-06T01:49:29.885 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-06T01:49:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.886 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_promtail": { 2023-11-06T01:49:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_promtail", 2023-11-06T01:49:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.888 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-06T01:49:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.889 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Promtail container image", 2023-11-06T01:49:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.890 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_snmp_gateway": { 2023-11-06T01:49:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_snmp_gateway", 2023-11-06T01:49:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.891 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-06T01:49:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SNMP Gateway container image", 2023-11-06T01:49:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "container_init": { 2023-11-06T01:49:29.893 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_init", 2023-11-06T01:49:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.894 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Run podman/docker with `--init`", 2023-11-06T01:49:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.895 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "daemon_cache_timeout": { 2023-11-06T01:49:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "daemon_cache_timeout", 2023-11-06T01:49:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.896 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.897 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-06T01:49:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.898 INFO:teuthology.orchestra.run.smithi053.stdout: "default_cephadm_command_timeout": { 2023-11-06T01:49:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_cephadm_command_timeout", 2023-11-06T01:49:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "900", 2023-11-06T01:49:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-06T01:49:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.900 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "default_registry": { 2023-11-06T01:49:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_registry", 2023-11-06T01:49:29.901 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io", 2023-11-06T01:49:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.902 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-06T01:49:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "device_cache_timeout": { 2023-11-06T01:49:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_cache_timeout", 2023-11-06T01:49:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.904 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1800", 2023-11-06T01:49:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.905 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache device inventory", 2023-11-06T01:49:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.906 INFO:teuthology.orchestra.run.smithi053.stdout: "device_enhanced_scan": { 2023-11-06T01:49:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_enhanced_scan", 2023-11-06T01:49:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.907 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-06T01:49:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.908 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "facts_cache_timeout": { 2023-11-06T01:49:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "facts_cache_timeout", 2023-11-06T01:49:29.909 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.910 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache host facts data", 2023-11-06T01:49:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "host_check_interval": { 2023-11-06T01:49:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "host_check_interval", 2023-11-06T01:49:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.912 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to perform a host check", 2023-11-06T01:49:29.913 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "inventory_list_all": { 2023-11-06T01:49:29.914 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "inventory_list_all", 2023-11-06T01:49:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.916 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-06T01:49:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.918 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.919 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.920 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "log_refresh_metadata": { 2023-11-06T01:49:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_refresh_metadata", 2023-11-06T01:49:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.922 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-06T01:49:29.923 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.925 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-06T01:49:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.926 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.927 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.928 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.929 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.930 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.931 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.932 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-06T01:49:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-06T01:49:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.934 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-06T01:49:29.935 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-06T01:49:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-06T01:49:29.936 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "*", 2023-11-06T01:49:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.937 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-06T01:49:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.938 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "max_count_per_host": { 2023-11-06T01:49:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_count_per_host", 2023-11-06T01:49:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.939 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.940 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of daemons per service per host", 2023-11-06T01:49:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.941 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd_draining_count": { 2023-11-06T01:49:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_osd_draining_count", 2023-11-06T01:49:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:29.942 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-06T01:49:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.943 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "migration_current": { 2023-11-06T01:49:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "migration_current", 2023-11-06T01:49:29.944 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.945 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "internal - do not modify", 2023-11-06T01:49:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2023-11-06T01:49:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2023-11-06T01:49:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.947 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "root", 2023-11-06T01:49:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.948 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm-package", 2023-11-06T01:49:29.949 INFO:teuthology.orchestra.run.smithi053.stdout: "root" 2023-11-06T01:49:29.949 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.949 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "mode for remote execution of cephadm", 2023-11-06T01:49:29.949 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.949 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.950 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.950 INFO:teuthology.orchestra.run.smithi053.stdout: "prometheus_alerts_path": { 2023-11-06T01:49:29.950 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus_alerts_path", 2023-11-06T01:49:29.950 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.950 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-06T01:49:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-06T01:49:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_insecure": { 2023-11-06T01:49:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_insecure", 2023-11-06T01:49:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-06T01:49:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_password": { 2023-11-06T01:49:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_password", 2023-11-06T01:49:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-06T01:49:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_url": { 2023-11-06T01:49:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_url", 2023-11-06T01:49:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-06T01:49:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_username": { 2023-11-06T01:49:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_username", 2023-11-06T01:49:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-06T01:49:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "secure_monitoring_stack": { 2023-11-06T01:49:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "secure_monitoring_stack", 2023-11-06T01:49:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-06T01:49:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "service_discovery_port": { 2023-11-06T01:49:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "service_discovery_port", 2023-11-06T01:49:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8765", 2023-11-06T01:49:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "cephadm service discovery port", 2023-11-06T01:49:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "ssh_config_file": { 2023-11-06T01:49:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssh_config_file", 2023-11-06T01:49:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-06T01:49:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "use_agent": { 2023-11-06T01:49:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_agent", 2023-11-06T01:49:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-06T01:49:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "use_repo_digest": { 2023-11-06T01:49:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_repo_digest", 2023-11-06T01:49:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-06T01:49:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_failed_host_check": { 2023-11-06T01:49:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_failed_host_check", 2023-11-06T01:49:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if the host check fails", 2023-11-06T01:49:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_daemons": { 2023-11-06T01:49:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_daemons", 2023-11-06T01:49:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-06T01:49:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_hosts": { 2023-11-06T01:49:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_hosts", 2023-11-06T01:49:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-06T01:49:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crash", 2023-11-06T01:49:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "retain_interval": { 2023-11-06T01:49:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retain_interval", 2023-11-06T01:49:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "31536000", 2023-11-06T01:49:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-06T01:49:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_recent_interval": { 2023-11-06T01:49:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_recent_interval", 2023-11-06T01:49:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1209600", 2023-11-06T01:49:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-06T01:49:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dashboard", 2023-11-06T01:49:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-06T01:49:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-06T01:49:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_HOST": { 2023-11-06T01:49:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-06T01:49:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-06T01:49:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-06T01:49:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_ENABLED": { 2023-11-06T01:49:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_ENABLED", 2023-11-06T01:49:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-06T01:49:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-06T01:49:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "ENABLE_BROWSABLE_API": { 2023-11-06T01:49:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-06T01:49:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-06T01:49:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-06T01:49:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-06T01:49:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-06T01:49:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-06T01:49:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-06T01:49:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-06T01:49:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-06T01:49:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-06T01:49:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-06T01:49:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-06T01:49:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-06T01:49:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.038 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.038 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.038 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.038 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.038 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.039 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.039 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.039 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.039 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.039 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.040 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-06T01:49:30.040 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-06T01:49:30.040 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.040 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.040 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.041 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.041 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.041 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.041 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.041 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.042 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.042 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.042 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.042 INFO:teuthology.orchestra.run.smithi053.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-06T01:49:30.043 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-06T01:49:30.043 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.043 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.043 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.043 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.043 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.044 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.044 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.044 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.044 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.044 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.045 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.045 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_PASSWORD": { 2023-11-06T01:49:30.045 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-06T01:49:30.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.046 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.046 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.046 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2023-11-06T01:49:30.046 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.046 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.047 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.047 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.047 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.047 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.047 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.048 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-06T01:49:30.048 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-06T01:49:30.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.048 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.048 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.049 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.049 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.049 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.049 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.049 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.049 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.050 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.050 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.050 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_URL": { 2023-11-06T01:49:30.050 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_URL", 2023-11-06T01:49:30.051 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.051 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.051 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.051 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.051 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.052 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.052 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.052 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.052 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.052 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.052 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.053 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_USERNAME": { 2023-11-06T01:49:30.053 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-06T01:49:30.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.053 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.054 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.054 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2023-11-06T01:49:30.054 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.054 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.054 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.055 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.055 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.055 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.055 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.056 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-06T01:49:30.056 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-06T01:49:30.056 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.056 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.056 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.056 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.057 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.057 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.057 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.057 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.057 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.058 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.058 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.058 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-06T01:49:30.058 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-06T01:49:30.059 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.059 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.059 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.059 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.059 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.060 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.060 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.060 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.060 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.060 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.060 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.061 INFO:teuthology.orchestra.run.smithi053.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-06T01:49:30.061 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-06T01:49:30.061 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.061 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.062 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.062 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.062 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.062 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.062 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.062 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.063 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.063 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.063 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.063 INFO:teuthology.orchestra.run.smithi053.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-06T01:49:30.064 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-06T01:49:30.064 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.064 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.064 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.064 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.064 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.065 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.065 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.065 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.065 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.065 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.066 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.066 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_HOST": { 2023-11-06T01:49:30.066 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-06T01:49:30.066 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.067 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.067 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.067 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.067 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.067 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.068 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.068 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.068 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.068 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.068 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.069 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-06T01:49:30.069 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-06T01:49:30.069 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.069 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.069 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.070 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.070 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.070 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.070 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.070 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.070 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.071 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.071 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.071 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-06T01:49:30.071 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-06T01:49:30.072 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.072 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.072 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.072 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.072 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.073 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.073 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.073 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.073 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.074 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.074 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-06T01:49:30.074 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-06T01:49:30.074 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.075 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.075 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.075 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.075 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.075 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.075 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.076 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.076 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.076 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.076 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.077 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-06T01:49:30.077 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-06T01:49:30.077 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.077 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.077 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.078 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.078 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.078 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.078 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.078 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.078 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.079 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.079 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.079 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-06T01:49:30.079 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-06T01:49:30.080 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.080 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.080 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.080 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.080 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.081 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.081 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.081 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.081 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.081 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.081 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.082 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-06T01:49:30.082 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-06T01:49:30.082 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.082 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.083 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.083 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.083 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.083 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.083 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.084 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.084 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.084 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.084 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.085 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-06T01:49:30.085 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-06T01:49:30.085 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.085 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.085 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.085 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.086 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.086 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.086 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.086 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.086 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.087 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.087 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.087 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-06T01:49:30.087 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-06T01:49:30.088 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.088 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.088 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.088 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.088 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.089 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.089 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.089 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.089 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.089 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.090 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.090 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_ENABLED": { 2023-11-06T01:49:30.090 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-06T01:49:30.090 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.091 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.091 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.091 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.091 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.091 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.092 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.092 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.092 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.092 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.092 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.093 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-06T01:49:30.093 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-06T01:49:30.093 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.093 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.093 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.094 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-06T01:49:30.094 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.094 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.094 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.094 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.095 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.095 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.095 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.095 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-06T01:49:30.096 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-06T01:49:30.096 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.096 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.096 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.096 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:30.097 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.097 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.097 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.097 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.097 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.097 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.098 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.098 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-06T01:49:30.098 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-06T01:49:30.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.099 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.099 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.099 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8", 2023-11-06T01:49:30.099 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.099 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.100 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.100 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.100 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.100 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.100 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.101 INFO:teuthology.orchestra.run.smithi053.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-06T01:49:30.101 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-06T01:49:30.101 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.101 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.101 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.102 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "45", 2023-11-06T01:49:30.102 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.102 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.102 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.102 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.103 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.103 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.103 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.103 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ACCESS_KEY": { 2023-11-06T01:49:30.103 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-06T01:49:30.104 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.104 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.104 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.104 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.104 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.105 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.105 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.105 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.105 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.105 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.105 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.106 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-06T01:49:30.106 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-06T01:49:30.106 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.106 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.107 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.107 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2023-11-06T01:49:30.107 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.107 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.107 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.108 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.108 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.108 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.108 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.108 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SECRET_KEY": { 2023-11-06T01:49:30.109 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-06T01:49:30.109 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.109 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.109 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.109 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.110 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.110 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.110 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.110 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.110 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.111 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.111 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.111 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SSL_VERIFY": { 2023-11-06T01:49:30.111 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-06T01:49:30.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.112 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.112 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.112 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.112 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.112 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.113 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.113 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.113 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.113 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.113 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.114 INFO:teuthology.orchestra.run.smithi053.stdout: "UNSAFE_TLS_v1_2": { 2023-11-06T01:49:30.114 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-06T01:49:30.114 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.114 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.114 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.115 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.115 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.115 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.115 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.115 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.116 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.116 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.116 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.116 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-06T01:49:30.117 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-06T01:49:30.117 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.117 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.117 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.117 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:30.117 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.118 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.118 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.118 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.118 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.118 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.119 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.119 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-06T01:49:30.119 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-06T01:49:30.119 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.120 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.120 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.120 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:30.120 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.120 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.120 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.121 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.121 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.121 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.121 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.122 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-06T01:49:30.122 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-06T01:49:30.122 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.122 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.122 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.123 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:30.123 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.123 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.123 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.123 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.123 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.124 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.124 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.124 INFO:teuthology.orchestra.run.smithi053.stdout: "cross_origin_url": { 2023-11-06T01:49:30.124 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cross_origin_url", 2023-11-06T01:49:30.125 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.125 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.125 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.125 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.125 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.126 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.126 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.126 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.126 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.126 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.127 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.127 INFO:teuthology.orchestra.run.smithi053.stdout: "crt_file": { 2023-11-06T01:49:30.127 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crt_file", 2023-11-06T01:49:30.127 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.128 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.128 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.128 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.128 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.128 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.128 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.129 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.129 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.129 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.129 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "debug": { 2023-11-06T01:49:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "debug", 2023-11-06T01:49:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable/disable debug options", 2023-11-06T01:49:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "jwt_token_ttl": { 2023-11-06T01:49:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "jwt_token_ttl", 2023-11-06T01:49:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "28800", 2023-11-06T01:49:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2023-11-06T01:49:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2023-11-06T01:49:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: "motd": { 2023-11-06T01:49:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "motd", 2023-11-06T01:49:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "The message of the day", 2023-11-06T01:49:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect_resolve_ip_addr": { 2023-11-06T01:49:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "redirect_resolve_ip_addr", 2023-11-06T01:49:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2023-11-06T01:49:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2023-11-06T01:49:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2023-11-06T01:49:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2023-11-06T01:49:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2023-11-06T01:49:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8080", 2023-11-06T01:49:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2023-11-06T01:49:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2023-11-06T01:49:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl_server_port": { 2023-11-06T01:49:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl_server_port", 2023-11-06T01:49:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8443", 2023-11-06T01:49:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2023-11-06T01:49:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2023-11-06T01:49:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "redirect", 2023-11-06T01:49:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect" 2023-11-06T01:49:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2023-11-06T01:49:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2023-11-06T01:49:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2023-11-06T01:49:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2023-11-06T01:49:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2023-11-06T01:49:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "url_prefix": { 2023-11-06T01:49:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url_prefix", 2023-11-06T01:49:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.175 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "devicehealth", 2023-11-06T01:49:30.175 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_monitoring": { 2023-11-06T01:49:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_monitoring", 2023-11-06T01:49:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "monitor device health metrics", 2023-11-06T01:49:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: "mark_out_threshold": { 2023-11-06T01:49:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mark_out_threshold", 2023-11-06T01:49:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2419200", 2023-11-06T01:49:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-06T01:49:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": { 2023-11-06T01:49:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_name", 2023-11-06T01:49:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "device_health_metrics", 2023-11-06T01:49:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-06T01:49:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: "retention_period": { 2023-11-06T01:49:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retention_period", 2023-11-06T01:49:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15552000", 2023-11-06T01:49:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain device health metrics", 2023-11-06T01:49:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_frequency": { 2023-11-06T01:49:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_frequency", 2023-11-06T01:49:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2023-11-06T01:49:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-06T01:49:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: "self_heal": { 2023-11-06T01:49:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "self_heal", 2023-11-06T01:49:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-06T01:49:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and check device health", 2023-11-06T01:49:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_threshold": { 2023-11-06T01:49:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_threshold", 2023-11-06T01:49:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7257600", 2023-11-06T01:49:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-06T01:49:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "diskprediction_local", 2023-11-06T01:49:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "predict_interval": { 2023-11-06T01:49:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predict_interval", 2023-11-06T01:49:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2023-11-06T01:49:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: "predictor_model": { 2023-11-06T01:49:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predictor_model", 2023-11-06T01:49:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "prophetstor", 2023-11-06T01:49:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.232 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.232 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.233 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "influx", 2023-11-06T01:49:30.233 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": false, 2023-11-06T01:49:30.233 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "influxdb python module not found", 2023-11-06T01:49:30.233 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.234 INFO:teuthology.orchestra.run.smithi053.stdout: "batch_size": { 2023-11-06T01:49:30.234 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "batch_size", 2023-11-06T01:49:30.234 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.234 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.234 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.235 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5000", 2023-11-06T01:49:30.235 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.235 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.235 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.235 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-06T01:49:30.236 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.236 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.236 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.236 INFO:teuthology.orchestra.run.smithi053.stdout: "database": { 2023-11-06T01:49:30.236 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "database", 2023-11-06T01:49:30.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.237 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.237 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.237 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "ceph", 2023-11-06T01:49:30.237 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.238 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.238 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.238 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-06T01:49:30.238 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.238 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.239 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.239 INFO:teuthology.orchestra.run.smithi053.stdout: "hostname": { 2023-11-06T01:49:30.239 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "hostname", 2023-11-06T01:49:30.239 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.239 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.240 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.240 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.240 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.240 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.240 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.241 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server hostname", 2023-11-06T01:49:30.241 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.241 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.241 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.241 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:30.242 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:30.242 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.242 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.242 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.242 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "30", 2023-11-06T01:49:30.243 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "5", 2023-11-06T01:49:30.243 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.243 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.243 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-06T01:49:30.243 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.244 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.244 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.244 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.244 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.244 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.245 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.245 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.245 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.245 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.245 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.246 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.246 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.246 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.246 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.246 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.247 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.247 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.247 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.247 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.247 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.247 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.248 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.248 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.248 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.248 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.249 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.249 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.249 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.249 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.249 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.249 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.250 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.250 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.250 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.250 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.251 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.251 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.251 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.251 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.252 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.252 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.252 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.252 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.252 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.253 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.253 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.253 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.253 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.253 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.254 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.254 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.254 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.254 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.255 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.255 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.255 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.255 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.256 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.256 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.256 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.256 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.256 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.257 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.257 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.257 INFO:teuthology.orchestra.run.smithi053.stdout: "password": { 2023-11-06T01:49:30.257 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "password", 2023-11-06T01:49:30.257 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.258 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.258 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.258 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.258 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.258 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.259 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.259 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "password of InfluxDB server user", 2023-11-06T01:49:30.259 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.259 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.259 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.260 INFO:teuthology.orchestra.run.smithi053.stdout: "port": { 2023-11-06T01:49:30.260 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "port", 2023-11-06T01:49:30.260 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.260 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.261 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.261 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8086", 2023-11-06T01:49:30.261 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.261 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.261 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.262 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server port", 2023-11-06T01:49:30.262 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.262 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.262 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.262 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2023-11-06T01:49:30.263 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2023-11-06T01:49:30.263 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.263 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.263 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.263 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "false", 2023-11-06T01:49:30.264 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.264 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.264 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.264 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-06T01:49:30.264 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.265 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.265 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.265 INFO:teuthology.orchestra.run.smithi053.stdout: "threads": { 2023-11-06T01:49:30.265 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threads", 2023-11-06T01:49:30.265 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.266 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.266 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.266 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:30.266 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:30.266 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "32", 2023-11-06T01:49:30.267 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.267 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-06T01:49:30.267 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.267 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.268 INFO:teuthology.orchestra.run.smithi053.stdout: "username": { 2023-11-06T01:49:30.268 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "username", 2023-11-06T01:49:30.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.268 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.269 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.269 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.269 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.269 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.269 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.269 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "username of InfluxDB server user", 2023-11-06T01:49:30.270 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.270 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.270 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.270 INFO:teuthology.orchestra.run.smithi053.stdout: "verify_ssl": { 2023-11-06T01:49:30.271 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "verify_ssl", 2023-11-06T01:49:30.271 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.271 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.271 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.271 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "true", 2023-11-06T01:49:30.272 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.272 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.272 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.272 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-06T01:49:30.272 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.272 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.273 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.274 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "insights", 2023-11-06T01:49:30.274 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.274 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.274 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.274 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.275 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.275 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.275 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.276 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.276 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.276 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.276 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.276 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.277 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.277 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.277 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.277 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.277 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.278 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.278 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.278 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.278 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.278 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.279 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.279 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.279 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.279 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.280 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.280 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.280 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.280 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.280 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.281 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.281 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.281 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.281 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.282 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.282 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.282 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.282 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.283 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.283 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.283 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.283 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.283 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.283 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.284 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.284 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.284 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.284 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.284 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.285 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.285 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.285 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.285 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.286 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.286 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.286 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.286 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.286 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.287 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.287 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.287 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.287 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.287 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.287 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.288 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.289 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "iostat", 2023-11-06T01:49:30.289 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.289 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.289 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.289 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.289 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.290 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.290 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.290 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.290 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.290 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.291 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.291 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.291 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.291 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.291 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.291 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.292 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.292 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.292 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.292 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.292 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.293 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.293 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.293 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.293 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.294 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.294 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.294 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.294 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.295 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.295 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.295 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.295 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.295 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.296 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.296 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.296 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.296 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.297 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.297 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.297 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.297 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.297 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.298 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.298 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.298 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.298 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.298 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.299 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.299 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.299 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.299 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.299 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.299 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.300 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.300 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.300 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.300 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.300 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.301 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.301 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.301 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.301 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.301 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.301 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.302 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.302 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.302 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.302 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.303 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "k8sevents", 2023-11-06T01:49:30.303 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.303 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.303 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.304 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_event_retention_days": { 2023-11-06T01:49:30.304 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ceph_event_retention_days", 2023-11-06T01:49:30.304 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.304 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.304 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.305 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7", 2023-11-06T01:49:30.305 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.305 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.305 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.305 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-06T01:49:30.305 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.306 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.306 INFO:teuthology.orchestra.run.smithi053.stdout: "config_check_secs": { 2023-11-06T01:49:30.306 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_check_secs", 2023-11-06T01:49:30.307 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.307 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.307 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.307 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:30.307 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "10", 2023-11-06T01:49:30.308 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.308 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.308 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-06T01:49:30.308 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.308 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.308 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.309 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.309 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.309 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.310 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.310 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.310 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.310 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.311 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.311 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.311 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.311 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.311 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.312 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.312 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.312 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.312 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.312 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.313 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.313 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.313 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.313 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.313 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.314 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.314 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.314 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.314 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.314 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.315 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.315 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.315 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.315 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.316 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.316 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.316 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.316 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.316 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.317 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.317 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.317 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.317 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.317 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.317 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.318 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.318 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.318 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.318 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.318 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.319 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.319 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.319 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.319 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.320 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.320 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.320 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.320 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.320 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.321 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.321 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.321 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.321 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.321 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.321 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.322 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.322 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.323 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "localpool", 2023-11-06T01:49:30.323 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.323 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.323 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.324 INFO:teuthology.orchestra.run.smithi053.stdout: "failure_domain": { 2023-11-06T01:49:30.324 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "failure_domain", 2023-11-06T01:49:30.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.324 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.324 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.324 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "host", 2023-11-06T01:49:30.325 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.325 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.325 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.325 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "failure domain for any created local pool", 2023-11-06T01:49:30.325 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-06T01:49:30.326 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.326 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.326 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.326 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.326 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.327 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.327 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.327 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.327 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.327 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.328 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.328 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.328 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.328 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.328 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.328 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.329 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.329 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.329 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.329 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.329 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.330 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.330 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.330 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.330 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.331 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.331 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.331 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.331 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.331 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.331 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.332 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.332 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.332 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.332 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.333 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.333 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.333 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.333 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.333 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.334 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.334 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.334 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.334 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.334 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.335 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.335 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.335 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.335 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.335 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.336 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.336 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.336 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.336 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.336 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.337 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.337 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.337 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.337 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.337 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.338 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.338 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.338 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.338 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.338 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.339 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.339 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.339 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.339 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": { 2023-11-06T01:49:30.339 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_size", 2023-11-06T01:49:30.340 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.340 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.340 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.340 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.340 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.341 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.341 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.341 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default min_size for any created local pool", 2023-11-06T01:49:30.341 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-06T01:49:30.341 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.342 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.342 INFO:teuthology.orchestra.run.smithi053.stdout: "num_rep": { 2023-11-06T01:49:30.342 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "num_rep", 2023-11-06T01:49:30.342 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.343 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.343 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.343 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3", 2023-11-06T01:49:30.343 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.343 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.344 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.344 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default replica count for any created local pool", 2023-11-06T01:49:30.344 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.344 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.344 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.345 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": { 2023-11-06T01:49:30.345 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_num", 2023-11-06T01:49:30.345 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.345 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.345 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.346 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "128", 2023-11-06T01:49:30.346 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.346 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.346 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.346 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default pg_num for any created local pool", 2023-11-06T01:49:30.347 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.347 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.347 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.347 INFO:teuthology.orchestra.run.smithi053.stdout: "prefix": { 2023-11-06T01:49:30.348 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prefix", 2023-11-06T01:49:30.348 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.348 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.348 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.348 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.348 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.349 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.349 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.349 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name prefix for any created local pool", 2023-11-06T01:49:30.349 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.349 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.350 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.350 INFO:teuthology.orchestra.run.smithi053.stdout: "subtree": { 2023-11-06T01:49:30.350 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "subtree", 2023-11-06T01:49:30.350 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.351 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.351 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.351 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "rack", 2023-11-06T01:49:30.351 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.351 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.351 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.352 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-06T01:49:30.352 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-06T01:49:30.352 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.352 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.353 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mds_autoscaler", 2023-11-06T01:49:30.353 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.354 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.354 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.354 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.354 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.354 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.355 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.355 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.355 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.355 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.355 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.355 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.356 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.356 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.356 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.356 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.356 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.357 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.357 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.357 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.357 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.357 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.358 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.358 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.358 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.358 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.359 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.359 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.359 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.359 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.359 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.360 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.360 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.360 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.360 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.360 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.361 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.361 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.361 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.361 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.361 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.362 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.362 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.362 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.362 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.362 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.363 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.363 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.363 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.363 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.363 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.364 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.364 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.364 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.364 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.365 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.365 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.365 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.365 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.365 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.365 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.366 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.366 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.366 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.366 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.366 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.367 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.367 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.368 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirroring", 2023-11-06T01:49:30.368 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.368 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.368 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.368 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.369 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.369 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.369 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.369 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.370 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.370 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.370 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.370 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.370 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.370 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.371 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.371 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.371 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.371 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.371 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.372 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.372 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.372 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.372 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.373 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.373 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.373 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.374 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.374 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.374 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.374 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.374 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.375 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.375 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.375 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.375 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.376 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.376 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.376 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.377 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.377 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.377 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.377 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.377 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.377 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.378 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.378 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.378 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.378 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.378 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.379 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.379 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.379 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.380 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.380 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.380 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.380 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.380 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.381 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.381 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.381 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.381 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.381 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.382 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.383 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "nfs", 2023-11-06T01:49:30.383 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.383 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.383 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.383 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.383 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.384 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.384 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.384 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.384 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.385 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.385 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.385 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.385 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.385 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.386 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.386 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.386 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.386 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.386 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.386 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.387 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.387 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.387 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.388 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.388 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.388 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.388 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.388 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.389 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.389 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.389 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.389 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.389 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.389 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.390 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.390 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.390 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.391 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.391 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.391 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.391 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.392 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.392 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.392 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.392 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.392 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.392 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.393 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.393 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.393 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.393 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.393 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.394 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.394 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.394 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.395 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.395 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.395 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.395 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.395 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.396 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.396 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.396 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.396 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.396 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.397 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2023-11-06T01:49:30.397 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.397 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.398 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.398 INFO:teuthology.orchestra.run.smithi053.stdout: "fail_fs": { 2023-11-06T01:49:30.398 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "fail_fs", 2023-11-06T01:49:30.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.398 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.399 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.399 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.399 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.399 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.400 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-06T01:49:30.400 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.400 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.400 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.400 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.401 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.401 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.401 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.401 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.402 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.402 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.402 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.402 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.402 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.403 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.403 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.403 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.403 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.403 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.404 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.404 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.404 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.404 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.404 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.405 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.405 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.405 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.405 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.405 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.406 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.406 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.406 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.406 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.406 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.407 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.407 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.407 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.407 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.408 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.408 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.408 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.408 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.408 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.409 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.409 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.409 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.409 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.409 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.409 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.410 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.410 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.410 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.410 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.410 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.411 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.411 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.411 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.412 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.412 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.412 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.412 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.412 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.412 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.413 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.413 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.413 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.414 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator": { 2023-11-06T01:49:30.414 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2023-11-06T01:49:30.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.414 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.415 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.415 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.415 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.415 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.415 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.415 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm", 2023-11-06T01:49:30.416 INFO:teuthology.orchestra.run.smithi053.stdout: "rook", 2023-11-06T01:49:30.416 INFO:teuthology.orchestra.run.smithi053.stdout: "test_orchestrator" 2023-11-06T01:49:30.416 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.416 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Orchestrator backend", 2023-11-06T01:49:30.416 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.417 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.417 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.418 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_perf_query", 2023-11-06T01:49:30.418 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.418 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.418 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.418 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.419 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.419 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.419 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.419 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.419 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.420 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.420 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.420 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.420 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.420 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.420 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.421 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.421 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.421 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.421 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.421 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.422 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.422 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.422 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.422 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.423 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.423 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.423 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.423 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.424 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.424 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.424 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.424 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.424 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.425 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.425 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.425 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.425 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.425 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.426 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.426 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.426 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.426 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.426 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.427 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.427 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.427 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.427 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.427 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.428 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.428 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.428 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.428 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.428 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.429 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.429 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.429 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.429 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.429 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.430 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.430 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.430 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.430 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.431 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.431 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.431 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.431 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.431 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.432 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.432 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_support", 2023-11-06T01:49:30.432 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.433 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.433 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.433 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.433 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.433 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.434 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.434 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.434 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.434 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.434 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.435 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.435 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.435 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.435 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.435 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.436 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.436 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.436 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.436 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.436 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.436 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.437 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.437 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.437 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.437 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.438 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.438 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.438 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.438 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.438 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.439 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.439 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.439 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.439 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.439 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.441 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.441 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.442 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.442 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.442 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.442 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.442 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.443 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.443 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.443 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.443 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.443 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.444 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.444 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.444 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.444 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.444 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.444 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.445 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.445 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.445 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.446 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.446 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.446 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.446 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.446 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.447 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.447 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.447 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.447 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.447 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.448 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_autoscaler", 2023-11-06T01:49:30.448 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.449 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.449 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.449 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.449 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.449 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.449 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.450 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.450 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.450 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.450 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.450 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.451 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.451 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.451 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.451 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.451 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.452 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.452 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.452 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.452 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.452 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.453 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.453 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.453 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.453 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.454 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.454 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.454 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.454 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.454 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.455 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.455 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.455 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.455 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.455 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.456 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.456 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.456 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.456 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.456 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.457 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.457 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.457 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.457 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.457 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.457 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.458 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.458 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.458 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.458 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.458 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.459 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.459 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.459 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.459 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.460 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.460 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.460 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.460 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.460 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.460 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.461 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.461 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.461 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.461 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.461 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.462 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.462 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.462 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:30.462 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:30.463 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.463 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.463 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.463 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:30.463 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.464 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.464 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.464 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.464 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.464 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.464 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.465 INFO:teuthology.orchestra.run.smithi053.stdout: "threshold": { 2023-11-06T01:49:30.465 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threshold", 2023-11-06T01:49:30.465 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:30.465 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.466 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.466 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2023-11-06T01:49:30.466 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1.0", 2023-11-06T01:49:30.466 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.466 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.467 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "scaling threshold", 2023-11-06T01:49:30.467 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-06T01:49:30.467 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.467 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.468 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "progress", 2023-11-06T01:49:30.468 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.469 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.469 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.469 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_pg_recovery_event": { 2023-11-06T01:49:30.469 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_pg_recovery_event", 2023-11-06T01:49:30.469 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.470 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.470 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.470 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.470 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.470 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.470 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.471 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-06T01:49:30.471 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.471 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.471 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.472 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2023-11-06T01:49:30.472 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2023-11-06T01:49:30.472 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.472 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.472 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.472 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.473 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.473 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.473 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.473 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.473 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.474 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.474 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.474 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.475 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.475 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.475 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.475 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.475 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.475 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.476 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.476 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.476 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.476 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.476 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.477 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.477 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.477 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.477 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.477 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.477 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.478 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.478 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.478 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.478 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.479 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.479 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.479 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.479 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.479 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.480 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.480 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.480 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.480 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.480 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.480 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.481 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.481 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.481 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.481 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.482 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.482 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.482 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.482 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.482 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.483 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.483 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.483 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.483 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.483 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.483 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.484 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.484 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.484 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.484 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.484 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.485 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.485 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.485 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.485 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.485 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.486 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.486 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.486 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.486 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.486 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.487 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.487 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.487 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.487 INFO:teuthology.orchestra.run.smithi053.stdout: "max_completed_events": { 2023-11-06T01:49:30.488 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_completed_events", 2023-11-06T01:49:30.488 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.488 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.488 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.488 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "50", 2023-11-06T01:49:30.489 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.489 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.489 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.489 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "number of past completed events to remember", 2023-11-06T01:49:30.489 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.489 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.490 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.490 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:30.490 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:30.490 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.491 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.491 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.491 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:30.491 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.491 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.492 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.492 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long the module is going to sleep", 2023-11-06T01:49:30.492 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.492 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.492 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.492 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.493 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.493 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus", 2023-11-06T01:49:30.493 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.494 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.494 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.494 INFO:teuthology.orchestra.run.smithi053.stdout: "cache": { 2023-11-06T01:49:30.494 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cache", 2023-11-06T01:49:30.494 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.495 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.495 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.495 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.495 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.495 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.496 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.496 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.496 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.496 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.496 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.497 INFO:teuthology.orchestra.run.smithi053.stdout: "exclude_perf_counters": { 2023-11-06T01:49:30.497 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "exclude_perf_counters", 2023-11-06T01:49:30.497 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.497 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.497 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.498 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.498 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.498 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.498 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.498 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-06T01:49:30.499 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-06T01:49:30.499 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.499 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.499 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.500 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.500 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.500 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.500 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.500 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.500 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.501 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.501 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.501 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.501 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.501 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.502 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.502 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.502 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.502 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.502 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.503 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.503 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.503 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.503 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.503 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.504 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.504 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.504 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.504 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.504 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.505 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.505 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.505 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.505 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.505 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.506 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.506 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.506 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.506 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.506 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.507 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.507 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.507 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.507 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.507 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.508 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.508 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.508 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.508 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.508 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.509 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.509 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.509 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.509 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.509 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.509 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.510 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.510 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.510 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.510 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.511 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.511 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.511 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.511 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.511 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.512 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.512 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.512 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.512 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.512 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools": { 2023-11-06T01:49:30.513 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools", 2023-11-06T01:49:30.513 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.513 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.513 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.513 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.514 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.514 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.514 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.514 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.514 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.515 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.515 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.515 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-06T01:49:30.515 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-06T01:49:30.516 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.516 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.516 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.516 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "300", 2023-11-06T01:49:30.516 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.517 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.517 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.517 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.517 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.517 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.517 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.518 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_interval": { 2023-11-06T01:49:30.518 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_interval", 2023-11-06T01:49:30.518 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:30.518 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.519 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.519 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15.0", 2023-11-06T01:49:30.519 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.519 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.519 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.519 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.520 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.520 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.520 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.520 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2023-11-06T01:49:30.521 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2023-11-06T01:49:30.521 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.521 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.521 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.521 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2023-11-06T01:49:30.522 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.522 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.522 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.522 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-06T01:49:30.522 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.523 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.523 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.523 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2023-11-06T01:49:30.523 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2023-11-06T01:49:30.523 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.524 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.524 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.524 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "9283", 2023-11-06T01:49:30.524 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.524 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.525 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.525 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-06T01:49:30.525 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.525 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.525 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.526 INFO:teuthology.orchestra.run.smithi053.stdout: "stale_cache_strategy": { 2023-11-06T01:49:30.526 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stale_cache_strategy", 2023-11-06T01:49:30.526 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.526 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.527 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.527 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "log", 2023-11-06T01:49:30.527 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.527 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.527 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.528 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.528 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.528 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.528 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.528 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2023-11-06T01:49:30.529 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2023-11-06T01:49:30.529 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.529 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.529 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.529 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "default", 2023-11-06T01:49:30.530 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.530 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.530 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.530 INFO:teuthology.orchestra.run.smithi053.stdout: "default", 2023-11-06T01:49:30.530 INFO:teuthology.orchestra.run.smithi053.stdout: "error" 2023-11-06T01:49:30.531 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.531 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.531 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.531 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.531 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.532 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2023-11-06T01:49:30.532 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2023-11-06T01:49:30.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.532 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.533 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2023-11-06T01:49:30.533 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2023-11-06T01:49:30.533 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2023-11-06T01:49:30.533 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.533 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.534 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.534 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.534 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.534 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.534 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.535 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_support", 2023-11-06T01:49:30.535 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.535 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.535 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.536 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.536 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.536 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.537 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.537 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.537 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.537 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.537 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.538 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.538 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.538 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.538 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.538 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.539 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.539 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.539 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.539 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.539 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.540 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.540 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.540 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.540 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.540 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.541 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.541 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.541 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.541 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.541 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.542 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.542 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.542 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.542 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.543 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.543 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.543 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.543 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.543 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.543 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.544 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.544 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.544 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.544 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.544 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.545 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.545 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.545 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.545 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.545 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.546 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.546 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.546 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.546 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.547 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.547 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.547 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.547 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.548 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.548 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.548 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.548 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.548 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.549 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.549 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.549 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_snap_create": { 2023-11-06T01:49:30.549 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_snap_create", 2023-11-06T01:49:30.550 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.550 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.550 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.550 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:30.550 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.550 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.551 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.551 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.551 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.551 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.551 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.552 INFO:teuthology.orchestra.run.smithi053.stdout: "mirror_snapshot_schedule": { 2023-11-06T01:49:30.552 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirror_snapshot_schedule", 2023-11-06T01:49:30.552 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.552 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.553 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.553 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.553 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.553 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.553 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.554 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.554 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.554 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.554 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.555 INFO:teuthology.orchestra.run.smithi053.stdout: "trash_purge_schedule": { 2023-11-06T01:49:30.555 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "trash_purge_schedule", 2023-11-06T01:49:30.555 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.555 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.555 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.556 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.556 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.556 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.556 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.556 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.556 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.557 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.557 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.558 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "restful", 2023-11-06T01:49:30.558 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.558 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.558 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.559 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_auth": { 2023-11-06T01:49:30.559 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_auth", 2023-11-06T01:49:30.559 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.559 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.559 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.560 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.560 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.560 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.560 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.560 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.560 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.561 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.561 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.561 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2023-11-06T01:49:30.561 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2023-11-06T01:49:30.562 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.562 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.562 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.562 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.562 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.563 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.563 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.563 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.563 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.563 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.563 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.564 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.564 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.564 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.564 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.565 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.565 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.565 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.565 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.565 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.566 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.566 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.566 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.566 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.566 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.566 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.567 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.567 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.567 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.567 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.567 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.568 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.568 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.568 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.569 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.569 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.569 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.569 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.569 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.570 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.570 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.570 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.570 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.571 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.571 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.571 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.571 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.571 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.572 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.572 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.572 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.572 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.572 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.573 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.573 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.573 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.573 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.573 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.574 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.574 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.574 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.575 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.575 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.575 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.575 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.576 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.576 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.576 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.576 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.576 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.577 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.577 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.577 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2023-11-06T01:49:30.577 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2023-11-06T01:49:30.577 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.578 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.578 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.578 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.578 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.578 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.579 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.579 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.579 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.579 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.579 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.580 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2023-11-06T01:49:30.580 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2023-11-06T01:49:30.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.580 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.581 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.581 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.581 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.581 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.581 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.582 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.582 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.582 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.583 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rgw", 2023-11-06T01:49:30.583 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.583 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.584 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.584 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.584 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.584 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.584 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.585 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.585 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.585 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.585 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.585 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.586 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.586 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.586 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.586 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.586 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.587 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.587 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.587 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.587 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.587 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.588 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.588 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.588 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.588 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.588 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.589 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.589 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.589 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.589 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.589 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.590 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.590 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.590 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.591 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.591 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.591 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.591 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.592 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.592 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.592 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.592 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.592 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.593 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.593 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.593 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.593 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.593 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.594 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.594 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.594 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.594 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.595 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.595 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.595 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.595 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.596 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.596 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.596 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.596 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.597 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.597 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.597 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.597 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.598 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rook", 2023-11-06T01:49:30.598 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.598 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.598 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.598 INFO:teuthology.orchestra.run.smithi053.stdout: "drive_group_interval": { 2023-11-06T01:49:30.599 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "drive_group_interval", 2023-11-06T01:49:30.599 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:30.599 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.599 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.599 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "300.0", 2023-11-06T01:49:30.600 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.600 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.600 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.600 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-06T01:49:30.600 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.601 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.601 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.601 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.601 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.602 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.602 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.602 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.602 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.602 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.602 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.603 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.603 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.603 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.603 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.603 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.604 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.604 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.604 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.604 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.604 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.605 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.605 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.605 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.605 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.605 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.606 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.606 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.606 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.606 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.606 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.607 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.607 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.607 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.607 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.608 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.608 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.608 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.608 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.608 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.609 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.609 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.609 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.609 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.609 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.610 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.610 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.610 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.610 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.610 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.611 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.611 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.611 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.611 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.611 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: "storage_class": { 2023-11-06T01:49:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "storage_class", 2023-11-06T01:49:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "local", 2023-11-06T01:49:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-06T01:49:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "selftest", 2023-11-06T01:49:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "roption1": { 2023-11-06T01:49:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption1", 2023-11-06T01:49:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "roption2": { 2023-11-06T01:49:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption2", 2023-11-06T01:49:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "xyz", 2023-11-06T01:49:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption1": { 2023-11-06T01:49:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption1", 2023-11-06T01:49:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption2": { 2023-11-06T01:49:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption2", 2023-11-06T01:49:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption3": { 2023-11-06T01:49:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption3", 2023-11-06T01:49:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption4": { 2023-11-06T01:49:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption4", 2023-11-06T01:49:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption5": { 2023-11-06T01:49:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption5", 2023-11-06T01:49:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption6": { 2023-11-06T01:49:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption6", 2023-11-06T01:49:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption7": { 2023-11-06T01:49:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption7", 2023-11-06T01:49:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "42", 2023-11-06T01:49:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "testkey": { 2023-11-06T01:49:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testkey", 2023-11-06T01:49:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "testlkey": { 2023-11-06T01:49:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testlkey", 2023-11-06T01:49:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "testnewline": { 2023-11-06T01:49:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testnewline", 2023-11-06T01:49:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snap_schedule", 2023-11-06T01:49:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_m_granularity": { 2023-11-06T01:49:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_m_granularity", 2023-11-06T01:49:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow minute scheduled snapshots", 2023-11-06T01:49:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "dump_on_update": { 2023-11-06T01:49:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dump_on_update", 2023-11-06T01:49:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "dump database to debug log on update", 2023-11-06T01:49:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stats", 2023-11-06T01:49:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "status", 2023-11-06T01:49:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telegraf", 2023-11-06T01:49:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "address": { 2023-11-06T01:49:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "address", 2023-11-06T01:49:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-06T01:49:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15", 2023-11-06T01:49:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.729 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.729 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.729 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.729 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.729 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.729 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.730 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.730 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.730 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.730 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.730 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.731 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.731 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.731 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.731 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.731 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.732 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.732 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.732 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.732 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.732 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.733 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.733 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.733 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.733 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.734 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.734 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telemetry", 2023-11-06T01:49:30.734 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.734 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_basic": { 2023-11-06T01:49:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_basic", 2023-11-06T01:49:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.736 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.736 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.736 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.736 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.736 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.737 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-06T01:49:30.737 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.737 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.737 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.737 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_crash": { 2023-11-06T01:49:30.738 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_crash", 2023-11-06T01:49:30.738 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.738 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.738 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.738 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.739 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.739 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.739 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.739 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-06T01:49:30.739 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.739 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.740 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.740 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_device": { 2023-11-06T01:49:30.740 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_device", 2023-11-06T01:49:30.740 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.741 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.741 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.741 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.741 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.741 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.742 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.742 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-06T01:49:30.742 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.742 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.742 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.743 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_ident": { 2023-11-06T01:49:30.743 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_ident", 2023-11-06T01:49:30.743 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.743 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.744 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.744 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.744 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.744 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.744 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-06T01:49:30.745 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.745 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.745 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.745 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_perf": { 2023-11-06T01:49:30.746 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_perf", 2023-11-06T01:49:30.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.746 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.746 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.747 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.747 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.747 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.747 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-06T01:49:30.747 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.748 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.748 INFO:teuthology.orchestra.run.smithi053.stdout: "contact": { 2023-11-06T01:49:30.748 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "contact", 2023-11-06T01:49:30.748 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.749 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.749 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.749 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.749 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.749 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.749 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.750 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.750 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.750 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.750 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": { 2023-11-06T01:49:30.751 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "description", 2023-11-06T01:49:30.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.751 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.751 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.752 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.752 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.752 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.752 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.752 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.753 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.753 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.753 INFO:teuthology.orchestra.run.smithi053.stdout: "device_url": { 2023-11-06T01:49:30.753 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_url", 2023-11-06T01:49:30.754 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.754 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.754 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.754 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-06T01:49:30.754 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.755 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.755 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.755 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.755 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.755 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.755 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.756 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2023-11-06T01:49:30.756 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2023-11-06T01:49:30.756 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.756 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.757 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.757 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.757 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.757 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.757 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.758 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.758 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.758 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.758 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.759 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:30.759 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:30.759 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.759 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.759 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "24", 2023-11-06T01:49:30.760 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "8", 2023-11-06T01:49:30.760 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.760 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.760 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.760 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.761 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.761 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.761 INFO:teuthology.orchestra.run.smithi053.stdout: "last_opt_revision": { 2023-11-06T01:49:30.761 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "last_opt_revision", 2023-11-06T01:49:30.762 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.762 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.763 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.763 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1", 2023-11-06T01:49:30.764 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.764 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.764 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.764 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.764 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.765 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.765 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard": { 2023-11-06T01:49:30.765 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard", 2023-11-06T01:49:30.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.766 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.766 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.766 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.767 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.767 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.767 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.767 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.767 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.767 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.768 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard_description": { 2023-11-06T01:49:30.768 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard_description", 2023-11-06T01:49:30.768 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.769 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.769 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.769 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.769 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.770 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.770 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.770 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.770 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.770 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.771 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.771 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.771 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.771 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.772 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.772 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.772 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.772 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.772 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.773 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.773 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.773 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.773 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.773 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.773 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.774 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.774 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.774 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.774 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.775 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.775 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.775 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.775 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.775 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.776 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.776 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.776 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.776 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.776 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.777 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.777 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.777 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.777 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.778 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.778 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.778 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.778 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.779 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.779 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.779 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.779 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.779 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.780 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.780 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.780 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.780 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.780 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.780 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.781 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.781 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.781 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.781 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.782 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.782 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.782 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.782 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.782 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.783 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.783 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.783 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.783 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.783 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.783 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.784 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.784 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.784 INFO:teuthology.orchestra.run.smithi053.stdout: "organization": { 2023-11-06T01:49:30.784 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "organization", 2023-11-06T01:49:30.785 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.785 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.785 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.785 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.785 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.786 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.786 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.786 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.786 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.786 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.787 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.787 INFO:teuthology.orchestra.run.smithi053.stdout: "proxy": { 2023-11-06T01:49:30.787 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "proxy", 2023-11-06T01:49:30.787 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.788 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.788 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.788 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.788 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.788 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.788 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.789 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.789 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.789 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.789 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.790 INFO:teuthology.orchestra.run.smithi053.stdout: "url": { 2023-11-06T01:49:30.790 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url", 2023-11-06T01:49:30.790 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.790 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.790 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.790 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-06T01:49:30.791 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.791 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.791 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.791 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.791 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.792 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.792 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.793 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "test_orchestrator", 2023-11-06T01:49:30.793 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.793 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.793 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.793 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.794 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.794 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.794 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.794 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.795 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.795 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.795 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.795 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.795 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.795 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.796 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.796 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.796 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.796 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.796 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.797 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.797 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.797 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.797 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.798 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.798 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.798 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.798 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.798 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.799 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.799 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.799 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.799 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.800 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.800 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.800 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.800 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.800 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.801 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.801 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.801 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.801 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.801 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.802 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.802 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.802 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.802 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.802 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.802 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.803 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.803 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.803 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.803 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.803 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.804 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.804 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.804 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.804 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.805 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.805 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.805 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.805 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.805 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.806 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.806 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.806 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.807 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "volumes", 2023-11-06T01:49:30.807 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.807 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.807 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.808 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.808 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.808 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.808 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.809 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.809 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.809 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.809 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.809 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.809 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.810 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.810 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.810 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.810 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.810 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.811 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.811 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.811 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.811 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.812 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.812 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.812 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.812 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.813 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.813 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.813 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.813 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.813 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.814 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.814 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.814 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.814 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.815 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.815 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.815 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.815 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.815 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.815 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.816 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.816 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.816 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.816 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.816 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.817 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.817 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.817 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.817 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.817 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.818 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.818 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.818 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.819 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.819 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.819 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.819 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.820 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.820 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.820 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.820 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.820 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.821 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.821 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_clones": { 2023-11-06T01:49:30.821 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_clones", 2023-11-06T01:49:30.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.821 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.822 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4", 2023-11-06T01:49:30.822 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.822 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.822 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.823 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-06T01:49:30.823 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.823 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.824 INFO:teuthology.orchestra.run.smithi053.stdout: "snapshot_clone_delay": { 2023-11-06T01:49:30.824 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snapshot_clone_delay", 2023-11-06T01:49:30.824 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.824 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.825 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:30.825 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.825 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.825 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.825 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-06T01:49:30.825 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.826 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.826 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.827 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix", 2023-11-06T01:49:30.827 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.827 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.827 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.827 INFO:teuthology.orchestra.run.smithi053.stdout: "discovery_interval": { 2023-11-06T01:49:30.828 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "discovery_interval", 2023-11-06T01:49:30.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:30.828 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.828 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "100", 2023-11-06T01:49:30.829 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.829 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.829 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.829 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.829 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.830 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.830 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.830 INFO:teuthology.orchestra.run.smithi053.stdout: "identifier": { 2023-11-06T01:49:30.830 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "identifier", 2023-11-06T01:49:30.831 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.831 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.831 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.831 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.831 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.833 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:30.833 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:30.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.833 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.834 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:30.834 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.834 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.834 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.834 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.835 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.835 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.835 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.835 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.836 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.836 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.836 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.836 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.836 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.837 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.837 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.837 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.837 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.837 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.837 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.838 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.838 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.838 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.838 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.838 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.839 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.839 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.839 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.839 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.839 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.840 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.840 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.840 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.840 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.840 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.841 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.841 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.841 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.841 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.841 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.841 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.842 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.842 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.845 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.845 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.845 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.846 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.846 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.846 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.846 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.846 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.847 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.847 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.847 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.847 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.847 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.848 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.848 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.848 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.849 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.849 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.849 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.849 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.849 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.850 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.850 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.850 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.850 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.850 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.851 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.851 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.851 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_host": { 2023-11-06T01:49:30.851 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_host", 2023-11-06T01:49:30.851 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.852 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.852 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.852 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.852 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.852 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.853 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.853 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.853 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.853 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.853 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.854 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_port": { 2023-11-06T01:49:30.854 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_port", 2023-11-06T01:49:30.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.854 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.855 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10051", 2023-11-06T01:49:30.855 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.855 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.855 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.856 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.856 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.856 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.856 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.856 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_sender": { 2023-11-06T01:49:30.857 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_sender", 2023-11-06T01:49:30.857 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.857 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.857 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.857 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-06T01:49:30.858 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.858 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.858 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.858 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.858 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.859 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.859 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.860 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-06T01:49:30.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.860 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.860 INFO:teuthology.orchestra.run.smithi053.stdout: "modules": [ 2023-11-06T01:49:30.860 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm", 2023-11-06T01:49:30.861 INFO:teuthology.orchestra.run.smithi053.stdout: "dashboard", 2023-11-06T01:49:30.861 INFO:teuthology.orchestra.run.smithi053.stdout: "iostat", 2023-11-06T01:49:30.861 INFO:teuthology.orchestra.run.smithi053.stdout: "nfs", 2023-11-06T01:49:30.861 INFO:teuthology.orchestra.run.smithi053.stdout: "restful" 2023-11-06T01:49:30.861 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.861 INFO:teuthology.orchestra.run.smithi053.stdout: "available_modules": [ 2023-11-06T01:49:30.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.862 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "alerts", 2023-11-06T01:49:30.862 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.862 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.862 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.863 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:30.863 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:30.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.863 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.863 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.864 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:30.864 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.864 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.864 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.864 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How frequently to reexamine health status", 2023-11-06T01:49:30.864 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.865 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.865 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.865 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.865 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.865 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.866 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.866 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.866 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.866 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.867 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.867 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.867 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.867 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.867 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.868 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.868 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.868 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.868 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.868 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.869 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.869 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.869 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.869 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.870 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.870 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.870 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.870 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.870 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.871 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.871 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.871 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.871 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.871 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.872 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.872 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.872 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.873 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.873 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.873 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.873 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.873 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.873 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.874 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.874 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.874 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.874 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.874 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.875 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.875 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.875 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.875 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.875 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.876 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.876 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.876 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.876 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.876 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.877 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.877 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.877 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.877 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.877 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.877 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.878 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.878 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_destination": { 2023-11-06T01:49:30.878 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_destination", 2023-11-06T01:49:30.878 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.879 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.879 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.879 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.879 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.879 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.880 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email address to send alerts to", 2023-11-06T01:49:30.880 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.880 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.880 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.881 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_from_name": { 2023-11-06T01:49:30.881 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_from_name", 2023-11-06T01:49:30.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.881 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.881 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "Ceph", 2023-11-06T01:49:30.882 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.882 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.882 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.882 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email From: name", 2023-11-06T01:49:30.882 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.883 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.883 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.883 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_host": { 2023-11-06T01:49:30.883 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_host", 2023-11-06T01:49:30.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.884 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.884 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.884 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.884 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.884 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.885 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.885 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP server", 2023-11-06T01:49:30.885 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.885 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.885 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.886 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_password": { 2023-11-06T01:49:30.886 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_password", 2023-11-06T01:49:30.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.886 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.887 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.887 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.887 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.887 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.887 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Password to authenticate with", 2023-11-06T01:49:30.888 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.888 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.888 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.888 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_port": { 2023-11-06T01:49:30.889 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_port", 2023-11-06T01:49:30.889 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.889 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.889 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.889 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "465", 2023-11-06T01:49:30.889 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.890 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.890 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.890 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP port", 2023-11-06T01:49:30.890 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.890 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.891 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.891 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_sender": { 2023-11-06T01:49:30.891 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_sender", 2023-11-06T01:49:30.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.892 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.892 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.892 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.892 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.892 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.893 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.893 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP envelope sender", 2023-11-06T01:49:30.893 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.893 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.893 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.894 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_ssl": { 2023-11-06T01:49:30.894 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_ssl", 2023-11-06T01:49:30.894 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.894 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.894 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.895 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.895 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.895 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.895 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.895 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-06T01:49:30.896 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.896 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.896 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_user": { 2023-11-06T01:49:30.896 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_user", 2023-11-06T01:49:30.897 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.897 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.897 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.897 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.898 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.898 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.898 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "User to authenticate as", 2023-11-06T01:49:30.898 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.898 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.899 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.899 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "balancer", 2023-11-06T01:49:30.900 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.900 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.900 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.900 INFO:teuthology.orchestra.run.smithi053.stdout: "active": { 2023-11-06T01:49:30.900 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "active", 2023-11-06T01:49:30.901 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.901 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.901 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.901 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.901 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.901 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.902 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.902 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically balance PGs across cluster", 2023-11-06T01:49:30.902 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.902 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.902 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.903 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_time": { 2023-11-06T01:49:30.903 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_time", 2023-11-06T01:49:30.903 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.903 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.904 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.904 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0000", 2023-11-06T01:49:30.904 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.904 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.904 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.905 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "beginning time of day to automatically balance", 2023-11-06T01:49:30.905 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-06T01:49:30.905 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.905 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.905 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_weekday": { 2023-11-06T01:49:30.906 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_weekday", 2023-11-06T01:49:30.906 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:30.906 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.906 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:30.907 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2023-11-06T01:49:30.907 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2023-11-06T01:49:30.907 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.907 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-06T01:49:30.907 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-06T01:49:30.907 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.908 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.908 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_max_iterations": { 2023-11-06T01:49:30.908 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_max_iterations", 2023-11-06T01:49:30.908 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:30.909 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.909 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.909 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "25", 2023-11-06T01:49:30.909 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:30.909 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "250", 2023-11-06T01:49:30.909 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.910 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-06T01:49:30.910 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.910 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.910 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.911 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_metrics": { 2023-11-06T01:49:30.911 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_metrics", 2023-11-06T01:49:30.911 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.911 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.911 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.912 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "pgs,objects,bytes", 2023-11-06T01:49:30.912 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.912 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.912 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.912 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-06T01:49:30.912 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-06T01:49:30.913 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.913 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.913 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_step": { 2023-11-06T01:49:30.913 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_step", 2023-11-06T01:49:30.914 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:30.914 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.914 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.914 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.5", 2023-11-06T01:49:30.914 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0.001", 2023-11-06T01:49:30.915 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "0.999", 2023-11-06T01:49:30.915 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.915 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "aggressiveness of optimization", 2023-11-06T01:49:30.915 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-06T01:49:30.915 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.915 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.916 INFO:teuthology.orchestra.run.smithi053.stdout: "end_time": { 2023-11-06T01:49:30.916 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_time", 2023-11-06T01:49:30.916 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.916 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.917 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.917 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2359", 2023-11-06T01:49:30.917 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.917 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.917 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.917 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ending time of day to automatically balance", 2023-11-06T01:49:30.918 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-06T01:49:30.918 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.918 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.918 INFO:teuthology.orchestra.run.smithi053.stdout: "end_weekday": { 2023-11-06T01:49:30.919 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_weekday", 2023-11-06T01:49:30.919 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:30.919 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.919 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.919 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:30.919 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2023-11-06T01:49:30.920 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2023-11-06T01:49:30.920 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.920 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-06T01:49:30.920 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-06T01:49:30.920 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.921 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.921 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:30.921 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:30.921 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.921 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.922 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.922 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.922 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.922 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.922 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.923 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.923 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.923 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.923 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.923 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.924 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.924 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.924 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.924 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.924 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.924 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.925 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:30.925 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:30.925 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.925 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.926 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.926 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.926 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.926 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.926 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.927 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.927 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.927 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.927 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.927 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:30.928 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:30.928 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.928 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.928 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:30.929 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.929 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.929 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.929 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:30.929 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:30.930 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:30.930 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:30.930 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:30.930 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:30.930 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.930 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.931 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.931 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.931 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.931 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:30.932 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:30.932 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.932 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.932 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.932 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.932 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.933 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.933 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.933 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:30.933 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.933 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.934 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.934 INFO:teuthology.orchestra.run.smithi053.stdout: "min_score": { 2023-11-06T01:49:30.934 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_score", 2023-11-06T01:49:30.934 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:30.935 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.935 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.935 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:30.935 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.935 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.935 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.936 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-06T01:49:30.936 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.936 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.936 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2023-11-06T01:49:30.937 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2023-11-06T01:49:30.937 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.937 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.937 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "upmap", 2023-11-06T01:49:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-compat", 2023-11-06T01:49:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: "none", 2023-11-06T01:49:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap" 2023-11-06T01:49:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Balancer mode", 2023-11-06T01:49:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_ids": { 2023-11-06T01:49:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_ids", 2023-11-06T01:49:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-06T01:49:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-06T01:49:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_deviation": { 2023-11-06T01:49:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_deviation", 2023-11-06T01:49:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-06T01:49:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-06T01:49:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_optimizations": { 2023-11-06T01:49:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_optimizations", 2023-11-06T01:49:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-06T01:49:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:30.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:30.951 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephadm", 2023-11-06T01:49:30.951 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:30.951 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:30.951 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:30.952 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_down_multiplier": { 2023-11-06T01:49:30.952 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_down_multiplier", 2023-11-06T01:49:30.952 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:30.952 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.952 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.953 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2023-11-06T01:49:30.953 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.953 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.953 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.954 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-06T01:49:30.954 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.954 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.954 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.955 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_refresh_rate": { 2023-11-06T01:49:30.955 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_refresh_rate", 2023-11-06T01:49:30.955 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.955 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.955 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.956 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "20", 2023-11-06T01:49:30.956 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.956 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.956 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.956 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-06T01:49:30.956 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.957 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.957 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.957 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_starting_port": { 2023-11-06T01:49:30.957 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_starting_port", 2023-11-06T01:49:30.958 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:30.958 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.958 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.958 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4721", 2023-11-06T01:49:30.958 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.959 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.959 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.959 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-06T01:49:30.959 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.959 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.959 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.960 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_ptrace": { 2023-11-06T01:49:30.960 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_ptrace", 2023-11-06T01:49:30.960 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.960 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.961 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.961 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.961 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.961 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.961 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.962 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-06T01:49:30.962 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-06T01:49:30.962 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.962 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.963 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_interval": { 2023-11-06T01:49:30.963 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_interval", 2023-11-06T01:49:30.963 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:30.963 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.963 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.964 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:30.964 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.964 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.964 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.964 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-06T01:49:30.964 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.965 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.965 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.965 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_memory_target_ratio": { 2023-11-06T01:49:30.965 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_memory_target_ratio", 2023-11-06T01:49:30.966 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:30.966 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.966 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.966 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.7", 2023-11-06T01:49:30.966 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.967 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.967 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.967 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-06T01:49:30.967 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.967 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.968 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.968 INFO:teuthology.orchestra.run.smithi053.stdout: "cgroups_split": { 2023-11-06T01:49:30.968 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cgroups_split", 2023-11-06T01:49:30.968 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.968 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.969 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.969 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.969 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.969 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.969 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.970 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-06T01:49:30.970 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.970 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.970 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.971 INFO:teuthology.orchestra.run.smithi053.stdout: "config_checks_enabled": { 2023-11-06T01:49:30.971 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_checks_enabled", 2023-11-06T01:49:30.971 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.971 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.971 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.971 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:30.972 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.972 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.972 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.972 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-06T01:49:30.972 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.972 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.973 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.973 INFO:teuthology.orchestra.run.smithi053.stdout: "config_dashboard": { 2023-11-06T01:49:30.973 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_dashboard", 2023-11-06T01:49:30.973 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:30.974 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.974 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.974 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:30.974 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.974 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.975 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.975 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-06T01:49:30.975 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.975 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.975 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.976 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_alertmanager": { 2023-11-06T01:49:30.976 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_alertmanager", 2023-11-06T01:49:30.976 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.976 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.976 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.976 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-06T01:49:30.977 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.977 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.977 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.977 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:30.977 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.977 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.978 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.978 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_base": { 2023-11-06T01:49:30.978 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_base", 2023-11-06T01:49:30.978 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.979 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.979 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:30.979 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-06T01:49:30.979 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.979 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.980 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.980 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Container image name, without the tag", 2023-11-06T01:49:30.980 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.980 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.980 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.981 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_elasticsearch": { 2023-11-06T01:49:30.981 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_elasticsearch", 2023-11-06T01:49:30.981 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.981 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.982 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.982 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-06T01:49:30.982 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.982 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.982 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.983 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "elasticsearch container image", 2023-11-06T01:49:30.983 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.983 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.983 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.983 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_grafana": { 2023-11-06T01:49:30.984 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_grafana", 2023-11-06T01:49:30.984 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.984 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.984 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.984 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-06T01:49:30.985 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.985 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.985 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.985 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:30.985 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.986 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.986 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.986 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_haproxy": { 2023-11-06T01:49:30.986 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_haproxy", 2023-11-06T01:49:30.987 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.987 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.987 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.987 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-06T01:49:30.987 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.988 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.988 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.988 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "HAproxy container image", 2023-11-06T01:49:30.988 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.988 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.988 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.989 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_agent": { 2023-11-06T01:49:30.989 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_agent", 2023-11-06T01:49:30.989 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.989 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.990 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.990 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-06T01:49:30.990 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.990 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.990 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.991 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger agent container image", 2023-11-06T01:49:30.991 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.991 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.991 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.991 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_collector": { 2023-11-06T01:49:30.992 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_collector", 2023-11-06T01:49:30.992 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.992 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.992 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.992 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-06T01:49:30.992 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.993 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.993 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.993 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger collector container image", 2023-11-06T01:49:30.993 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.993 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.994 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.994 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_query": { 2023-11-06T01:49:30.994 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_query", 2023-11-06T01:49:30.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.995 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.995 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.995 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-06T01:49:30.995 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.995 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.995 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.996 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger query container image", 2023-11-06T01:49:30.996 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.996 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.996 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.997 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_keepalived": { 2023-11-06T01:49:30.997 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_keepalived", 2023-11-06T01:49:30.997 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:30.997 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:30.997 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:30.998 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-06T01:49:30.998 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:30.998 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:30.998 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:30.998 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Keepalived container image", 2023-11-06T01:49:30.998 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:30.999 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:30.999 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:30.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:30.999 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_loki": { 2023-11-06T01:49:30.999 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_loki", 2023-11-06T01:49:31.000 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.000 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.000 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.000 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-06T01:49:31.000 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.001 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.001 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.001 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Loki container image", 2023-11-06T01:49:31.001 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.001 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.002 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_node_exporter": { 2023-11-06T01:49:31.002 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_node_exporter", 2023-11-06T01:49:31.002 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.003 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.003 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-06T01:49:31.003 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.003 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.003 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.004 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:31.004 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.004 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.004 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.005 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_nvmeof": { 2023-11-06T01:49:31.005 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_nvmeof", 2023-11-06T01:49:31.005 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.005 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.005 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.005 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-11-06T01:49:31.006 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.006 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.006 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.006 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Nvme-of container image", 2023-11-06T01:49:31.006 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.007 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.007 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.007 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_prometheus": { 2023-11-06T01:49:31.007 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_prometheus", 2023-11-06T01:49:31.007 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.008 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.008 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.008 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-06T01:49:31.008 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.008 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.009 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.009 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2023-11-06T01:49:31.009 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.009 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.010 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_promtail": { 2023-11-06T01:49:31.010 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_promtail", 2023-11-06T01:49:31.010 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.010 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.011 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.011 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-06T01:49:31.011 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.011 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.011 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.011 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Promtail container image", 2023-11-06T01:49:31.012 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.012 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.012 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_snmp_gateway": { 2023-11-06T01:49:31.013 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_snmp_gateway", 2023-11-06T01:49:31.013 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.013 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.013 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.013 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-06T01:49:31.014 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.014 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.014 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.014 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SNMP Gateway container image", 2023-11-06T01:49:31.014 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.014 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.015 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.015 INFO:teuthology.orchestra.run.smithi053.stdout: "container_init": { 2023-11-06T01:49:31.015 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_init", 2023-11-06T01:49:31.015 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.016 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.016 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.016 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.016 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.016 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.017 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.017 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Run podman/docker with `--init`", 2023-11-06T01:49:31.017 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.017 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.017 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.018 INFO:teuthology.orchestra.run.smithi053.stdout: "daemon_cache_timeout": { 2023-11-06T01:49:31.018 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "daemon_cache_timeout", 2023-11-06T01:49:31.018 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.018 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.019 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.019 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:31.019 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.019 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.019 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.020 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-06T01:49:31.020 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.020 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.020 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.020 INFO:teuthology.orchestra.run.smithi053.stdout: "default_cephadm_command_timeout": { 2023-11-06T01:49:31.021 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_cephadm_command_timeout", 2023-11-06T01:49:31.021 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.021 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.021 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.021 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "900", 2023-11-06T01:49:31.022 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.022 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.022 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.022 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-06T01:49:31.022 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.023 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.023 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.023 INFO:teuthology.orchestra.run.smithi053.stdout: "default_registry": { 2023-11-06T01:49:31.023 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_registry", 2023-11-06T01:49:31.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.024 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.024 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.024 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io", 2023-11-06T01:49:31.024 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.025 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.025 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.025 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-06T01:49:31.025 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.025 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.026 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.026 INFO:teuthology.orchestra.run.smithi053.stdout: "device_cache_timeout": { 2023-11-06T01:49:31.026 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_cache_timeout", 2023-11-06T01:49:31.026 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.026 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.027 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.027 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1800", 2023-11-06T01:49:31.027 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.027 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.027 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.028 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache device inventory", 2023-11-06T01:49:31.028 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.028 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.028 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.029 INFO:teuthology.orchestra.run.smithi053.stdout: "device_enhanced_scan": { 2023-11-06T01:49:31.029 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_enhanced_scan", 2023-11-06T01:49:31.029 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.029 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.029 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.029 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.030 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.030 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.030 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.030 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-06T01:49:31.030 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.031 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.031 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.031 INFO:teuthology.orchestra.run.smithi053.stdout: "facts_cache_timeout": { 2023-11-06T01:49:31.031 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "facts_cache_timeout", 2023-11-06T01:49:31.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.032 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.032 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.032 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:31.032 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.032 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.033 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.033 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache host facts data", 2023-11-06T01:49:31.033 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.033 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.033 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.034 INFO:teuthology.orchestra.run.smithi053.stdout: "host_check_interval": { 2023-11-06T01:49:31.034 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "host_check_interval", 2023-11-06T01:49:31.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.034 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.035 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.035 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:31.035 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.035 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.035 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.036 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to perform a host check", 2023-11-06T01:49:31.036 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.036 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.036 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.036 INFO:teuthology.orchestra.run.smithi053.stdout: "inventory_list_all": { 2023-11-06T01:49:31.037 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "inventory_list_all", 2023-11-06T01:49:31.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.037 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.037 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.037 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.038 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.038 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.038 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.038 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-06T01:49:31.038 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.039 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.039 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.039 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.039 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.039 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.040 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.040 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.040 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.040 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.040 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.041 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.041 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.041 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.041 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.041 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.041 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.042 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.042 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.042 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.042 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.042 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.043 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.043 INFO:teuthology.orchestra.run.smithi053.stdout: "log_refresh_metadata": { 2023-11-06T01:49:31.043 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_refresh_metadata", 2023-11-06T01:49:31.043 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.043 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.044 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.044 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.044 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.044 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.044 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.045 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-06T01:49:31.045 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.045 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.045 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.046 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.046 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.046 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.046 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.046 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.046 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.047 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.047 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.047 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.048 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-06T01:49:31.048 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.048 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.049 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.049 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.049 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.050 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.050 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.050 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.050 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.050 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.051 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.051 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.051 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.051 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.053 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.053 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.054 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.054 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.054 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.054 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.054 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.055 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.055 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.055 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.055 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.056 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.056 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.056 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.056 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.056 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.057 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.057 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.057 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.057 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.057 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.058 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.058 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-06T01:49:31.058 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-06T01:49:31.058 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.058 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.059 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.059 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.059 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.059 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.059 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.060 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-06T01:49:31.060 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.060 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.060 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.061 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-06T01:49:31.061 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-06T01:49:31.061 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.061 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.061 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.062 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "*", 2023-11-06T01:49:31.062 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.062 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.062 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.062 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-06T01:49:31.063 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.063 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.063 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.063 INFO:teuthology.orchestra.run.smithi053.stdout: "max_count_per_host": { 2023-11-06T01:49:31.064 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_count_per_host", 2023-11-06T01:49:31.064 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.064 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.064 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.064 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:31.065 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.065 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.065 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.065 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of daemons per service per host", 2023-11-06T01:49:31.065 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.066 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.066 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.066 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd_draining_count": { 2023-11-06T01:49:31.066 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_osd_draining_count", 2023-11-06T01:49:31.067 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.067 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.067 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.067 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:31.068 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.068 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.068 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.068 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-06T01:49:31.068 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "migration_current": { 2023-11-06T01:49:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "migration_current", 2023-11-06T01:49:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "internal - do not modify", 2023-11-06T01:49:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2023-11-06T01:49:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2023-11-06T01:49:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "root", 2023-11-06T01:49:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm-package", 2023-11-06T01:49:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "root" 2023-11-06T01:49:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "mode for remote execution of cephadm", 2023-11-06T01:49:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "prometheus_alerts_path": { 2023-11-06T01:49:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus_alerts_path", 2023-11-06T01:49:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-06T01:49:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-06T01:49:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_insecure": { 2023-11-06T01:49:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_insecure", 2023-11-06T01:49:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-06T01:49:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_password": { 2023-11-06T01:49:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_password", 2023-11-06T01:49:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-06T01:49:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_url": { 2023-11-06T01:49:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_url", 2023-11-06T01:49:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-06T01:49:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_username": { 2023-11-06T01:49:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_username", 2023-11-06T01:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-06T01:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "secure_monitoring_stack": { 2023-11-06T01:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "secure_monitoring_stack", 2023-11-06T01:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-06T01:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: "service_discovery_port": { 2023-11-06T01:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "service_discovery_port", 2023-11-06T01:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8765", 2023-11-06T01:49:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "cephadm service discovery port", 2023-11-06T01:49:31.098 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.098 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.098 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.098 INFO:teuthology.orchestra.run.smithi053.stdout: "ssh_config_file": { 2023-11-06T01:49:31.099 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssh_config_file", 2023-11-06T01:49:31.099 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.099 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.099 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.099 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-06T01:49:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: "use_agent": { 2023-11-06T01:49:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_agent", 2023-11-06T01:49:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-06T01:49:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.104 INFO:teuthology.orchestra.run.smithi053.stdout: "use_repo_digest": { 2023-11-06T01:49:31.104 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_repo_digest", 2023-11-06T01:49:31.104 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.106 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.106 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-06T01:49:31.106 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.106 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.107 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.107 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_failed_host_check": { 2023-11-06T01:49:31.107 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_failed_host_check", 2023-11-06T01:49:31.108 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.108 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.108 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.108 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if the host check fails", 2023-11-06T01:49:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.110 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.110 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.110 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_daemons": { 2023-11-06T01:49:31.111 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_daemons", 2023-11-06T01:49:31.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.111 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.111 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.111 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-06T01:49:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_hosts": { 2023-11-06T01:49:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_hosts", 2023-11-06T01:49:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-06T01:49:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crash", 2023-11-06T01:49:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "retain_interval": { 2023-11-06T01:49:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retain_interval", 2023-11-06T01:49:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "31536000", 2023-11-06T01:49:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-06T01:49:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_recent_interval": { 2023-11-06T01:49:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_recent_interval", 2023-11-06T01:49:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1209600", 2023-11-06T01:49:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-06T01:49:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dashboard", 2023-11-06T01:49:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-06T01:49:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-06T01:49:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_HOST": { 2023-11-06T01:49:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-06T01:49:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-06T01:49:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-06T01:49:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_ENABLED": { 2023-11-06T01:49:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_ENABLED", 2023-11-06T01:49:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-06T01:49:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-06T01:49:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "ENABLE_BROWSABLE_API": { 2023-11-06T01:49:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-06T01:49:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-06T01:49:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-06T01:49:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-06T01:49:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-06T01:49:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-06T01:49:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-06T01:49:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-06T01:49:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-06T01:49:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-06T01:49:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-06T01:49:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-06T01:49:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-06T01:49:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-06T01:49:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-06T01:49:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-06T01:49:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-06T01:49:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_PASSWORD": { 2023-11-06T01:49:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-06T01:49:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2023-11-06T01:49:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-06T01:49:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-06T01:49:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_URL": { 2023-11-06T01:49:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_URL", 2023-11-06T01:49:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_USERNAME": { 2023-11-06T01:49:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-06T01:49:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2023-11-06T01:49:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-06T01:49:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-06T01:49:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-06T01:49:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-06T01:49:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-06T01:49:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-06T01:49:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-06T01:49:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-06T01:49:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_HOST": { 2023-11-06T01:49:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-06T01:49:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.206 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.206 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.206 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.206 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.206 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.207 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.207 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.207 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.207 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.207 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-06T01:49:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-06T01:49:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-06T01:49:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-06T01:49:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-06T01:49:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-06T01:49:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-06T01:49:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-06T01:49:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-06T01:49:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-06T01:49:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-06T01:49:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-06T01:49:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-06T01:49:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-06T01:49:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-06T01:49:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-06T01:49:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_ENABLED": { 2023-11-06T01:49:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-06T01:49:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-06T01:49:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-06T01:49:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-06T01:49:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-06T01:49:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-06T01:49:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-06T01:49:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-06T01:49:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8", 2023-11-06T01:49:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-06T01:49:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-06T01:49:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "45", 2023-11-06T01:49:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ACCESS_KEY": { 2023-11-06T01:49:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-06T01:49:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-06T01:49:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-06T01:49:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2023-11-06T01:49:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SECRET_KEY": { 2023-11-06T01:49:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-06T01:49:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SSL_VERIFY": { 2023-11-06T01:49:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-06T01:49:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: "UNSAFE_TLS_v1_2": { 2023-11-06T01:49:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-06T01:49:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.259 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.259 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.259 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.259 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.259 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.259 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.260 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.260 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.260 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-06T01:49:31.260 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-06T01:49:31.261 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.261 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.261 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.261 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:31.261 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.262 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.262 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.262 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.262 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.262 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.263 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.263 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-06T01:49:31.263 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-06T01:49:31.263 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.264 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.264 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.264 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:31.264 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.264 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.265 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.265 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.265 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.265 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.265 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.266 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-06T01:49:31.266 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-06T01:49:31.266 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.266 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.267 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.267 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:31.267 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.267 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.267 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.268 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.268 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.268 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.268 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.269 INFO:teuthology.orchestra.run.smithi053.stdout: "cross_origin_url": { 2023-11-06T01:49:31.269 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cross_origin_url", 2023-11-06T01:49:31.269 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.269 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.269 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.270 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.270 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.270 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.270 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.270 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.271 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.271 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.271 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.271 INFO:teuthology.orchestra.run.smithi053.stdout: "crt_file": { 2023-11-06T01:49:31.271 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crt_file", 2023-11-06T01:49:31.272 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.272 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.272 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.272 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.272 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.273 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.273 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.273 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.273 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.273 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.274 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.274 INFO:teuthology.orchestra.run.smithi053.stdout: "debug": { 2023-11-06T01:49:31.274 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "debug", 2023-11-06T01:49:31.274 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.275 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.275 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.275 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.275 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.276 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.276 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable/disable debug options", 2023-11-06T01:49:31.276 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.276 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.276 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.277 INFO:teuthology.orchestra.run.smithi053.stdout: "jwt_token_ttl": { 2023-11-06T01:49:31.277 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "jwt_token_ttl", 2023-11-06T01:49:31.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.277 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.278 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.278 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "28800", 2023-11-06T01:49:31.278 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.278 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.278 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.279 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.279 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.279 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.279 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.280 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2023-11-06T01:49:31.280 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2023-11-06T01:49:31.280 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.280 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.280 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.280 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.281 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.281 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.281 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.281 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.281 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.281 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.282 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.282 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.282 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.283 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.283 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.283 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.283 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.284 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.284 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.284 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.284 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.284 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.284 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.285 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.285 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.285 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.285 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.286 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.286 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.286 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.286 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.287 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.287 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.288 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.288 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.288 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.288 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.288 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.289 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.289 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.289 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.289 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.290 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.290 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.290 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.291 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.291 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.292 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.292 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.292 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.292 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.293 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.293 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.293 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.293 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.294 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.294 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.294 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.294 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.295 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.295 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.295 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.295 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.296 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.296 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.296 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.296 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.297 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.297 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.297 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.297 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.297 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.298 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.298 INFO:teuthology.orchestra.run.smithi053.stdout: "motd": { 2023-11-06T01:49:31.298 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "motd", 2023-11-06T01:49:31.298 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.299 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.299 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.299 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.299 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.299 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.300 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.300 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "The message of the day", 2023-11-06T01:49:31.300 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.300 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.300 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.301 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect_resolve_ip_addr": { 2023-11-06T01:49:31.301 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "redirect_resolve_ip_addr", 2023-11-06T01:49:31.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.301 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.301 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.302 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.302 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.302 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.302 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.302 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.303 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.303 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.303 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.303 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2023-11-06T01:49:31.304 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2023-11-06T01:49:31.304 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.304 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.304 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.304 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2023-11-06T01:49:31.305 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.305 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.305 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.305 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.305 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.306 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.306 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2023-11-06T01:49:31.306 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2023-11-06T01:49:31.307 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.307 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.307 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.307 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8080", 2023-11-06T01:49:31.307 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.307 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.308 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.308 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.308 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.308 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.308 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.309 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2023-11-06T01:49:31.309 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2023-11-06T01:49:31.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.310 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.310 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.310 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.310 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.311 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.311 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.311 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.311 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.311 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.312 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl_server_port": { 2023-11-06T01:49:31.312 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl_server_port", 2023-11-06T01:49:31.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.312 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.312 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.313 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8443", 2023-11-06T01:49:31.313 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.313 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.313 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.313 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.314 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.314 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.314 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.314 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2023-11-06T01:49:31.315 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2023-11-06T01:49:31.315 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.315 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.315 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.315 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "redirect", 2023-11-06T01:49:31.315 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.316 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.316 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.316 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.316 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect" 2023-11-06T01:49:31.317 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.317 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.317 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.317 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.318 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.318 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2023-11-06T01:49:31.318 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2023-11-06T01:49:31.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.319 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.319 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2023-11-06T01:49:31.320 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2023-11-06T01:49:31.320 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2023-11-06T01:49:31.320 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.320 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.320 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.321 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.321 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.321 INFO:teuthology.orchestra.run.smithi053.stdout: "url_prefix": { 2023-11-06T01:49:31.322 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url_prefix", 2023-11-06T01:49:31.322 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.322 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.322 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.323 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.323 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.323 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.323 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.324 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.324 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.324 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.324 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.325 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "devicehealth", 2023-11-06T01:49:31.326 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.326 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.326 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.326 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_monitoring": { 2023-11-06T01:49:31.327 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_monitoring", 2023-11-06T01:49:31.327 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.327 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.327 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.328 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.328 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.328 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.328 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.329 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "monitor device health metrics", 2023-11-06T01:49:31.329 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.329 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.329 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.330 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.330 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.330 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.330 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.330 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.331 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.331 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.331 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.331 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.331 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.332 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.332 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.332 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.332 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.332 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.333 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.333 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.333 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.333 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.334 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.334 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.334 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.334 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.335 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.335 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.335 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.335 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.335 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.336 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.336 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.336 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.336 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.336 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.337 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.337 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.337 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.337 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.338 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.338 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.338 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.338 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.338 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.339 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.339 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.339 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.339 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.339 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.339 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.340 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.340 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.340 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.340 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.340 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.341 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.341 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.341 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.341 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.341 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.342 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.342 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.342 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.342 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.343 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.343 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.343 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.343 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.344 INFO:teuthology.orchestra.run.smithi053.stdout: "mark_out_threshold": { 2023-11-06T01:49:31.344 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mark_out_threshold", 2023-11-06T01:49:31.344 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.344 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.344 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.344 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2419200", 2023-11-06T01:49:31.345 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.345 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.345 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.345 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-06T01:49:31.345 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.346 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.346 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.346 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": { 2023-11-06T01:49:31.346 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_name", 2023-11-06T01:49:31.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.347 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.347 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "device_health_metrics", 2023-11-06T01:49:31.347 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.347 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.348 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.348 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-06T01:49:31.348 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.348 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.348 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.349 INFO:teuthology.orchestra.run.smithi053.stdout: "retention_period": { 2023-11-06T01:49:31.349 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retention_period", 2023-11-06T01:49:31.349 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.349 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.349 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.350 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15552000", 2023-11-06T01:49:31.350 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.350 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.350 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.351 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain device health metrics", 2023-11-06T01:49:31.351 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.351 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.351 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.352 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_frequency": { 2023-11-06T01:49:31.352 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_frequency", 2023-11-06T01:49:31.352 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.352 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.352 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.353 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2023-11-06T01:49:31.353 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.353 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.353 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.353 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-06T01:49:31.354 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.354 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.354 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.354 INFO:teuthology.orchestra.run.smithi053.stdout: "self_heal": { 2023-11-06T01:49:31.355 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "self_heal", 2023-11-06T01:49:31.355 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.355 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.355 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.355 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.356 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.356 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.356 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.356 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-06T01:49:31.357 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.357 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.357 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:31.358 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:31.358 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.358 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.358 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.358 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:31.359 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.359 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.359 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.360 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and check device health", 2023-11-06T01:49:31.360 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.360 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.360 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.361 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_threshold": { 2023-11-06T01:49:31.361 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_threshold", 2023-11-06T01:49:31.361 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.361 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.362 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7257600", 2023-11-06T01:49:31.362 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.362 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.363 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.363 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-06T01:49:31.363 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.363 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.364 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.365 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "diskprediction_local", 2023-11-06T01:49:31.365 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.365 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.366 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.366 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.366 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.366 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.366 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.367 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.367 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.367 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.367 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.367 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.368 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.368 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.368 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.368 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.368 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "predict_interval": { 2023-11-06T01:49:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predict_interval", 2023-11-06T01:49:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2023-11-06T01:49:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "predictor_model": { 2023-11-06T01:49:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predictor_model", 2023-11-06T01:49:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "prophetstor", 2023-11-06T01:49:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2023-11-06T01:49:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "influx", 2023-11-06T01:49:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": false, 2023-11-06T01:49:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "influxdb python module not found", 2023-11-06T01:49:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "batch_size": { 2023-11-06T01:49:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "batch_size", 2023-11-06T01:49:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5000", 2023-11-06T01:49:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-06T01:49:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "database": { 2023-11-06T01:49:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "database", 2023-11-06T01:49:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "ceph", 2023-11-06T01:49:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-06T01:49:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "hostname": { 2023-11-06T01:49:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "hostname", 2023-11-06T01:49:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server hostname", 2023-11-06T01:49:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "30", 2023-11-06T01:49:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "5", 2023-11-06T01:49:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-06T01:49:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "password": { 2023-11-06T01:49:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "password", 2023-11-06T01:49:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "password of InfluxDB server user", 2023-11-06T01:49:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "port": { 2023-11-06T01:49:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "port", 2023-11-06T01:49:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8086", 2023-11-06T01:49:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server port", 2023-11-06T01:49:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2023-11-06T01:49:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2023-11-06T01:49:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "false", 2023-11-06T01:49:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-06T01:49:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "threads": { 2023-11-06T01:49:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threads", 2023-11-06T01:49:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "32", 2023-11-06T01:49:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-06T01:49:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "username": { 2023-11-06T01:49:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "username", 2023-11-06T01:49:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "username of InfluxDB server user", 2023-11-06T01:49:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.430 INFO:teuthology.orchestra.run.smithi053.stdout: "verify_ssl": { 2023-11-06T01:49:31.430 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "verify_ssl", 2023-11-06T01:49:31.430 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.431 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.431 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.431 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "true", 2023-11-06T01:49:31.431 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.431 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.432 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.432 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-06T01:49:31.432 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.432 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.433 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.434 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "insights", 2023-11-06T01:49:31.434 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.434 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.435 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.435 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.435 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "iostat", 2023-11-06T01:49:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "k8sevents", 2023-11-06T01:49:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_event_retention_days": { 2023-11-06T01:49:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ceph_event_retention_days", 2023-11-06T01:49:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7", 2023-11-06T01:49:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-06T01:49:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "config_check_secs": { 2023-11-06T01:49:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_check_secs", 2023-11-06T01:49:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "10", 2023-11-06T01:49:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-06T01:49:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "localpool", 2023-11-06T01:49:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "failure_domain": { 2023-11-06T01:49:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "failure_domain", 2023-11-06T01:49:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "host", 2023-11-06T01:49:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "failure domain for any created local pool", 2023-11-06T01:49:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-06T01:49:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": { 2023-11-06T01:49:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_size", 2023-11-06T01:49:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default min_size for any created local pool", 2023-11-06T01:49:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-06T01:49:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "num_rep": { 2023-11-06T01:49:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "num_rep", 2023-11-06T01:49:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3", 2023-11-06T01:49:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default replica count for any created local pool", 2023-11-06T01:49:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": { 2023-11-06T01:49:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_num", 2023-11-06T01:49:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "128", 2023-11-06T01:49:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default pg_num for any created local pool", 2023-11-06T01:49:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "prefix": { 2023-11-06T01:49:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prefix", 2023-11-06T01:49:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name prefix for any created local pool", 2023-11-06T01:49:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "subtree": { 2023-11-06T01:49:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "subtree", 2023-11-06T01:49:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "rack", 2023-11-06T01:49:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-06T01:49:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-06T01:49:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mds_autoscaler", 2023-11-06T01:49:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.524 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.524 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.524 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.524 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.524 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.524 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.525 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.525 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.525 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.525 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.525 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.526 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.526 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.526 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.526 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.526 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.527 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.527 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.527 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.527 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.527 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.528 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.528 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.528 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.528 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.529 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.529 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.529 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.529 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.529 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.530 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.530 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.530 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.530 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.531 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.531 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.531 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.531 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.532 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.532 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.532 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.533 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.533 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.533 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.533 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.533 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.534 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.534 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.534 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.534 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.535 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.535 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.535 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.536 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.536 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.536 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.537 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.537 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.537 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.537 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.538 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.538 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.538 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.538 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.539 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.539 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.540 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirroring", 2023-11-06T01:49:31.540 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.540 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.540 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.540 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.541 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.541 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.541 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.541 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.542 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.542 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.542 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.543 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.543 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.543 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.543 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.543 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.544 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.544 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.544 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.544 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.544 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.545 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.545 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.545 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.545 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.546 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.546 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.546 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.546 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.547 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.547 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.547 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.547 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.547 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.547 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.548 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.548 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.548 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.548 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.549 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.549 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.549 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.549 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.549 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.550 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.550 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.550 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.550 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.550 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.551 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.551 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.551 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.551 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.551 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.551 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.552 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.552 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.552 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.552 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.553 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.553 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.553 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.553 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.553 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.553 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.554 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.554 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.554 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.554 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.554 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.555 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "nfs", 2023-11-06T01:49:31.555 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.556 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.556 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.556 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.556 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.556 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.557 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.557 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.557 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.557 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.557 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.557 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.558 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.558 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.558 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.558 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.558 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.559 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.559 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.559 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.559 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.559 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.559 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.560 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.560 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.560 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.560 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.561 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.561 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.561 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.561 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.561 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.561 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.562 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.562 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.562 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.562 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.563 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.563 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.563 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.563 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.563 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.564 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.564 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.564 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.564 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.564 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.564 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.565 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.565 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.565 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.565 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.566 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.566 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.566 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.566 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.567 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.567 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.567 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.567 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.568 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.568 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.568 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.568 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.569 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.569 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.569 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.569 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.570 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2023-11-06T01:49:31.571 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.571 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.571 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.571 INFO:teuthology.orchestra.run.smithi053.stdout: "fail_fs": { 2023-11-06T01:49:31.572 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "fail_fs", 2023-11-06T01:49:31.572 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.572 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.572 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.572 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.573 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.573 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.573 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.573 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-06T01:49:31.574 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.574 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.574 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.575 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.575 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.575 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.575 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.576 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.576 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.576 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.576 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.576 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.577 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.577 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.577 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.577 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.578 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.578 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.578 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.579 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.579 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.579 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.580 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.580 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.580 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.580 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.581 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.581 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.581 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.581 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.581 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.582 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.582 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.582 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.582 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.583 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.583 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.583 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.584 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.584 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.584 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.584 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.584 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.585 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.585 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.585 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.585 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.585 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.585 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.586 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.586 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.586 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.586 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.587 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.587 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.587 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.587 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.587 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.587 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.588 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.588 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.588 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.588 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.588 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.589 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator": { 2023-11-06T01:49:31.589 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2023-11-06T01:49:31.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.589 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.590 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.590 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.590 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.590 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.590 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm", 2023-11-06T01:49:31.591 INFO:teuthology.orchestra.run.smithi053.stdout: "rook", 2023-11-06T01:49:31.591 INFO:teuthology.orchestra.run.smithi053.stdout: "test_orchestrator" 2023-11-06T01:49:31.591 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.591 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Orchestrator backend", 2023-11-06T01:49:31.591 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.592 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.592 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.593 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_perf_query", 2023-11-06T01:49:31.593 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.593 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.593 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.593 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.594 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.594 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.594 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.594 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.594 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.595 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.595 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.595 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.595 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.595 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.596 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.596 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.596 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.596 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.596 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.597 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.597 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.597 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.597 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.598 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.598 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.598 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.598 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.598 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.598 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.599 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.599 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.599 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.599 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.599 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.600 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.600 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.600 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.600 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.601 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.601 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.601 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.601 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.602 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.602 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.602 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.602 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.602 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.603 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.603 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.603 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.603 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.603 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.604 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.604 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.604 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.605 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.605 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.605 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.605 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.605 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.606 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.606 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.606 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.606 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.607 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.607 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.607 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.608 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_support", 2023-11-06T01:49:31.609 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.609 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.609 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.609 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.609 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.610 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.610 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.610 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.610 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.611 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.611 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.611 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.611 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.612 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.612 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.612 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.612 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.613 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.613 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.613 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.613 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.614 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.614 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.614 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.614 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.615 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.615 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.615 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.615 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.615 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.616 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.616 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.616 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.616 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.616 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.617 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.617 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.617 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.617 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.618 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.618 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.618 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.618 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.618 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.619 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.619 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.619 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.619 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.619 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.620 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.620 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.620 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.620 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.620 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.620 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.621 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.621 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.621 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.621 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.622 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.622 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.622 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.622 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.622 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.623 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.623 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.623 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.623 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.624 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_autoscaler", 2023-11-06T01:49:31.624 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.624 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.625 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.625 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.625 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.625 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.625 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.626 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.626 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.626 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.626 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.626 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.627 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.627 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.627 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.627 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.627 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.627 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.628 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.628 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.628 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.628 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.628 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.629 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.629 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.629 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.629 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.630 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.630 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.630 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.630 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.630 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.631 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.631 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.631 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.631 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.631 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.632 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.632 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.633 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.633 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.633 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.633 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.633 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.633 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.634 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.634 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.634 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.634 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.634 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.635 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.635 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.635 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.635 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.636 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.636 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.636 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.636 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.637 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.637 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.637 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.637 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.638 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.638 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.638 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.638 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.638 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.639 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:31.639 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:31.639 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.639 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.640 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.640 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:31.640 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.640 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.641 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.641 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.641 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.642 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.642 INFO:teuthology.orchestra.run.smithi053.stdout: "threshold": { 2023-11-06T01:49:31.642 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threshold", 2023-11-06T01:49:31.643 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:31.643 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.643 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.643 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2023-11-06T01:49:31.643 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1.0", 2023-11-06T01:49:31.644 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.644 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.644 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "scaling threshold", 2023-11-06T01:49:31.644 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-06T01:49:31.645 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.645 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.646 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "progress", 2023-11-06T01:49:31.646 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.647 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.647 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.647 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_pg_recovery_event": { 2023-11-06T01:49:31.647 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_pg_recovery_event", 2023-11-06T01:49:31.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.647 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.648 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.648 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.648 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.648 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.648 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.649 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-06T01:49:31.649 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.649 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.649 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.649 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2023-11-06T01:49:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2023-11-06T01:49:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "max_completed_events": { 2023-11-06T01:49:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_completed_events", 2023-11-06T01:49:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "50", 2023-11-06T01:49:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "number of past completed events to remember", 2023-11-06T01:49:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2023-11-06T01:49:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2023-11-06T01:49:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2023-11-06T01:49:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long the module is going to sleep", 2023-11-06T01:49:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus", 2023-11-06T01:49:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "cache": { 2023-11-06T01:49:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cache", 2023-11-06T01:49:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "exclude_perf_counters": { 2023-11-06T01:49:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "exclude_perf_counters", 2023-11-06T01:49:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-06T01:49:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-06T01:49:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools": { 2023-11-06T01:49:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools", 2023-11-06T01:49:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-06T01:49:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-06T01:49:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "300", 2023-11-06T01:49:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_interval": { 2023-11-06T01:49:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_interval", 2023-11-06T01:49:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15.0", 2023-11-06T01:49:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2023-11-06T01:49:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2023-11-06T01:49:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2023-11-06T01:49:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-06T01:49:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2023-11-06T01:49:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2023-11-06T01:49:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "9283", 2023-11-06T01:49:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-06T01:49:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: "stale_cache_strategy": { 2023-11-06T01:49:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stale_cache_strategy", 2023-11-06T01:49:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "log", 2023-11-06T01:49:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2023-11-06T01:49:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2023-11-06T01:49:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "default", 2023-11-06T01:49:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: "default", 2023-11-06T01:49:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: "error" 2023-11-06T01:49:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2023-11-06T01:49:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2023-11-06T01:49:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2023-11-06T01:49:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2023-11-06T01:49:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2023-11-06T01:49:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_support", 2023-11-06T01:49:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.730 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.730 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.730 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.730 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.730 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.734 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.734 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.734 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.734 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.736 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.736 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.736 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.736 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.736 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_snap_create": { 2023-11-06T01:49:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_snap_create", 2023-11-06T01:49:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2023-11-06T01:49:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: "mirror_snapshot_schedule": { 2023-11-06T01:49:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirror_snapshot_schedule", 2023-11-06T01:49:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: "trash_purge_schedule": { 2023-11-06T01:49:31.743 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "trash_purge_schedule", 2023-11-06T01:49:31.743 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.743 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.743 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.745 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.745 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.746 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "restful", 2023-11-06T01:49:31.746 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.746 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.747 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.747 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_auth": { 2023-11-06T01:49:31.747 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_auth", 2023-11-06T01:49:31.747 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.749 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.749 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.749 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.749 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2023-11-06T01:49:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2023-11-06T01:49:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.751 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.751 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.753 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.753 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.753 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.754 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.754 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.754 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.754 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2023-11-06T01:49:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2023-11-06T01:49:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2023-11-06T01:49:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2023-11-06T01:49:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rgw", 2023-11-06T01:49:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.778 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.778 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.779 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.779 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.779 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.779 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.780 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.780 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.780 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.780 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.780 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.781 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.781 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.781 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.782 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.782 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.782 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.782 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.782 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.783 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.783 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.783 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.783 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.784 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.784 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.784 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.784 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.784 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.785 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.785 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.785 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.786 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.786 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.786 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.786 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.787 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.787 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.787 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.787 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.787 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.788 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.788 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.788 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.789 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.789 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.790 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rook", 2023-11-06T01:49:31.790 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.790 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.791 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.791 INFO:teuthology.orchestra.run.smithi053.stdout: "drive_group_interval": { 2023-11-06T01:49:31.791 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "drive_group_interval", 2023-11-06T01:49:31.791 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:31.791 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.792 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "300.0", 2023-11-06T01:49:31.792 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.792 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.792 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.793 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-06T01:49:31.793 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.793 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.794 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.794 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.794 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.794 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.795 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.795 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.795 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.795 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.795 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.796 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.796 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.796 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.796 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.796 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.797 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.797 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.797 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.797 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.797 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.798 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.798 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.798 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.799 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.799 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.799 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.799 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.799 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.800 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.800 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.800 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.800 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.801 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.801 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.801 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.801 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.801 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.802 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.802 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.802 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.802 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.802 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.803 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.803 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.803 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.803 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.803 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.804 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.804 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.804 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.805 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.805 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.805 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.806 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.806 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.806 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.806 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.806 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.807 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.807 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.807 INFO:teuthology.orchestra.run.smithi053.stdout: "storage_class": { 2023-11-06T01:49:31.807 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "storage_class", 2023-11-06T01:49:31.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.808 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.808 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.808 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "local", 2023-11-06T01:49:31.808 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.808 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.809 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.809 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-06T01:49:31.809 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.809 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.809 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.810 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "selftest", 2023-11-06T01:49:31.811 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.811 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.811 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.811 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.811 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.812 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.812 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.813 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.813 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.813 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.813 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.813 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.814 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.814 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.814 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.814 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.814 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.814 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.815 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.815 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.815 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.815 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.816 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.816 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.816 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.816 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.816 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.816 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.817 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.817 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.817 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.817 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.818 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.818 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.818 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.819 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.819 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.819 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.820 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.820 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.820 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.820 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.821 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.821 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.821 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.821 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.821 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.822 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.822 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.822 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.822 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.823 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.823 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.823 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.824 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.824 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.824 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.825 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.825 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.825 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.825 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.826 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.826 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.826 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.827 INFO:teuthology.orchestra.run.smithi053.stdout: "roption1": { 2023-11-06T01:49:31.827 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption1", 2023-11-06T01:49:31.827 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.827 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.827 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.828 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.828 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.828 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.828 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.828 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.829 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.829 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.829 INFO:teuthology.orchestra.run.smithi053.stdout: "roption2": { 2023-11-06T01:49:31.830 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption2", 2023-11-06T01:49:31.830 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.830 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.830 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.831 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "xyz", 2023-11-06T01:49:31.831 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.831 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.831 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.831 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.832 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.832 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.832 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.832 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption1": { 2023-11-06T01:49:31.832 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption1", 2023-11-06T01:49:31.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.833 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.833 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.833 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.834 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.834 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.834 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.834 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.834 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption2": { 2023-11-06T01:49:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption2", 2023-11-06T01:49:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption3": { 2023-11-06T01:49:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption3", 2023-11-06T01:49:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2023-11-06T01:49:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption4": { 2023-11-06T01:49:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption4", 2023-11-06T01:49:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption5": { 2023-11-06T01:49:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption5", 2023-11-06T01:49:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption6": { 2023-11-06T01:49:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption6", 2023-11-06T01:49:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption7": { 2023-11-06T01:49:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption7", 2023-11-06T01:49:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2023-11-06T01:49:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "42", 2023-11-06T01:49:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: "testkey": { 2023-11-06T01:49:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testkey", 2023-11-06T01:49:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: "testlkey": { 2023-11-06T01:49:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testlkey", 2023-11-06T01:49:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: "testnewline": { 2023-11-06T01:49:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testnewline", 2023-11-06T01:49:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.862 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snap_schedule", 2023-11-06T01:49:31.862 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.862 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.863 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.863 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_m_granularity": { 2023-11-06T01:49:31.863 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_m_granularity", 2023-11-06T01:49:31.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.863 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.863 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow minute scheduled snapshots", 2023-11-06T01:49:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "dump_on_update": { 2023-11-06T01:49:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dump_on_update", 2023-11-06T01:49:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "dump database to debug log on update", 2023-11-06T01:49:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stats", 2023-11-06T01:49:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "status", 2023-11-06T01:49:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telegraf", 2023-11-06T01:49:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "address": { 2023-11-06T01:49:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "address", 2023-11-06T01:49:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-06T01:49:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15", 2023-11-06T01:49:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telemetry", 2023-11-06T01:49:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_basic": { 2023-11-06T01:49:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_basic", 2023-11-06T01:49:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-06T01:49:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_crash": { 2023-11-06T01:49:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_crash", 2023-11-06T01:49:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-06T01:49:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_device": { 2023-11-06T01:49:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_device", 2023-11-06T01:49:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2023-11-06T01:49:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-06T01:49:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_ident": { 2023-11-06T01:49:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_ident", 2023-11-06T01:49:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-06T01:49:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_perf": { 2023-11-06T01:49:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_perf", 2023-11-06T01:49:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-06T01:49:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.957 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.957 INFO:teuthology.orchestra.run.smithi053.stdout: "contact": { 2023-11-06T01:49:31.958 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "contact", 2023-11-06T01:49:31.958 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.958 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.958 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.958 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.959 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.959 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.959 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.959 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.960 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.960 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.960 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.960 INFO:teuthology.orchestra.run.smithi053.stdout: "description": { 2023-11-06T01:49:31.961 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "description", 2023-11-06T01:49:31.961 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.961 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.961 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.962 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.962 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.962 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.962 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.962 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "device_url": { 2023-11-06T01:49:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_url", 2023-11-06T01:49:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-06T01:49:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2023-11-06T01:49:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2023-11-06T01:49:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "24", 2023-11-06T01:49:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "8", 2023-11-06T01:49:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "last_opt_revision": { 2023-11-06T01:49:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "last_opt_revision", 2023-11-06T01:49:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1", 2023-11-06T01:49:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard": { 2023-11-06T01:49:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard", 2023-11-06T01:49:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard_description": { 2023-11-06T01:49:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard_description", 2023-11-06T01:49:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: "organization": { 2023-11-06T01:49:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "organization", 2023-11-06T01:49:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "proxy": { 2023-11-06T01:49:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "proxy", 2023-11-06T01:49:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "url": { 2023-11-06T01:49:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url", 2023-11-06T01:49:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-06T01:49:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "test_orchestrator", 2023-11-06T01:49:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:32.007 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:32.007 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:32.007 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:32.007 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:32.007 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.008 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.008 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:32.008 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.008 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.008 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:32.009 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:32.009 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:32.009 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:32.009 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:32.009 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:32.010 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:32.010 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:32.010 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.010 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.010 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.011 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.011 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:32.011 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:32.011 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:32.012 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.012 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.012 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:32.012 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.012 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.012 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.013 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.013 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.013 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.013 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.014 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:32.014 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:32.014 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:32.014 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.014 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.015 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:32.015 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.015 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.015 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:32.015 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:32.015 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:32.016 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:32.016 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:32.016 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:32.016 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:32.016 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:32.017 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.017 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.017 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.017 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.018 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:32.018 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:32.018 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:32.018 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.018 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.019 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:32.019 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.019 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.019 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.019 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.020 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.020 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.020 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.020 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.020 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:32.021 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "volumes", 2023-11-06T01:49:32.021 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:32.021 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:32.021 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:32.022 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:32.022 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:32.022 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:32.022 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.022 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.023 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:32.023 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.023 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.023 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:32.023 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:32.023 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:32.024 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:32.024 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:32.024 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:32.024 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:32.024 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:32.025 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.025 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.025 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.025 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.026 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:32.026 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:32.026 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:32.026 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.026 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.026 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:32.027 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.027 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.027 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.027 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.027 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.028 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.028 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.029 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:32.029 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:32.029 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:32.029 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.029 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.029 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:32.030 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.030 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.030 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:32.031 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:32.031 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:32.031 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:32.031 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:32.031 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:32.031 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:32.032 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:32.032 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.032 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.032 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.033 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.033 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:32.034 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:32.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:32.034 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.034 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.035 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:32.035 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.035 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.035 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.036 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.036 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.036 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.036 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.037 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_clones": { 2023-11-06T01:49:32.037 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_clones", 2023-11-06T01:49:32.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:32.037 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.038 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:32.038 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4", 2023-11-06T01:49:32.038 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.038 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.038 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.038 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-06T01:49:32.039 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.039 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.039 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.039 INFO:teuthology.orchestra.run.smithi053.stdout: "snapshot_clone_delay": { 2023-11-06T01:49:32.040 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snapshot_clone_delay", 2023-11-06T01:49:32.040 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:32.040 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.041 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:32.041 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2023-11-06T01:49:32.041 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.041 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.042 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.042 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-06T01:49:32.042 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.042 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.043 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.043 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.043 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:32.044 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix", 2023-11-06T01:49:32.044 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2023-11-06T01:49:32.044 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2023-11-06T01:49:32.044 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2023-11-06T01:49:32.045 INFO:teuthology.orchestra.run.smithi053.stdout: "discovery_interval": { 2023-11-06T01:49:32.045 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "discovery_interval", 2023-11-06T01:49:32.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2023-11-06T01:49:32.045 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.045 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:32.046 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "100", 2023-11-06T01:49:32.046 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.046 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.046 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.046 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.047 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.047 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.047 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.047 INFO:teuthology.orchestra.run.smithi053.stdout: "identifier": { 2023-11-06T01:49:32.047 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "identifier", 2023-11-06T01:49:32.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:32.048 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.048 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:32.048 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:32.048 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.049 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.049 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.049 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.049 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.049 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.049 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.050 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2023-11-06T01:49:32.050 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2023-11-06T01:49:32.050 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2023-11-06T01:49:32.051 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.051 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:32.051 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2023-11-06T01:49:32.051 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.051 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.052 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.052 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.052 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.052 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.052 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.053 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2023-11-06T01:49:32.053 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2023-11-06T01:49:32.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:32.053 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.053 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.054 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:32.054 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.054 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.054 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:32.054 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:32.055 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:32.055 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:32.055 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:32.055 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:32.056 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:32.056 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:32.056 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.056 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.056 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.056 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.057 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2023-11-06T01:49:32.057 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2023-11-06T01:49:32.057 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:32.057 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.058 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.058 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:32.058 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.058 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.058 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.059 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.059 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.059 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.059 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.059 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2023-11-06T01:49:32.060 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2023-11-06T01:49:32.060 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:32.060 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.060 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.060 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2023-11-06T01:49:32.061 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.061 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.061 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2023-11-06T01:49:32.061 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2023-11-06T01:49:32.061 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2023-11-06T01:49:32.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2023-11-06T01:49:32.062 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2023-11-06T01:49:32.062 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2023-11-06T01:49:32.062 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2023-11-06T01:49:32.062 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:32.062 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.063 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.063 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.063 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.063 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2023-11-06T01:49:32.064 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2023-11-06T01:49:32.064 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2023-11-06T01:49:32.064 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.064 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2023-11-06T01:49:32.064 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2023-11-06T01:49:32.065 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.065 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.065 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.065 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.066 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.066 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.066 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.066 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_host": { 2023-11-06T01:49:32.067 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_host", 2023-11-06T01:49:32.067 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:32.067 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.067 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:32.068 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2023-11-06T01:49:32.068 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.068 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.068 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.068 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.069 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.069 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.069 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.070 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_port": { 2023-11-06T01:49:32.070 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_port", 2023-11-06T01:49:32.070 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2023-11-06T01:49:32.070 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.070 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:32.071 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10051", 2023-11-06T01:49:32.071 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.071 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_sender": { 2023-11-06T01:49:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_sender", 2023-11-06T01:49:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2023-11-06T01:49:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2023-11-06T01:49:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2023-11-06T01:49:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-06T01:49:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2023-11-06T01:49:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2023-11-06T01:49:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2023-11-06T01:49:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2023-11-06T01:49:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2023-11-06T01:49:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2023-11-06T01:49:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2023-11-06T01:49:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "services": { 2023-11-06T01:49:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "dashboard": "https://172.21.15.53:8443/" 2023-11-06T01:49:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: "always_on_modules": { 2023-11-06T01:49:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: "octopus": [ 2023-11-06T01:49:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: "balancer", 2023-11-06T01:49:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "crash", 2023-11-06T01:49:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "devicehealth", 2023-11-06T01:49:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator", 2023-11-06T01:49:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscaler", 2023-11-06T01:49:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "progress", 2023-11-06T01:49:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_support", 2023-11-06T01:49:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "status", 2023-11-06T01:49:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "telemetry", 2023-11-06T01:49:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "volumes" 2023-11-06T01:49:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "pacific": [ 2023-11-06T01:49:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "balancer", 2023-11-06T01:49:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "crash", 2023-11-06T01:49:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "devicehealth", 2023-11-06T01:49:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator", 2023-11-06T01:49:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscaler", 2023-11-06T01:49:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "progress", 2023-11-06T01:49:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_support", 2023-11-06T01:49:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "status", 2023-11-06T01:49:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "telemetry", 2023-11-06T01:49:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "volumes" 2023-11-06T01:49:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "quincy": [ 2023-11-06T01:49:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "balancer", 2023-11-06T01:49:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "crash", 2023-11-06T01:49:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "devicehealth", 2023-11-06T01:49:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator", 2023-11-06T01:49:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscaler", 2023-11-06T01:49:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "progress", 2023-11-06T01:49:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_support", 2023-11-06T01:49:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "status", 2023-11-06T01:49:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "telemetry", 2023-11-06T01:49:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "volumes" 2023-11-06T01:49:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2023-11-06T01:49:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "reef": [ 2023-11-06T01:49:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "balancer", 2023-11-06T01:49:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "crash", 2023-11-06T01:49:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "devicehealth", 2023-11-06T01:49:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator", 2023-11-06T01:49:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscaler", 2023-11-06T01:49:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "progress", 2023-11-06T01:49:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_support", 2023-11-06T01:49:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: "status", 2023-11-06T01:49:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: "telemetry", 2023-11-06T01:49:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: "volumes" 2023-11-06T01:49:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-06T01:49:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "last_failure_osd_epoch": 3, 2023-11-06T01:49:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "active_clients": [ 2023-11-06T01:49:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "devicehealth", 2023-11-06T01:49:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2023-11-06T01:49:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2023-11-06T01:49:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2023-11-06T01:49:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4069330909 2023-11-06T01:49:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-06T01:49:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "libcephsqlite", 2023-11-06T01:49:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2023-11-06T01:49:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2023-11-06T01:49:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2023-11-06T01:49:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1030194757 2023-11-06T01:49:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-06T01:49:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_support", 2023-11-06T01:49:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2023-11-06T01:49:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2023-11-06T01:49:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2023-11-06T01:49:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 439783877 2023-11-06T01:49:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-06T01:49:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "volumes", 2023-11-06T01:49:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2023-11-06T01:49:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-11-06T01:49:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2023-11-06T01:49:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2023-11-06T01:49:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2297095197 2023-11-06T01:49:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-06T01:49:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-06T01:49:32.099 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:49:32.101 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:28 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/4015473385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:49:32.102 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:28 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3821951875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:49:32.102 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:28 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1989121313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:49:32.102 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:29 smithi062 ceph-mon[103924]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 1.61k op/s 2023-11-06T01:49:32.102 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:29 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1915206644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-06T01:49:32.102 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:31 smithi062 ceph-mon[103924]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.78k op/s 2023-11-06T01:49:32.103 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:28 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/4015473385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:49:32.103 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:28 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3821951875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:49:32.103 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:28 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1989121313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:49:32.104 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:29 smithi163 ceph-mon[104416]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 1.61k op/s 2023-11-06T01:49:32.104 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:29 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1915206644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-06T01:49:32.104 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:31 smithi163 ceph-mon[104416]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.78k op/s 2023-11-06T01:49:32.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/1915206644 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6c400a0570 msgr2=0x7f6c400a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:32.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6c400a0570 0x7f6c400a2a30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400b580 tx=0x7f6c44005fb0 comp rx=0 tx=0).stop 2023-11-06T01:49:32.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/1915206644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c540ffc70 msgr2=0x7f6c54195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:49:32.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c540ffc70 0x7f6c54195910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000e970 tx=0x7f6c5000ec80 comp rx=0 tx=0).stop 2023-11-06T01:49:32.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/1915206644 shutdown_connections 2023-11-06T01:49:32.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f6c400a0570 0x7f6c400a2a30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:32.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54101800 0x7f6c5419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:32.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c540ffc70 0x7f6c54195910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:32.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 --2- 172.21.15.53:0/1915206644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54100e60 0x7f6c54195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:49:32.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/1915206644 >> 172.21.15.53:0/1915206644 conn(0x7f6c540fb1c0 msgr2=0x7f6c540fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:49:32.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/1915206644 shutdown_connections 2023-11-06T01:49:32.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:49:28.471+0000 7f6c5b8b3700 1 -- 172.21.15.53:0/1915206644 wait complete. 2023-11-06T01:49:32.115 DEBUG:tasks.check_counter:Getting stats from a 2023-11-06T01:49:32.115 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-11-06T01:49:32.116 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:29 smithi053 ceph-mon[99472]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 1.61k op/s 2023-11-06T01:49:32.116 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:29 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1915206644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-06T01:49:32.117 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:31 smithi053 ceph-mon[99472]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.78k op/s 2023-11-06T01:49:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:49:32.841 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:32 smithi053 ceph-mon[99472]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 1.68k op/s 2023-11-06T01:49:32.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:32 smithi062 ceph-mon[103924]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 1.68k op/s 2023-11-06T01:49:32.986 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:32 smithi163 ceph-mon[104416]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 1.68k op/s 2023-11-06T01:49:35.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:35 smithi062 ceph-mon[103924]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2023-11-06T01:49:35.400 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:35 smithi053 ceph-mon[99472]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2023-11-06T01:49:35.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:35 smithi163 ceph-mon[104416]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2023-11-06T01:49:35.979 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:49:35.979 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:49:35.979 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1845, 2023-11-06T01:49:35.979 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1845, 2023-11-06T01:49:35.980 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 4752047, 2023-11-06T01:49:35.980 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 573000, 2023-11-06T01:49:35.980 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2023-11-06T01:49:35.980 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2023-11-06T01:49:35.980 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.171283415, 2023-11-06T01:49:35.980 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.045323098, 2023-11-06T01:49:35.981 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.048191573, 2023-11-06T01:49:35.981 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.062775548, 2023-11-06T01:49:35.981 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:35.981 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1844, 2023-11-06T01:49:35.981 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.291602609, 2023-11-06T01:49:35.982 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000158135 2023-11-06T01:49:35.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.982 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:35.982 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:35.982 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:35.983 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:35.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.983 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-11-06T01:49:35.983 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2023-11-06T01:49:35.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.984 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:49:35.984 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1324, 2023-11-06T01:49:35.984 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1299, 2023-11-06T01:49:35.984 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 2441490, 2023-11-06T01:49:35.984 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 475349, 2023-11-06T01:49:35.984 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2023-11-06T01:49:35.985 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2023-11-06T01:49:35.985 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.129413509, 2023-11-06T01:49:35.985 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.037563384, 2023-11-06T01:49:35.985 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.040525475, 2023-11-06T01:49:35.985 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.043884979, 2023-11-06T01:49:35.986 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:35.986 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1298, 2023-11-06T01:49:35.986 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.333016942, 2023-11-06T01:49:35.986 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000256561 2023-11-06T01:49:35.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.987 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:35.987 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:35.987 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:35.987 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:35.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.987 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 635440, 2023-11-06T01:49:35.988 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 159616 2023-11-06T01:49:35.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.988 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:49:35.988 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 360, 2023-11-06T01:49:35.988 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 570, 2023-11-06T01:49:35.989 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 88410, 2023-11-06T01:49:35.989 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 640042, 2023-11-06T01:49:35.989 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 7, 2023-11-06T01:49:35.989 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2023-11-06T01:49:35.989 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.050497004, 2023-11-06T01:49:35.990 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.022037634, 2023-11-06T01:49:35.990 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.010006370, 2023-11-06T01:49:35.990 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.013975890, 2023-11-06T01:49:35.990 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:35.990 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 568, 2023-11-06T01:49:35.991 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.177300750, 2023-11-06T01:49:35.991 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000312149 2023-11-06T01:49:35.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.991 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:35.991 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:35.991 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:35.992 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:35.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.992 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-06T01:49:35.992 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 528704 2023-11-06T01:49:35.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.993 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2023-11-06T01:49:35.993 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2023-11-06T01:49:35.993 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2023-11-06T01:49:35.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.993 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2023-11-06T01:49:35.993 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2023-11-06T01:49:35.994 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2023-11-06T01:49:35.994 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5377, 2023-11-06T01:49:35.994 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.149645017, 2023-11-06T01:49:35.994 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000027830 2023-11-06T01:49:35.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:35.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.995 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2023-11-06T01:49:35.995 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2023-11-06T01:49:35.995 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2023-11-06T01:49:35.995 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:35.996 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:35.996 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:35.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:35.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.996 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2023-11-06T01:49:35.996 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2023-11-06T01:49:35.997 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2023-11-06T01:49:35.997 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2023-11-06T01:49:35.997 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:35.997 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:35.997 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:35.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:35.998 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2023-11-06T01:49:35.998 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2023-11-06T01:49:35.998 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2023-11-06T01:49:35.998 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2023-11-06T01:49:35.999 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2023-11-06T01:49:35.999 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2023-11-06T01:49:35.999 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2023-11-06T01:49:35.999 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 903, 2023-11-06T01:49:35.999 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 0, 2023-11-06T01:49:35.999 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 798, 2023-11-06T01:49:36.000 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 105, 2023-11-06T01:49:36.000 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 902, 2023-11-06T01:49:36.000 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2023-11-06T01:49:36.000 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2023-11-06T01:49:36.000 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2023-11-06T01:49:36.001 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 3, 2023-11-06T01:49:36.001 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2023-11-06T01:49:36.001 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2023-11-06T01:49:36.001 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2023-11-06T01:49:36.001 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2023-11-06T01:49:36.002 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2023-11-06T01:49:36.002 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2023-11-06T01:49:36.002 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2023-11-06T01:49:36.002 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2023-11-06T01:49:36.002 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2023-11-06T01:49:36.002 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2023-11-06T01:49:36.003 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2023-11-06T01:49:36.003 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2023-11-06T01:49:36.003 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2023-11-06T01:49:36.003 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2023-11-06T01:49:36.003 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2023-11-06T01:49:36.004 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2023-11-06T01:49:36.004 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 0, 2023-11-06T01:49:36.004 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 0, 2023-11-06T01:49:36.004 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2023-11-06T01:49:36.004 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2023-11-06T01:49:36.004 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2023-11-06T01:49:36.005 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T01:49:36.005 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2023-11-06T01:49:36.005 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2023-11-06T01:49:36.005 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2023-11-06T01:49:36.005 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2023-11-06T01:49:36.006 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2023-11-06T01:49:36.006 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2023-11-06T01:49:36.006 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2023-11-06T01:49:36.006 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2023-11-06T01:49:36.006 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T01:49:36.007 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T01:49:36.007 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T01:49:36.007 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2023-11-06T01:49:36.007 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2023-11-06T01:49:36.007 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2023-11-06T01:49:36.008 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2023-11-06T01:49:36.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.008 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2023-11-06T01:49:36.008 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2023-11-06T01:49:36.008 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2023-11-06T01:49:36.008 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2023-11-06T01:49:36.009 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2023-11-06T01:49:36.009 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2023-11-06T01:49:36.009 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2023-11-06T01:49:36.009 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2023-11-06T01:49:36.009 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2023-11-06T01:49:36.010 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2023-11-06T01:49:36.010 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2023-11-06T01:49:36.010 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2023-11-06T01:49:36.010 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2023-11-06T01:49:36.010 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2023-11-06T01:49:36.011 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2023-11-06T01:49:36.011 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2023-11-06T01:49:36.011 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2023-11-06T01:49:36.011 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2023-11-06T01:49:36.011 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2023-11-06T01:49:36.012 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2023-11-06T01:49:36.012 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2023-11-06T01:49:36.012 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2023-11-06T01:49:36.012 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2023-11-06T01:49:36.012 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2023-11-06T01:49:36.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.013 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2023-11-06T01:49:36.013 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2023-11-06T01:49:36.013 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2023-11-06T01:49:36.013 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2023-11-06T01:49:36.013 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 3197, 2023-11-06T01:49:36.014 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2023-11-06T01:49:36.014 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2023-11-06T01:49:36.014 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2023-11-06T01:49:36.014 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2023-11-06T01:49:36.014 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2023-11-06T01:49:36.015 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 4, 2023-11-06T01:49:36.015 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2023-11-06T01:49:36.015 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2023-11-06T01:49:36.015 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2023-11-06T01:49:36.015 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 9938524, 2023-11-06T01:49:36.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 9937386, 2023-11-06T01:49:36.016 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2023-11-06T01:49:36.016 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.016 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.016 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.017 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 3197 2023-11-06T01:49:36.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.017 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2023-11-06T01:49:36.017 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 78, 2023-11-06T01:49:36.018 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 147, 2023-11-06T01:49:36.018 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 69, 2023-11-06T01:49:36.018 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 14, 2023-11-06T01:49:36.018 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 17, 2023-11-06T01:49:36.018 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 3, 2023-11-06T01:49:36.019 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 903, 2023-11-06T01:49:36.019 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 976, 2023-11-06T01:49:36.019 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 73, 2023-11-06T01:49:36.019 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2023-11-06T01:49:36.019 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2023-11-06T01:49:36.019 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2023-11-06T01:49:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 53040, 2023-11-06T01:49:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 207156 2023-11-06T01:49:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2023-11-06T01:49:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2023-11-06T01:49:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2023-11-06T01:49:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2023-11-06T01:49:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2023-11-06T01:49:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2023-11-06T01:49:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2023-11-06T01:49:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2023-11-06T01:49:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2023-11-06T01:49:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2023-11-06T01:49:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2023-11-06T01:49:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2023-11-06T01:49:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2023-11-06T01:49:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2023-11-06T01:49:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2023-11-06T01:49:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2023-11-06T01:49:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2023-11-06T01:49:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2023-11-06T01:49:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2023-11-06T01:49:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2023-11-06T01:49:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2023-11-06T01:49:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2023-11-06T01:49:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2023-11-06T01:49:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2023-11-06T01:49:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2023-11-06T01:49:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2023-11-06T01:49:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2023-11-06T01:49:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2023-11-06T01:49:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2023-11-06T01:49:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2023-11-06T01:49:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2023-11-06T01:49:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2023-11-06T01:49:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2023-11-06T01:49:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2023-11-06T01:49:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2023-11-06T01:49:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2023-11-06T01:49:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2023-11-06T01:49:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2023-11-06T01:49:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2023-11-06T01:49:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2023-11-06T01:49:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 1, 2023-11-06T01:49:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 0, 2023-11-06T01:49:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2023-11-06T01:49:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2023-11-06T01:49:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2023-11-06T01:49:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2023-11-06T01:49:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 2407, 2023-11-06T01:49:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2023-11-06T01:49:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2023-11-06T01:49:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2023-11-06T01:49:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2023-11-06T01:49:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:49:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:49:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:49:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:49:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:49:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:49:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:49:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:49:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:49:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:49:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:49:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:49:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:49:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2023-11-06T01:49:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:49:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2023-11-06T01:49:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:49:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:49:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:49:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:49:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:49:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:49:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:49:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2023-11-06T01:49:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:49:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:49:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:49:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2023-11-06T01:49:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2023-11-06T01:49:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2023-11-06T01:49:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:49:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:49:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:49:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:49:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 201574, 2023-11-06T01:49:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 126, 2023-11-06T01:49:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2023-11-06T01:49:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2023-11-06T01:49:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2023-11-06T01:49:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2023-11-06T01:49:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:49:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2023-11-06T01:49:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:49:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2023-11-06T01:49:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 7720, 2023-11-06T01:49:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 162, 2023-11-06T01:49:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:49:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:49:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2023-11-06T01:49:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2023-11-06T01:49:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 845059, 2023-11-06T01:49:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 27332, 2023-11-06T01:49:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2023-11-06T01:49:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2023-11-06T01:49:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2023-11-06T01:49:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2023-11-06T01:49:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2023-11-06T01:49:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2023-11-06T01:49:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2023-11-06T01:49:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 3214, 2023-11-06T01:49:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 21, 2023-11-06T01:49:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 3, 2023-11-06T01:49:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 3214, 2023-11-06T01:49:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2023-11-06T01:49:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3214, 2023-11-06T01:49:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 8.217844092, 2023-11-06T01:49:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002556889 2023-11-06T01:49:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2023-11-06T01:49:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2023-11-06T01:49:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3214, 2023-11-06T01:49:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 3215 2023-11-06T01:49:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 3214, 2023-11-06T01:49:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 3214, 2023-11-06T01:49:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2023-11-06T01:49:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2023-11-06T01:49:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2023-11-06T01:49:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 1950, 2023-11-06T01:49:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2023-11-06T01:49:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 1262, 2023-11-06T01:49:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2023-11-06T01:49:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2023-11-06T01:49:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2023-11-06T01:49:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2023-11-06T01:49:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2023-11-06T01:49:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2023-11-06T01:49:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2023-11-06T01:49:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2023-11-06T01:49:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2023-11-06T01:49:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2023-11-06T01:49:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 1, 2023-11-06T01:49:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2023-11-06T01:49:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2023-11-06T01:49:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2023-11-06T01:49:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2023-11-06T01:49:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2023-11-06T01:49:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2023-11-06T01:49:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2023-11-06T01:49:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T01:49:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2023-11-06T01:49:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2023-11-06T01:49:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2023-11-06T01:49:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2023-11-06T01:49:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2023-11-06T01:49:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2023-11-06T01:49:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2023-11-06T01:49:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2023-11-06T01:49:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2023-11-06T01:49:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2023-11-06T01:49:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2023-11-06T01:49:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2023-11-06T01:49:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2023-11-06T01:49:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2023-11-06T01:49:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2023-11-06T01:49:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2023-11-06T01:49:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2023-11-06T01:49:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2023-11-06T01:49:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2023-11-06T01:49:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 84, 2023-11-06T01:49:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2023-11-06T01:49:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 2, 2023-11-06T01:49:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 6, 2023-11-06T01:49:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 8, 2023-11-06T01:49:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 2, 2023-11-06T01:49:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2023-11-06T01:49:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2023-11-06T01:49:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2023-11-06T01:49:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2023-11-06T01:49:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2023-11-06T01:49:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2023-11-06T01:49:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2023-11-06T01:49:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2023-11-06T01:49:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2023-11-06T01:49:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2023-11-06T01:49:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2023-11-06T01:49:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2023-11-06T01:49:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2023-11-06T01:49:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2023-11-06T01:49:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2023-11-06T01:49:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2023-11-06T01:49:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2023-11-06T01:49:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2023-11-06T01:49:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:49:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-06T01:49:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 3525, 2023-11-06T01:49:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 6945912, 2023-11-06T01:49:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 3525, 2023-11-06T01:49:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-06T01:49:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-06T01:49:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 3525, 2023-11-06T01:49:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 6945912, 2023-11-06T01:49:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2023-11-06T01:49:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-06T01:49:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-06T01:49:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-06T01:49:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 3214, 2023-11-06T01:49:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 5744225, 2023-11-06T01:49:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 3214, 2023-11-06T01:49:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 5744225, 2023-11-06T01:49:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2023-11-06T01:49:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2023-11-06T01:49:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-06T01:49:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-06T01:49:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 3214, 2023-11-06T01:49:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 3214, 2023-11-06T01:49:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 3214, 2023-11-06T01:49:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 3214, 2023-11-06T01:49:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2023-11-06T01:49:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2023-11-06T01:49:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-06T01:49:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-06T01:49:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-06T01:49:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-06T01:49:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2023-11-06T01:49:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2023-11-06T01:49:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x556000b5c500": { 2023-11-06T01:49:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2023-11-06T01:49:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-06T01:49:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-06T01:49:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-06T01:49:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-06T01:49:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2023-11-06T01:49:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2023-11-06T01:49:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:36.099 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:49:36.346 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-11-06T01:49:36.347 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-11-06T01:49:36.347 DEBUG:tasks.check_counter:Getting stats from d 2023-11-06T01:49:36.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-11-06T01:49:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:49:37.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:37 smithi062 ceph-mon[103924]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 1.24k op/s 2023-11-06T01:49:37.414 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:37 smithi053 ceph-mon[99472]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 1.24k op/s 2023-11-06T01:49:37.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:37 smithi163 ceph-mon[104416]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 1.24k op/s 2023-11-06T01:49:38.264 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:49:38.264 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:49:38.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 0, 2023-11-06T01:49:38.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 2, 2023-11-06T01:49:38.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 0, 2023-11-06T01:49:38.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 0, 2023-11-06T01:49:38.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2023-11-06T01:49:38.266 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 0, 2023-11-06T01:49:38.266 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.001445772, 2023-11-06T01:49:38.266 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000000000, 2023-11-06T01:49:38.266 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000258436, 2023-11-06T01:49:38.267 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-06T01:49:38.267 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:38.267 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:38.267 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:38.267 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:38.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:38.268 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:38.268 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:38.268 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:38.268 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:38.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:38.269 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-06T01:49:38.269 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2023-11-06T01:49:38.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:38.270 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:49:38.270 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 6, 2023-11-06T01:49:38.271 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 217, 2023-11-06T01:49:38.271 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 3264, 2023-11-06T01:49:38.271 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 190448, 2023-11-06T01:49:38.271 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2023-11-06T01:49:38.272 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2023-11-06T01:49:38.272 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.014843433, 2023-11-06T01:49:38.273 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.011118858, 2023-11-06T01:49:38.273 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.002008744, 2023-11-06T01:49:38.273 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-06T01:49:38.273 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:38.273 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 216, 2023-11-06T01:49:38.274 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.050244195, 2023-11-06T01:49:38.274 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000232612 2023-11-06T01:49:38.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:38.274 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:38.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:38.275 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:38.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:38.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:38.275 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-11-06T01:49:38.276 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 190448 2023-11-06T01:49:38.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:38.276 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:49:38.276 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 284, 2023-11-06T01:49:38.276 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 281, 2023-11-06T01:49:38.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 488208, 2023-11-06T01:49:38.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 160800, 2023-11-06T01:49:38.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 2, 2023-11-06T01:49:38.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2023-11-06T01:49:38.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.040356574, 2023-11-06T01:49:38.278 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.015419523, 2023-11-06T01:49:38.278 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.016632371, 2023-11-06T01:49:38.278 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.007662111, 2023-11-06T01:49:38.278 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:38.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 280, 2023-11-06T01:49:38.279 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.057705086, 2023-11-06T01:49:38.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000206089 2023-11-06T01:49:38.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:38.279 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:38.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:38.280 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:38.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:38.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:38.280 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 488208, 2023-11-06T01:49:38.281 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 160800 2023-11-06T01:49:38.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:38.281 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2023-11-06T01:49:38.281 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 0, 2023-11-06T01:49:38.281 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2023-11-06T01:49:38.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:38.282 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2023-11-06T01:49:38.283 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2023-11-06T01:49:38.283 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2023-11-06T01:49:38.283 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:49:38.283 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:49:38.284 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:49:38.284 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:49:38.284 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:49:38.284 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:49:38.284 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:49:38.285 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:49:38.285 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:49:38.285 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:49:38.285 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:49:38.286 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:49:38.286 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:49:38.286 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2023-11-06T01:49:38.286 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:49:38.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2023-11-06T01:49:38.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:49:38.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:49:38.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:49:38.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:49:38.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:49:38.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:49:38.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:49:38.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2023-11-06T01:49:38.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:49:38.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:49:38.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:49:38.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2023-11-06T01:49:38.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2023-11-06T01:49:38.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2023-11-06T01:49:38.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:49:38.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:49:38.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:49:38.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:49:38.290 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 149752, 2023-11-06T01:49:38.291 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 31, 2023-11-06T01:49:38.291 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2023-11-06T01:49:38.291 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2023-11-06T01:49:38.291 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2023-11-06T01:49:38.292 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2023-11-06T01:49:38.292 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:49:38.292 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2023-11-06T01:49:38.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:49:38.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2023-11-06T01:49:38.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 0, 2023-11-06T01:49:38.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 0, 2023-11-06T01:49:38.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:49:38.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:49:38.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2023-11-06T01:49:38.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2023-11-06T01:49:38.294 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 928, 2023-11-06T01:49:38.295 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2, 2023-11-06T01:49:38.295 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2023-11-06T01:49:38.295 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2023-11-06T01:49:38.295 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2023-11-06T01:49:38.296 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2023-11-06T01:49:38.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:38.296 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:49:38.296 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:38.296 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-06T01:49:38.297 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:38.297 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 286, 2023-11-06T01:49:38.297 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 448081, 2023-11-06T01:49:38.298 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:38.298 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 286, 2023-11-06T01:49:38.298 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-06T01:49:38.299 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-06T01:49:38.299 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 286, 2023-11-06T01:49:38.299 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 448081, 2023-11-06T01:49:38.299 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:38.300 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:38.300 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:38.300 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:38.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:38.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:38.301 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:49:38.674 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-11-06T01:49:38.675 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-11-06T01:49:38.675 DEBUG:tasks.check_counter:Getting stats from g 2023-11-06T01:49:38.675 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-11-06T01:49:39.238 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:49:39.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:39 smithi062 ceph-mon[103924]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 766 op/s 2023-11-06T01:49:39.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:39 smithi053 ceph-mon[99472]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 766 op/s 2023-11-06T01:49:39.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:39 smithi163 ceph-mon[104416]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 766 op/s 2023-11-06T01:49:40.481 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:49:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:49:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1319, 2023-11-06T01:49:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1293, 2023-11-06T01:49:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 974818, 2023-11-06T01:49:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 473696, 2023-11-06T01:49:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2023-11-06T01:49:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2023-11-06T01:49:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.152720731, 2023-11-06T01:49:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.053524642, 2023-11-06T01:49:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.044609526, 2023-11-06T01:49:40.485 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.044858595, 2023-11-06T01:49:40.486 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:40.486 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1292, 2023-11-06T01:49:40.486 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.366212462, 2023-11-06T01:49:40.486 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000283446 2023-11-06T01:49:40.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.487 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:40.487 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.487 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.487 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.488 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 635712, 2023-11-06T01:49:40.488 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 160208 2023-11-06T01:49:40.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.488 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:49:40.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2058, 2023-11-06T01:49:40.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 2269, 2023-11-06T01:49:40.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1680623, 2023-11-06T01:49:40.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 1083336, 2023-11-06T01:49:40.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2023-11-06T01:49:40.490 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2023-11-06T01:49:40.490 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.165501645, 2023-11-06T01:49:40.490 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.046570557, 2023-11-06T01:49:40.491 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.040180419, 2023-11-06T01:49:40.491 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.066742163, 2023-11-06T01:49:40.491 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:40.491 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2267, 2023-11-06T01:49:40.491 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.332649489, 2023-11-06T01:49:40.492 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000146735 2023-11-06T01:49:40.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.492 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:40.492 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.492 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.493 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.493 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-06T01:49:40.493 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 443552 2023-11-06T01:49:40.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.494 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:49:40.494 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 8, 2023-11-06T01:49:40.494 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 8, 2023-11-06T01:49:40.494 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 4573, 2023-11-06T01:49:40.494 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 1704, 2023-11-06T01:49:40.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 7, 2023-11-06T01:49:40.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2023-11-06T01:49:40.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.003331326, 2023-11-06T01:49:40.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000123304, 2023-11-06T01:49:40.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.002519173, 2023-11-06T01:49:40.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000084156, 2023-11-06T01:49:40.496 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:40.496 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7, 2023-11-06T01:49:40.496 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002925339, 2023-11-06T01:49:40.496 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000417905 2023-11-06T01:49:40.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.497 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:40.497 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.497 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.497 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.498 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-11-06T01:49:40.498 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2023-11-06T01:49:40.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.498 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2023-11-06T01:49:40.499 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2023-11-06T01:49:40.499 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2023-11-06T01:49:40.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.499 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2023-11-06T01:49:40.499 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2023-11-06T01:49:40.500 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2023-11-06T01:49:40.500 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5100, 2023-11-06T01:49:40.500 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.138205283, 2023-11-06T01:49:40.500 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000027099 2023-11-06T01:49:40.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:40.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.501 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2023-11-06T01:49:40.501 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2023-11-06T01:49:40.501 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2023-11-06T01:49:40.501 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.501 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.502 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.502 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:40.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.502 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2023-11-06T01:49:40.503 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2023-11-06T01:49:40.503 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2023-11-06T01:49:40.503 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2023-11-06T01:49:40.503 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.503 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.504 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.504 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2023-11-06T01:49:40.504 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2023-11-06T01:49:40.504 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2023-11-06T01:49:40.504 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2023-11-06T01:49:40.505 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2023-11-06T01:49:40.505 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2023-11-06T01:49:40.505 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2023-11-06T01:49:40.505 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 753, 2023-11-06T01:49:40.506 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 217, 2023-11-06T01:49:40.506 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 427, 2023-11-06T01:49:40.506 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 109, 2023-11-06T01:49:40.506 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 442, 2023-11-06T01:49:40.507 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2023-11-06T01:49:40.507 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2023-11-06T01:49:40.507 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2023-11-06T01:49:40.507 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 8, 2023-11-06T01:49:40.507 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2023-11-06T01:49:40.507 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2023-11-06T01:49:40.508 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2023-11-06T01:49:40.508 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2023-11-06T01:49:40.508 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2023-11-06T01:49:40.508 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2023-11-06T01:49:40.508 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2023-11-06T01:49:40.509 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2023-11-06T01:49:40.509 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2023-11-06T01:49:40.509 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2023-11-06T01:49:40.509 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2023-11-06T01:49:40.509 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2023-11-06T01:49:40.510 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2023-11-06T01:49:40.510 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2023-11-06T01:49:40.510 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2023-11-06T01:49:40.510 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2023-11-06T01:49:40.510 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 0, 2023-11-06T01:49:40.511 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 0, 2023-11-06T01:49:40.511 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2023-11-06T01:49:40.511 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2023-11-06T01:49:40.511 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2023-11-06T01:49:40.511 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T01:49:40.512 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2023-11-06T01:49:40.512 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2023-11-06T01:49:40.512 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2023-11-06T01:49:40.512 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2023-11-06T01:49:40.512 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2023-11-06T01:49:40.512 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2023-11-06T01:49:40.513 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2023-11-06T01:49:40.513 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2023-11-06T01:49:40.513 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T01:49:40.513 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T01:49:40.513 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T01:49:40.514 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2023-11-06T01:49:40.514 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2023-11-06T01:49:40.514 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2023-11-06T01:49:40.514 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2023-11-06T01:49:40.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.515 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2023-11-06T01:49:40.515 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2023-11-06T01:49:40.515 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2023-11-06T01:49:40.515 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2023-11-06T01:49:40.515 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2023-11-06T01:49:40.516 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2023-11-06T01:49:40.516 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2023-11-06T01:49:40.516 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2023-11-06T01:49:40.516 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2023-11-06T01:49:40.516 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2023-11-06T01:49:40.517 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2023-11-06T01:49:40.517 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2023-11-06T01:49:40.517 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2023-11-06T01:49:40.517 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2023-11-06T01:49:40.517 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2023-11-06T01:49:40.518 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2023-11-06T01:49:40.518 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2023-11-06T01:49:40.518 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2023-11-06T01:49:40.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2023-11-06T01:49:40.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2023-11-06T01:49:40.519 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2023-11-06T01:49:40.519 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2023-11-06T01:49:40.519 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2023-11-06T01:49:40.519 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2023-11-06T01:49:40.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.520 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2023-11-06T01:49:40.520 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2023-11-06T01:49:40.520 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2023-11-06T01:49:40.520 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2023-11-06T01:49:40.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 477, 2023-11-06T01:49:40.521 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2023-11-06T01:49:40.521 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2023-11-06T01:49:40.521 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2023-11-06T01:49:40.521 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2023-11-06T01:49:40.521 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2023-11-06T01:49:40.521 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 1, 2023-11-06T01:49:40.522 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2023-11-06T01:49:40.522 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2023-11-06T01:49:40.522 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2023-11-06T01:49:40.522 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 5352553, 2023-11-06T01:49:40.522 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 5351979, 2023-11-06T01:49:40.523 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2023-11-06T01:49:40.523 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.523 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.523 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.524 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 477 2023-11-06T01:49:40.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.524 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2023-11-06T01:49:40.524 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 745, 2023-11-06T01:49:40.525 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 745, 2023-11-06T01:49:40.525 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 0, 2023-11-06T01:49:40.525 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 35, 2023-11-06T01:49:40.525 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 35, 2023-11-06T01:49:40.525 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 0, 2023-11-06T01:49:40.525 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 753, 2023-11-06T01:49:40.526 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 753, 2023-11-06T01:49:40.526 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 0, 2023-11-06T01:49:40.526 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2023-11-06T01:49:40.526 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2023-11-06T01:49:40.526 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2023-11-06T01:49:40.527 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 51584, 2023-11-06T01:49:40.527 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 207156 2023-11-06T01:49:40.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.527 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2023-11-06T01:49:40.527 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2023-11-06T01:49:40.528 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2023-11-06T01:49:40.528 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2023-11-06T01:49:40.528 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2023-11-06T01:49:40.528 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2023-11-06T01:49:40.528 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2023-11-06T01:49:40.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.529 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.529 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2023-11-06T01:49:40.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.530 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.530 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2023-11-06T01:49:40.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.531 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.531 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2023-11-06T01:49:40.532 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.532 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.532 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.532 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2023-11-06T01:49:40.532 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.533 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.533 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.533 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2023-11-06T01:49:40.533 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.534 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.534 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.534 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2023-11-06T01:49:40.534 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.535 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.535 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.535 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2023-11-06T01:49:40.535 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.536 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.536 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.536 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2023-11-06T01:49:40.536 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.536 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.537 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.537 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2023-11-06T01:49:40.537 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.537 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.538 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.538 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2023-11-06T01:49:40.538 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.538 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.539 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.539 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2023-11-06T01:49:40.539 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.539 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.540 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.540 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2023-11-06T01:49:40.540 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.540 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.541 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.541 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2023-11-06T01:49:40.541 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.541 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.542 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.542 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2023-11-06T01:49:40.542 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.542 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.543 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.543 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2023-11-06T01:49:40.543 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.543 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.544 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.544 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2023-11-06T01:49:40.544 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.544 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.545 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.545 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2023-11-06T01:49:40.545 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.545 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.546 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.546 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2023-11-06T01:49:40.546 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.546 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.547 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.547 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2023-11-06T01:49:40.547 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.547 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.548 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.548 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2023-11-06T01:49:40.548 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.548 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.549 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.549 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2023-11-06T01:49:40.549 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.549 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.549 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.550 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2023-11-06T01:49:40.550 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.550 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.550 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.551 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2023-11-06T01:49:40.551 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.551 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.551 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.552 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2023-11-06T01:49:40.552 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.552 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.552 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.553 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2023-11-06T01:49:40.553 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.553 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.553 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.554 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2023-11-06T01:49:40.554 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.554 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.554 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.555 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2023-11-06T01:49:40.555 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.555 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.555 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.556 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2023-11-06T01:49:40.556 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.556 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.556 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.557 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2023-11-06T01:49:40.557 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2023-11-06T01:49:40.557 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2023-11-06T01:49:40.557 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.557 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.558 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.558 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:40.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.558 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2023-11-06T01:49:40.559 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2023-11-06T01:49:40.559 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 1, 2023-11-06T01:49:40.559 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 0, 2023-11-06T01:49:40.559 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2023-11-06T01:49:40.559 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2023-11-06T01:49:40.560 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2023-11-06T01:49:40.560 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2023-11-06T01:49:40.560 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 2410, 2023-11-06T01:49:40.560 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2023-11-06T01:49:40.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.560 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2023-11-06T01:49:40.561 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2023-11-06T01:49:40.561 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2023-11-06T01:49:40.561 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:49:40.561 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:49:40.561 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:49:40.562 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:49:40.562 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:49:40.562 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:49:40.562 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:49:40.563 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:49:40.563 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:49:40.563 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:49:40.563 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:49:40.563 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:49:40.564 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:49:40.564 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2023-11-06T01:49:40.564 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:49:40.564 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2023-11-06T01:49:40.564 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:49:40.564 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:49:40.565 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:49:40.565 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:49:40.565 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:49:40.565 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:49:40.565 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:49:40.566 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2023-11-06T01:49:40.566 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:49:40.566 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:49:40.566 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:49:40.566 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2023-11-06T01:49:40.567 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2023-11-06T01:49:40.567 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2023-11-06T01:49:40.567 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:49:40.567 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:49:40.567 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:49:40.568 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:49:40.568 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 224803, 2023-11-06T01:49:40.568 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 794, 2023-11-06T01:49:40.568 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2023-11-06T01:49:40.568 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2023-11-06T01:49:40.569 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2023-11-06T01:49:40.569 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2023-11-06T01:49:40.569 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:49:40.569 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2023-11-06T01:49:40.569 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:49:40.570 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2023-11-06T01:49:40.570 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 7720, 2023-11-06T01:49:40.570 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 162, 2023-11-06T01:49:40.570 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:49:40.570 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:49:40.571 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2023-11-06T01:49:40.571 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2023-11-06T01:49:40.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 2213090, 2023-11-06T01:49:40.571 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 18660, 2023-11-06T01:49:40.571 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2023-11-06T01:49:40.571 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2023-11-06T01:49:40.572 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2023-11-06T01:49:40.572 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2023-11-06T01:49:40.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.572 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2023-11-06T01:49:40.572 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2023-11-06T01:49:40.573 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2023-11-06T01:49:40.573 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 3069, 2023-11-06T01:49:40.573 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 21, 2023-11-06T01:49:40.573 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 3, 2023-11-06T01:49:40.573 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 3069, 2023-11-06T01:49:40.574 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2023-11-06T01:49:40.574 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3069, 2023-11-06T01:49:40.574 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 24.199541123, 2023-11-06T01:49:40.574 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.007885155 2023-11-06T01:49:40.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.575 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2023-11-06T01:49:40.575 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2023-11-06T01:49:40.575 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3069, 2023-11-06T01:49:40.575 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 3070 2023-11-06T01:49:40.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.576 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 3069, 2023-11-06T01:49:40.576 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 3069, 2023-11-06T01:49:40.576 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2023-11-06T01:49:40.576 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2023-11-06T01:49:40.576 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2023-11-06T01:49:40.577 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 1904, 2023-11-06T01:49:40.577 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2023-11-06T01:49:40.577 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 1163, 2023-11-06T01:49:40.577 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2023-11-06T01:49:40.577 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2023-11-06T01:49:40.577 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2023-11-06T01:49:40.578 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2023-11-06T01:49:40.578 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2023-11-06T01:49:40.578 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2023-11-06T01:49:40.578 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2023-11-06T01:49:40.579 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2023-11-06T01:49:40.579 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2023-11-06T01:49:40.579 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2023-11-06T01:49:40.579 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 1, 2023-11-06T01:49:40.579 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2023-11-06T01:49:40.580 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2023-11-06T01:49:40.580 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2023-11-06T01:49:40.580 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2023-11-06T01:49:40.580 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2023-11-06T01:49:40.580 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2023-11-06T01:49:40.580 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2023-11-06T01:49:40.581 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T01:49:40.581 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2023-11-06T01:49:40.581 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2023-11-06T01:49:40.581 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2023-11-06T01:49:40.581 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2023-11-06T01:49:40.582 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2023-11-06T01:49:40.582 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2023-11-06T01:49:40.582 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2023-11-06T01:49:40.582 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2023-11-06T01:49:40.582 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2023-11-06T01:49:40.583 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2023-11-06T01:49:40.583 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2023-11-06T01:49:40.583 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2023-11-06T01:49:40.583 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2023-11-06T01:49:40.583 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2023-11-06T01:49:40.584 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2023-11-06T01:49:40.584 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2023-11-06T01:49:40.584 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2023-11-06T01:49:40.584 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2023-11-06T01:49:40.584 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2023-11-06T01:49:40.585 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 84, 2023-11-06T01:49:40.585 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2023-11-06T01:49:40.585 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 2, 2023-11-06T01:49:40.585 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 6, 2023-11-06T01:49:40.585 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 8, 2023-11-06T01:49:40.586 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 2, 2023-11-06T01:49:40.586 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2023-11-06T01:49:40.586 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2023-11-06T01:49:40.586 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2023-11-06T01:49:40.586 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2023-11-06T01:49:40.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.587 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2023-11-06T01:49:40.587 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2023-11-06T01:49:40.587 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2023-11-06T01:49:40.587 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2023-11-06T01:49:40.587 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2023-11-06T01:49:40.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.588 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2023-11-06T01:49:40.588 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2023-11-06T01:49:40.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.588 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2023-11-06T01:49:40.589 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2023-11-06T01:49:40.589 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2023-11-06T01:49:40.589 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2023-11-06T01:49:40.589 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2023-11-06T01:49:40.589 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2023-11-06T01:49:40.590 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2023-11-06T01:49:40.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.590 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:49:40.590 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:40.590 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-06T01:49:40.591 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:40.591 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 3381, 2023-11-06T01:49:40.591 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 2336930, 2023-11-06T01:49:40.591 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:40.591 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 3381, 2023-11-06T01:49:40.592 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-06T01:49:40.592 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-06T01:49:40.592 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 3381, 2023-11-06T01:49:40.592 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2336930, 2023-11-06T01:49:40.592 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:40.593 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.593 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.593 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.593 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:40.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.594 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2023-11-06T01:49:40.594 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:40.594 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-06T01:49:40.594 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:40.594 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-06T01:49:40.595 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-06T01:49:40.595 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:40.595 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:40.595 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 3069, 2023-11-06T01:49:40.595 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 1158254, 2023-11-06T01:49:40.595 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 3069, 2023-11-06T01:49:40.596 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1158254, 2023-11-06T01:49:40.596 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:40.596 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.596 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.596 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.597 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:40.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.597 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2023-11-06T01:49:40.597 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:40.597 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2023-11-06T01:49:40.598 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:40.598 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-06T01:49:40.598 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-06T01:49:40.598 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:40.598 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:40.599 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 3069, 2023-11-06T01:49:40.599 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 3069, 2023-11-06T01:49:40.599 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 3069, 2023-11-06T01:49:40.599 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 3069, 2023-11-06T01:49:40.599 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:40.600 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.600 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.600 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:40.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.601 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2023-11-06T01:49:40.601 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:40.601 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2023-11-06T01:49:40.601 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:40.601 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-06T01:49:40.602 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-06T01:49:40.602 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:40.602 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:40.602 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-06T01:49:40.602 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-06T01:49:40.603 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2023-11-06T01:49:40.603 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2023-11-06T01:49:40.603 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:40.603 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.603 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.604 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:40.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:40.604 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x562b1f7c8500": { 2023-11-06T01:49:40.604 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2023-11-06T01:49:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-06T01:49:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-06T01:49:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-06T01:49:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-06T01:49:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2023-11-06T01:49:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2023-11-06T01:49:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:40.608 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:49:40.904 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-11-06T01:49:40.904 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-11-06T01:49:40.904 DEBUG:tasks.check_counter:Getting stats from j 2023-11-06T01:49:40.905 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-11-06T01:49:41.469 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.a/config 2023-11-06T01:49:41.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:41 smithi163 ceph-mon[104416]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.8 MiB/s wr, 283 op/s 2023-11-06T01:49:41.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:41 smithi053 ceph-mon[99472]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.8 MiB/s wr, 283 op/s 2023-11-06T01:49:41.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:41 smithi062 ceph-mon[103924]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.8 MiB/s wr, 283 op/s 2023-11-06T01:49:42.747 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-06T01:49:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:49:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 286, 2023-11-06T01:49:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 283, 2023-11-06T01:49:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 488752, 2023-11-06T01:49:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 161984, 2023-11-06T01:49:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2023-11-06T01:49:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 0, 2023-11-06T01:49:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.041036963, 2023-11-06T01:49:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.015080753, 2023-11-06T01:49:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.016620937, 2023-11-06T01:49:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.007476502, 2023-11-06T01:49:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 282, 2023-11-06T01:49:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.089756101, 2023-11-06T01:49:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000318284 2023-11-06T01:49:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 488752, 2023-11-06T01:49:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 161984 2023-11-06T01:49:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:49:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2, 2023-11-06T01:49:42.753 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 217, 2023-11-06T01:49:42.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 320, 2023-11-06T01:49:42.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 190352, 2023-11-06T01:49:42.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2023-11-06T01:49:42.755 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2023-11-06T01:49:42.755 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.014137988, 2023-11-06T01:49:42.755 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.010995578, 2023-11-06T01:49:42.755 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.001339613, 2023-11-06T01:49:42.755 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-06T01:49:42.756 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:42.756 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 215, 2023-11-06T01:49:42.756 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.053729697, 2023-11-06T01:49:42.756 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000249905 2023-11-06T01:49:42.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:42.757 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:42.757 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:42.757 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:42.758 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:42.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:42.758 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-06T01:49:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 190352 2023-11-06T01:49:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:49:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 4, 2023-11-06T01:49:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 4, 2023-11-06T01:49:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 2944, 2023-11-06T01:49:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 448, 2023-11-06T01:49:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 2, 2023-11-06T01:49:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2023-11-06T01:49:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.000741015, 2023-11-06T01:49:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000045581, 2023-11-06T01:49:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000678595, 2023-11-06T01:49:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-06T01:49:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3, 2023-11-06T01:49:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000670085, 2023-11-06T01:49:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000223361 2023-11-06T01:49:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-11-06T01:49:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2023-11-06T01:49:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2023-11-06T01:49:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 0, 2023-11-06T01:49:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2023-11-06T01:49:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2023-11-06T01:49:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2023-11-06T01:49:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2023-11-06T01:49:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:49:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:49:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:49:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:49:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:49:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:49:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:49:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:49:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:49:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:49:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:49:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:49:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:49:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2023-11-06T01:49:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:49:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2023-11-06T01:49:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:49:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:49:42.772 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:49:42.772 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:49:42.772 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:49:42.772 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:49:42.772 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:49:42.773 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2023-11-06T01:49:42.773 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:49:42.773 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:49:42.773 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:49:42.773 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2023-11-06T01:49:42.774 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2023-11-06T01:49:42.774 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2023-11-06T01:49:42.774 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:49:42.774 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:49:42.774 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:49:42.775 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:49:42.775 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 147937, 2023-11-06T01:49:42.775 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 28, 2023-11-06T01:49:42.775 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2023-11-06T01:49:42.775 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2023-11-06T01:49:42.775 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2023-11-06T01:49:42.776 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2023-11-06T01:49:42.776 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:49:42.776 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2023-11-06T01:49:42.776 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:49:42.776 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2023-11-06T01:49:42.777 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 0, 2023-11-06T01:49:42.777 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 0, 2023-11-06T01:49:42.777 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:49:42.777 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:49:42.777 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2023-11-06T01:49:42.778 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2023-11-06T01:49:42.778 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 928, 2023-11-06T01:49:42.778 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2, 2023-11-06T01:49:42.778 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2023-11-06T01:49:42.778 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2023-11-06T01:49:42.779 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2023-11-06T01:49:42.779 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2023-11-06T01:49:42.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-06T01:49:42.779 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:49:42.779 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-06T01:49:42.780 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-06T01:49:42.780 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-06T01:49:42.780 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 288, 2023-11-06T01:49:42.780 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 448341, 2023-11-06T01:49:42.780 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:42.781 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 288, 2023-11-06T01:49:42.781 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-06T01:49:42.781 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-06T01:49:42.781 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 288, 2023-11-06T01:49:42.781 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 448341, 2023-11-06T01:49:42.782 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-06T01:49:42.782 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-06T01:49:42.782 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-06T01:49:42.782 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-06T01:49:42.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:42.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-06T01:49:42.783 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-06T01:49:42.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:42 smithi053 ceph-mon[99472]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-11-06T01:49:42.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:42 smithi062 ceph-mon[103924]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-11-06T01:49:42.986 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:42 smithi163 ceph-mon[104416]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-11-06T01:49:43.225 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-11-06T01:49:43.225 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-11-06T01:49:43.226 DEBUG:tasks.check_counter:Getting stats from b 2023-11-06T01:49:43.226 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-11-06T01:49:43.904 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:49:45.265 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-11-06T01:49:45.265 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:49:45.266 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1921385, 2023-11-06T01:49:45.266 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1921323, 2023-11-06T01:49:45.266 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 310097724, 2023-11-06T01:49:45.266 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 311180364, 2023-11-06T01:49:45.266 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 10, 2023-11-06T01:49:45.267 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2023-11-06T01:49:45.267 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 84.703545285, 2023-11-06T01:49:45.267 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 33.395406873, 2023-11-06T01:49:45.267 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 60.488748340, 2023-11-06T01:49:45.267 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.031070038, 2023-11-06T01:49:45.268 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:45.268 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1921323, 2023-11-06T01:49:45.268 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 41.357339788, 2023-11-06T01:49:45.268 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000021525 2023-11-06T01:49:45.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.269 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:45.269 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3840554, 2023-11-06T01:49:45.269 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.476933880, 2023-11-06T01:49:45.269 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000124 2023-11-06T01:49:45.269 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.270 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-06T01:49:45.270 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2023-11-06T01:49:45.270 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.270 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:49:45.270 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1061, 2023-11-06T01:49:45.271 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1145, 2023-11-06T01:49:45.271 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1015337, 2023-11-06T01:49:45.271 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 600794, 2023-11-06T01:49:45.271 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 11, 2023-11-06T01:49:45.271 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 8, 2023-11-06T01:49:45.271 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.104808629, 2023-11-06T01:49:45.272 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.035968946, 2023-11-06T01:49:45.272 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.051886154, 2023-11-06T01:49:45.272 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.018665966, 2023-11-06T01:49:45.272 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:45.272 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1144, 2023-11-06T01:49:45.273 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.270623994, 2023-11-06T01:49:45.273 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000236559 2023-11-06T01:49:45.273 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.273 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:45.273 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 666, 2023-11-06T01:49:45.274 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000148184, 2023-11-06T01:49:45.274 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000222 2023-11-06T01:49:45.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.274 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 671344, 2023-11-06T01:49:45.274 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 166688 2023-11-06T01:49:45.275 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.275 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:49:45.275 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 640253, 2023-11-06T01:49:45.275 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 641810, 2023-11-06T01:49:45.275 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 101649081, 2023-11-06T01:49:45.276 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 105778064, 2023-11-06T01:49:45.276 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 16, 2023-11-06T01:49:45.276 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2023-11-06T01:49:45.276 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 27.072733482, 2023-11-06T01:49:45.276 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 10.206250383, 2023-11-06T01:49:45.277 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 19.488333297, 2023-11-06T01:49:45.277 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.008354868, 2023-11-06T01:49:45.277 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:45.277 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 641807, 2023-11-06T01:49:45.277 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 21.511751994, 2023-11-06T01:49:45.278 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000033517 2023-11-06T01:49:45.278 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.278 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:45.278 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1280258, 2023-11-06T01:49:45.278 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.169705711, 2023-11-06T01:49:45.279 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000132 2023-11-06T01:49:45.279 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.279 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-11-06T01:49:45.279 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 538304 2023-11-06T01:49:45.279 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.280 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2023-11-06T01:49:45.280 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2023-11-06T01:49:45.280 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2023-11-06T01:49:45.280 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.280 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2023-11-06T01:49:45.280 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2023-11-06T01:49:45.281 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2023-11-06T01:49:45.281 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1469, 2023-11-06T01:49:45.281 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.956073593, 2023-11-06T01:49:45.281 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000650832 2023-11-06T01:49:45.281 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:45.282 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.282 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2023-11-06T01:49:45.282 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2023-11-06T01:49:45.282 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2023-11-06T01:49:45.282 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 719, 2023-11-06T01:49:45.283 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.036230586, 2023-11-06T01:49:45.283 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000050390 2023-11-06T01:49:45.283 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:45.283 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.283 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2023-11-06T01:49:45.284 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 941, 2023-11-06T01:49:45.284 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 803, 2023-11-06T01:49:45.284 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2023-11-06T01:49:45.284 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 803, 2023-11-06T01:49:45.284 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.732629335, 2023-11-06T01:49:45.285 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002157695 2023-11-06T01:49:45.285 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.285 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2023-11-06T01:49:45.285 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 138, 2023-11-06T01:49:45.285 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 1, 2023-11-06T01:49:45.286 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2023-11-06T01:49:45.286 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 11, 2023-11-06T01:49:45.286 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2023-11-06T01:49:45.286 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2023-11-06T01:49:45.286 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 92, 2023-11-06T01:49:45.287 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 12, 2023-11-06T01:49:45.287 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 56, 2023-11-06T01:49:45.287 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 24, 2023-11-06T01:49:45.287 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 74, 2023-11-06T01:49:45.287 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 565, 2023-11-06T01:49:45.287 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 39, 2023-11-06T01:49:45.288 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 39, 2023-11-06T01:49:45.288 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 11, 2023-11-06T01:49:45.288 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 1564, 2023-11-06T01:49:45.288 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 1278, 2023-11-06T01:49:45.288 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 138, 2023-11-06T01:49:45.289 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 7, 2023-11-06T01:49:45.289 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2023-11-06T01:49:45.289 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2023-11-06T01:49:45.289 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 35, 2023-11-06T01:49:45.289 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 1363, 2023-11-06T01:49:45.290 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2023-11-06T01:49:45.290 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 2, 2023-11-06T01:49:45.290 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 571, 2023-11-06T01:49:45.290 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 16, 2023-11-06T01:49:45.290 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 822, 2023-11-06T01:49:45.291 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2023-11-06T01:49:45.291 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 259, 2023-11-06T01:49:45.291 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2023-11-06T01:49:45.291 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 1039, 2023-11-06T01:49:45.291 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 1693659912, 2023-11-06T01:49:45.292 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2023-11-06T01:49:45.292 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 259, 2023-11-06T01:49:45.292 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2023-11-06T01:49:45.292 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T01:49:45.292 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2023-11-06T01:49:45.292 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2023-11-06T01:49:45.293 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2023-11-06T01:49:45.293 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2023-11-06T01:49:45.293 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 259, 2023-11-06T01:49:45.293 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 2, 2023-11-06T01:49:45.293 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 38, 2023-11-06T01:49:45.294 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 694, 2023-11-06T01:49:45.294 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T01:49:45.294 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T01:49:45.294 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T01:49:45.294 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 564, 2023-11-06T01:49:45.295 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 348, 2023-11-06T01:49:45.295 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 95, 2023-11-06T01:49:45.295 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 606 2023-11-06T01:49:45.295 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.295 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2023-11-06T01:49:45.296 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2023-11-06T01:49:45.296 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2023-11-06T01:49:45.296 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2023-11-06T01:49:45.296 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 31, 2023-11-06T01:49:45.296 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 5, 2023-11-06T01:49:45.296 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2023-11-06T01:49:45.297 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2023-11-06T01:49:45.297 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2023-11-06T01:49:45.297 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 207, 2023-11-06T01:49:45.297 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 207, 2023-11-06T01:49:45.297 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2023-11-06T01:49:45.298 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2023-11-06T01:49:45.298 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2023-11-06T01:49:45.298 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2023-11-06T01:49:45.298 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2023-11-06T01:49:45.298 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2023-11-06T01:49:45.299 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2023-11-06T01:49:45.299 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2023-11-06T01:49:45.299 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 2, 2023-11-06T01:49:45.299 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2023-11-06T01:49:45.299 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2023-11-06T01:49:45.300 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2023-11-06T01:49:45.300 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2023-11-06T01:49:45.300 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.300 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2023-11-06T01:49:45.300 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 1853, 2023-11-06T01:49:45.301 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2023-11-06T01:49:45.301 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2023-11-06T01:49:45.301 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 1853, 2023-11-06T01:49:45.301 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2023-11-06T01:49:45.301 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2023-11-06T01:49:45.301 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 2, 2023-11-06T01:49:45.302 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2023-11-06T01:49:45.302 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2023-11-06T01:49:45.302 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 2, 2023-11-06T01:49:45.302 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2023-11-06T01:49:45.302 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2023-11-06T01:49:45.303 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2023-11-06T01:49:45.303 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 8033173, 2023-11-06T01:49:45.303 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2023-11-06T01:49:45.303 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2023-11-06T01:49:45.303 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 614, 2023-11-06T01:49:45.304 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 7.273841751, 2023-11-06T01:49:45.304 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.011846647 2023-11-06T01:49:45.304 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.304 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2023-11-06T01:49:45.304 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.305 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2023-11-06T01:49:45.305 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 283, 2023-11-06T01:49:45.305 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 1036, 2023-11-06T01:49:45.305 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 753, 2023-11-06T01:49:45.305 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 52, 2023-11-06T01:49:45.306 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 66, 2023-11-06T01:49:45.306 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 14, 2023-11-06T01:49:45.306 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 292, 2023-11-06T01:49:45.306 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 1056, 2023-11-06T01:49:45.306 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 764, 2023-11-06T01:49:45.307 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 248, 2023-11-06T01:49:45.307 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 754, 2023-11-06T01:49:45.307 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 506, 2023-11-06T01:49:45.307 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 57772, 2023-11-06T01:49:45.307 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 198964 2023-11-06T01:49:45.308 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.308 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2023-11-06T01:49:45.308 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 1051, 2023-11-06T01:49:45.308 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 41, 2023-11-06T01:49:45.308 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 941, 2023-11-06T01:49:45.309 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 286, 2023-11-06T01:49:45.309 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 99, 2023-11-06T01:49:45.309 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2023-11-06T01:49:45.309 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 206, 2023-11-06T01:49:45.309 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.451143482, 2023-11-06T01:49:45.309 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002190016 2023-11-06T01:49:45.310 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.310 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2023-11-06T01:49:45.310 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 21, 2023-11-06T01:49:45.310 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.058981843, 2023-11-06T01:49:45.310 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002808659 2023-11-06T01:49:45.311 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.311 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2023-11-06T01:49:45.311 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.311 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.311 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.312 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.312 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2023-11-06T01:49:45.312 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.312 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.312 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2023-11-06T01:49:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 289, 2023-11-06T01:49:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.511601134, 2023-11-06T01:49:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001770246 2023-11-06T01:49:45.314 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.314 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2023-11-06T01:49:45.314 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.314 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.314 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.314 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.315 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2023-11-06T01:49:45.315 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.315 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.315 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.315 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.316 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2023-11-06T01:49:45.316 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.316 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.316 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.316 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.317 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2023-11-06T01:49:45.317 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.317 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.317 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.317 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.318 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2023-11-06T01:49:45.318 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.318 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.318 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.318 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.319 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2023-11-06T01:49:45.319 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.319 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.319 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.319 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.320 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2023-11-06T01:49:45.320 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.320 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.320 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.320 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.321 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2023-11-06T01:49:45.321 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.321 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.321 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.321 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.322 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2023-11-06T01:49:45.322 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.322 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.322 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.322 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.323 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2023-11-06T01:49:45.323 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 71, 2023-11-06T01:49:45.323 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.062687352, 2023-11-06T01:49:45.323 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000882920 2023-11-06T01:49:45.323 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.324 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2023-11-06T01:49:45.324 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6, 2023-11-06T01:49:45.324 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.011400467, 2023-11-06T01:49:45.324 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001900077 2023-11-06T01:49:45.324 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.325 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2023-11-06T01:49:45.325 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 206, 2023-11-06T01:49:45.325 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.629160291, 2023-11-06T01:49:45.325 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.003054176 2023-11-06T01:49:45.325 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.326 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2023-11-06T01:49:45.326 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.326 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.326 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.326 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.327 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2023-11-06T01:49:45.327 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.327 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.327 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.327 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.328 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2023-11-06T01:49:45.328 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.328 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.328 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.328 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.328 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2023-11-06T01:49:45.329 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.329 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.329 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.329 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.329 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2023-11-06T01:49:45.330 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.330 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.331 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.331 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.331 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2023-11-06T01:49:45.331 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.332 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.332 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.332 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.332 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2023-11-06T01:49:45.332 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.333 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.333 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.333 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.333 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2023-11-06T01:49:45.333 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.334 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.334 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.334 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.334 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2023-11-06T01:49:45.334 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.335 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.335 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.335 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.335 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2023-11-06T01:49:45.336 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.336 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.336 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.336 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.336 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2023-11-06T01:49:45.337 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.337 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.337 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.337 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2023-11-06T01:49:45.338 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4, 2023-11-06T01:49:45.338 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.007654766, 2023-11-06T01:49:45.338 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001913691 2023-11-06T01:49:45.338 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.338 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2023-11-06T01:49:45.339 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2023-11-06T01:49:45.339 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2023-11-06T01:49:45.339 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.339 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.339 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.340 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:45.340 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.340 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2023-11-06T01:49:45.340 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2023-11-06T01:49:45.340 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 1, 2023-11-06T01:49:45.341 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 0, 2023-11-06T01:49:45.341 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2023-11-06T01:49:45.341 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2023-11-06T01:49:45.341 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 75, 2023-11-06T01:49:45.341 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 75, 2023-11-06T01:49:45.342 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 2377, 2023-11-06T01:49:45.342 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2023-11-06T01:49:45.342 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.342 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2023-11-06T01:49:45.342 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2023-11-06T01:49:45.343 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2023-11-06T01:49:45.343 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:49:45.343 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:49:45.343 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:49:45.343 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:49:45.344 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:49:45.344 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:49:45.344 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:49:45.344 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:49:45.344 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:49:45.345 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:49:45.345 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:49:45.345 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:49:45.345 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:49:45.345 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2023-11-06T01:49:45.346 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:49:45.346 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2023-11-06T01:49:45.346 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:49:45.346 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:49:45.346 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:49:45.347 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:49:45.347 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:49:45.347 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:49:45.347 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:49:45.347 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2023-11-06T01:49:45.348 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:49:45.348 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:49:45.348 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:49:45.348 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2023-11-06T01:49:45.348 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2023-11-06T01:49:45.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2023-11-06T01:49:45.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:49:45.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:49:45.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:49:45.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:49:45.349 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 299749, 2023-11-06T01:49:45.350 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 182, 2023-11-06T01:49:45.350 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 352, 2023-11-06T01:49:45.350 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 4, 2023-11-06T01:49:45.350 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2023-11-06T01:49:45.350 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2023-11-06T01:49:45.351 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:49:45.351 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2023-11-06T01:49:45.351 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:49:45.351 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2023-11-06T01:49:45.351 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 7720, 2023-11-06T01:49:45.352 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 162, 2023-11-06T01:49:45.352 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:49:45.352 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:49:45.352 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2023-11-06T01:49:45.352 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2023-11-06T01:49:45.353 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 355150, 2023-11-06T01:49:45.353 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 4249, 2023-11-06T01:49:45.353 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2023-11-06T01:49:45.353 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2023-11-06T01:49:45.353 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2023-11-06T01:49:45.354 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2023-11-06T01:49:45.354 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.354 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2023-11-06T01:49:45.354 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2023-11-06T01:49:45.354 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2023-11-06T01:49:45.355 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 1541, 2023-11-06T01:49:45.355 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 3935785, 2023-11-06T01:49:45.355 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 27, 2023-11-06T01:49:45.355 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 1541, 2023-11-06T01:49:45.355 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2023-11-06T01:49:45.355 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1541, 2023-11-06T01:49:45.356 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 12.687804051, 2023-11-06T01:49:45.356 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.008233487 2023-11-06T01:49:45.356 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.356 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2023-11-06T01:49:45.356 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2023-11-06T01:49:45.357 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1541, 2023-11-06T01:49:45.357 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1709 2023-11-06T01:49:45.357 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.357 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 1541, 2023-11-06T01:49:45.357 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 347, 2023-11-06T01:49:45.358 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 1194, 2023-11-06T01:49:45.358 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2023-11-06T01:49:45.358 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2023-11-06T01:49:45.358 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 0, 2023-11-06T01:49:45.358 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 10, 2023-11-06T01:49:45.359 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 87, 2023-11-06T01:49:45.359 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 701, 2023-11-06T01:49:45.359 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 132, 2023-11-06T01:49:45.359 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2023-11-06T01:49:45.359 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2023-11-06T01:49:45.360 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2023-11-06T01:49:45.360 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2023-11-06T01:49:45.360 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 219, 2023-11-06T01:49:45.360 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2023-11-06T01:49:45.360 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2023-11-06T01:49:45.360 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2023-11-06T01:49:45.361 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 260, 2023-11-06T01:49:45.361 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 20, 2023-11-06T01:49:45.361 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2023-11-06T01:49:45.361 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2023-11-06T01:49:45.361 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2023-11-06T01:49:45.362 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2023-11-06T01:49:45.362 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2023-11-06T01:49:45.362 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2023-11-06T01:49:45.362 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T01:49:45.362 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2023-11-06T01:49:45.363 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2023-11-06T01:49:45.363 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2023-11-06T01:49:45.363 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2023-11-06T01:49:45.363 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2023-11-06T01:49:45.363 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2023-11-06T01:49:45.364 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2023-11-06T01:49:45.364 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2023-11-06T01:49:45.364 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2023-11-06T01:49:45.364 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2023-11-06T01:49:45.364 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2023-11-06T01:49:45.364 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2023-11-06T01:49:45.365 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2023-11-06T01:49:45.365 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2023-11-06T01:49:45.365 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2023-11-06T01:49:45.365 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2023-11-06T01:49:45.365 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2023-11-06T01:49:45.366 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2023-11-06T01:49:45.366 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2023-11-06T01:49:45.366 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 84, 2023-11-06T01:49:45.366 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2023-11-06T01:49:45.366 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 2, 2023-11-06T01:49:45.367 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 12, 2023-11-06T01:49:45.367 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 12, 2023-11-06T01:49:45.367 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2023-11-06T01:49:45.367 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2023-11-06T01:49:45.367 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 207, 2023-11-06T01:49:45.368 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2023-11-06T01:49:45.368 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 71 2023-11-06T01:49:45.368 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.368 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2023-11-06T01:49:45.368 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2023-11-06T01:49:45.368 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2023-11-06T01:49:45.369 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2023-11-06T01:49:45.369 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2023-11-06T01:49:45.369 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.369 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2023-11-06T01:49:45.369 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 984 2023-11-06T01:49:45.370 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.370 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2023-11-06T01:49:45.370 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2023-11-06T01:49:45.370 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 6, 2023-11-06T01:49:45.370 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2023-11-06T01:49:45.371 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 6, 2023-11-06T01:49:45.371 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 207, 2023-11-06T01:49:45.371 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2023-11-06T01:49:45.371 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.371 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:49:45.372 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:45.372 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-11-06T01:49:45.372 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:45.372 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 2562695, 2023-11-06T01:49:45.372 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 182099125, 2023-11-06T01:49:45.372 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:45.373 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 2562695, 2023-11-06T01:49:45.373 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:45.373 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:45.373 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2562695, 2023-11-06T01:49:45.373 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 182099125, 2023-11-06T01:49:45.374 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:45.374 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.374 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.374 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.374 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:45.375 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.375 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2023-11-06T01:49:45.375 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:45.375 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-11-06T01:49:45.375 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:45.376 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:45.376 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:45.376 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:45.376 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:45.376 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 1541, 2023-11-06T01:49:45.376 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 3956676, 2023-11-06T01:49:45.377 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1541, 2023-11-06T01:49:45.377 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 3956676, 2023-11-06T01:49:45.377 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:45.377 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.377 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.378 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.378 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:45.378 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.378 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2023-11-06T01:49:45.378 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:45.379 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2023-11-06T01:49:45.379 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:45.379 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:45.379 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:45.379 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:45.380 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:45.380 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 1541, 2023-11-06T01:49:45.380 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 1541, 2023-11-06T01:49:45.380 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1541, 2023-11-06T01:49:45.380 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1541, 2023-11-06T01:49:45.381 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:45.381 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.381 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.381 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.381 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:45.381 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.382 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2023-11-06T01:49:45.382 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:45.382 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2023-11-06T01:49:45.382 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:45.382 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 207, 2023-11-06T01:49:45.383 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 20907, 2023-11-06T01:49:45.383 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:45.383 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 207, 2023-11-06T01:49:45.383 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:45.383 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:45.384 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 87, 2023-11-06T01:49:45.384 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 20907, 2023-11-06T01:49:45.384 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:45.384 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.384 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.384 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.385 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:45.385 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:45.385 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x558b38b7e0a0": { 2023-11-06T01:49:45.385 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 5097, 2023-11-06T01:49:45.385 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2023-11-06T01:49:45.386 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:45.386 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 1853, 2023-11-06T01:49:45.386 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 3838869, 2023-11-06T01:49:45.386 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:45.386 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 1853, 2023-11-06T01:49:45.387 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:45.387 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:45.387 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 614, 2023-11-06T01:49:45.387 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 3833772, 2023-11-06T01:49:45.387 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:45.388 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:45.388 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:45.388 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:45.388 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:45.388 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:45.388 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-11-06T01:49:45.389 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:45 smithi053 ceph-mon[99472]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:49:45.390 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:45 smithi053 ceph-mon[99472]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-11-06T01:49:45.390 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:45 smithi062 ceph-mon[103924]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:49:45.390 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:45 smithi062 ceph-mon[103924]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-11-06T01:49:45.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:45 smithi163 ceph-mon[104416]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-11-06T01:49:45.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:45 smithi163 ceph-mon[104416]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-11-06T01:49:45.673 INFO:tasks.check_counter:Daemon mds.b mds.exported=2 2023-11-06T01:49:45.674 INFO:tasks.check_counter:Daemon mds.b mds.imported=16 2023-11-06T01:49:45.674 DEBUG:tasks.check_counter:Getting stats from e 2023-11-06T01:49:45.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-11-06T01:49:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:49:47.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:47 smithi062 ceph-mon[103924]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 KiB/s wr, 15 op/s 2023-11-06T01:49:47.430 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-11-06T01:49:47.430 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:49:47.431 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 5290, 2023-11-06T01:49:47.431 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 6182, 2023-11-06T01:49:47.431 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1530912, 2023-11-06T01:49:47.431 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 6375475, 2023-11-06T01:49:47.431 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 10, 2023-11-06T01:49:47.432 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2023-11-06T01:49:47.432 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.382781702, 2023-11-06T01:49:47.432 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.133439964, 2023-11-06T01:49:47.432 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.200879714, 2023-11-06T01:49:47.432 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.062476268, 2023-11-06T01:49:47.433 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:47.433 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6184, 2023-11-06T01:49:47.433 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.129483444, 2023-11-06T01:49:47.433 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000182646 2023-11-06T01:49:47.433 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.434 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:47.434 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 7298, 2023-11-06T01:49:47.434 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.001859234, 2023-11-06T01:49:47.434 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000254 2023-11-06T01:49:47.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.435 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-06T01:49:47.436 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2023-11-06T01:49:47.436 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.437 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:49:47.437 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1448, 2023-11-06T01:49:47.437 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1307, 2023-11-06T01:49:47.437 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1061334, 2023-11-06T01:49:47.437 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 2144409, 2023-11-06T01:49:47.438 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 11, 2023-11-06T01:49:47.438 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 9, 2023-11-06T01:49:47.438 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.125942489, 2023-11-06T01:49:47.438 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.043679632, 2023-11-06T01:49:47.439 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.059070963, 2023-11-06T01:49:47.439 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.027258150, 2023-11-06T01:49:47.439 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:47.439 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1307, 2023-11-06T01:49:47.439 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.419953568, 2023-11-06T01:49:47.440 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000321311 2023-11-06T01:49:47.440 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.440 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:47.440 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 687, 2023-11-06T01:49:47.441 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000138532, 2023-11-06T01:49:47.441 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000201 2023-11-06T01:49:47.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.441 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 670080, 2023-11-06T01:49:47.442 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 170032 2023-11-06T01:49:47.442 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.442 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:49:47.442 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 8487, 2023-11-06T01:49:47.442 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 9723, 2023-11-06T01:49:47.443 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 2395613, 2023-11-06T01:49:47.443 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 5993545, 2023-11-06T01:49:47.443 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 15, 2023-11-06T01:49:47.443 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 7, 2023-11-06T01:49:47.443 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.498898676, 2023-11-06T01:49:47.444 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.206521450, 2023-11-06T01:49:47.444 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.388104573, 2023-11-06T01:49:47.444 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000934492, 2023-11-06T01:49:47.444 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:47.444 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 9717, 2023-11-06T01:49:47.445 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.464263582, 2023-11-06T01:49:47.445 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000150690 2023-11-06T01:49:47.445 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.445 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:47.445 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 15637, 2023-11-06T01:49:47.446 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.003192372, 2023-11-06T01:49:47.446 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000204 2023-11-06T01:49:47.446 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.446 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-06T01:49:47.447 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 454816 2023-11-06T01:49:47.447 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.447 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2023-11-06T01:49:47.447 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2023-11-06T01:49:47.447 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2023-11-06T01:49:47.448 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.448 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2023-11-06T01:49:47.448 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2023-11-06T01:49:47.448 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2023-11-06T01:49:47.448 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4776, 2023-11-06T01:49:47.449 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 5.045342255, 2023-11-06T01:49:47.449 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001056394 2023-11-06T01:49:47.449 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:47.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.449 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2023-11-06T01:49:47.450 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2023-11-06T01:49:47.450 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2023-11-06T01:49:47.450 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2, 2023-11-06T01:49:47.451 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000016533, 2023-11-06T01:49:47.451 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000008266 2023-11-06T01:49:47.451 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:47.451 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.451 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2023-11-06T01:49:47.452 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 6618, 2023-11-06T01:49:47.452 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 6616, 2023-11-06T01:49:47.452 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2023-11-06T01:49:47.452 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6616, 2023-11-06T01:49:47.453 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 37.252124968, 2023-11-06T01:49:47.453 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.005630611 2023-11-06T01:49:47.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.453 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2023-11-06T01:49:47.453 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 2, 2023-11-06T01:49:47.454 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2023-11-06T01:49:47.454 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2023-11-06T01:49:47.454 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 11, 2023-11-06T01:49:47.454 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2023-11-06T01:49:47.454 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2023-11-06T01:49:47.455 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 911, 2023-11-06T01:49:47.455 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2023-11-06T01:49:47.455 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 867, 2023-11-06T01:49:47.455 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 44, 2023-11-06T01:49:47.455 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 910, 2023-11-06T01:49:47.456 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 71, 2023-11-06T01:49:47.456 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 66, 2023-11-06T01:49:47.456 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 66, 2023-11-06T01:49:47.456 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 5, 2023-11-06T01:49:47.456 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 24619, 2023-11-06T01:49:47.457 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 16388, 2023-11-06T01:49:47.457 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 1, 2023-11-06T01:49:47.457 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 4, 2023-11-06T01:49:47.457 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2023-11-06T01:49:47.457 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2023-11-06T01:49:47.458 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 4782, 2023-11-06T01:49:47.458 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 9134, 2023-11-06T01:49:47.458 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 1, 2023-11-06T01:49:47.458 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 3, 2023-11-06T01:49:47.458 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 70, 2023-11-06T01:49:47.459 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 5, 2023-11-06T01:49:47.459 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 115, 2023-11-06T01:49:47.459 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2023-11-06T01:49:47.459 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 118, 2023-11-06T01:49:47.459 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2023-11-06T01:49:47.460 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 1039, 2023-11-06T01:49:47.460 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 1693659912, 2023-11-06T01:49:47.460 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2023-11-06T01:49:47.460 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 118, 2023-11-06T01:49:47.460 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2023-11-06T01:49:47.461 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T01:49:47.461 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2023-11-06T01:49:47.461 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 1, 2023-11-06T01:49:47.462 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2023-11-06T01:49:47.462 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 1, 2023-11-06T01:49:47.462 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 117, 2023-11-06T01:49:47.462 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 10, 2023-11-06T01:49:47.462 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 792, 2023-11-06T01:49:47.463 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 1318, 2023-11-06T01:49:47.463 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T01:49:47.463 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T01:49:47.463 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T01:49:47.463 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 1994, 2023-11-06T01:49:47.464 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 725, 2023-11-06T01:49:47.464 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 68, 2023-11-06T01:49:47.464 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 2373 2023-11-06T01:49:47.464 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.464 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2023-11-06T01:49:47.465 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2023-11-06T01:49:47.465 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2023-11-06T01:49:47.465 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2023-11-06T01:49:47.465 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 6, 2023-11-06T01:49:47.465 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 7, 2023-11-06T01:49:47.465 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2023-11-06T01:49:47.466 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2023-11-06T01:49:47.466 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2023-11-06T01:49:47.466 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 580, 2023-11-06T01:49:47.466 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2023-11-06T01:49:47.467 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 2314, 2023-11-06T01:49:47.467 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2023-11-06T01:49:47.467 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2023-11-06T01:49:47.467 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2023-11-06T01:49:47.468 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2023-11-06T01:49:47.468 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2023-11-06T01:49:47.468 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2023-11-06T01:49:47.468 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2023-11-06T01:49:47.468 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 3, 2023-11-06T01:49:47.469 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2023-11-06T01:49:47.469 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2023-11-06T01:49:47.469 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2023-11-06T01:49:47.469 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2023-11-06T01:49:47.469 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.470 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2023-11-06T01:49:47.470 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 3210, 2023-11-06T01:49:47.470 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2023-11-06T01:49:47.470 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2023-11-06T01:49:47.471 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 3210, 2023-11-06T01:49:47.471 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2023-11-06T01:49:47.471 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2023-11-06T01:49:47.471 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 4, 2023-11-06T01:49:47.471 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2023-11-06T01:49:47.472 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2023-11-06T01:49:47.472 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 4, 2023-11-06T01:49:47.472 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2023-11-06T01:49:47.472 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2023-11-06T01:49:47.472 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2023-11-06T01:49:47.473 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 9953554, 2023-11-06T01:49:47.473 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2023-11-06T01:49:47.473 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2023-11-06T01:49:47.473 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2312, 2023-11-06T01:49:47.474 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 10.049775258, 2023-11-06T01:49:47.474 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004346788 2023-11-06T01:49:47.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.474 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2023-11-06T01:49:47.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.475 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2023-11-06T01:49:47.475 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 86, 2023-11-06T01:49:47.475 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 153, 2023-11-06T01:49:47.475 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 67, 2023-11-06T01:49:47.476 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 23, 2023-11-06T01:49:47.476 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 24, 2023-11-06T01:49:47.476 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 1, 2023-11-06T01:49:47.476 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 911, 2023-11-06T01:49:47.476 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 2082, 2023-11-06T01:49:47.477 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 1171, 2023-11-06T01:49:47.477 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 66, 2023-11-06T01:49:47.477 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 305, 2023-11-06T01:49:47.477 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 239, 2023-11-06T01:49:47.477 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 56296, 2023-11-06T01:49:47.478 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 198964 2023-11-06T01:49:47.478 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.478 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2023-11-06T01:49:47.478 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 15100, 2023-11-06T01:49:47.478 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 4, 2023-11-06T01:49:47.479 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 6618, 2023-11-06T01:49:47.479 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 1121, 2023-11-06T01:49:47.479 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 50, 2023-11-06T01:49:47.479 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2023-11-06T01:49:47.479 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 30, 2023-11-06T01:49:47.480 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.038499775, 2023-11-06T01:49:47.480 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001283325 2023-11-06T01:49:47.480 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.480 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2023-11-06T01:49:47.480 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4, 2023-11-06T01:49:47.481 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.010688327, 2023-11-06T01:49:47.481 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002672081 2023-11-06T01:49:47.481 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.481 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2023-11-06T01:49:47.481 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.482 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.482 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.482 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.482 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2023-11-06T01:49:47.482 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 580, 2023-11-06T01:49:47.483 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.107500395, 2023-11-06T01:49:47.483 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.003633621 2023-11-06T01:49:47.483 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.483 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2023-11-06T01:49:47.483 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2994, 2023-11-06T01:49:47.484 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 5.415888203, 2023-11-06T01:49:47.484 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001808913 2023-11-06T01:49:47.484 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.484 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2023-11-06T01:49:47.484 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.484 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.485 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.485 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.485 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2023-11-06T01:49:47.485 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.486 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.486 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.486 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2023-11-06T01:49:47.486 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.487 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.487 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.487 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.487 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2023-11-06T01:49:47.487 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.488 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.488 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.488 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.488 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2023-11-06T01:49:47.488 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.489 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.489 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.489 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.489 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2023-11-06T01:49:47.489 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.489 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.490 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.490 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.490 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2023-11-06T01:49:47.490 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.491 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.491 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.491 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.491 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2023-11-06T01:49:47.491 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.492 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.492 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.492 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.492 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2023-11-06T01:49:47.492 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.493 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.493 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.493 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.493 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2023-11-06T01:49:47.493 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 89, 2023-11-06T01:49:47.494 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.215376904, 2023-11-06T01:49:47.494 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002419965 2023-11-06T01:49:47.494 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.494 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2023-11-06T01:49:47.494 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 25, 2023-11-06T01:49:47.495 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.076593199, 2023-11-06T01:49:47.495 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.003063727 2023-11-06T01:49:47.495 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.495 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2023-11-06T01:49:47.495 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2314, 2023-11-06T01:49:47.496 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 27.648189080, 2023-11-06T01:49:47.496 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.011948223 2023-11-06T01:49:47.496 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.496 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2023-11-06T01:49:47.496 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.497 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.497 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.497 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.497 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2023-11-06T01:49:47.497 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.498 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.498 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.498 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.498 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2023-11-06T01:49:47.498 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.499 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.499 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.499 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.499 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2023-11-06T01:49:47.499 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.500 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.500 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.500 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.500 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2023-11-06T01:49:47.500 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.501 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.501 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.501 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.501 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2023-11-06T01:49:47.501 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.501 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.502 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.502 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.502 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2023-11-06T01:49:47.502 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.503 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.503 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.503 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.503 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2023-11-06T01:49:47.503 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.504 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.504 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.504 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.504 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2023-11-06T01:49:47.504 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.505 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.505 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.505 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.505 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2023-11-06T01:49:47.505 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.506 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.506 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.506 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.506 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2023-11-06T01:49:47.507 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.507 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.507 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.507 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.507 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2023-11-06T01:49:47.508 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 580, 2023-11-06T01:49:47.508 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.739389085, 2023-11-06T01:49:47.508 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002998946 2023-11-06T01:49:47.508 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.508 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2023-11-06T01:49:47.509 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2023-11-06T01:49:47.509 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2023-11-06T01:49:47.509 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.509 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.509 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.510 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:47.510 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.510 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2023-11-06T01:49:47.510 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2023-11-06T01:49:47.511 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 1, 2023-11-06T01:49:47.511 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 0, 2023-11-06T01:49:47.511 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2023-11-06T01:49:47.511 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2023-11-06T01:49:47.511 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 428, 2023-11-06T01:49:47.512 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 428, 2023-11-06T01:49:47.512 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 953, 2023-11-06T01:49:47.512 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2023-11-06T01:49:47.512 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.512 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2023-11-06T01:49:47.513 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2023-11-06T01:49:47.513 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2023-11-06T01:49:47.513 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:49:47.513 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:49:47.513 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:49:47.514 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:49:47.514 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:49:47.514 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:49:47.514 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:49:47.514 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:49:47.515 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:49:47.515 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:49:47.515 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:49:47.515 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:49:47.516 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:49:47.516 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2023-11-06T01:49:47.516 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:49:47.516 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2023-11-06T01:49:47.516 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:49:47.517 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:49:47.517 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:49:47.517 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:49:47.517 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:49:47.517 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:49:47.518 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:49:47.518 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2023-11-06T01:49:47.518 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:49:47.518 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:49:47.518 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:49:47.519 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2023-11-06T01:49:47.519 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2023-11-06T01:49:47.519 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2023-11-06T01:49:47.519 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:49:47.519 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:49:47.520 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:49:47.520 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:49:47.520 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 292096, 2023-11-06T01:49:47.520 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 179, 2023-11-06T01:49:47.520 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2023-11-06T01:49:47.521 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2023-11-06T01:49:47.521 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2023-11-06T01:49:47.521 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2023-11-06T01:49:47.521 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:49:47.521 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2023-11-06T01:49:47.522 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:49:47.522 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2023-11-06T01:49:47.522 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 7720, 2023-11-06T01:49:47.522 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 162, 2023-11-06T01:49:47.522 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:49:47.523 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:49:47.523 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2023-11-06T01:49:47.523 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2023-11-06T01:49:47.523 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 957253, 2023-11-06T01:49:47.523 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 33880, 2023-11-06T01:49:47.523 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2023-11-06T01:49:47.524 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2023-11-06T01:49:47.524 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2023-11-06T01:49:47.524 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2023-11-06T01:49:47.524 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.524 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2023-11-06T01:49:47.525 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2023-11-06T01:49:47.525 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2023-11-06T01:49:47.525 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 2687, 2023-11-06T01:49:47.525 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 5835798, 2023-11-06T01:49:47.525 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 33, 2023-11-06T01:49:47.526 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 2687, 2023-11-06T01:49:47.526 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2023-11-06T01:49:47.526 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2687, 2023-11-06T01:49:47.526 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 12.723801027, 2023-11-06T01:49:47.526 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004735318 2023-11-06T01:49:47.527 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.527 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2023-11-06T01:49:47.527 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2023-11-06T01:49:47.527 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2687, 2023-11-06T01:49:47.527 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2901 2023-11-06T01:49:47.528 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.528 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 2687, 2023-11-06T01:49:47.528 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 118, 2023-11-06T01:49:47.528 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 2569, 2023-11-06T01:49:47.528 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2023-11-06T01:49:47.529 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2023-11-06T01:49:47.529 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 0, 2023-11-06T01:49:47.529 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 10, 2023-11-06T01:49:47.529 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 0, 2023-11-06T01:49:47.529 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 2312, 2023-11-06T01:49:47.530 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 65, 2023-11-06T01:49:47.530 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2023-11-06T01:49:47.530 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2023-11-06T01:49:47.530 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2023-11-06T01:49:47.530 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2023-11-06T01:49:47.531 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 7, 2023-11-06T01:49:47.531 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2023-11-06T01:49:47.531 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2023-11-06T01:49:47.531 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2023-11-06T01:49:47.531 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 118, 2023-11-06T01:49:47.532 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 20, 2023-11-06T01:49:47.532 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2023-11-06T01:49:47.532 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2023-11-06T01:49:47.532 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2023-11-06T01:49:47.532 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2023-11-06T01:49:47.533 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2023-11-06T01:49:47.533 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2023-11-06T01:49:47.533 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T01:49:47.533 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2023-11-06T01:49:47.534 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2023-11-06T01:49:47.534 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2023-11-06T01:49:47.534 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2023-11-06T01:49:47.534 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2023-11-06T01:49:47.534 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2023-11-06T01:49:47.535 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2023-11-06T01:49:47.535 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2023-11-06T01:49:47.535 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2023-11-06T01:49:47.535 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2023-11-06T01:49:47.535 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2023-11-06T01:49:47.536 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2023-11-06T01:49:47.536 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2023-11-06T01:49:47.536 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2023-11-06T01:49:47.536 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2023-11-06T01:49:47.536 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2023-11-06T01:49:47.537 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2023-11-06T01:49:47.537 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2023-11-06T01:49:47.537 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2023-11-06T01:49:47.537 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 84, 2023-11-06T01:49:47.537 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2023-11-06T01:49:47.538 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 2, 2023-11-06T01:49:47.538 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 12, 2023-11-06T01:49:47.538 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 12, 2023-11-06T01:49:47.538 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2023-11-06T01:49:47.538 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2023-11-06T01:49:47.539 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 220, 2023-11-06T01:49:47.539 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 0, 2023-11-06T01:49:47.539 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 149 2023-11-06T01:49:47.539 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.539 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2023-11-06T01:49:47.540 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2023-11-06T01:49:47.540 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2023-11-06T01:49:47.540 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2023-11-06T01:49:47.540 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2023-11-06T01:49:47.540 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.541 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2023-11-06T01:49:47.541 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 6625 2023-11-06T01:49:47.541 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.541 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2023-11-06T01:49:47.541 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2023-11-06T01:49:47.542 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2023-11-06T01:49:47.542 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2023-11-06T01:49:47.542 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2023-11-06T01:49:47.542 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2023-11-06T01:49:47.542 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2023-11-06T01:49:47.543 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.543 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:49:47.543 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:47.543 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-11-06T01:49:47.543 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:47.544 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 15221, 2023-11-06T01:49:47.544 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 3597194, 2023-11-06T01:49:47.544 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:47.544 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 15221, 2023-11-06T01:49:47.544 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:47.545 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:47.545 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 15221, 2023-11-06T01:49:47.545 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 3597194, 2023-11-06T01:49:47.545 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:47.545 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.546 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.546 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.546 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:47.546 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.546 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2023-11-06T01:49:47.547 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:47.547 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-11-06T01:49:47.547 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:47.547 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:47.547 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:47.547 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:47.548 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:47.548 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 2687, 2023-11-06T01:49:47.548 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 5835798, 2023-11-06T01:49:47.548 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2687, 2023-11-06T01:49:47.548 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 5835798, 2023-11-06T01:49:47.549 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:47.549 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.549 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.549 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.549 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:47.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.550 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2023-11-06T01:49:47.550 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:47.550 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2023-11-06T01:49:47.551 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:47.551 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:47.551 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:47.551 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:47.551 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:47.551 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 2687, 2023-11-06T01:49:47.552 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 2687, 2023-11-06T01:49:47.552 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2687, 2023-11-06T01:49:47.552 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2687, 2023-11-06T01:49:47.552 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:47.553 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.553 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.553 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.553 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:47.553 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.554 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2023-11-06T01:49:47.554 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:47.554 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2023-11-06T01:49:47.554 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:47.554 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:47.555 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:47.555 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:47.555 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:47.555 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:47.556 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:47.556 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2023-11-06T01:49:47.556 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2023-11-06T01:49:47.556 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:47.556 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.556 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.557 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.557 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:47.557 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:47.557 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x555cb99720a0": { 2023-11-06T01:49:47.557 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 9408, 2023-11-06T01:49:47.558 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2023-11-06T01:49:47.558 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:47.558 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 3210, 2023-11-06T01:49:47.558 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 5759250, 2023-11-06T01:49:47.559 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:47.559 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 3210, 2023-11-06T01:49:47.559 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:47.559 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:47.559 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2312, 2023-11-06T01:49:47.560 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 5749842, 2023-11-06T01:49:47.560 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:47.560 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:47.560 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:47.560 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:47.561 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:47.561 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:47.561 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-11-06T01:49:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:47 smithi163 ceph-mon[104416]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 KiB/s wr, 15 op/s 2023-11-06T01:49:47.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:47 smithi053 ceph-mon[99472]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 KiB/s wr, 15 op/s 2023-11-06T01:49:47.745 INFO:tasks.check_counter:Daemon mds.e mds.exported=3 2023-11-06T01:49:47.746 INFO:tasks.check_counter:Daemon mds.e mds.imported=5 2023-11-06T01:49:47.746 DEBUG:tasks.check_counter:Getting stats from h 2023-11-06T01:49:47.746 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-11-06T01:49:48.354 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:49:49.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:49 smithi062 ceph-mon[103924]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-11-06T01:49:49.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:49 smithi163 ceph-mon[104416]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-11-06T01:49:49.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:49 smithi053 ceph-mon[99472]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-11-06T01:49:51.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:51 smithi163 ceph-mon[104416]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2023-11-06T01:49:51.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:51 smithi053 ceph-mon[99472]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2023-11-06T01:49:51.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:51 smithi062 ceph-mon[103924]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2023-11-06T01:49:52.387 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-11-06T01:49:52.388 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:49:52.388 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 3238, 2023-11-06T01:49:52.388 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 3239, 2023-11-06T01:49:52.388 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 2286026, 2023-11-06T01:49:52.389 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 1006714, 2023-11-06T01:49:52.389 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2023-11-06T01:49:52.389 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2023-11-06T01:49:52.389 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.243858452, 2023-11-06T01:49:52.389 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.060682096, 2023-11-06T01:49:52.390 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.061558715, 2023-11-06T01:49:52.390 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.104106987, 2023-11-06T01:49:52.390 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:52.390 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3237, 2023-11-06T01:49:52.390 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.543171336, 2023-11-06T01:49:52.391 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000167800 2023-11-06T01:49:52.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.391 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:52.391 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.391 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.392 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.392 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.392 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-06T01:49:52.392 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2023-11-06T01:49:52.392 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.393 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:49:52.393 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 327, 2023-11-06T01:49:52.393 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 294, 2023-11-06T01:49:52.393 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 669292, 2023-11-06T01:49:52.393 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 164300, 2023-11-06T01:49:52.393 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 3, 2023-11-06T01:49:52.394 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2023-11-06T01:49:52.394 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.042018620, 2023-11-06T01:49:52.394 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.013868310, 2023-11-06T01:49:52.394 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.019687524, 2023-11-06T01:49:52.394 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.008782594, 2023-11-06T01:49:52.395 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:52.395 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 294, 2023-11-06T01:49:52.395 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.071531944, 2023-11-06T01:49:52.395 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000243305 2023-11-06T01:49:52.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.396 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:52.396 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.396 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.396 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.396 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.396 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 668736, 2023-11-06T01:49:52.397 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 163616 2023-11-06T01:49:52.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.397 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:49:52.397 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 3, 2023-11-06T01:49:52.397 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 217, 2023-11-06T01:49:52.398 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 588, 2023-11-06T01:49:52.398 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 534423, 2023-11-06T01:49:52.398 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2023-11-06T01:49:52.398 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2023-11-06T01:49:52.398 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.011218964, 2023-11-06T01:49:52.399 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.007430053, 2023-11-06T01:49:52.399 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001875250, 2023-11-06T01:49:52.399 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000032531, 2023-11-06T01:49:52.399 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:52.399 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 215, 2023-11-06T01:49:52.399 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.090677568, 2023-11-06T01:49:52.400 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000421756 2023-11-06T01:49:52.400 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.400 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:52.400 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.400 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.401 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.401 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.401 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-06T01:49:52.401 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 534112 2023-11-06T01:49:52.402 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.402 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2023-11-06T01:49:52.402 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2023-11-06T01:49:52.402 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2023-11-06T01:49:52.402 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.403 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2023-11-06T01:49:52.403 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2023-11-06T01:49:52.403 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2023-11-06T01:49:52.403 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 5377, 2023-11-06T01:49:52.403 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.139599027, 2023-11-06T01:49:52.404 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000025962 2023-11-06T01:49:52.404 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:52.404 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.404 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2023-11-06T01:49:52.404 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2023-11-06T01:49:52.405 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2023-11-06T01:49:52.405 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.405 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.405 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.405 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:52.406 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.406 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2023-11-06T01:49:52.406 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2023-11-06T01:49:52.406 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2023-11-06T01:49:52.406 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2023-11-06T01:49:52.407 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.407 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.407 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.407 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.408 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2023-11-06T01:49:52.408 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2023-11-06T01:49:52.408 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2023-11-06T01:49:52.408 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2023-11-06T01:49:52.408 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2023-11-06T01:49:52.409 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2023-11-06T01:49:52.409 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2023-11-06T01:49:52.409 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 293, 2023-11-06T01:49:52.409 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 2, 2023-11-06T01:49:52.409 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 124, 2023-11-06T01:49:52.410 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 167, 2023-11-06T01:49:52.410 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 290, 2023-11-06T01:49:52.410 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 3, 2023-11-06T01:49:52.410 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2023-11-06T01:49:52.411 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2023-11-06T01:49:52.411 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 8, 2023-11-06T01:49:52.411 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2023-11-06T01:49:52.411 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2023-11-06T01:49:52.411 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2023-11-06T01:49:52.412 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2023-11-06T01:49:52.412 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2023-11-06T01:49:52.412 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2023-11-06T01:49:52.412 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2023-11-06T01:49:52.412 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2023-11-06T01:49:52.412 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2023-11-06T01:49:52.413 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2023-11-06T01:49:52.413 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2023-11-06T01:49:52.413 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2023-11-06T01:49:52.413 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2023-11-06T01:49:52.413 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2023-11-06T01:49:52.414 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2023-11-06T01:49:52.414 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2023-11-06T01:49:52.414 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2023-11-06T01:49:52.414 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2023-11-06T01:49:52.415 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2023-11-06T01:49:52.415 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2023-11-06T01:49:52.415 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2023-11-06T01:49:52.415 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T01:49:52.415 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2023-11-06T01:49:52.416 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2023-11-06T01:49:52.416 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2023-11-06T01:49:52.416 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2023-11-06T01:49:52.416 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2023-11-06T01:49:52.416 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2023-11-06T01:49:52.417 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2023-11-06T01:49:52.417 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2023-11-06T01:49:52.417 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T01:49:52.417 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T01:49:52.417 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T01:49:52.417 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2023-11-06T01:49:52.418 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2023-11-06T01:49:52.418 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2023-11-06T01:49:52.418 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2023-11-06T01:49:52.418 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.418 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2023-11-06T01:49:52.419 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2023-11-06T01:49:52.419 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2023-11-06T01:49:52.419 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2023-11-06T01:49:52.419 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2023-11-06T01:49:52.419 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2023-11-06T01:49:52.420 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2023-11-06T01:49:52.420 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2023-11-06T01:49:52.420 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2023-11-06T01:49:52.420 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2023-11-06T01:49:52.421 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2023-11-06T01:49:52.421 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2023-11-06T01:49:52.421 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2023-11-06T01:49:52.421 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2023-11-06T01:49:52.421 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2023-11-06T01:49:52.421 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2023-11-06T01:49:52.422 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2023-11-06T01:49:52.422 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2023-11-06T01:49:52.422 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2023-11-06T01:49:52.422 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2023-11-06T01:49:52.423 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2023-11-06T01:49:52.423 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2023-11-06T01:49:52.423 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2023-11-06T01:49:52.423 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2023-11-06T01:49:52.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.423 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2023-11-06T01:49:52.424 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2023-11-06T01:49:52.424 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2023-11-06T01:49:52.424 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2023-11-06T01:49:52.424 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 1020, 2023-11-06T01:49:52.424 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2023-11-06T01:49:52.425 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2023-11-06T01:49:52.425 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2023-11-06T01:49:52.425 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2023-11-06T01:49:52.425 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2023-11-06T01:49:52.425 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 1, 2023-11-06T01:49:52.426 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2023-11-06T01:49:52.426 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2023-11-06T01:49:52.426 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2023-11-06T01:49:52.426 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 5574019, 2023-11-06T01:49:52.426 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 5573445, 2023-11-06T01:49:52.427 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2023-11-06T01:49:52.427 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.427 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.427 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.428 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 1020 2023-11-06T01:49:52.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.428 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2023-11-06T01:49:52.428 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 236, 2023-11-06T01:49:52.429 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 266, 2023-11-06T01:49:52.429 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 30, 2023-11-06T01:49:52.429 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 16, 2023-11-06T01:49:52.429 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 20, 2023-11-06T01:49:52.429 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 4, 2023-11-06T01:49:52.430 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 293, 2023-11-06T01:49:52.430 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 300, 2023-11-06T01:49:52.430 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 7, 2023-11-06T01:49:52.430 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2023-11-06T01:49:52.431 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2023-11-06T01:49:52.431 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2023-11-06T01:49:52.431 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 51444, 2023-11-06T01:49:52.431 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 207156 2023-11-06T01:49:52.431 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.432 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2023-11-06T01:49:52.432 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2023-11-06T01:49:52.432 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2023-11-06T01:49:52.432 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2023-11-06T01:49:52.432 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2023-11-06T01:49:52.432 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2023-11-06T01:49:52.433 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2023-11-06T01:49:52.433 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.433 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.433 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.434 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2023-11-06T01:49:52.434 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.434 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.434 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.435 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.435 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2023-11-06T01:49:52.435 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.435 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.435 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.435 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.436 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2023-11-06T01:49:52.436 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.436 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.436 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.436 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.437 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2023-11-06T01:49:52.437 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.437 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.437 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.438 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.438 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2023-11-06T01:49:52.438 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.438 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.438 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.438 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.439 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2023-11-06T01:49:52.439 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.439 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.439 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.439 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.440 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2023-11-06T01:49:52.440 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.440 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.440 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.440 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.441 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2023-11-06T01:49:52.441 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.441 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.441 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.442 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2023-11-06T01:49:52.442 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.442 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.442 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.442 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.443 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2023-11-06T01:49:52.443 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.443 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.443 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.443 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.444 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2023-11-06T01:49:52.444 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.444 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.444 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.444 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.445 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2023-11-06T01:49:52.445 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.445 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.445 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.445 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.446 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2023-11-06T01:49:52.446 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.446 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.446 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.447 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.447 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2023-11-06T01:49:52.447 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.447 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.447 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.448 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.448 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2023-11-06T01:49:52.448 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.448 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.448 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.449 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2023-11-06T01:49:52.449 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.449 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.449 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.450 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.450 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2023-11-06T01:49:52.450 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.450 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.450 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.451 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.451 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2023-11-06T01:49:52.451 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.451 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.451 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.452 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.452 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2023-11-06T01:49:52.452 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.452 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.452 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.453 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2023-11-06T01:49:52.453 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.453 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.453 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.454 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2023-11-06T01:49:52.454 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.454 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.454 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.455 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.455 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2023-11-06T01:49:52.455 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.455 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.455 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.456 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.456 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2023-11-06T01:49:52.456 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.456 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.456 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.457 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2023-11-06T01:49:52.457 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.457 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.457 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.458 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.458 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2023-11-06T01:49:52.458 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.458 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.458 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.459 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.459 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2023-11-06T01:49:52.459 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.459 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.459 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.460 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.460 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2023-11-06T01:49:52.460 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.460 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.460 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.461 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.461 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2023-11-06T01:49:52.461 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.461 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.462 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.462 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.462 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2023-11-06T01:49:52.462 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2023-11-06T01:49:52.462 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2023-11-06T01:49:52.463 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.463 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.463 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.463 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:52.464 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.464 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2023-11-06T01:49:52.464 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2023-11-06T01:49:52.464 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 2, 2023-11-06T01:49:52.464 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 1, 2023-11-06T01:49:52.464 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2023-11-06T01:49:52.465 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2023-11-06T01:49:52.465 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2023-11-06T01:49:52.465 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2023-11-06T01:49:52.465 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 2388, 2023-11-06T01:49:52.465 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2023-11-06T01:49:52.466 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.466 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2023-11-06T01:49:52.466 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2023-11-06T01:49:52.466 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2023-11-06T01:49:52.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:49:52.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:49:52.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:49:52.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:49:52.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:49:52.468 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:49:52.468 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:49:52.468 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:49:52.468 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:49:52.468 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:49:52.469 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:49:52.469 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:49:52.469 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:49:52.469 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2023-11-06T01:49:52.469 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:49:52.470 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2023-11-06T01:49:52.470 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:49:52.470 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:49:52.470 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:49:52.471 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:49:52.471 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:49:52.471 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:49:52.471 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:49:52.471 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2023-11-06T01:49:52.471 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:49:52.472 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:49:52.472 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:49:52.472 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2023-11-06T01:49:52.472 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2023-11-06T01:49:52.473 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2023-11-06T01:49:52.473 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:49:52.473 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:49:52.473 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:49:52.473 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:49:52.474 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 189741, 2023-11-06T01:49:52.474 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 274, 2023-11-06T01:49:52.474 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2023-11-06T01:49:52.474 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2023-11-06T01:49:52.474 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2023-11-06T01:49:52.475 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2023-11-06T01:49:52.475 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:49:52.475 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2023-11-06T01:49:52.475 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:49:52.475 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2023-11-06T01:49:52.476 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 7720, 2023-11-06T01:49:52.476 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 162, 2023-11-06T01:49:52.476 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:49:52.476 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:49:52.476 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2023-11-06T01:49:52.477 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2023-11-06T01:49:52.477 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 749886, 2023-11-06T01:49:52.477 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 6622, 2023-11-06T01:49:52.477 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2023-11-06T01:49:52.477 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2023-11-06T01:49:52.478 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2023-11-06T01:49:52.478 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2023-11-06T01:49:52.478 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.478 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2023-11-06T01:49:52.479 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2023-11-06T01:49:52.479 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2023-11-06T01:49:52.479 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 3240, 2023-11-06T01:49:52.479 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 21, 2023-11-06T01:49:52.479 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 3, 2023-11-06T01:49:52.480 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 3240, 2023-11-06T01:49:52.480 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2023-11-06T01:49:52.480 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3240, 2023-11-06T01:49:52.480 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 20.705679312, 2023-11-06T01:49:52.480 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.006390641 2023-11-06T01:49:52.480 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.481 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2023-11-06T01:49:52.481 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2023-11-06T01:49:52.481 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3240, 2023-11-06T01:49:52.481 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3241 2023-11-06T01:49:52.481 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.482 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 3240, 2023-11-06T01:49:52.482 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 3240, 2023-11-06T01:49:52.482 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2023-11-06T01:49:52.482 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2023-11-06T01:49:52.482 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2023-11-06T01:49:52.483 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 1971, 2023-11-06T01:49:52.483 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2023-11-06T01:49:52.483 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 1267, 2023-11-06T01:49:52.483 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2023-11-06T01:49:52.483 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2023-11-06T01:49:52.483 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2023-11-06T01:49:52.484 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2023-11-06T01:49:52.484 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2023-11-06T01:49:52.484 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2023-11-06T01:49:52.484 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2023-11-06T01:49:52.485 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2023-11-06T01:49:52.485 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2023-11-06T01:49:52.485 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2023-11-06T01:49:52.485 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1, 2023-11-06T01:49:52.485 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2023-11-06T01:49:52.486 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2023-11-06T01:49:52.486 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2023-11-06T01:49:52.486 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2023-11-06T01:49:52.486 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2023-11-06T01:49:52.486 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2023-11-06T01:49:52.487 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2023-11-06T01:49:52.487 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T01:49:52.487 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2023-11-06T01:49:52.487 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2023-11-06T01:49:52.487 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2023-11-06T01:49:52.488 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2023-11-06T01:49:52.488 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2023-11-06T01:49:52.488 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2023-11-06T01:49:52.488 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2023-11-06T01:49:52.488 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2023-11-06T01:49:52.489 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2023-11-06T01:49:52.489 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2023-11-06T01:49:52.489 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2023-11-06T01:49:52.489 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2023-11-06T01:49:52.489 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2023-11-06T01:49:52.490 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2023-11-06T01:49:52.490 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2023-11-06T01:49:52.490 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2023-11-06T01:49:52.490 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2023-11-06T01:49:52.490 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2023-11-06T01:49:52.491 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2023-11-06T01:49:52.491 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 84, 2023-11-06T01:49:52.491 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2023-11-06T01:49:52.491 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 2, 2023-11-06T01:49:52.491 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 4, 2023-11-06T01:49:52.492 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 5, 2023-11-06T01:49:52.492 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 1, 2023-11-06T01:49:52.492 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2023-11-06T01:49:52.492 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2023-11-06T01:49:52.492 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2023-11-06T01:49:52.492 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2023-11-06T01:49:52.493 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.493 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2023-11-06T01:49:52.493 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2023-11-06T01:49:52.493 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2023-11-06T01:49:52.493 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2023-11-06T01:49:52.494 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2023-11-06T01:49:52.494 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.494 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2023-11-06T01:49:52.494 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2023-11-06T01:49:52.494 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.495 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2023-11-06T01:49:52.495 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2023-11-06T01:49:52.495 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2023-11-06T01:49:52.495 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2023-11-06T01:49:52.495 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2023-11-06T01:49:52.495 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2023-11-06T01:49:52.496 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2023-11-06T01:49:52.496 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.496 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:49:52.496 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:52.496 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-11-06T01:49:52.497 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:52.497 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 3564, 2023-11-06T01:49:52.497 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 2615749, 2023-11-06T01:49:52.497 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:52.497 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 3564, 2023-11-06T01:49:52.498 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:52.498 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:52.498 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3564, 2023-11-06T01:49:52.498 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2615749, 2023-11-06T01:49:52.498 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:52.499 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.499 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.499 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.499 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:52.499 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.500 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2023-11-06T01:49:52.500 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:52.500 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-11-06T01:49:52.500 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:52.500 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:52.501 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:52.501 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:52.501 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:52.501 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3240, 2023-11-06T01:49:52.502 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 1379720, 2023-11-06T01:49:52.502 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3240, 2023-11-06T01:49:52.502 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1379720, 2023-11-06T01:49:52.502 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:52.502 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.503 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.503 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.503 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:52.503 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.503 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2023-11-06T01:49:52.504 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:52.504 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2023-11-06T01:49:52.504 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:52.504 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:52.504 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:52.505 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:52.505 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:52.505 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3240, 2023-11-06T01:49:52.505 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 3240, 2023-11-06T01:49:52.505 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3240, 2023-11-06T01:49:52.506 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 3240, 2023-11-06T01:49:52.506 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:52.506 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.506 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.507 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.507 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:52.507 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.507 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2023-11-06T01:49:52.507 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:52.508 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2023-11-06T01:49:52.508 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:52.508 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:52.508 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:52.508 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:52.509 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:52.509 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:52.509 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:52.509 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2023-11-06T01:49:52.509 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2023-11-06T01:49:52.510 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:52.510 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.510 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.510 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.510 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:52.511 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:52.511 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x56285a9125a0": { 2023-11-06T01:49:52.511 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:52.511 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2023-11-06T01:49:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2023-11-06T01:49:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2023-11-06T01:49:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:52.514 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:52.514 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:52.514 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:52.514 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:52.514 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-11-06T01:49:52.723 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:52 smithi062 ceph-mon[103924]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 17 op/s 2023-11-06T01:49:52.757 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-11-06T01:49:52.758 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-11-06T01:49:52.758 DEBUG:tasks.check_counter:Getting stats from k 2023-11-06T01:49:52.759 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-11-06T01:49:52.842 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:52 smithi053 ceph-mon[99472]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 17 op/s 2023-11-06T01:49:52.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:52 smithi163 ceph-mon[104416]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 17 op/s 2023-11-06T01:49:53.497 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.b/config 2023-11-06T01:49:55.172 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-11-06T01:49:55.172 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:49:55.172 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 638641, 2023-11-06T01:49:55.172 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 639222, 2023-11-06T01:49:55.173 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 104266856, 2023-11-06T01:49:55.173 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 103349610, 2023-11-06T01:49:55.173 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 17, 2023-11-06T01:49:55.173 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 7, 2023-11-06T01:49:55.174 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 33.508527112, 2023-11-06T01:49:55.174 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 14.316520280, 2023-11-06T01:49:55.174 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 23.761609567, 2023-11-06T01:49:55.174 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.011064201, 2023-11-06T01:49:55.174 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:55.175 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 639216, 2023-11-06T01:49:55.175 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 10.981893227, 2023-11-06T01:49:55.175 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000017180 2023-11-06T01:49:55.175 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.175 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:55.176 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1276852, 2023-11-06T01:49:55.176 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.166116338, 2023-11-06T01:49:55.176 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000130 2023-11-06T01:49:55.176 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.177 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-06T01:49:55.177 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2023-11-06T01:49:55.177 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.177 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:49:55.177 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2570575, 2023-11-06T01:49:55.177 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2570426, 2023-11-06T01:49:55.178 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 411880247, 2023-11-06T01:49:55.178 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 412446320, 2023-11-06T01:49:55.178 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 10, 2023-11-06T01:49:55.178 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 5, 2023-11-06T01:49:55.178 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 94.068941315, 2023-11-06T01:49:55.179 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 35.375418377, 2023-11-06T01:49:55.179 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 66.316413049, 2023-11-06T01:49:55.179 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.005818905, 2023-11-06T01:49:55.179 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:55.179 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2570429, 2023-11-06T01:49:55.180 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 62.715191401, 2023-11-06T01:49:55.180 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000024398 2023-11-06T01:49:55.180 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.180 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:55.180 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 5140195, 2023-11-06T01:49:55.181 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.575634668, 2023-11-06T01:49:55.181 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000111 2023-11-06T01:49:55.181 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.181 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-11-06T01:49:55.181 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 542752 2023-11-06T01:49:55.182 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.182 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:49:55.182 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 547, 2023-11-06T01:49:55.182 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 522, 2023-11-06T01:49:55.182 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 836625, 2023-11-06T01:49:55.183 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 374141, 2023-11-06T01:49:55.183 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 10, 2023-11-06T01:49:55.183 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 8, 2023-11-06T01:49:55.183 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.056551638, 2023-11-06T01:49:55.183 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.019606490, 2023-11-06T01:49:55.184 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.028574841, 2023-11-06T01:49:55.184 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.009958483, 2023-11-06T01:49:55.184 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:55.184 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 521, 2023-11-06T01:49:55.184 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.237075161, 2023-11-06T01:49:55.185 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000455038 2023-11-06T01:49:55.185 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.185 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:55.185 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 165, 2023-11-06T01:49:55.185 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000040800, 2023-11-06T01:49:55.186 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000247 2023-11-06T01:49:55.186 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.186 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 658288, 2023-11-06T01:49:55.186 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 170400 2023-11-06T01:49:55.186 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.187 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2023-11-06T01:49:55.187 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2023-11-06T01:49:55.187 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2023-11-06T01:49:55.187 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.187 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2023-11-06T01:49:55.188 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2023-11-06T01:49:55.188 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2023-11-06T01:49:55.188 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 648, 2023-11-06T01:49:55.188 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.179986576, 2023-11-06T01:49:55.188 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000277757 2023-11-06T01:49:55.189 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:55.189 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.189 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2023-11-06T01:49:55.189 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2023-11-06T01:49:55.189 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2023-11-06T01:49:55.190 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2, 2023-11-06T01:49:55.190 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000016102, 2023-11-06T01:49:55.190 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000008051 2023-11-06T01:49:55.190 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:55.190 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.191 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2023-11-06T01:49:55.191 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 214, 2023-11-06T01:49:55.191 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 211, 2023-11-06T01:49:55.191 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2023-11-06T01:49:55.191 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 211, 2023-11-06T01:49:55.192 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.171947713, 2023-11-06T01:49:55.192 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000814918 2023-11-06T01:49:55.192 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.192 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2023-11-06T01:49:55.193 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 3, 2023-11-06T01:49:55.193 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 1, 2023-11-06T01:49:55.193 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2023-11-06T01:49:55.193 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 11, 2023-11-06T01:49:55.194 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2023-11-06T01:49:55.194 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2023-11-06T01:49:55.194 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 427, 2023-11-06T01:49:55.194 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 6, 2023-11-06T01:49:55.195 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 3, 2023-11-06T01:49:55.195 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 418, 2023-11-06T01:49:55.195 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 418, 2023-11-06T01:49:55.195 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 332, 2023-11-06T01:49:55.195 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 395, 2023-11-06T01:49:55.195 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 395, 2023-11-06T01:49:55.196 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 16, 2023-11-06T01:49:55.196 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 385, 2023-11-06T01:49:55.196 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 346, 2023-11-06T01:49:55.196 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 3, 2023-11-06T01:49:55.196 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 5, 2023-11-06T01:49:55.197 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2023-11-06T01:49:55.197 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2023-11-06T01:49:55.197 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2023-11-06T01:49:55.197 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 793, 2023-11-06T01:49:55.198 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2023-11-06T01:49:55.198 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 9, 2023-11-06T01:49:55.198 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 347, 2023-11-06T01:49:55.198 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 7, 2023-11-06T01:49:55.199 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 745, 2023-11-06T01:49:55.199 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2023-11-06T01:49:55.199 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 174, 2023-11-06T01:49:55.199 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2023-11-06T01:49:55.199 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 1039, 2023-11-06T01:49:55.200 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 1693659912, 2023-11-06T01:49:55.200 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2023-11-06T01:49:55.200 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 174, 2023-11-06T01:49:55.200 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2023-11-06T01:49:55.200 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T01:49:55.201 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2023-11-06T01:49:55.201 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2023-11-06T01:49:55.201 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2023-11-06T01:49:55.201 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2023-11-06T01:49:55.201 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 174, 2023-11-06T01:49:55.202 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2023-11-06T01:49:55.202 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 5, 2023-11-06T01:49:55.202 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 53, 2023-11-06T01:49:55.202 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T01:49:55.203 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T01:49:55.203 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T01:49:55.203 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 139, 2023-11-06T01:49:55.203 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 113, 2023-11-06T01:49:55.203 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 12, 2023-11-06T01:49:55.204 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 21 2023-11-06T01:49:55.204 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.204 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2023-11-06T01:49:55.204 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2023-11-06T01:49:55.204 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2023-11-06T01:49:55.205 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2023-11-06T01:49:55.205 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 13, 2023-11-06T01:49:55.205 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 5, 2023-11-06T01:49:55.205 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2023-11-06T01:49:55.205 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2023-11-06T01:49:55.206 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2023-11-06T01:49:55.206 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2023-11-06T01:49:55.206 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2023-11-06T01:49:55.206 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2023-11-06T01:49:55.206 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2023-11-06T01:49:55.207 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2023-11-06T01:49:55.207 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2023-11-06T01:49:55.207 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2023-11-06T01:49:55.207 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2023-11-06T01:49:55.207 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2023-11-06T01:49:55.208 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2023-11-06T01:49:55.208 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 9, 2023-11-06T01:49:55.208 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2023-11-06T01:49:55.208 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2023-11-06T01:49:55.208 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2023-11-06T01:49:55.209 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2023-11-06T01:49:55.209 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.209 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2023-11-06T01:49:55.209 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 486, 2023-11-06T01:49:55.209 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2023-11-06T01:49:55.210 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2023-11-06T01:49:55.210 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 486, 2023-11-06T01:49:55.210 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2023-11-06T01:49:55.210 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2023-11-06T01:49:55.210 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 1, 2023-11-06T01:49:55.211 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2023-11-06T01:49:55.211 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2023-11-06T01:49:55.211 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 1, 2023-11-06T01:49:55.211 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2023-11-06T01:49:55.211 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2023-11-06T01:49:55.211 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2023-11-06T01:49:55.212 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 5420661, 2023-11-06T01:49:55.212 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2023-11-06T01:49:55.212 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2023-11-06T01:49:55.212 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 223, 2023-11-06T01:49:55.212 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 4.493886668, 2023-11-06T01:49:55.213 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.020151958 2023-11-06T01:49:55.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.213 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2023-11-06T01:49:55.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.213 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2023-11-06T01:49:55.214 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 430, 2023-11-06T01:49:55.214 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 761, 2023-11-06T01:49:55.214 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 331, 2023-11-06T01:49:55.214 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 45, 2023-11-06T01:49:55.214 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 46, 2023-11-06T01:49:55.215 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 1, 2023-11-06T01:49:55.215 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 427, 2023-11-06T01:49:55.215 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 771, 2023-11-06T01:49:55.215 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 344, 2023-11-06T01:49:55.215 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 395, 2023-11-06T01:49:55.216 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 928, 2023-11-06T01:49:55.216 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 533, 2023-11-06T01:49:55.216 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 56592, 2023-11-06T01:49:55.216 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 198964 2023-11-06T01:49:55.216 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.217 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2023-11-06T01:49:55.217 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 223, 2023-11-06T01:49:55.217 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 2, 2023-11-06T01:49:55.217 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 214, 2023-11-06T01:49:55.217 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 22, 2023-11-06T01:49:55.218 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 5, 2023-11-06T01:49:55.218 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2023-11-06T01:49:55.218 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2023-11-06T01:49:55.218 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000907693, 2023-11-06T01:49:55.218 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000907693 2023-11-06T01:49:55.219 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.219 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2023-11-06T01:49:55.219 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.219 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.219 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.220 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.220 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2023-11-06T01:49:55.220 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.220 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.220 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.221 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.221 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2023-11-06T01:49:55.221 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.221 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.221 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.222 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.222 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2023-11-06T01:49:55.222 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 166, 2023-11-06T01:49:55.222 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.116261806, 2023-11-06T01:49:55.222 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000700372 2023-11-06T01:49:55.223 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.223 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2023-11-06T01:49:55.223 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.223 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.223 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.224 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.224 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2023-11-06T01:49:55.224 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.224 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.224 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.225 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.225 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2023-11-06T01:49:55.225 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.225 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.225 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.226 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.226 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2023-11-06T01:49:55.226 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.226 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.226 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.227 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.227 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2023-11-06T01:49:55.227 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.227 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.227 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.228 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2023-11-06T01:49:55.228 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.228 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.228 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.229 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.229 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2023-11-06T01:49:55.229 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.229 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.230 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.230 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2023-11-06T01:49:55.230 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.230 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.231 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.231 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.231 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2023-11-06T01:49:55.231 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.231 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.232 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.232 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2023-11-06T01:49:55.232 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 22, 2023-11-06T01:49:55.232 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.020940224, 2023-11-06T01:49:55.233 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000951828 2023-11-06T01:49:55.233 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.233 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2023-11-06T01:49:55.233 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 22, 2023-11-06T01:49:55.233 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.033837990, 2023-11-06T01:49:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001538090 2023-11-06T01:49:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2023-11-06T01:49:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.235 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.235 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.235 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2023-11-06T01:49:55.235 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.235 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.237 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.237 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.237 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2023-11-06T01:49:55.238 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.238 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.238 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.238 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.238 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2023-11-06T01:49:55.239 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.239 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.239 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.239 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.239 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2023-11-06T01:49:55.240 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.240 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.240 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.240 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2023-11-06T01:49:55.241 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.241 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.241 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.241 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.241 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2023-11-06T01:49:55.242 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.242 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.242 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.242 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.242 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2023-11-06T01:49:55.243 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.243 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.243 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.243 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.243 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2023-11-06T01:49:55.244 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.244 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.244 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.244 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.244 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2023-11-06T01:49:55.245 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.245 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.245 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.245 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2023-11-06T01:49:55.246 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.246 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.246 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.246 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.246 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2023-11-06T01:49:55.247 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.247 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.247 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.247 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.247 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2023-11-06T01:49:55.248 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.248 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.248 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.248 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.248 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2023-11-06T01:49:55.249 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2023-11-06T01:49:55.249 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2023-11-06T01:49:55.249 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.249 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.249 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.250 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:55.250 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.250 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2023-11-06T01:49:55.250 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2023-11-06T01:49:55.250 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 1, 2023-11-06T01:49:55.251 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 0, 2023-11-06T01:49:55.251 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2023-11-06T01:49:55.251 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2023-11-06T01:49:55.251 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 58, 2023-11-06T01:49:55.251 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 58, 2023-11-06T01:49:55.251 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 2390, 2023-11-06T01:49:55.252 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2023-11-06T01:49:55.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.252 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2023-11-06T01:49:55.252 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2023-11-06T01:49:55.252 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2023-11-06T01:49:55.253 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:49:55.253 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:49:55.253 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:49:55.254 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:49:55.254 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:49:55.254 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:49:55.254 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:49:55.255 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:49:55.255 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:49:55.255 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:49:55.255 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:49:55.255 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:49:55.256 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:49:55.256 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2023-11-06T01:49:55.256 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:49:55.256 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2023-11-06T01:49:55.256 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:49:55.257 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:49:55.257 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:49:55.257 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:49:55.257 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:49:55.257 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:49:55.257 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:49:55.258 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2023-11-06T01:49:55.258 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:49:55.258 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:49:55.258 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:49:55.258 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2023-11-06T01:49:55.259 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2023-11-06T01:49:55.259 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2023-11-06T01:49:55.259 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:49:55.259 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:49:55.259 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:49:55.260 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:49:55.260 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 296407, 2023-11-06T01:49:55.260 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 521, 2023-11-06T01:49:55.260 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2023-11-06T01:49:55.260 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2023-11-06T01:49:55.261 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2023-11-06T01:49:55.261 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2023-11-06T01:49:55.261 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:49:55.261 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2023-11-06T01:49:55.261 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:49:55.262 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2023-11-06T01:49:55.262 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 7720, 2023-11-06T01:49:55.262 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 162, 2023-11-06T01:49:55.262 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:49:55.262 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:49:55.263 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2023-11-06T01:49:55.263 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2023-11-06T01:49:55.263 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 1473530, 2023-11-06T01:49:55.263 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 14000, 2023-11-06T01:49:55.263 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2023-11-06T01:49:55.263 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2023-11-06T01:49:55.264 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2023-11-06T01:49:55.264 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2023-11-06T01:49:55.264 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.264 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2023-11-06T01:49:55.264 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2023-11-06T01:49:55.265 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2023-11-06T01:49:55.265 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 500, 2023-11-06T01:49:55.265 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 1259494, 2023-11-06T01:49:55.265 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 27, 2023-11-06T01:49:55.265 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 500, 2023-11-06T01:49:55.266 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2023-11-06T01:49:55.266 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 500, 2023-11-06T01:49:55.266 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 6.320902355, 2023-11-06T01:49:55.266 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.012641804 2023-11-06T01:49:55.267 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.267 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2023-11-06T01:49:55.267 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2023-11-06T01:49:55.267 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 500, 2023-11-06T01:49:55.268 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 544 2023-11-06T01:49:55.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.268 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 500, 2023-11-06T01:49:55.268 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 175, 2023-11-06T01:49:55.268 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 325, 2023-11-06T01:49:55.269 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2023-11-06T01:49:55.269 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2023-11-06T01:49:55.269 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 0, 2023-11-06T01:49:55.269 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 10, 2023-11-06T01:49:55.269 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 0, 2023-11-06T01:49:55.270 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 223, 2023-11-06T01:49:55.270 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 54, 2023-11-06T01:49:55.270 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2023-11-06T01:49:55.270 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2023-11-06T01:49:55.271 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2023-11-06T01:49:55.271 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2023-11-06T01:49:55.271 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 6, 2023-11-06T01:49:55.271 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2023-11-06T01:49:55.271 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2023-11-06T01:49:55.272 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2023-11-06T01:49:55.272 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 174, 2023-11-06T01:49:55.272 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 20, 2023-11-06T01:49:55.272 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2023-11-06T01:49:55.272 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2023-11-06T01:49:55.273 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2023-11-06T01:49:55.273 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2023-11-06T01:49:55.273 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2023-11-06T01:49:55.273 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2023-11-06T01:49:55.273 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T01:49:55.274 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2023-11-06T01:49:55.274 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2023-11-06T01:49:55.274 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2023-11-06T01:49:55.274 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2023-11-06T01:49:55.274 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2023-11-06T01:49:55.275 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2023-11-06T01:49:55.275 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2023-11-06T01:49:55.275 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2023-11-06T01:49:55.275 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2023-11-06T01:49:55.275 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2023-11-06T01:49:55.276 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2023-11-06T01:49:55.276 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2023-11-06T01:49:55.276 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2023-11-06T01:49:55.276 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2023-11-06T01:49:55.276 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2023-11-06T01:49:55.277 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2023-11-06T01:49:55.277 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2023-11-06T01:49:55.277 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2023-11-06T01:49:55.277 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2023-11-06T01:49:55.277 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 84, 2023-11-06T01:49:55.278 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2023-11-06T01:49:55.278 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 2, 2023-11-06T01:49:55.278 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 12, 2023-11-06T01:49:55.278 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 12, 2023-11-06T01:49:55.278 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2023-11-06T01:49:55.279 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2023-11-06T01:49:55.279 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 44, 2023-11-06T01:49:55.279 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2023-11-06T01:49:55.279 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 11 2023-11-06T01:49:55.279 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.280 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2023-11-06T01:49:55.280 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2023-11-06T01:49:55.280 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2023-11-06T01:49:55.280 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2023-11-06T01:49:55.280 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2023-11-06T01:49:55.281 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.281 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2023-11-06T01:49:55.281 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 225 2023-11-06T01:49:55.281 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.281 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2023-11-06T01:49:55.281 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2023-11-06T01:49:55.282 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2023-11-06T01:49:55.282 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2023-11-06T01:49:55.282 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2023-11-06T01:49:55.282 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2023-11-06T01:49:55.282 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2023-11-06T01:49:55.283 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.283 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:49:55.283 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:55.283 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-11-06T01:49:55.283 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:55.284 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 3209761, 2023-11-06T01:49:55.284 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 228084918, 2023-11-06T01:49:55.284 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:55.284 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 3209761, 2023-11-06T01:49:55.284 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:55.285 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:55.285 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3209761, 2023-11-06T01:49:55.285 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 228084918, 2023-11-06T01:49:55.285 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:55.285 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.286 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.286 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.286 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:55.286 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.286 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2023-11-06T01:49:55.287 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:55.287 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-11-06T01:49:55.287 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:55.287 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:55.287 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:55.288 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:55.288 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:55.288 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 500, 2023-11-06T01:49:55.288 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 1259478, 2023-11-06T01:49:55.288 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 500, 2023-11-06T01:49:55.288 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1259478, 2023-11-06T01:49:55.289 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:55.289 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.289 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.289 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.289 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:55.290 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.290 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2023-11-06T01:49:55.290 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:55.290 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2023-11-06T01:49:55.290 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:55.291 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:55.291 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:55.291 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:55.291 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:55.291 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 500, 2023-11-06T01:49:55.292 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 500, 2023-11-06T01:49:55.292 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 500, 2023-11-06T01:49:55.292 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 500, 2023-11-06T01:49:55.292 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:55.292 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.293 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.293 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.293 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:55.293 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.293 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2023-11-06T01:49:55.294 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:55.294 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2023-11-06T01:49:55.294 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:55.294 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2023-11-06T01:49:55.294 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2023-11-06T01:49:55.295 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:55.295 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:55.295 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:55.295 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:55.295 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2023-11-06T01:49:55.296 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2023-11-06T01:49:55.296 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:55.296 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.296 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.296 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.297 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:55.297 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-11-06T01:49:55.297 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x5575cdd680a0": { 2023-11-06T01:49:55.297 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-11-06T01:49:55.297 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2023-11-06T01:49:55.298 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-11-06T01:49:55.298 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 486, 2023-11-06T01:49:55.298 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1226357, 2023-11-06T01:49:55.298 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:55.298 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 486, 2023-11-06T01:49:55.299 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-11-06T01:49:55.299 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-11-06T01:49:55.299 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 223, 2023-11-06T01:49:55.299 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1226357, 2023-11-06T01:49:55.299 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-11-06T01:49:55.300 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-11-06T01:49:55.300 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-11-06T01:49:55.300 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-11-06T01:49:55.300 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:55.301 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-11-06T01:49:55.301 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-11-06T01:49:55.301 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:55 smithi062 ceph-mon[103924]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-11-06T01:49:55.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:55 smithi163 ceph-mon[104416]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-11-06T01:49:55.564 INFO:tasks.check_counter:Daemon mds.k mds.exported=9 2023-11-06T01:49:55.564 INFO:tasks.check_counter:Daemon mds.k mds.imported=7 2023-11-06T01:49:55.564 DEBUG:tasks.check_counter:Getting stats from c 2023-11-06T01:49:55.565 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-11-06T01:49:55.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:55 smithi053 ceph-mon[99472]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-11-06T01:49:56.219 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:49:57.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:57 smithi163 ceph-mon[104416]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 16 op/s 2023-11-06T01:49:57.513 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-11-06T01:49:57.514 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:49:57.514 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1283510, 2023-11-06T01:49:57.514 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1282866, 2023-11-06T01:49:57.514 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 208657323, 2023-11-06T01:49:57.515 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 204660694, 2023-11-06T01:49:57.515 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 11, 2023-11-06T01:49:57.515 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 4, 2023-11-06T01:49:57.515 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 54.557484784, 2023-11-06T01:49:57.515 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 22.549446172, 2023-11-06T01:49:57.515 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 40.741298822, 2023-11-06T01:49:57.516 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.019846406, 2023-11-06T01:49:57.516 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:57.516 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1282864, 2023-11-06T01:49:57.516 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 15.354435451, 2023-11-06T01:49:57.516 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000011968 2023-11-06T01:49:57.517 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.517 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:57.517 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2564768, 2023-11-06T01:49:57.517 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.295318523, 2023-11-06T01:49:57.517 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000115 2023-11-06T01:49:57.518 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.518 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-06T01:49:57.518 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 0 2023-11-06T01:49:57.518 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.518 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:49:57.519 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1306149, 2023-11-06T01:49:57.519 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1306432, 2023-11-06T01:49:57.519 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 212620859, 2023-11-06T01:49:57.519 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 207293098, 2023-11-06T01:49:57.519 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 13, 2023-11-06T01:49:57.520 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 8, 2023-11-06T01:49:57.520 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 35.694799235, 2023-11-06T01:49:57.520 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 13.712628661, 2023-11-06T01:49:57.520 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 24.912487496, 2023-11-06T01:49:57.520 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.008498162, 2023-11-06T01:49:57.520 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:57.521 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1306431, 2023-11-06T01:49:57.521 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 26.520983545, 2023-11-06T01:49:57.521 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000020300 2023-11-06T01:49:57.521 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.521 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:57.522 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2611428, 2023-11-06T01:49:57.522 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.228428606, 2023-11-06T01:49:57.522 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000087 2023-11-06T01:49:57.522 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.522 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 677840, 2023-11-06T01:49:57.523 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 170176 2023-11-06T01:49:57.523 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.523 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:49:57.523 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 638342, 2023-11-06T01:49:57.523 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 638311, 2023-11-06T01:49:57.524 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 103618253, 2023-11-06T01:49:57.524 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 101633633, 2023-11-06T01:49:57.524 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 14, 2023-11-06T01:49:57.524 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 5, 2023-11-06T01:49:57.524 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 18.582148790, 2023-11-06T01:49:57.525 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 7.090716829, 2023-11-06T01:49:57.525 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 13.198850474, 2023-11-06T01:49:57.525 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.006104188, 2023-11-06T01:49:57.525 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:57.525 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 638310, 2023-11-06T01:49:57.526 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 13.607241830, 2023-11-06T01:49:57.526 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000021317 2023-11-06T01:49:57.526 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.526 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:57.526 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1275827, 2023-11-06T01:49:57.527 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.119870186, 2023-11-06T01:49:57.527 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000093 2023-11-06T01:49:57.527 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.527 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-11-06T01:49:57.527 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 543984 2023-11-06T01:49:57.528 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.528 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2023-11-06T01:49:57.528 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2023-11-06T01:49:57.528 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2023-11-06T01:49:57.528 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.528 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2023-11-06T01:49:57.529 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2023-11-06T01:49:57.529 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2023-11-06T01:49:57.529 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1056, 2023-11-06T01:49:57.529 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.197811616, 2023-11-06T01:49:57.529 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000187321 2023-11-06T01:49:57.530 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:49:57.530 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.530 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2023-11-06T01:49:57.530 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2023-11-06T01:49:57.530 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2023-11-06T01:49:57.531 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 186, 2023-11-06T01:49:57.531 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.005787854, 2023-11-06T01:49:57.531 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000031117 2023-11-06T01:49:57.531 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:49:57.531 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.531 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2023-11-06T01:49:57.532 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 1168, 2023-11-06T01:49:57.532 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 443, 2023-11-06T01:49:57.532 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2023-11-06T01:49:57.532 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 443, 2023-11-06T01:49:57.532 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.504014286, 2023-11-06T01:49:57.533 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001137729 2023-11-06T01:49:57.533 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.533 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2023-11-06T01:49:57.533 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 725, 2023-11-06T01:49:57.533 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 0, 2023-11-06T01:49:57.534 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 0, 2023-11-06T01:49:57.534 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 11, 2023-11-06T01:49:57.534 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 0, 2023-11-06T01:49:57.534 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2023-11-06T01:49:57.534 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 277, 2023-11-06T01:49:57.535 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 0, 2023-11-06T01:49:57.535 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 277, 2023-11-06T01:49:57.535 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 0, 2023-11-06T01:49:57.535 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 275, 2023-11-06T01:49:57.535 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 0, 2023-11-06T01:49:57.536 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 227, 2023-11-06T01:49:57.536 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 227, 2023-11-06T01:49:57.536 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 11, 2023-11-06T01:49:57.536 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 1613, 2023-11-06T01:49:57.536 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 722, 2023-11-06T01:49:57.536 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 724, 2023-11-06T01:49:57.537 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 7, 2023-11-06T01:49:57.537 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2023-11-06T01:49:57.537 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2023-11-06T01:49:57.537 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 0, 2023-11-06T01:49:57.537 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 615, 2023-11-06T01:49:57.538 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2023-11-06T01:49:57.538 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 2, 2023-11-06T01:49:57.538 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 0, 2023-11-06T01:49:57.538 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 8, 2023-11-06T01:49:57.538 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 231, 2023-11-06T01:49:57.539 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2023-11-06T01:49:57.539 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 147, 2023-11-06T01:49:57.539 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2023-11-06T01:49:57.539 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 1039, 2023-11-06T01:49:57.539 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 1693659912, 2023-11-06T01:49:57.539 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 0, 2023-11-06T01:49:57.540 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 147, 2023-11-06T01:49:57.540 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2023-11-06T01:49:57.540 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T01:49:57.540 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2023-11-06T01:49:57.540 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 0, 2023-11-06T01:49:57.541 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 0, 2023-11-06T01:49:57.541 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 0, 2023-11-06T01:49:57.541 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 147, 2023-11-06T01:49:57.541 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 0, 2023-11-06T01:49:57.541 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 7, 2023-11-06T01:49:57.542 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 206, 2023-11-06T01:49:57.542 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 1, 2023-11-06T01:49:57.542 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T01:49:57.542 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T01:49:57.542 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 422, 2023-11-06T01:49:57.543 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 300, 2023-11-06T01:49:57.543 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 32, 2023-11-06T01:49:57.543 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 108 2023-11-06T01:49:57.543 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.543 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2023-11-06T01:49:57.543 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 0, 2023-11-06T01:49:57.544 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2023-11-06T01:49:57.544 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2023-11-06T01:49:57.544 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 7, 2023-11-06T01:49:57.544 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 11, 2023-11-06T01:49:57.544 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2023-11-06T01:49:57.545 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2023-11-06T01:49:57.545 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2023-11-06T01:49:57.545 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 27, 2023-11-06T01:49:57.545 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 27, 2023-11-06T01:49:57.545 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 0, 2023-11-06T01:49:57.546 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2023-11-06T01:49:57.546 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2023-11-06T01:49:57.546 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2023-11-06T01:49:57.546 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2023-11-06T01:49:57.546 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2023-11-06T01:49:57.547 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2023-11-06T01:49:57.547 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 0, 2023-11-06T01:49:57.547 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 2, 2023-11-06T01:49:57.547 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2023-11-06T01:49:57.547 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 0, 2023-11-06T01:49:57.548 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2023-11-06T01:49:57.548 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2023-11-06T01:49:57.548 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.548 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2023-11-06T01:49:57.548 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 1023, 2023-11-06T01:49:57.548 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 0, 2023-11-06T01:49:57.549 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 0, 2023-11-06T01:49:57.549 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 1023, 2023-11-06T01:49:57.549 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2023-11-06T01:49:57.549 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 0, 2023-11-06T01:49:57.549 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 1, 2023-11-06T01:49:57.550 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 0, 2023-11-06T01:49:57.550 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 0, 2023-11-06T01:49:57.550 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 1, 2023-11-06T01:49:57.550 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2023-11-06T01:49:57.550 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 0, 2023-11-06T01:49:57.551 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 4194304, 2023-11-06T01:49:57.551 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 5578398, 2023-11-06T01:49:57.551 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 0, 2023-11-06T01:49:57.551 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2023-11-06T01:49:57.551 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 408, 2023-11-06T01:49:57.552 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 10.591011894, 2023-11-06T01:49:57.552 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.025958362 2023-11-06T01:49:57.552 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.552 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 0 2023-11-06T01:49:57.552 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.553 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2023-11-06T01:49:57.553 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 246, 2023-11-06T01:49:57.553 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 273, 2023-11-06T01:49:57.553 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 27, 2023-11-06T01:49:57.553 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 27, 2023-11-06T01:49:57.554 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 28, 2023-11-06T01:49:57.554 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 1, 2023-11-06T01:49:57.554 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 276, 2023-11-06T01:49:57.554 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 308, 2023-11-06T01:49:57.554 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 32, 2023-11-06T01:49:57.555 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 227, 2023-11-06T01:49:57.555 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 472, 2023-11-06T01:49:57.555 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 245, 2023-11-06T01:49:57.555 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 55292, 2023-11-06T01:49:57.555 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 198964 2023-11-06T01:49:57.556 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.556 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2023-11-06T01:49:57.556 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 1310, 2023-11-06T01:49:57.556 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 0, 2023-11-06T01:49:57.556 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 1168, 2023-11-06T01:49:57.556 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 17, 2023-11-06T01:49:57.557 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 47, 2023-11-06T01:49:57.557 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2023-11-06T01:49:57.557 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 26, 2023-11-06T01:49:57.557 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.072740990, 2023-11-06T01:49:57.557 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.002797730 2023-11-06T01:49:57.558 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.558 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2023-11-06T01:49:57.558 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 9, 2023-11-06T01:49:57.558 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.031322893, 2023-11-06T01:49:57.558 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.003480321 2023-11-06T01:49:57.559 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.559 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2023-11-06T01:49:57.559 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.559 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.559 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.559 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.560 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2023-11-06T01:49:57.560 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.560 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.560 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.560 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.561 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2023-11-06T01:49:57.561 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 315, 2023-11-06T01:49:57.561 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.285875417, 2023-11-06T01:49:57.561 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000907541 2023-11-06T01:49:57.561 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.562 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2023-11-06T01:49:57.562 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.562 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.562 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.562 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.563 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2023-11-06T01:49:57.563 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1, 2023-11-06T01:49:57.563 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000138102, 2023-11-06T01:49:57.563 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000138102 2023-11-06T01:49:57.563 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.563 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2023-11-06T01:49:57.564 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.564 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.564 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.564 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.565 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2023-11-06T01:49:57.565 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.565 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.565 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.565 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.565 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2023-11-06T01:49:57.566 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.566 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.566 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.566 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.566 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2023-11-06T01:49:57.567 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.567 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.567 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.567 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.567 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2023-11-06T01:49:57.568 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.568 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.568 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.568 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.568 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2023-11-06T01:49:57.568 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.569 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.569 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.569 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.569 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2023-11-06T01:49:57.569 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.570 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.570 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.570 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.570 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2023-11-06T01:49:57.570 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 37, 2023-11-06T01:49:57.571 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.033882319, 2023-11-06T01:49:57.571 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000915738 2023-11-06T01:49:57.571 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.571 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2023-11-06T01:49:57.571 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 26, 2023-11-06T01:49:57.572 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.018778389, 2023-11-06T01:49:57.572 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000722245 2023-11-06T01:49:57.572 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.572 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2023-11-06T01:49:57.572 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1, 2023-11-06T01:49:57.573 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000674028, 2023-11-06T01:49:57.573 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000674028 2023-11-06T01:49:57.573 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.573 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2023-11-06T01:49:57.573 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.574 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.574 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.574 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.574 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2023-11-06T01:49:57.574 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.575 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.575 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.575 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.575 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2023-11-06T01:49:57.575 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.576 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.576 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.576 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.576 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2023-11-06T01:49:57.576 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.577 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.577 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.577 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.577 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2023-11-06T01:49:57.577 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.577 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.578 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.578 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.578 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2023-11-06T01:49:57.578 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1, 2023-11-06T01:49:57.579 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.009519762, 2023-11-06T01:49:57.579 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.009519762 2023-11-06T01:49:57.579 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.579 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2023-11-06T01:49:57.579 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.579 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.580 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.580 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.580 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2023-11-06T01:49:57.580 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.580 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.581 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.581 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.581 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2023-11-06T01:49:57.581 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.581 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.582 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.582 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.582 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2023-11-06T01:49:57.582 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.582 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.583 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.583 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.583 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2023-11-06T01:49:57.583 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.583 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.584 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.584 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.584 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2023-11-06T01:49:57.584 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 27, 2023-11-06T01:49:57.584 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.051082386, 2023-11-06T01:49:57.585 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001891940 2023-11-06T01:49:57.585 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.585 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2023-11-06T01:49:57.585 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2023-11-06T01:49:57.585 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2023-11-06T01:49:57.586 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.586 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.586 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.586 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:49:57.586 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.587 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2023-11-06T01:49:57.587 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 1, 2023-11-06T01:49:57.587 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 2, 2023-11-06T01:49:57.587 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 1, 2023-11-06T01:49:57.587 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 1, 2023-11-06T01:49:57.587 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2023-11-06T01:49:57.588 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 53, 2023-11-06T01:49:57.588 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 53, 2023-11-06T01:49:57.588 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 955, 2023-11-06T01:49:57.588 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2023-11-06T01:49:57.588 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.589 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2023-11-06T01:49:57.589 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2023-11-06T01:49:57.589 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2023-11-06T01:49:57.589 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:49:57.589 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:49:57.590 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:49:57.590 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:49:57.590 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:49:57.590 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:49:57.590 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:49:57.591 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:49:57.591 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:49:57.591 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:49:57.591 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:49:57.591 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:49:57.592 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:49:57.592 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2023-11-06T01:49:57.592 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:49:57.592 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2023-11-06T01:49:57.592 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:49:57.592 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:49:57.593 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:49:57.593 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:49:57.593 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:49:57.593 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:49:57.593 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:49:57.594 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2023-11-06T01:49:57.594 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:49:57.594 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:49:57.594 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:49:57.594 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2023-11-06T01:49:57.595 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2023-11-06T01:49:57.595 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2023-11-06T01:49:57.595 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:49:57.595 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:49:57.595 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:49:57.595 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:49:57.596 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 340136, 2023-11-06T01:49:57.596 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 372, 2023-11-06T01:49:57.596 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 792, 2023-11-06T01:49:57.596 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 9, 2023-11-06T01:49:57.596 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2023-11-06T01:49:57.597 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2023-11-06T01:49:57.597 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:49:57.597 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2023-11-06T01:49:57.597 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:49:57.597 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2023-11-06T01:49:57.598 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 7720, 2023-11-06T01:49:57.598 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 162, 2023-11-06T01:49:57.598 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:49:57.598 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:49:57.598 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2023-11-06T01:49:57.599 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2023-11-06T01:49:57.599 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 879360, 2023-11-06T01:49:57.599 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 9404, 2023-11-06T01:49:57.599 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2023-11-06T01:49:57.599 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2023-11-06T01:49:57.600 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2023-11-06T01:49:57.600 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2023-11-06T01:49:57.600 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.600 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2023-11-06T01:49:57.600 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2023-11-06T01:49:57.600 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2023-11-06T01:49:57.601 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 857, 2023-11-06T01:49:57.601 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 1435621, 2023-11-06T01:49:57.601 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 27, 2023-11-06T01:49:57.601 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 857, 2023-11-06T01:49:57.601 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2023-11-06T01:49:57.602 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 857, 2023-11-06T01:49:57.602 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 12.819706021, 2023-11-06T01:49:57.602 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.014958816 2023-11-06T01:49:57.602 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.603 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2023-11-06T01:49:57.603 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2023-11-06T01:49:57.603 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 857, 2023-11-06T01:49:57.603 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 996 2023-11-06T01:49:57.603 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.604 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 857, 2023-11-06T01:49:57.604 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 172, 2023-11-06T01:49:57.604 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 685, 2023-11-06T01:49:57.604 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2023-11-06T01:49:57.604 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2023-11-06T01:49:57.605 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 0, 2023-11-06T01:49:57.605 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 10, 2023-11-06T01:49:57.605 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 25, 2023-11-06T01:49:57.605 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 433, 2023-11-06T01:49:57.605 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 91, 2023-11-06T01:49:57.606 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2023-11-06T01:49:57.606 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2023-11-06T01:49:57.606 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2023-11-06T01:49:57.606 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 0, 2023-11-06T01:49:57.606 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 39, 2023-11-06T01:49:57.607 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2023-11-06T01:49:57.607 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2023-11-06T01:49:57.607 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2023-11-06T01:49:57.607 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 147, 2023-11-06T01:49:57.607 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 20, 2023-11-06T01:49:57.608 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2023-11-06T01:49:57.608 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2023-11-06T01:49:57.608 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2023-11-06T01:49:57.608 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2023-11-06T01:49:57.608 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2023-11-06T01:49:57.609 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2023-11-06T01:49:57.609 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T01:49:57.609 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2023-11-06T01:49:57.609 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2023-11-06T01:49:57.609 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 1, 2023-11-06T01:49:57.610 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2023-11-06T01:49:57.610 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2023-11-06T01:49:57.610 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2023-11-06T01:49:57.610 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2023-11-06T01:49:57.610 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2023-11-06T01:49:57.611 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2023-11-06T01:49:57.611 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2023-11-06T01:49:57.611 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2023-11-06T01:49:57.611 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2023-11-06T01:49:57.611 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2023-11-06T01:49:57.612 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2023-11-06T01:49:57.612 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2023-11-06T01:49:57.612 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2023-11-06T01:49:57.612 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2023-11-06T01:49:57.612 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2023-11-06T01:49:57.613 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2023-11-06T01:49:57.613 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 84, 2023-11-06T01:49:57.613 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2023-11-06T01:49:57.613 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 2, 2023-11-06T01:49:57.613 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 12, 2023-11-06T01:49:57.613 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 12, 2023-11-06T01:49:57.614 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 0, 2023-11-06T01:49:57.614 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2023-11-06T01:49:57.614 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 185, 2023-11-06T01:49:57.614 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 0, 2023-11-06T01:49:57.614 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 45 2023-11-06T01:49:57.615 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.615 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2023-11-06T01:49:57.615 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 1, 2023-11-06T01:49:57.615 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2023-11-06T01:49:57.615 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2023-11-06T01:49:57.616 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2023-11-06T01:49:57.616 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.616 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2023-11-06T01:49:57.616 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 1170 2023-11-06T01:49:57.616 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.617 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2023-11-06T01:49:57.617 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2023-11-06T01:49:57.617 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 3, 2023-11-06T01:49:57.617 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2023-11-06T01:49:57.617 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 3, 2023-11-06T01:49:57.618 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 27, 2023-11-06T01:49:57.618 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2023-11-06T01:49:57.618 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.618 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:49:57.618 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:49:57.619 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2023-11-06T01:49:57.619 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:49:57.619 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 3227998, 2023-11-06T01:49:57.619 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 234355729, 2023-11-06T01:49:57.619 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:57.620 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 3227998, 2023-11-06T01:49:57.620 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:49:57.620 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:49:57.620 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3227998, 2023-11-06T01:49:57.620 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 234355729, 2023-11-06T01:49:57.621 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:49:57.621 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.621 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.621 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.621 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:49:57.621 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.622 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2023-11-06T01:49:57.622 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:49:57.622 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2023-11-06T01:49:57.622 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:49:57.622 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:49:57.623 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:49:57.623 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:57.623 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:57.623 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 857, 2023-11-06T01:49:57.624 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 1438348, 2023-11-06T01:49:57.624 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 857, 2023-11-06T01:49:57.624 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 1438348, 2023-11-06T01:49:57.624 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:49:57.624 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.625 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.625 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.625 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:49:57.625 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.625 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2023-11-06T01:49:57.626 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:49:57.626 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2023-11-06T01:49:57.626 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:49:57.626 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:49:57.626 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:49:57.627 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:57.627 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:49:57.627 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 857, 2023-11-06T01:49:57.627 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 857, 2023-11-06T01:49:57.627 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 857, 2023-11-06T01:49:57.628 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 857, 2023-11-06T01:49:57.628 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:49:57.628 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.628 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.628 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.629 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:49:57.629 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.629 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2023-11-06T01:49:57.629 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:49:57.629 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2023-11-06T01:49:57.629 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:49:57.630 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 27, 2023-11-06T01:49:57.630 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 2727, 2023-11-06T01:49:57.630 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:57.630 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 27, 2023-11-06T01:49:57.630 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:49:57.631 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:49:57.631 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 25, 2023-11-06T01:49:57.631 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 2727, 2023-11-06T01:49:57.631 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:49:57.631 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.632 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.632 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.632 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:49:57.632 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:57.632 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x55dfba6de0a0": { 2023-11-06T01:49:57.633 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:49:57.633 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2023-11-06T01:49:57.633 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:49:57.633 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 1023, 2023-11-06T01:49:57.633 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 1384094, 2023-11-06T01:49:57.634 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:49:57.634 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 1023, 2023-11-06T01:49:57.634 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:49:57.634 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:49:57.634 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 408, 2023-11-06T01:49:57.635 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 1384094, 2023-11-06T01:49:57.635 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:49:57.635 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:57.635 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:57.635 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:57.635 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:49:57.636 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:49:57.636 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-11-06T01:49:57.636 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:57 smithi053 ceph-mon[99472]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 16 op/s 2023-11-06T01:49:57.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:57 smithi062 ceph-mon[103924]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 16 op/s 2023-11-06T01:49:57.877 INFO:tasks.check_counter:Daemon mds.c mds.exported=2 2023-11-06T01:49:57.878 INFO:tasks.check_counter:Daemon mds.c mds.imported=8 2023-11-06T01:49:57.878 DEBUG:tasks.check_counter:Getting stats from f 2023-11-06T01:49:57.878 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-11-06T01:49:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:49:59.236 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:49:59 smithi163 ceph-mon[104416]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-11-06T01:49:59.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:49:59 smithi053 ceph-mon[99472]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-11-06T01:49:59.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:49:59 smithi062 ceph-mon[103924]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-11-06T01:49:59.970 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-11-06T01:49:59.970 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:49:59.971 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1161, 2023-11-06T01:49:59.971 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1163, 2023-11-06T01:49:59.971 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 4575673, 2023-11-06T01:49:59.971 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 361081, 2023-11-06T01:49:59.971 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 6, 2023-11-06T01:49:59.971 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 2, 2023-11-06T01:49:59.972 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.102407630, 2023-11-06T01:49:59.972 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.025945380, 2023-11-06T01:49:59.972 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.027401936, 2023-11-06T01:49:59.972 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.039111877, 2023-11-06T01:49:59.972 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:59.973 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1161, 2023-11-06T01:49:59.973 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.254870010, 2023-11-06T01:49:59.973 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000219526 2023-11-06T01:49:59.973 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.973 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:59.974 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:59.974 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:59.974 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:59.974 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.974 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-06T01:49:59.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 0 2023-11-06T01:49:59.975 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.975 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:49:59.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1620, 2023-11-06T01:49:59.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1837, 2023-11-06T01:49:59.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 1290825, 2023-11-06T01:49:59.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 956023, 2023-11-06T01:49:59.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 5, 2023-11-06T01:49:59.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 4, 2023-11-06T01:49:59.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.120425386, 2023-11-06T01:49:59.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.038340794, 2023-11-06T01:49:59.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.026267804, 2023-11-06T01:49:59.977 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.046769995, 2023-11-06T01:49:59.977 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:59.977 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1835, 2023-11-06T01:49:59.977 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.347040909, 2023-11-06T01:49:59.977 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000189123 2023-11-06T01:49:59.978 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:59.978 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:59.978 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:59.979 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:59.979 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.979 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-06T01:49:59.979 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 452448 2023-11-06T01:49:59.979 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.980 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:49:59.980 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 955, 2023-11-06T01:49:59.980 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 921, 2023-11-06T01:49:59.980 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 5024249, 2023-11-06T01:49:59.980 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 359777, 2023-11-06T01:49:59.981 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 3, 2023-11-06T01:49:59.981 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 2, 2023-11-06T01:49:59.981 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.087741756, 2023-11-06T01:49:59.981 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.028411192, 2023-11-06T01:49:59.981 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.032067115, 2023-11-06T01:49:59.981 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.024216065, 2023-11-06T01:49:59.982 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:49:59.982 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 921, 2023-11-06T01:49:59.982 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.115761318, 2023-11-06T01:49:59.982 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000125690 2023-11-06T01:49:59.982 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.983 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:49:59.983 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:59.983 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:59.983 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:59.983 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.984 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 670048, 2023-11-06T01:49:59.984 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 165712 2023-11-06T01:49:59.984 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.984 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2023-11-06T01:49:59.984 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2023-11-06T01:49:59.985 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2023-11-06T01:49:59.985 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.985 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2023-11-06T01:49:59.985 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2023-11-06T01:49:59.985 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2023-11-06T01:49:59.985 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 5675, 2023-11-06T01:49:59.986 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.159086007, 2023-11-06T01:49:59.986 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000028032 2023-11-06T01:49:59.986 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:49:59.986 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.986 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2023-11-06T01:49:59.987 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2023-11-06T01:49:59.987 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2023-11-06T01:49:59.987 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:59.987 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:59.987 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:59.988 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:49:59.988 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.988 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2023-11-06T01:49:59.988 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 0, 2023-11-06T01:49:59.988 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 0, 2023-11-06T01:49:59.989 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2023-11-06T01:49:59.989 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:49:59.989 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:49:59.989 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:49:59.989 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:49:59.990 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2023-11-06T01:49:59.990 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 0, 2023-11-06T01:49:59.990 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 0, 2023-11-06T01:49:59.990 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 0, 2023-11-06T01:49:59.990 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 0, 2023-11-06T01:49:59.991 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 0, 2023-11-06T01:49:59.991 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2023-11-06T01:49:59.991 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 228, 2023-11-06T01:49:59.991 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 84, 2023-11-06T01:49:59.991 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 135, 2023-11-06T01:49:59.992 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 9, 2023-11-06T01:49:59.992 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 107, 2023-11-06T01:49:59.992 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 20, 2023-11-06T01:49:59.992 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 0, 2023-11-06T01:49:59.992 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 0, 2023-11-06T01:49:59.993 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 6, 2023-11-06T01:49:59.993 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 0, 2023-11-06T01:49:59.993 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 0, 2023-11-06T01:49:59.993 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 0, 2023-11-06T01:49:59.993 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 0, 2023-11-06T01:49:59.994 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2023-11-06T01:49:59.994 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2023-11-06T01:49:59.994 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 0, 2023-11-06T01:49:59.994 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 0, 2023-11-06T01:49:59.994 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2023-11-06T01:49:59.995 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 0, 2023-11-06T01:49:59.995 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 0, 2023-11-06T01:49:59.995 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 0, 2023-11-06T01:49:59.995 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 0, 2023-11-06T01:49:59.995 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2023-11-06T01:49:59.996 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 0, 2023-11-06T01:49:59.996 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2023-11-06T01:49:59.996 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 1038, 2023-11-06T01:49:59.996 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 1747014300, 2023-11-06T01:49:59.996 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 0, 2023-11-06T01:49:59.996 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 0, 2023-11-06T01:49:59.997 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2023-11-06T01:49:59.997 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T01:49:59.997 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2023-11-06T01:49:59.997 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 0, 2023-11-06T01:49:59.997 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 0, 2023-11-06T01:49:59.998 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 0, 2023-11-06T01:49:59.998 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 0, 2023-11-06T01:49:59.998 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 0, 2023-11-06T01:49:59.998 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 0, 2023-11-06T01:49:59.998 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 0, 2023-11-06T01:49:59.999 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T01:49:59.999 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T01:49:59.999 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T01:49:59.999 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 0, 2023-11-06T01:49:59.999 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 0, 2023-11-06T01:50:00.000 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 0, 2023-11-06T01:50:00.000 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 0 2023-11-06T01:50:00.000 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.000 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2023-11-06T01:50:00.000 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 0, 2023-11-06T01:50:00.000 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2023-11-06T01:50:00.001 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2023-11-06T01:50:00.001 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 0, 2023-11-06T01:50:00.001 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 0, 2023-11-06T01:50:00.001 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2023-11-06T01:50:00.002 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2023-11-06T01:50:00.002 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2023-11-06T01:50:00.002 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 0, 2023-11-06T01:50:00.002 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 0, 2023-11-06T01:50:00.002 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 0, 2023-11-06T01:50:00.003 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2023-11-06T01:50:00.003 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2023-11-06T01:50:00.003 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2023-11-06T01:50:00.003 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2023-11-06T01:50:00.003 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2023-11-06T01:50:00.004 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2023-11-06T01:50:00.004 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 0, 2023-11-06T01:50:00.004 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 0, 2023-11-06T01:50:00.004 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2023-11-06T01:50:00.004 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 0, 2023-11-06T01:50:00.005 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2023-11-06T01:50:00.005 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2023-11-06T01:50:00.005 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.005 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2023-11-06T01:50:00.005 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 0, 2023-11-06T01:50:00.005 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 0, 2023-11-06T01:50:00.006 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 0, 2023-11-06T01:50:00.006 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 3387, 2023-11-06T01:50:00.006 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2023-11-06T01:50:00.006 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 0, 2023-11-06T01:50:00.006 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 0, 2023-11-06T01:50:00.007 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 0, 2023-11-06T01:50:00.007 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 0, 2023-11-06T01:50:00.007 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 2, 2023-11-06T01:50:00.007 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2023-11-06T01:50:00.007 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 0, 2023-11-06T01:50:00.008 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 13342214, 2023-11-06T01:50:00.008 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 13465319, 2023-11-06T01:50:00.008 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 13459192, 2023-11-06T01:50:00.008 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2023-11-06T01:50:00.008 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.009 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.009 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.009 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.009 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 3387 2023-11-06T01:50:00.009 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.010 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2023-11-06T01:50:00.010 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 164, 2023-11-06T01:50:00.010 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 1317, 2023-11-06T01:50:00.010 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 1153, 2023-11-06T01:50:00.010 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 22, 2023-11-06T01:50:00.011 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 74, 2023-11-06T01:50:00.011 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 52, 2023-11-06T01:50:00.011 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 228, 2023-11-06T01:50:00.011 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 1385, 2023-11-06T01:50:00.011 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 1157, 2023-11-06T01:50:00.012 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 0, 2023-11-06T01:50:00.012 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 0, 2023-11-06T01:50:00.012 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 0, 2023-11-06T01:50:00.012 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 49108, 2023-11-06T01:50:00.013 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 207156 2023-11-06T01:50:00.013 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.013 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2023-11-06T01:50:00.013 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 0, 2023-11-06T01:50:00.013 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 0, 2023-11-06T01:50:00.014 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 0, 2023-11-06T01:50:00.014 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 0, 2023-11-06T01:50:00.014 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 0, 2023-11-06T01:50:00.014 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2023-11-06T01:50:00.015 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.015 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.015 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.015 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.015 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2023-11-06T01:50:00.015 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.016 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.016 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.016 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.016 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2023-11-06T01:50:00.016 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.017 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.017 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.017 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.017 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2023-11-06T01:50:00.017 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.018 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.018 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.018 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.018 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2023-11-06T01:50:00.018 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.019 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.019 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.019 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.019 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2023-11-06T01:50:00.020 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.020 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.020 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.020 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.020 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2023-11-06T01:50:00.021 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.021 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.021 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.021 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.021 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2023-11-06T01:50:00.021 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.022 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.022 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.022 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.022 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2023-11-06T01:50:00.023 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.023 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.023 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.023 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.023 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2023-11-06T01:50:00.024 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.024 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.024 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.024 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.024 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2023-11-06T01:50:00.025 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.025 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.025 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.025 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.025 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2023-11-06T01:50:00.026 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.026 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.026 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.026 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.026 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2023-11-06T01:50:00.027 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.027 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.027 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.027 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.027 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2023-11-06T01:50:00.028 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.028 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.028 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.028 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.028 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2023-11-06T01:50:00.029 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.029 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.029 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.029 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.029 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2023-11-06T01:50:00.030 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.030 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.030 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.030 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.030 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2023-11-06T01:50:00.031 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.031 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.031 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.031 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.031 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2023-11-06T01:50:00.032 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.032 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.032 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.032 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.032 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2023-11-06T01:50:00.033 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.033 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.033 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.033 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.033 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2023-11-06T01:50:00.034 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.034 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.034 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.034 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.034 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2023-11-06T01:50:00.035 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.035 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.035 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.035 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.035 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2023-11-06T01:50:00.036 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.036 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.036 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.036 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.036 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2023-11-06T01:50:00.037 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.037 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.037 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.037 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.037 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2023-11-06T01:50:00.037 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.038 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.038 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.038 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.038 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2023-11-06T01:50:00.038 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.039 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.039 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.039 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.039 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2023-11-06T01:50:00.039 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.039 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.040 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.040 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.040 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2023-11-06T01:50:00.040 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.040 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.041 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.041 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.041 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2023-11-06T01:50:00.041 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.041 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.041 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.042 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.042 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2023-11-06T01:50:00.042 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.042 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.042 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.043 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.043 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2023-11-06T01:50:00.043 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2023-11-06T01:50:00.043 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2023-11-06T01:50:00.043 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.043 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.044 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.044 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:00.044 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.044 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2023-11-06T01:50:00.044 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 1, 2023-11-06T01:50:00.045 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 2, 2023-11-06T01:50:00.045 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 1, 2023-11-06T01:50:00.045 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 2, 2023-11-06T01:50:00.045 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2023-11-06T01:50:00.045 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 0, 2023-11-06T01:50:00.045 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 0, 2023-11-06T01:50:00.046 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 3651, 2023-11-06T01:50:00.046 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2023-11-06T01:50:00.046 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.046 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2023-11-06T01:50:00.046 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2023-11-06T01:50:00.047 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2023-11-06T01:50:00.047 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:50:00.047 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:50:00.047 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:50:00.047 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:50:00.048 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:50:00.048 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:50:00.048 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:50:00.048 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:50:00.048 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:50:00.048 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:50:00.049 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:50:00.049 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:50:00.049 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:50:00.049 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2023-11-06T01:50:00.049 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:50:00.050 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2023-11-06T01:50:00.050 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:50:00.050 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:50:00.050 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:50:00.050 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:50:00.051 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:50:00.051 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:50:00.051 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:50:00.051 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2023-11-06T01:50:00.051 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:50:00.052 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:50:00.052 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:50:00.052 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2023-11-06T01:50:00.052 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2023-11-06T01:50:00.052 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2023-11-06T01:50:00.052 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:50:00.053 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:50:00.053 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:50:00.053 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:50:00.053 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 194674, 2023-11-06T01:50:00.053 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 193, 2023-11-06T01:50:00.053 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2023-11-06T01:50:00.054 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2023-11-06T01:50:00.054 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2023-11-06T01:50:00.054 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2023-11-06T01:50:00.054 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:50:00.055 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2023-11-06T01:50:00.055 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:50:00.055 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2023-11-06T01:50:00.055 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 7720, 2023-11-06T01:50:00.055 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 162, 2023-11-06T01:50:00.055 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:50:00.056 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:50:00.056 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2023-11-06T01:50:00.056 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2023-11-06T01:50:00.056 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 548425, 2023-11-06T01:50:00.056 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 4227, 2023-11-06T01:50:00.056 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2023-11-06T01:50:00.057 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2023-11-06T01:50:00.057 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2023-11-06T01:50:00.057 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2023-11-06T01:50:00.057 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.057 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2023-11-06T01:50:00.057 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2023-11-06T01:50:00.058 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2023-11-06T01:50:00.058 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 3404, 2023-11-06T01:50:00.058 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 26, 2023-11-06T01:50:00.058 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 4, 2023-11-06T01:50:00.058 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 3404, 2023-11-06T01:50:00.059 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2023-11-06T01:50:00.059 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3404, 2023-11-06T01:50:00.059 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 10.151766395, 2023-11-06T01:50:00.059 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.002982305 2023-11-06T01:50:00.059 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.060 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2023-11-06T01:50:00.060 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2023-11-06T01:50:00.060 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3404, 2023-11-06T01:50:00.060 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 3405 2023-11-06T01:50:00.060 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.060 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 3404, 2023-11-06T01:50:00.061 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 3404, 2023-11-06T01:50:00.061 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 0, 2023-11-06T01:50:00.061 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2023-11-06T01:50:00.061 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2023-11-06T01:50:00.061 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 2059, 2023-11-06T01:50:00.062 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 0, 2023-11-06T01:50:00.062 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 1342, 2023-11-06T01:50:00.062 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 0, 2023-11-06T01:50:00.062 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 0, 2023-11-06T01:50:00.062 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2023-11-06T01:50:00.062 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2023-11-06T01:50:00.063 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2023-11-06T01:50:00.063 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 0, 2023-11-06T01:50:00.063 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 0, 2023-11-06T01:50:00.063 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2023-11-06T01:50:00.063 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2023-11-06T01:50:00.064 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2023-11-06T01:50:00.064 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 2, 2023-11-06T01:50:00.064 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 0, 2023-11-06T01:50:00.064 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2023-11-06T01:50:00.064 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2023-11-06T01:50:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2023-11-06T01:50:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2023-11-06T01:50:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2023-11-06T01:50:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2023-11-06T01:50:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T01:50:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2023-11-06T01:50:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2023-11-06T01:50:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 0, 2023-11-06T01:50:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2023-11-06T01:50:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2023-11-06T01:50:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2023-11-06T01:50:00.067 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2023-11-06T01:50:00.067 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2023-11-06T01:50:00.067 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2023-11-06T01:50:00.067 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2023-11-06T01:50:00.067 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2023-11-06T01:50:00.067 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2023-11-06T01:50:00.068 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2023-11-06T01:50:00.068 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2023-11-06T01:50:00.068 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2023-11-06T01:50:00.068 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2023-11-06T01:50:00.068 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2023-11-06T01:50:00.069 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2023-11-06T01:50:00.069 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2023-11-06T01:50:00.069 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 84, 2023-11-06T01:50:00.069 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2023-11-06T01:50:00.069 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 2, 2023-11-06T01:50:00.069 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 5, 2023-11-06T01:50:00.070 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 5, 2023-11-06T01:50:00.070 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 0, 2023-11-06T01:50:00.070 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2023-11-06T01:50:00.070 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 0, 2023-11-06T01:50:00.070 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 2, 2023-11-06T01:50:00.071 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 0 2023-11-06T01:50:00.071 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.071 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2023-11-06T01:50:00.071 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 0, 2023-11-06T01:50:00.071 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2023-11-06T01:50:00.071 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2023-11-06T01:50:00.072 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2023-11-06T01:50:00.072 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.072 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2023-11-06T01:50:00.072 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 0 2023-11-06T01:50:00.072 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.073 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2023-11-06T01:50:00.073 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2023-11-06T01:50:00.073 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 0, 2023-11-06T01:50:00.073 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2023-11-06T01:50:00.073 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 0, 2023-11-06T01:50:00.074 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 0, 2023-11-06T01:50:00.074 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2023-11-06T01:50:00.074 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.074 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:50:00.074 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:00.075 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2023-11-06T01:50:00.075 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:00.075 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 3732, 2023-11-06T01:50:00.075 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 10535259, 2023-11-06T01:50:00.075 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:00.075 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 3732, 2023-11-06T01:50:00.076 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:50:00.076 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:50:00.076 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3732, 2023-11-06T01:50:00.076 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 10535259, 2023-11-06T01:50:00.076 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:00.077 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.077 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.077 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.077 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:00.077 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.077 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2023-11-06T01:50:00.078 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:00.078 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2023-11-06T01:50:00.078 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:00.078 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:50:00.078 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:50:00.079 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:00.079 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:50:00.079 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 3404, 2023-11-06T01:50:00.079 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 9271025, 2023-11-06T01:50:00.079 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3404, 2023-11-06T01:50:00.079 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 9271025, 2023-11-06T01:50:00.080 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:00.080 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.080 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.080 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.080 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:00.081 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.081 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2023-11-06T01:50:00.081 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:00.081 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2023-11-06T01:50:00.081 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:00.081 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:50:00.082 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:50:00.082 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:00.082 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:50:00.082 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 3404, 2023-11-06T01:50:00.082 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 3404, 2023-11-06T01:50:00.083 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3404, 2023-11-06T01:50:00.083 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 3404, 2023-11-06T01:50:00.083 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:00.083 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.083 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.084 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.084 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:00.084 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.084 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2023-11-06T01:50:00.084 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:00.084 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2023-11-06T01:50:00.085 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:00.085 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:50:00.085 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:50:00.085 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:00.085 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:50:00.086 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:50:00.086 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:50:00.086 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2023-11-06T01:50:00.086 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2023-11-06T01:50:00.086 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:00.087 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.087 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.087 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.087 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:00.087 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:00.087 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x55cfa69fc640": { 2023-11-06T01:50:00.088 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:00.088 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2023-11-06T01:50:00.088 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:00.088 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:50:00.088 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:50:00.088 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:00.089 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:50:00.089 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:50:00.089 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:50:00.089 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2023-11-06T01:50:00.089 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2023-11-06T01:50:00.090 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:00.090 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:00.090 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:00.090 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:00.090 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:00.091 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:00.091 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-11-06T01:50:00.288 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:00 smithi163 ceph-mon[104416]: overall HEALTH_OK 2023-11-06T01:50:00.293 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-11-06T01:50:00.293 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-11-06T01:50:00.293 DEBUG:tasks.check_counter:Getting stats from i 2023-11-06T01:50:00.294 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-11-06T01:50:00.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:00 smithi053 ceph-mon[99472]: overall HEALTH_OK 2023-11-06T01:50:00.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:00 smithi062 ceph-mon[103924]: overall HEALTH_OK 2023-11-06T01:50:00.864 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:50:01.373 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:01 smithi163 ceph-mon[104416]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-11-06T01:50:01.471 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:01 smithi053 ceph-mon[99472]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-11-06T01:50:01.506 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:01 smithi062 ceph-mon[103924]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-11-06T01:50:02.088 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-11-06T01:50:02.088 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:50:02.089 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 2566304, 2023-11-06T01:50:02.089 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 2567325, 2023-11-06T01:50:02.089 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 406588087, 2023-11-06T01:50:02.089 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 425272461, 2023-11-06T01:50:02.089 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 10, 2023-11-06T01:50:02.090 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 4, 2023-11-06T01:50:02.090 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 89.647970343, 2023-11-06T01:50:02.090 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 38.009594944, 2023-11-06T01:50:02.090 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 62.908466767, 2023-11-06T01:50:02.090 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.125720515, 2023-11-06T01:50:02.091 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:50:02.091 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2567327, 2023-11-06T01:50:02.091 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 39.116363162, 2023-11-06T01:50:02.091 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000015236 2023-11-06T01:50:02.091 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.091 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:50:02.092 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 5128634, 2023-11-06T01:50:02.092 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.482896847, 2023-11-06T01:50:02.092 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000094 2023-11-06T01:50:02.092 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.092 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-06T01:50:02.093 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 0 2023-11-06T01:50:02.093 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.093 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:50:02.093 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 11304, 2023-11-06T01:50:02.093 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 12693, 2023-11-06T01:50:02.094 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 4053592, 2023-11-06T01:50:02.094 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 6894748, 2023-11-06T01:50:02.094 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 14, 2023-11-06T01:50:02.094 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 5, 2023-11-06T01:50:02.095 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.703773332, 2023-11-06T01:50:02.095 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.228902575, 2023-11-06T01:50:02.095 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.419144704, 2023-11-06T01:50:02.095 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.157710382, 2023-11-06T01:50:02.095 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:50:02.096 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 12693, 2023-11-06T01:50:02.096 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 3.217181199, 2023-11-06T01:50:02.096 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000253461 2023-11-06T01:50:02.096 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.096 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:50:02.097 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 19754, 2023-11-06T01:50:02.097 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.002897813, 2023-11-06T01:50:02.097 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000146 2023-11-06T01:50:02.097 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.097 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 687040, 2023-11-06T01:50:02.098 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 174208 2023-11-06T01:50:02.098 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.098 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:50:02.098 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1284060, 2023-11-06T01:50:02.098 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1284253, 2023-11-06T01:50:02.099 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 203515159, 2023-11-06T01:50:02.099 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 212076347, 2023-11-06T01:50:02.099 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 219, 2023-11-06T01:50:02.099 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 10, 2023-11-06T01:50:02.099 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 44.496888341, 2023-11-06T01:50:02.100 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 16.858675370, 2023-11-06T01:50:02.100 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 31.884005611, 2023-11-06T01:50:02.100 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.092729550, 2023-11-06T01:50:02.100 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:50:02.101 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1284247, 2023-11-06T01:50:02.101 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 29.611393959, 2023-11-06T01:50:02.101 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000023057 2023-11-06T01:50:02.101 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.101 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:50:02.102 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2565502, 2023-11-06T01:50:02.102 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.292941475, 2023-11-06T01:50:02.102 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000114 2023-11-06T01:50:02.102 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.102 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 56656, 2023-11-06T01:50:02.103 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 497456 2023-11-06T01:50:02.103 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.103 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2023-11-06T01:50:02.103 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2023-11-06T01:50:02.103 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2023-11-06T01:50:02.103 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.104 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2023-11-06T01:50:02.104 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2023-11-06T01:50:02.104 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2023-11-06T01:50:02.104 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3141, 2023-11-06T01:50:02.104 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 1.975646120, 2023-11-06T01:50:02.105 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000628986 2023-11-06T01:50:02.105 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:02.105 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.105 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2023-11-06T01:50:02.105 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2023-11-06T01:50:02.106 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2023-11-06T01:50:02.106 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 206, 2023-11-06T01:50:02.106 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.009929854, 2023-11-06T01:50:02.106 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000048203 2023-11-06T01:50:02.106 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:02.107 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.107 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2023-11-06T01:50:02.107 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 3015, 2023-11-06T01:50:02.107 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 2235, 2023-11-06T01:50:02.107 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2023-11-06T01:50:02.108 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2235, 2023-11-06T01:50:02.108 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 8.398919913, 2023-11-06T01:50:02.108 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.003757906 2023-11-06T01:50:02.108 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.108 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2023-11-06T01:50:02.109 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 780, 2023-11-06T01:50:02.109 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 5, 2023-11-06T01:50:02.109 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 2, 2023-11-06T01:50:02.109 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 623, 2023-11-06T01:50:02.109 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 3, 2023-11-06T01:50:02.110 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2023-11-06T01:50:02.110 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 167, 2023-11-06T01:50:02.110 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 6, 2023-11-06T01:50:02.110 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 150, 2023-11-06T01:50:02.110 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 11, 2023-11-06T01:50:02.111 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 159, 2023-11-06T01:50:02.111 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 1159, 2023-11-06T01:50:02.111 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 143, 2023-11-06T01:50:02.111 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 143, 2023-11-06T01:50:02.111 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 8, 2023-11-06T01:50:02.112 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 4486, 2023-11-06T01:50:02.112 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 2845, 2023-11-06T01:50:02.112 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 777, 2023-11-06T01:50:02.112 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 6, 2023-11-06T01:50:02.112 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2023-11-06T01:50:02.112 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2023-11-06T01:50:02.113 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 224, 2023-11-06T01:50:02.113 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 293, 2023-11-06T01:50:02.113 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2023-11-06T01:50:02.113 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 26, 2023-11-06T01:50:02.113 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 1132, 2023-11-06T01:50:02.114 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 6, 2023-11-06T01:50:02.114 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 207, 2023-11-06T01:50:02.114 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2023-11-06T01:50:02.114 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 301, 2023-11-06T01:50:02.114 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2023-11-06T01:50:02.115 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 1038, 2023-11-06T01:50:02.115 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 1747014300, 2023-11-06T01:50:02.115 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 0, 2023-11-06T01:50:02.115 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 301, 2023-11-06T01:50:02.115 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2023-11-06T01:50:02.116 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T01:50:02.116 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2023-11-06T01:50:02.116 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 41, 2023-11-06T01:50:02.116 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 2, 2023-11-06T01:50:02.116 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 39, 2023-11-06T01:50:02.116 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 260, 2023-11-06T01:50:02.117 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 10, 2023-11-06T01:50:02.117 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 228, 2023-11-06T01:50:02.117 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 1543, 2023-11-06T01:50:02.117 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T01:50:02.117 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T01:50:02.118 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T01:50:02.118 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 1313, 2023-11-06T01:50:02.118 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 1041, 2023-11-06T01:50:02.118 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 34, 2023-11-06T01:50:02.118 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 244 2023-11-06T01:50:02.119 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.119 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2023-11-06T01:50:02.119 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 0, 2023-11-06T01:50:02.119 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2023-11-06T01:50:02.119 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2023-11-06T01:50:02.119 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 6, 2023-11-06T01:50:02.120 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 35, 2023-11-06T01:50:02.120 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2023-11-06T01:50:02.120 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2023-11-06T01:50:02.120 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2023-11-06T01:50:02.120 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 85, 2023-11-06T01:50:02.121 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 52, 2023-11-06T01:50:02.121 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 132, 2023-11-06T01:50:02.121 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2023-11-06T01:50:02.121 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2023-11-06T01:50:02.121 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2023-11-06T01:50:02.122 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2023-11-06T01:50:02.122 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2023-11-06T01:50:02.122 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2023-11-06T01:50:02.122 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 2, 2023-11-06T01:50:02.122 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 26, 2023-11-06T01:50:02.123 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2023-11-06T01:50:02.123 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 3, 2023-11-06T01:50:02.123 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2023-11-06T01:50:02.123 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2023-11-06T01:50:02.123 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.124 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2023-11-06T01:50:02.124 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 3388, 2023-11-06T01:50:02.124 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 3375, 2023-11-06T01:50:02.124 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 3375, 2023-11-06T01:50:02.124 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 13, 2023-11-06T01:50:02.125 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2023-11-06T01:50:02.125 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 2, 2023-11-06T01:50:02.125 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 73, 2023-11-06T01:50:02.125 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 72, 2023-11-06T01:50:02.125 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 72, 2023-11-06T01:50:02.126 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 2, 2023-11-06T01:50:02.126 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2023-11-06T01:50:02.126 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 1, 2023-11-06T01:50:02.126 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 13342214, 2023-11-06T01:50:02.126 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 13471446, 2023-11-06T01:50:02.127 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 0, 2023-11-06T01:50:02.127 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2023-11-06T01:50:02.127 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1359, 2023-11-06T01:50:02.127 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 7.927627319, 2023-11-06T01:50:02.127 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.005833427 2023-11-06T01:50:02.127 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.128 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 0 2023-11-06T01:50:02.128 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.128 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2023-11-06T01:50:02.128 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 171, 2023-11-06T01:50:02.128 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 1326, 2023-11-06T01:50:02.129 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 1155, 2023-11-06T01:50:02.129 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 36, 2023-11-06T01:50:02.129 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 78, 2023-11-06T01:50:02.129 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 42, 2023-11-06T01:50:02.129 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 167, 2023-11-06T01:50:02.130 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 1722, 2023-11-06T01:50:02.130 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 1555, 2023-11-06T01:50:02.130 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 143, 2023-11-06T01:50:02.130 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 1309, 2023-11-06T01:50:02.130 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 1166, 2023-11-06T01:50:02.131 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 66804, 2023-11-06T01:50:02.131 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 207156 2023-11-06T01:50:02.131 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.131 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2023-11-06T01:50:02.131 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 3653, 2023-11-06T01:50:02.132 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 65, 2023-11-06T01:50:02.132 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 3015, 2023-11-06T01:50:02.132 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 444, 2023-11-06T01:50:02.132 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 127, 2023-11-06T01:50:02.132 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2023-11-06T01:50:02.133 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1062, 2023-11-06T01:50:02.133 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 6.162560817, 2023-11-06T01:50:02.133 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.005802787 2023-11-06T01:50:02.133 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.133 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2023-11-06T01:50:02.134 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 183, 2023-11-06T01:50:02.134 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.127801839, 2023-11-06T01:50:02.134 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000698370 2023-11-06T01:50:02.134 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.134 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2023-11-06T01:50:02.135 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.135 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.135 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.135 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.135 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2023-11-06T01:50:02.135 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 33, 2023-11-06T01:50:02.136 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.058890210, 2023-11-06T01:50:02.136 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001784551 2023-11-06T01:50:02.136 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.136 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2023-11-06T01:50:02.136 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 110, 2023-11-06T01:50:02.137 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.287564705, 2023-11-06T01:50:02.137 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.002614224 2023-11-06T01:50:02.137 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.137 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2023-11-06T01:50:02.137 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.138 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.138 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.138 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.138 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2023-11-06T01:50:02.138 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1, 2023-11-06T01:50:02.139 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000165827, 2023-11-06T01:50:02.139 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000165827 2023-11-06T01:50:02.139 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.139 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2023-11-06T01:50:02.140 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.140 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.140 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.140 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.140 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2023-11-06T01:50:02.140 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.141 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.141 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.141 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.141 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2023-11-06T01:50:02.141 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.142 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.142 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.142 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.142 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2023-11-06T01:50:02.142 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.143 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.143 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.143 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.143 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2023-11-06T01:50:02.143 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 35, 2023-11-06T01:50:02.144 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.253979582, 2023-11-06T01:50:02.144 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.007256559 2023-11-06T01:50:02.144 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.144 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2023-11-06T01:50:02.144 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.145 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.145 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.145 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.145 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2023-11-06T01:50:02.145 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.145 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.146 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.146 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.146 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2023-11-06T01:50:02.146 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 526, 2023-11-06T01:50:02.146 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.138393082, 2023-11-06T01:50:02.147 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000263104 2023-11-06T01:50:02.147 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.147 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2023-11-06T01:50:02.147 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 51, 2023-11-06T01:50:02.147 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.106074647, 2023-11-06T01:50:02.147 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.002079895 2023-11-06T01:50:02.148 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.148 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2023-11-06T01:50:02.148 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 172, 2023-11-06T01:50:02.148 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 1.150366192, 2023-11-06T01:50:02.148 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.006688175 2023-11-06T01:50:02.149 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.149 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2023-11-06T01:50:02.149 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.149 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.149 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.150 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.150 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2023-11-06T01:50:02.150 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.150 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.150 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.151 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.151 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2023-11-06T01:50:02.151 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.151 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.151 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.152 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.152 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2023-11-06T01:50:02.152 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.152 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.152 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.153 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.153 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2023-11-06T01:50:02.153 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.153 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.153 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.153 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.154 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2023-11-06T01:50:02.154 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1, 2023-11-06T01:50:02.154 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.002944474, 2023-11-06T01:50:02.154 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.002944474 2023-11-06T01:50:02.154 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.155 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2023-11-06T01:50:02.155 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.155 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.155 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.155 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.156 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2023-11-06T01:50:02.156 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.156 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.156 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.156 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.156 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2023-11-06T01:50:02.157 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.157 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.157 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.157 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.157 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2023-11-06T01:50:02.158 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 6, 2023-11-06T01:50:02.158 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.016996664, 2023-11-06T01:50:02.158 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.002832777 2023-11-06T01:50:02.158 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.158 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2023-11-06T01:50:02.159 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.159 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.159 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.159 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.159 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2023-11-06T01:50:02.160 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 55, 2023-11-06T01:50:02.160 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.093181874, 2023-11-06T01:50:02.160 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001694215 2023-11-06T01:50:02.160 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.160 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2023-11-06T01:50:02.160 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2023-11-06T01:50:02.161 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2023-11-06T01:50:02.161 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.161 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.161 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.161 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:02.162 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.162 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2023-11-06T01:50:02.162 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 1, 2023-11-06T01:50:02.162 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 2, 2023-11-06T01:50:02.162 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 1, 2023-11-06T01:50:02.163 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 1, 2023-11-06T01:50:02.163 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2023-11-06T01:50:02.163 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 3, 2023-11-06T01:50:02.163 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 3, 2023-11-06T01:50:02.163 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 967, 2023-11-06T01:50:02.164 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2023-11-06T01:50:02.164 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.164 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2023-11-06T01:50:02.164 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 360, 2023-11-06T01:50:02.164 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 360, 2023-11-06T01:50:02.165 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:50:02.165 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:50:02.165 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:50:02.165 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:50:02.165 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:50:02.165 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:50:02.166 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:50:02.166 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:50:02.166 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:50:02.166 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:50:02.166 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:50:02.167 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:50:02.167 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:50:02.167 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2023-11-06T01:50:02.167 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:50:02.167 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2023-11-06T01:50:02.168 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:50:02.168 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:50:02.168 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:50:02.168 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:50:02.168 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:50:02.169 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:50:02.169 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:50:02.169 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2023-11-06T01:50:02.169 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:50:02.169 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:50:02.169 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:50:02.170 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2023-11-06T01:50:02.170 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2023-11-06T01:50:02.170 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2023-11-06T01:50:02.170 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:50:02.170 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:50:02.171 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:50:02.171 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:50:02.171 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 311592, 2023-11-06T01:50:02.171 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 280, 2023-11-06T01:50:02.171 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2023-11-06T01:50:02.172 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2023-11-06T01:50:02.172 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2023-11-06T01:50:02.172 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2023-11-06T01:50:02.172 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:50:02.172 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2023-11-06T01:50:02.172 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:50:02.173 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2023-11-06T01:50:02.173 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 7720, 2023-11-06T01:50:02.173 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 162, 2023-11-06T01:50:02.173 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:50:02.173 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:50:02.174 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2023-11-06T01:50:02.174 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2023-11-06T01:50:02.174 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 607406, 2023-11-06T01:50:02.174 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 5700, 2023-11-06T01:50:02.174 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2023-11-06T01:50:02.175 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2023-11-06T01:50:02.175 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2023-11-06T01:50:02.175 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2023-11-06T01:50:02.175 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.175 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2023-11-06T01:50:02.176 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2023-11-06T01:50:02.176 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2023-11-06T01:50:02.176 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 4259, 2023-11-06T01:50:02.176 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 11352059, 2023-11-06T01:50:02.176 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 36, 2023-11-06T01:50:02.177 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 4259, 2023-11-06T01:50:02.177 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2023-11-06T01:50:02.177 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 4259, 2023-11-06T01:50:02.177 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 91.252872323, 2023-11-06T01:50:02.177 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.021425891 2023-11-06T01:50:02.178 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.178 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2023-11-06T01:50:02.178 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2023-11-06T01:50:02.178 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 4259, 2023-11-06T01:50:02.178 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 7443 2023-11-06T01:50:02.179 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.179 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 4259, 2023-11-06T01:50:02.179 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 333, 2023-11-06T01:50:02.179 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 3926, 2023-11-06T01:50:02.179 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2023-11-06T01:50:02.180 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2023-11-06T01:50:02.180 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 553, 2023-11-06T01:50:02.180 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 971, 2023-11-06T01:50:02.180 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 23, 2023-11-06T01:50:02.180 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 1380, 2023-11-06T01:50:02.181 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 245, 2023-11-06T01:50:02.181 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2023-11-06T01:50:02.181 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2023-11-06T01:50:02.181 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2023-11-06T01:50:02.181 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 3, 2023-11-06T01:50:02.182 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 583, 2023-11-06T01:50:02.182 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2023-11-06T01:50:02.182 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2023-11-06T01:50:02.182 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2023-11-06T01:50:02.182 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 310, 2023-11-06T01:50:02.182 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 1942, 2023-11-06T01:50:02.183 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2023-11-06T01:50:02.183 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2023-11-06T01:50:02.183 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2023-11-06T01:50:02.183 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2023-11-06T01:50:02.183 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2023-11-06T01:50:02.184 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2023-11-06T01:50:02.184 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T01:50:02.184 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2023-11-06T01:50:02.184 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2023-11-06T01:50:02.184 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 0, 2023-11-06T01:50:02.185 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2023-11-06T01:50:02.185 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2023-11-06T01:50:02.185 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2023-11-06T01:50:02.185 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2023-11-06T01:50:02.185 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2023-11-06T01:50:02.186 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2023-11-06T01:50:02.186 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2023-11-06T01:50:02.186 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2023-11-06T01:50:02.186 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2023-11-06T01:50:02.186 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2023-11-06T01:50:02.187 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2023-11-06T01:50:02.187 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2023-11-06T01:50:02.187 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2023-11-06T01:50:02.187 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2023-11-06T01:50:02.187 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2023-11-06T01:50:02.187 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2023-11-06T01:50:02.188 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 84, 2023-11-06T01:50:02.188 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2023-11-06T01:50:02.188 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 3, 2023-11-06T01:50:02.188 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 12, 2023-11-06T01:50:02.188 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 12, 2023-11-06T01:50:02.189 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 0, 2023-11-06T01:50:02.189 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2023-11-06T01:50:02.189 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 1357, 2023-11-06T01:50:02.189 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 14, 2023-11-06T01:50:02.189 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 62 2023-11-06T01:50:02.190 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.190 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2023-11-06T01:50:02.190 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 1, 2023-11-06T01:50:02.190 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2023-11-06T01:50:02.190 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2023-11-06T01:50:02.191 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2023-11-06T01:50:02.191 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.191 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2023-11-06T01:50:02.191 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 3113 2023-11-06T01:50:02.191 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.192 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2023-11-06T01:50:02.192 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2023-11-06T01:50:02.192 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 19, 2023-11-06T01:50:02.192 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2023-11-06T01:50:02.192 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 6, 2023-11-06T01:50:02.192 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 52, 2023-11-06T01:50:02.193 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2023-11-06T01:50:02.193 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.193 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:50:02.193 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:02.193 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2023-11-06T01:50:02.194 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:02.194 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 3861664, 2023-11-06T01:50:02.194 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 266579232, 2023-11-06T01:50:02.194 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:02.194 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 3861664, 2023-11-06T01:50:02.195 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:50:02.195 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:50:02.195 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3861664, 2023-11-06T01:50:02.195 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 266579232, 2023-11-06T01:50:02.195 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:02.196 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.196 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.196 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.196 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:02.196 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.196 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2023-11-06T01:50:02.197 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:02.197 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2023-11-06T01:50:02.197 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:02.197 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:50:02.197 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:50:02.198 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:02.198 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:50:02.198 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 4259, 2023-11-06T01:50:02.198 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 11357223, 2023-11-06T01:50:02.198 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 4259, 2023-11-06T01:50:02.199 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 11357223, 2023-11-06T01:50:02.199 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:02.199 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.199 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.199 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.200 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:02.200 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.200 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2023-11-06T01:50:02.200 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:02.200 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2023-11-06T01:50:02.201 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:02.201 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:50:02.201 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:50:02.201 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:02.201 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:50:02.202 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 4259, 2023-11-06T01:50:02.202 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 4259, 2023-11-06T01:50:02.202 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 4259, 2023-11-06T01:50:02.202 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 4259, 2023-11-06T01:50:02.202 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:02.203 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.203 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.203 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.203 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:02.203 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.204 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2023-11-06T01:50:02.204 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:02.204 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2023-11-06T01:50:02.204 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:02.204 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 52, 2023-11-06T01:50:02.205 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 5252, 2023-11-06T01:50:02.205 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:02.205 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 52, 2023-11-06T01:50:02.205 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:50:02.205 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:50:02.205 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 21, 2023-11-06T01:50:02.206 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 5252, 2023-11-06T01:50:02.206 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:02.206 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.206 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.206 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.207 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:02.207 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:02.207 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x55c8df9e20a0": { 2023-11-06T01:50:02.207 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:02.207 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2023-11-06T01:50:02.208 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:02.208 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 3388, 2023-11-06T01:50:02.208 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 9277142, 2023-11-06T01:50:02.208 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:02.208 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 3388, 2023-11-06T01:50:02.209 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:50:02.209 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:50:02.209 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 1359, 2023-11-06T01:50:02.209 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 9277142, 2023-11-06T01:50:02.209 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:02.209 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:02.210 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:02.210 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:02.210 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:02.210 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:02.210 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-11-06T01:50:02.428 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:02 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:50:02.429 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:02 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:50:02.429 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:02 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:50:02.429 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:02 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:50:02.431 INFO:tasks.check_counter:Daemon mds.i mds.exported=26 2023-11-06T01:50:02.431 INFO:tasks.check_counter:Daemon mds.i mds.imported=6 2023-11-06T01:50:02.431 DEBUG:tasks.check_counter:Getting stats from l 2023-11-06T01:50:02.432 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7e9f9af51acf98e0ccc3037487018e4ae40caa5 shell --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-11-06T01:50:02.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:02 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:50:02.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:02 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:50:02.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:02 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:50:02.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:02 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:50:02.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:02 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-06T01:50:02.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:02 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-06T01:50:02.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:02 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-06T01:50:02.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:02 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' 2023-11-06T01:50:03.012 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/mon.c/config 2023-11-06T01:50:03.435 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:03 smithi163 ceph-mon[104416]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-11-06T01:50:03.591 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:03 smithi053 ceph-mon[99472]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-11-06T01:50:03.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:03 smithi062 ceph-mon[103924]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-11-06T01:50:04.238 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-11-06T01:50:04.238 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T01:50:04.238 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 334, 2023-11-06T01:50:04.238 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 301, 2023-11-06T01:50:04.239 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 672434, 2023-11-06T01:50:04.239 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 167981, 2023-11-06T01:50:04.239 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 6, 2023-11-06T01:50:04.239 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 1, 2023-11-06T01:50:04.239 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.041556462, 2023-11-06T01:50:04.240 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.014228153, 2023-11-06T01:50:04.240 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.019446723, 2023-11-06T01:50:04.240 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.007572790, 2023-11-06T01:50:04.240 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:50:04.240 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 301, 2023-11-06T01:50:04.241 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.071758969, 2023-11-06T01:50:04.241 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000238401 2023-11-06T01:50:04.241 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.241 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:50:04.241 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.242 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.242 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.242 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.242 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 671600, 2023-11-06T01:50:04.242 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 167664 2023-11-06T01:50:04.243 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.243 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T01:50:04.243 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 934, 2023-11-06T01:50:04.243 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1154, 2023-11-06T01:50:04.243 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 229614, 2023-11-06T01:50:04.244 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 829729, 2023-11-06T01:50:04.244 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 7, 2023-11-06T01:50:04.244 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 4, 2023-11-06T01:50:04.244 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.080130171, 2023-11-06T01:50:04.244 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.022894291, 2023-11-06T01:50:04.244 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.016842132, 2023-11-06T01:50:04.245 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.033286734, 2023-11-06T01:50:04.245 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:50:04.245 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1152, 2023-11-06T01:50:04.245 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.225427793, 2023-11-06T01:50:04.245 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000195683 2023-11-06T01:50:04.246 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.246 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:50:04.246 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.247 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.248 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.248 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.248 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-06T01:50:04.248 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 539872 2023-11-06T01:50:04.249 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.249 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T01:50:04.249 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 2389, 2023-11-06T01:50:04.249 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 2391, 2023-11-06T01:50:04.249 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 4549693, 2023-11-06T01:50:04.250 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 743037, 2023-11-06T01:50:04.250 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 4, 2023-11-06T01:50:04.250 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 3, 2023-11-06T01:50:04.251 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.176776609, 2023-11-06T01:50:04.251 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.044547975, 2023-11-06T01:50:04.252 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.049175161, 2023-11-06T01:50:04.252 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.068717354, 2023-11-06T01:50:04.252 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T01:50:04.252 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2389, 2023-11-06T01:50:04.253 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.281054589, 2023-11-06T01:50:04.253 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000117645 2023-11-06T01:50:04.253 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.253 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2023-11-06T01:50:04.253 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.254 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.254 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.254 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.254 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-06T01:50:04.254 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 0 2023-11-06T01:50:04.255 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.255 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2023-11-06T01:50:04.255 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2023-11-06T01:50:04.255 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2023-11-06T01:50:04.255 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.256 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2023-11-06T01:50:04.256 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2023-11-06T01:50:04.256 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2023-11-06T01:50:04.256 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 5662, 2023-11-06T01:50:04.256 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.138521913, 2023-11-06T01:50:04.256 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000024465 2023-11-06T01:50:04.257 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:04.257 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.257 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2023-11-06T01:50:04.257 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2023-11-06T01:50:04.257 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2023-11-06T01:50:04.258 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.258 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.258 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.258 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:04.258 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.259 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2023-11-06T01:50:04.259 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 0, 2023-11-06T01:50:04.259 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 0, 2023-11-06T01:50:04.259 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2023-11-06T01:50:04.259 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.259 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.260 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.260 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.260 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2023-11-06T01:50:04.260 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 0, 2023-11-06T01:50:04.260 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 0, 2023-11-06T01:50:04.261 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 0, 2023-11-06T01:50:04.261 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 0, 2023-11-06T01:50:04.261 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 0, 2023-11-06T01:50:04.261 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2023-11-06T01:50:04.261 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 476, 2023-11-06T01:50:04.261 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 44, 2023-11-06T01:50:04.262 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 422, 2023-11-06T01:50:04.262 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 10, 2023-11-06T01:50:04.262 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 413, 2023-11-06T01:50:04.262 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 1, 2023-11-06T01:50:04.263 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 0, 2023-11-06T01:50:04.263 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 0, 2023-11-06T01:50:04.263 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 12, 2023-11-06T01:50:04.263 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 0, 2023-11-06T01:50:04.263 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 0, 2023-11-06T01:50:04.263 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 0, 2023-11-06T01:50:04.264 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 0, 2023-11-06T01:50:04.264 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2023-11-06T01:50:04.264 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2023-11-06T01:50:04.264 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 0, 2023-11-06T01:50:04.264 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 0, 2023-11-06T01:50:04.265 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2023-11-06T01:50:04.265 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 0, 2023-11-06T01:50:04.265 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 0, 2023-11-06T01:50:04.265 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 0, 2023-11-06T01:50:04.265 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 0, 2023-11-06T01:50:04.265 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2023-11-06T01:50:04.266 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 0, 2023-11-06T01:50:04.266 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2023-11-06T01:50:04.266 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 0, 2023-11-06T01:50:04.266 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 0, 2023-11-06T01:50:04.266 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 0, 2023-11-06T01:50:04.267 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 0, 2023-11-06T01:50:04.267 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2023-11-06T01:50:04.267 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T01:50:04.267 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2023-11-06T01:50:04.267 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 0, 2023-11-06T01:50:04.268 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 0, 2023-11-06T01:50:04.268 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 0, 2023-11-06T01:50:04.268 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 0, 2023-11-06T01:50:04.268 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 0, 2023-11-06T01:50:04.268 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 0, 2023-11-06T01:50:04.269 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 0, 2023-11-06T01:50:04.269 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T01:50:04.269 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T01:50:04.269 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T01:50:04.269 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 0, 2023-11-06T01:50:04.269 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 0, 2023-11-06T01:50:04.270 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 0, 2023-11-06T01:50:04.270 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 0 2023-11-06T01:50:04.270 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.270 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2023-11-06T01:50:04.270 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 0, 2023-11-06T01:50:04.271 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2023-11-06T01:50:04.271 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2023-11-06T01:50:04.271 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 0, 2023-11-06T01:50:04.271 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 0, 2023-11-06T01:50:04.271 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2023-11-06T01:50:04.272 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2023-11-06T01:50:04.272 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2023-11-06T01:50:04.272 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 0, 2023-11-06T01:50:04.272 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 0, 2023-11-06T01:50:04.272 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 0, 2023-11-06T01:50:04.273 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2023-11-06T01:50:04.273 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2023-11-06T01:50:04.273 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2023-11-06T01:50:04.273 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2023-11-06T01:50:04.273 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2023-11-06T01:50:04.274 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2023-11-06T01:50:04.274 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 0, 2023-11-06T01:50:04.274 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 0, 2023-11-06T01:50:04.274 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2023-11-06T01:50:04.274 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 0, 2023-11-06T01:50:04.274 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2023-11-06T01:50:04.275 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2023-11-06T01:50:04.275 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.275 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2023-11-06T01:50:04.275 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 0, 2023-11-06T01:50:04.275 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 0, 2023-11-06T01:50:04.276 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 0, 2023-11-06T01:50:04.276 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 1862, 2023-11-06T01:50:04.276 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2023-11-06T01:50:04.276 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 0, 2023-11-06T01:50:04.276 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 0, 2023-11-06T01:50:04.276 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 0, 2023-11-06T01:50:04.277 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 0, 2023-11-06T01:50:04.277 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 2, 2023-11-06T01:50:04.277 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2023-11-06T01:50:04.277 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 0, 2023-11-06T01:50:04.277 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 4194304, 2023-11-06T01:50:04.278 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 8046494, 2023-11-06T01:50:04.278 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 8045362, 2023-11-06T01:50:04.278 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2023-11-06T01:50:04.278 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.278 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.279 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.279 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.279 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 1862 2023-11-06T01:50:04.279 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.279 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2023-11-06T01:50:04.279 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 261, 2023-11-06T01:50:04.280 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 1028, 2023-11-06T01:50:04.280 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 767, 2023-11-06T01:50:04.280 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 37, 2023-11-06T01:50:04.280 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 62, 2023-11-06T01:50:04.280 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 25, 2023-11-06T01:50:04.281 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 476, 2023-11-06T01:50:04.281 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 1046, 2023-11-06T01:50:04.281 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 570, 2023-11-06T01:50:04.281 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 0, 2023-11-06T01:50:04.281 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 0, 2023-11-06T01:50:04.282 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 0, 2023-11-06T01:50:04.282 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 51060, 2023-11-06T01:50:04.282 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 207156 2023-11-06T01:50:04.282 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.282 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2023-11-06T01:50:04.283 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 0, 2023-11-06T01:50:04.283 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 0, 2023-11-06T01:50:04.283 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 0, 2023-11-06T01:50:04.283 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 0, 2023-11-06T01:50:04.283 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 0, 2023-11-06T01:50:04.283 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2023-11-06T01:50:04.284 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.284 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.284 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.284 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.284 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2023-11-06T01:50:04.285 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.285 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.285 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.285 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.285 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2023-11-06T01:50:04.285 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.286 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.286 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.286 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.286 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2023-11-06T01:50:04.286 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.287 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.287 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.287 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.287 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2023-11-06T01:50:04.287 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.288 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.288 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.288 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.288 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2023-11-06T01:50:04.288 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.288 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.289 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.289 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.289 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2023-11-06T01:50:04.289 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.289 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.290 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.290 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.290 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2023-11-06T01:50:04.290 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.290 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.291 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.291 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.291 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2023-11-06T01:50:04.291 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.291 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.292 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.292 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.292 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2023-11-06T01:50:04.292 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.292 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.292 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.293 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.293 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2023-11-06T01:50:04.293 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.293 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.293 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.294 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.294 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2023-11-06T01:50:04.294 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.294 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.294 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.295 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.295 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2023-11-06T01:50:04.295 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.295 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.295 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.296 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.296 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2023-11-06T01:50:04.296 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.296 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.296 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.296 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.297 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2023-11-06T01:50:04.297 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.297 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.297 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.297 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.298 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2023-11-06T01:50:04.298 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.298 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.298 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.298 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.299 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2023-11-06T01:50:04.299 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.299 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.299 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.299 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.300 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2023-11-06T01:50:04.300 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.300 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.300 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.300 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.301 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2023-11-06T01:50:04.301 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.301 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.301 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.301 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.301 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2023-11-06T01:50:04.302 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.302 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.302 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.302 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.302 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2023-11-06T01:50:04.303 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.303 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.303 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.303 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.303 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2023-11-06T01:50:04.304 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.304 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.304 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.304 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.304 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2023-11-06T01:50:04.305 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.305 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.305 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.305 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.305 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2023-11-06T01:50:04.306 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.306 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.306 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.306 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.306 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2023-11-06T01:50:04.307 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.307 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.307 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.307 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.307 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2023-11-06T01:50:04.307 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.308 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.308 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.308 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.308 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2023-11-06T01:50:04.308 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.309 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.309 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.309 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.309 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2023-11-06T01:50:04.309 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.310 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.310 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.310 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.310 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2023-11-06T01:50:04.310 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.311 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.311 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.311 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.311 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2023-11-06T01:50:04.311 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2023-11-06T01:50:04.311 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2023-11-06T01:50:04.312 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.312 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.312 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.312 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:04.312 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.313 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2023-11-06T01:50:04.313 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 1, 2023-11-06T01:50:04.313 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 1, 2023-11-06T01:50:04.313 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 0, 2023-11-06T01:50:04.313 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 1, 2023-11-06T01:50:04.314 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2023-11-06T01:50:04.314 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 0, 2023-11-06T01:50:04.314 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 0, 2023-11-06T01:50:04.314 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 2351, 2023-11-06T01:50:04.314 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2023-11-06T01:50:04.315 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.315 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2023-11-06T01:50:04.315 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2023-11-06T01:50:04.315 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2023-11-06T01:50:04.315 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T01:50:04.315 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2023-11-06T01:50:04.316 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T01:50:04.316 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2023-11-06T01:50:04.316 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T01:50:04.316 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T01:50:04.316 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T01:50:04.317 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T01:50:04.317 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T01:50:04.317 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2023-11-06T01:50:04.317 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T01:50:04.317 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T01:50:04.318 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2023-11-06T01:50:04.318 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2023-11-06T01:50:04.318 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2023-11-06T01:50:04.318 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2023-11-06T01:50:04.318 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T01:50:04.319 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T01:50:04.319 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T01:50:04.319 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T01:50:04.319 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T01:50:04.319 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2023-11-06T01:50:04.320 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2023-11-06T01:50:04.320 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2023-11-06T01:50:04.320 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T01:50:04.320 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T01:50:04.320 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2023-11-06T01:50:04.320 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2023-11-06T01:50:04.321 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2023-11-06T01:50:04.321 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2023-11-06T01:50:04.321 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T01:50:04.321 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2023-11-06T01:50:04.321 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T01:50:04.322 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2023-11-06T01:50:04.322 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 200339, 2023-11-06T01:50:04.322 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 307, 2023-11-06T01:50:04.322 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2023-11-06T01:50:04.322 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2023-11-06T01:50:04.323 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2023-11-06T01:50:04.323 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2023-11-06T01:50:04.323 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2023-11-06T01:50:04.323 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2023-11-06T01:50:04.323 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2023-11-06T01:50:04.324 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2023-11-06T01:50:04.324 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 7720, 2023-11-06T01:50:04.324 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 162, 2023-11-06T01:50:04.324 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T01:50:04.324 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2023-11-06T01:50:04.325 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2023-11-06T01:50:04.325 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2023-11-06T01:50:04.325 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 958754, 2023-11-06T01:50:04.325 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 7622, 2023-11-06T01:50:04.325 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2023-11-06T01:50:04.326 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2023-11-06T01:50:04.326 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2023-11-06T01:50:04.326 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2023-11-06T01:50:04.326 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.326 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2023-11-06T01:50:04.327 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2023-11-06T01:50:04.327 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2023-11-06T01:50:04.327 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 3322, 2023-11-06T01:50:04.327 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 21, 2023-11-06T01:50:04.327 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 3, 2023-11-06T01:50:04.327 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 3322, 2023-11-06T01:50:04.328 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2023-11-06T01:50:04.328 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3322, 2023-11-06T01:50:04.328 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 18.651571436, 2023-11-06T01:50:04.328 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.005614560 2023-11-06T01:50:04.328 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.329 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2023-11-06T01:50:04.329 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2023-11-06T01:50:04.329 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3322, 2023-11-06T01:50:04.329 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 3323 2023-11-06T01:50:04.329 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.330 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 3322, 2023-11-06T01:50:04.330 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 3322, 2023-11-06T01:50:04.330 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 0, 2023-11-06T01:50:04.330 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2023-11-06T01:50:04.330 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2023-11-06T01:50:04.331 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 1965, 2023-11-06T01:50:04.331 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 0, 2023-11-06T01:50:04.331 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 1355, 2023-11-06T01:50:04.331 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 0, 2023-11-06T01:50:04.331 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 0, 2023-11-06T01:50:04.331 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2023-11-06T01:50:04.332 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2023-11-06T01:50:04.332 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2023-11-06T01:50:04.332 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 0, 2023-11-06T01:50:04.332 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 0, 2023-11-06T01:50:04.332 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2023-11-06T01:50:04.333 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2023-11-06T01:50:04.333 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2023-11-06T01:50:04.333 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 1, 2023-11-06T01:50:04.333 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 0, 2023-11-06T01:50:04.333 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2023-11-06T01:50:04.334 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2023-11-06T01:50:04.334 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2023-11-06T01:50:04.334 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2023-11-06T01:50:04.334 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2023-11-06T01:50:04.334 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2023-11-06T01:50:04.335 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T01:50:04.335 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2023-11-06T01:50:04.335 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2023-11-06T01:50:04.335 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 0, 2023-11-06T01:50:04.335 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2023-11-06T01:50:04.336 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2023-11-06T01:50:04.336 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2023-11-06T01:50:04.336 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2023-11-06T01:50:04.336 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2023-11-06T01:50:04.336 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2023-11-06T01:50:04.336 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2023-11-06T01:50:04.337 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2023-11-06T01:50:04.337 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2023-11-06T01:50:04.337 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2023-11-06T01:50:04.337 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2023-11-06T01:50:04.337 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2023-11-06T01:50:04.338 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2023-11-06T01:50:04.338 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2023-11-06T01:50:04.338 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2023-11-06T01:50:04.338 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2023-11-06T01:50:04.338 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 84, 2023-11-06T01:50:04.339 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2023-11-06T01:50:04.339 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 2, 2023-11-06T01:50:04.339 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 5, 2023-11-06T01:50:04.339 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 7, 2023-11-06T01:50:04.339 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 2, 2023-11-06T01:50:04.339 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2023-11-06T01:50:04.340 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 0, 2023-11-06T01:50:04.340 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 2, 2023-11-06T01:50:04.340 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 0 2023-11-06T01:50:04.340 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.340 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2023-11-06T01:50:04.341 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 0, 2023-11-06T01:50:04.341 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2023-11-06T01:50:04.341 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2023-11-06T01:50:04.341 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2023-11-06T01:50:04.341 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.342 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2023-11-06T01:50:04.342 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 0 2023-11-06T01:50:04.342 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.342 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2023-11-06T01:50:04.342 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2023-11-06T01:50:04.343 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 0, 2023-11-06T01:50:04.343 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2023-11-06T01:50:04.343 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 0, 2023-11-06T01:50:04.343 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 0, 2023-11-06T01:50:04.343 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2023-11-06T01:50:04.343 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.344 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T01:50:04.344 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:04.344 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2023-11-06T01:50:04.344 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:04.344 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 3653, 2023-11-06T01:50:04.345 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 5103171, 2023-11-06T01:50:04.345 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:04.345 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 3653, 2023-11-06T01:50:04.345 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:50:04.345 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:50:04.345 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3653, 2023-11-06T01:50:04.346 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 5103171, 2023-11-06T01:50:04.346 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:04.346 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.346 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.346 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.347 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:04.347 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.347 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2023-11-06T01:50:04.347 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:04.347 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2023-11-06T01:50:04.348 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:04.348 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:50:04.348 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:50:04.348 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:04.348 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:50:04.349 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 3322, 2023-11-06T01:50:04.349 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 3852195, 2023-11-06T01:50:04.349 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3322, 2023-11-06T01:50:04.349 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 3852195, 2023-11-06T01:50:04.349 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:04.350 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.350 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.350 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.350 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:04.350 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.351 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2023-11-06T01:50:04.351 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:04.351 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2023-11-06T01:50:04.351 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:04.351 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:50:04.352 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:50:04.352 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:04.352 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:50:04.352 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 3322, 2023-11-06T01:50:04.352 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 3322, 2023-11-06T01:50:04.352 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3322, 2023-11-06T01:50:04.353 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 3322, 2023-11-06T01:50:04.353 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:04.353 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.353 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.353 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.354 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:04.354 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.354 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2023-11-06T01:50:04.354 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:04.354 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2023-11-06T01:50:04.355 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:04.355 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:50:04.355 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:50:04.355 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:04.355 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:50:04.355 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:50:04.356 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:50:04.356 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2023-11-06T01:50:04.356 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2023-11-06T01:50:04.356 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:04.356 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.357 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.357 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.357 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:04.357 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-11-06T01:50:04.357 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x55b347f1e5a0": { 2023-11-06T01:50:04.358 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2023-11-06T01:50:04.358 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2023-11-06T01:50:04.358 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2023-11-06T01:50:04.358 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2023-11-06T01:50:04.359 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2023-11-06T01:50:04.359 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2023-11-06T01:50:04.359 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2023-11-06T01:50:04.359 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2023-11-06T01:50:04.359 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2023-11-06T01:50:04.359 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2023-11-06T01:50:04.360 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2023-11-06T01:50:04.360 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2023-11-06T01:50:04.360 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2023-11-06T01:50:04.360 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2023-11-06T01:50:04.360 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2023-11-06T01:50:04.361 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:04.361 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-11-06T01:50:04.361 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-11-06T01:50:04.563 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-11-06T01:50:04.563 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-11-06T01:50:04.564 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-06T01:50:04.575 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-06T01:50:04.576 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:50:04.576 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T01:50:04.599 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:50:04.599 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T01:50:04.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:50:04.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.842+0000 7f8341a42700 1 -- 172.21.15.53:0/2507661369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f833c0ffc70 msgr2=0x7f833c100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:04.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.842+0000 7f8341a42700 1 --2- 172.21.15.53:0/2507661369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f833c0ffc70 0x7f833c100090 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f832c0099c0 tx=0x7f832c009cd0 comp rx=0 tx=0).stop 2023-11-06T01:50:04.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.842+0000 7f8341a42700 1 -- 172.21.15.53:0/2507661369 shutdown_connections 2023-11-06T01:50:04.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.842+0000 7f8341a42700 1 --2- 172.21.15.53:0/2507661369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f833c101800 0x7f833c1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:04.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.842+0000 7f8341a42700 1 --2- 172.21.15.53:0/2507661369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f833c0ffc70 0x7f833c100090 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:04.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.842+0000 7f8341a42700 1 --2- 172.21.15.53:0/2507661369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f833c100e60 0x7f833c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:04.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.842+0000 7f8341a42700 1 -- 172.21.15.53:0/2507661369 >> 172.21.15.53:0/2507661369 conn(0x7f833c0fb1c0 msgr2=0x7f833c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:04.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.842+0000 7f8341a42700 1 -- 172.21.15.53:0/2507661369 shutdown_connections 2023-11-06T01:50:04.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.842+0000 7f8341a42700 1 -- 172.21.15.53:0/2507661369 wait complete. 2023-11-06T01:50:04.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f8341a42700 1 Processor -- start 2023-11-06T01:50:04.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f8341a42700 1 -- start start 2023-11-06T01:50:04.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f8341a42700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f833c0ffc70 0x7f833c069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:04.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f8341a42700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f833c100e60 0x7f833c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:04.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f8341a42700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f833c101800 0x7f833c19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:04.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f8341a42700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c19e590 con 0x7f833c100e60 2023-11-06T01:50:04.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f8341a42700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c19e700 con 0x7f833c0ffc70 2023-11-06T01:50:04.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f8341a42700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c19e8c0 con 0x7f833c101800 2023-11-06T01:50:04.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f833affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f833c0ffc70 0x7f833c069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:04.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f833a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f833c100e60 0x7f833c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:04.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f833affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f833c0ffc70 0x7f833c069d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55642/0 (socket says 172.21.15.53:55642) 2023-11-06T01:50:04.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f833affd700 1 -- 172.21.15.53:0/2508416524 learned_addr learned my addr 172.21.15.53:0/2508416524 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:04.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f833b7fe700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f833c101800 0x7f833c19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:04.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f833affd700 1 -- 172.21.15.53:0/2508416524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f833c101800 msgr2=0x7f833c19ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:04.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f833affd700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f833c101800 0x7f833c19ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:04.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f833affd700 1 -- 172.21.15.53:0/2508416524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f833c100e60 msgr2=0x7f833c19b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:04.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f833affd700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f833c100e60 0x7f833c19b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:04.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f833affd700 1 -- 172.21.15.53:0/2508416524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0095f0 con 0x7f833c0ffc70 2023-11-06T01:50:04.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.843+0000 7f833a7fc700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f833c100e60 0x7f833c19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:50:04.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.844+0000 7f833affd700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f833c0ffc70 0x7f833c069d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f832c009fd0 tx=0x7f832c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:04.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.844+0000 7f8340a40700 1 -- 172.21.15.53:0/2508416524 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c01d070 con 0x7f833c0ffc70 2023-11-06T01:50:04.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.844+0000 7f8341a42700 1 -- 172.21.15.53:0/2508416524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833c19eb40 con 0x7f833c0ffc70 2023-11-06T01:50:04.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.844+0000 7f8340a40700 1 -- 172.21.15.53:0/2508416524 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f832c005030 con 0x7f833c0ffc70 2023-11-06T01:50:04.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.844+0000 7f8340a40700 1 -- 172.21.15.53:0/2508416524 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c004a90 con 0x7f833c0ffc70 2023-11-06T01:50:04.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.844+0000 7f8341a42700 1 -- 172.21.15.53:0/2508416524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c19efc0 con 0x7f833c0ffc70 2023-11-06T01:50:04.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.845+0000 7f8341a42700 1 -- 172.21.15.53:0/2508416524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833c107260 con 0x7f833c0ffc70 2023-11-06T01:50:04.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.846+0000 7f8340a40700 1 -- 172.21.15.53:0/2508416524 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f832c0051a0 con 0x7f833c0ffc70 2023-11-06T01:50:04.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.847+0000 7f8340a40700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83240a0570 0x7f83240a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:04.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.847+0000 7f833a7fc700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83240a0570 0x7f83240a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:04.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.847+0000 7f8340a40700 1 -- 172.21.15.53:0/2508416524 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f832c0ccb20 con 0x7f833c0ffc70 2023-11-06T01:50:04.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.847+0000 7f833a7fc700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83240a0570 0x7f83240a2a30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f833c069ab0 tx=0x7f8330009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:04.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:04.853+0000 7f8340a40700 1 -- 172.21.15.53:0/2508416524 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f832c096850 con 0x7f833c0ffc70 2023-11-06T01:50:05.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.055+0000 7f8341a42700 1 -- 172.21.15.53:0/2508416524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f833c02ce70 con 0x7f833c0ffc70 2023-11-06T01:50:05.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.058+0000 7f8340a40700 1 -- 172.21.15.53:0/2508416524 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22543 (secure 0 0 0) 0x7f832c026090 con 0x7f833c0ffc70 2023-11-06T01:50:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:50:05.056 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:33:27.113678+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:50:05.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 -- 172.21.15.53:0/2508416524 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83240a0570 msgr2=0x7f83240a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83240a0570 0x7f83240a2a30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f833c069ab0 tx=0x7f8330009260 comp rx=0 tx=0).stop 2023-11-06T01:50:05.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 -- 172.21.15.53:0/2508416524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f833c0ffc70 msgr2=0x7f833c069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f833c0ffc70 0x7f833c069d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f832c009fd0 tx=0x7f832c009580 comp rx=0 tx=0).stop 2023-11-06T01:50:05.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 -- 172.21.15.53:0/2508416524 shutdown_connections 2023-11-06T01:50:05.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f83240a0570 0x7f83240a2a30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f833c0ffc70 0x7f833c069d40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f833c101800 0x7f833c19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 --2- 172.21.15.53:0/2508416524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f833c100e60 0x7f833c19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 -- 172.21.15.53:0/2508416524 >> 172.21.15.53:0/2508416524 conn(0x7f833c0fb1c0 msgr2=0x7f833c1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:05.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 -- 172.21.15.53:0/2508416524 shutdown_connections 2023-11-06T01:50:05.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.060+0000 7f8341a42700 1 -- 172.21.15.53:0/2508416524 wait complete. 2023-11-06T01:50:05.071 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T01:28:35.924640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T01:32:24.604601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T01:32:25.457741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T01:50:05.073 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T01:50:05.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.205+0000 7f8fde82e700 1 -- 172.21.15.53:0/129246566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80ffc80 msgr2=0x7f8fd81000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.205+0000 7f8fde82e700 1 --2- 172.21.15.53:0/129246566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80ffc80 0x7f8fd81000a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80099c0 tx=0x7f8fc8009cd0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.205+0000 7f8fde82e700 1 -- 172.21.15.53:0/129246566 shutdown_connections 2023-11-06T01:50:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.205+0000 7f8fde82e700 1 --2- 172.21.15.53:0/129246566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fd8101810 0x7f8fd81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.205+0000 7f8fde82e700 1 --2- 172.21.15.53:0/129246566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80ffc80 0x7f8fd81000a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.205+0000 7f8fde82e700 1 --2- 172.21.15.53:0/129246566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fd8100e70 0x7f8fd81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.205+0000 7f8fde82e700 1 -- 172.21.15.53:0/129246566 >> 172.21.15.53:0/129246566 conn(0x7f8fd80fb1f0 msgr2=0x7f8fd80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:05.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.205+0000 7f8fde82e700 1 -- 172.21.15.53:0/129246566 shutdown_connections 2023-11-06T01:50:05.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.205+0000 7f8fde82e700 1 -- 172.21.15.53:0/129246566 wait complete. 2023-11-06T01:50:05.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fde82e700 1 Processor -- start 2023-11-06T01:50:05.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fde82e700 1 -- start start 2023-11-06T01:50:05.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fde82e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80ffc80 0x7f8fd8196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fde82e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fd8100e70 0x7f8fd8196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fde82e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fd8101810 0x7f8fd819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fde82e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd819e610 con 0x7f8fd80ffc80 2023-11-06T01:50:05.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fde82e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd819e750 con 0x7f8fd8100e70 2023-11-06T01:50:05.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fde82e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd819e8e0 con 0x7f8fd8101810 2023-11-06T01:50:05.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fd7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80ffc80 0x7f8fd8196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fd77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fd8100e70 0x7f8fd8196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fd7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80ffc80 0x7f8fd8196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44216/0 (socket says 172.21.15.53:44216) 2023-11-06T01:50:05.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fd7fff700 1 -- 172.21.15.53:0/3364155795 learned_addr learned my addr 172.21.15.53:0/3364155795 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:05.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.206+0000 7f8fdcdcb700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fd8101810 0x7f8fd819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fd7fff700 1 -- 172.21.15.53:0/3364155795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fd8101810 msgr2=0x7f8fd819e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fd7fff700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fd8101810 0x7f8fd819e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fd7fff700 1 -- 172.21.15.53:0/3364155795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fd8100e70 msgr2=0x7f8fd8196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fd7fff700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fd8100e70 0x7f8fd8196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fd7fff700 1 -- 172.21.15.53:0/3364155795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc80095f0 con 0x7f8fd80ffc80 2023-11-06T01:50:05.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fd77fe700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fd8100e70 0x7f8fd8196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:50:05.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fd7fff700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80ffc80 0x7f8fd8196a90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009fd0 tx=0x7f8fc8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:05.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fd57fa700 1 -- 172.21.15.53:0/3364155795 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc801d070 con 0x7f8fd80ffc80 2023-11-06T01:50:05.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fde82e700 1 -- 172.21.15.53:0/3364155795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd819eb60 con 0x7f8fd80ffc80 2023-11-06T01:50:05.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fd57fa700 1 -- 172.21.15.53:0/3364155795 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc8005030 con 0x7f8fd80ffc80 2023-11-06T01:50:05.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fd57fa700 1 -- 172.21.15.53:0/3364155795 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8004a90 con 0x7f8fd80ffc80 2023-11-06T01:50:05.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.207+0000 7f8fde82e700 1 -- 172.21.15.53:0/3364155795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd819efe0 con 0x7f8fd80ffc80 2023-11-06T01:50:05.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.208+0000 7f8fde82e700 1 -- 172.21.15.53:0/3364155795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd810b490 con 0x7f8fd80ffc80 2023-11-06T01:50:05.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.209+0000 7f8fd57fa700 1 -- 172.21.15.53:0/3364155795 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f8fc80051a0 con 0x7f8fd80ffc80 2023-11-06T01:50:05.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.210+0000 7f8fd57fa700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8fc40a0520 0x7f8fc40a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.210+0000 7f8fd77fe700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8fc40a0520 0x7f8fc40a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.210+0000 7f8fd57fa700 1 -- 172.21.15.53:0/3364155795 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8fc80ccb80 con 0x7f8fd80ffc80 2023-11-06T01:50:05.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.210+0000 7f8fd77fe700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8fc40a0520 0x7f8fc40a29e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0011490 tx=0x7f8fc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:05.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.214+0000 7f8fd57fa700 1 -- 172.21.15.53:0/3364155795 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8fc8096850 con 0x7f8fd80ffc80 2023-11-06T01:50:05.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:05 smithi053 ceph-mon[99472]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-11-06T01:50:05.341 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:05 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2508416524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:50:05.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.416+0000 7f8fde82e700 1 -- 172.21.15.53:0/3364155795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8fd819f590 con 0x7f8fd80ffc80 2023-11-06T01:50:05.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.416+0000 7f8fd57fa700 1 -- 172.21.15.53:0/3364155795 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8fc80260a0 con 0x7f8fd80ffc80 2023-11-06T01:50:05.415 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-11-06T01:50:05.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.418+0000 7f8fde82e700 1 -- 172.21.15.53:0/3364155795 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8fc40a0520 msgr2=0x7f8fc40a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.418+0000 7f8fde82e700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8fc40a0520 0x7f8fc40a29e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0011490 tx=0x7f8fc0009260 comp rx=0 tx=0).stop 2023-11-06T01:50:05.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.418+0000 7f8fde82e700 1 -- 172.21.15.53:0/3364155795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80ffc80 msgr2=0x7f8fd8196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.418+0000 7f8fde82e700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80ffc80 0x7f8fd8196a90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009fd0 tx=0x7f8fc8009580 comp rx=0 tx=0).stop 2023-11-06T01:50:05.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.418+0000 7f8fde82e700 1 -- 172.21.15.53:0/3364155795 shutdown_connections 2023-11-06T01:50:05.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.418+0000 7f8fde82e700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8fc40a0520 0x7f8fc40a29e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.418+0000 7f8fde82e700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fd8101810 0x7f8fd819e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.418+0000 7f8fde82e700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80ffc80 0x7f8fd8196a90 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.418+0000 7f8fde82e700 1 --2- 172.21.15.53:0/3364155795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fd8100e70 0x7f8fd8196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.418+0000 7f8fde82e700 1 -- 172.21.15.53:0/3364155795 >> 172.21.15.53:0/3364155795 conn(0x7f8fd80fb1f0 msgr2=0x7f8fd8067f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.419+0000 7f8fde82e700 1 -- 172.21.15.53:0/3364155795 shutdown_connections 2023-11-06T01:50:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.419+0000 7f8fde82e700 1 -- 172.21.15.53:0/3364155795 wait complete. 2023-11-06T01:50:05.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T01:50:05.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:05 smithi163 ceph-mon[104416]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-11-06T01:50:05.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:05 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2508416524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:50:05.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.560+0000 7f9eafdf9700 1 -- 172.21.15.53:0/2324330443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea8101840 msgr2=0x7f9ea81080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.560+0000 7f9eafdf9700 1 --2- 172.21.15.53:0/2324330443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea8101840 0x7f9ea81080f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000b0d0 tx=0x7f9ea000b3e0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.560+0000 7f9eafdf9700 1 -- 172.21.15.53:0/2324330443 shutdown_connections 2023-11-06T01:50:05.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.560+0000 7f9eafdf9700 1 --2- 172.21.15.53:0/2324330443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ea80ffcb0 0x7f9ea81000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.560+0000 7f9eafdf9700 1 --2- 172.21.15.53:0/2324330443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea8101840 0x7f9ea81080f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.560+0000 7f9eafdf9700 1 --2- 172.21.15.53:0/2324330443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8100ea0 0x7f9ea8101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.560+0000 7f9eafdf9700 1 -- 172.21.15.53:0/2324330443 >> 172.21.15.53:0/2324330443 conn(0x7f9ea80fb220 msgr2=0x7f9ea80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:05.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.560+0000 7f9eafdf9700 1 -- 172.21.15.53:0/2324330443 shutdown_connections 2023-11-06T01:50:05.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.560+0000 7f9eafdf9700 1 -- 172.21.15.53:0/2324330443 wait complete. 2023-11-06T01:50:05.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eafdf9700 1 Processor -- start 2023-11-06T01:50:05.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eafdf9700 1 -- start start 2023-11-06T01:50:05.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eafdf9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea80ffcb0 0x7f9ea8103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eafdf9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ea8100ea0 0x7f9ea8102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eafdf9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8101840 0x7f9ea81027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eafdf9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea8102f60 con 0x7f9ea80ffcb0 2023-11-06T01:50:05.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eafdf9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea81030d0 con 0x7f9ea8100ea0 2023-11-06T01:50:05.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eafdf9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea81a0c10 con 0x7f9ea8101840 2023-11-06T01:50:05.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eadb95700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea80ffcb0 0x7f9ea8103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9ead394700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ea8100ea0 0x7f9ea8102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eadb95700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea80ffcb0 0x7f9ea8103c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44230/0 (socket says 172.21.15.53:44230) 2023-11-06T01:50:05.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eae396700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8101840 0x7f9ea81027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.561+0000 7f9eadb95700 1 -- 172.21.15.53:0/3102904535 learned_addr learned my addr 172.21.15.53:0/3102904535 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:05.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9ead394700 1 -- 172.21.15.53:0/3102904535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8101840 msgr2=0x7f9ea81027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9ead394700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8101840 0x7f9ea81027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9ead394700 1 -- 172.21.15.53:0/3102904535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea80ffcb0 msgr2=0x7f9ea8103c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9ead394700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea80ffcb0 0x7f9ea8103c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9ead394700 1 -- 172.21.15.53:0/3102904535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea0009d00 con 0x7f9ea8100ea0 2023-11-06T01:50:05.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9eadb95700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea80ffcb0 0x7f9ea8103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:50:05.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9ead394700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ea8100ea0 0x7f9ea8102280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00e9f0 tx=0x7f9e9c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:05.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9e9affd700 1 -- 172.21.15.53:0/3102904535 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00c9c0 con 0x7f9ea8100ea0 2023-11-06T01:50:05.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9eafdf9700 1 -- 172.21.15.53:0/3102904535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea81a0ef0 con 0x7f9ea8100ea0 2023-11-06T01:50:05.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9e9affd700 1 -- 172.21.15.53:0/3102904535 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e9c010430 con 0x7f9ea8100ea0 2023-11-06T01:50:05.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9e9affd700 1 -- 172.21.15.53:0/3102904535 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c018680 con 0x7f9ea8100ea0 2023-11-06T01:50:05.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.562+0000 7f9eafdf9700 1 -- 172.21.15.53:0/3102904535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea81a1370 con 0x7f9ea8100ea0 2023-11-06T01:50:05.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.563+0000 7f9eafdf9700 1 -- 172.21.15.53:0/3102904535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea810b4d0 con 0x7f9ea8100ea0 2023-11-06T01:50:05.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.564+0000 7f9e9affd700 1 -- 172.21.15.53:0/3102904535 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f9e9c010860 con 0x7f9ea8100ea0 2023-11-06T01:50:05.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.565+0000 7f9e9affd700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e940a0520 0x7f9e940a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.565+0000 7f9eadb95700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e940a0520 0x7f9e940a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.565+0000 7f9e9affd700 1 -- 172.21.15.53:0/3102904535 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9e9c014070 con 0x7f9ea8100ea0 2023-11-06T01:50:05.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.565+0000 7f9eadb95700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e940a0520 0x7f9e940a29e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005950 tx=0x7f9ea4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:05.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.569+0000 7f9e9affd700 1 -- 172.21.15.53:0/3102904535 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9e9c0957a0 con 0x7f9ea8100ea0 2023-11-06T01:50:05.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:05 smithi062 ceph-mon[103924]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-11-06T01:50:05.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:05 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2508416524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:50:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.772+0000 7f9eafdf9700 1 -- 172.21.15.53:0/3102904535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ea81a19b0 con 0x7f9ea8100ea0 2023-11-06T01:50:05.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.773+0000 7f9e9affd700 1 -- 172.21.15.53:0/3102904535 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e9c098dc0 con 0x7f9ea8100ea0 2023-11-06T01:50:05.771 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-11-06T01:50:05.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.775+0000 7f9eafdf9700 1 -- 172.21.15.53:0/3102904535 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e940a0520 msgr2=0x7f9e940a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.775+0000 7f9eafdf9700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e940a0520 0x7f9e940a29e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005950 tx=0x7f9ea4009260 comp rx=0 tx=0).stop 2023-11-06T01:50:05.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.775+0000 7f9eafdf9700 1 -- 172.21.15.53:0/3102904535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ea8100ea0 msgr2=0x7f9ea8102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.775+0000 7f9eafdf9700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ea8100ea0 0x7f9ea8102280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00e9f0 tx=0x7f9e9c00ed00 comp rx=0 tx=0).stop 2023-11-06T01:50:05.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.776+0000 7f9eafdf9700 1 -- 172.21.15.53:0/3102904535 shutdown_connections 2023-11-06T01:50:05.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.776+0000 7f9eafdf9700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f9e940a0520 0x7f9e940a29e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.776+0000 7f9eafdf9700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8101840 0x7f9ea81027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.776+0000 7f9eafdf9700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ea80ffcb0 0x7f9ea8103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.776+0000 7f9eafdf9700 1 --2- 172.21.15.53:0/3102904535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ea8100ea0 0x7f9ea8102280 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.776+0000 7f9eafdf9700 1 -- 172.21.15.53:0/3102904535 >> 172.21.15.53:0/3102904535 conn(0x7f9ea80fb220 msgr2=0x7f9ea8068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:05.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.776+0000 7f9eafdf9700 1 -- 172.21.15.53:0/3102904535 shutdown_connections 2023-11-06T01:50:05.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.776+0000 7f9eafdf9700 1 -- 172.21.15.53:0/3102904535 wait complete. 2023-11-06T01:50:05.786 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T01:50:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.918+0000 7f437dd73700 1 -- 172.21.15.53:0/707240533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4378106ac0 msgr2=0x7f4378108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.918+0000 7f437dd73700 1 --2- 172.21.15.53:0/707240533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4378106ac0 0x7f4378108ed0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f437000b0a0 tx=0x7f437000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.918+0000 7f437dd73700 1 -- 172.21.15.53:0/707240533 shutdown_connections 2023-11-06T01:50:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.918+0000 7f437dd73700 1 --2- 172.21.15.53:0/707240533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4378106ac0 0x7f4378108ed0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.918+0000 7f437dd73700 1 --2- 172.21.15.53:0/707240533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4378101860 0x7f4378103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.918+0000 7f437dd73700 1 --2- 172.21.15.53:0/707240533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4378104190 0x7f4378106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.918+0000 7f437dd73700 1 -- 172.21.15.53:0/707240533 >> 172.21.15.53:0/707240533 conn(0x7f43780fb1f0 msgr2=0x7f43780fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.918+0000 7f437dd73700 1 -- 172.21.15.53:0/707240533 shutdown_connections 2023-11-06T01:50:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.918+0000 7f437dd73700 1 -- 172.21.15.53:0/707240533 wait complete. 2023-11-06T01:50:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f437dd73700 1 Processor -- start 2023-11-06T01:50:05.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f437dd73700 1 -- start start 2023-11-06T01:50:05.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f437dd73700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4378101860 0x7f4378196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f437dd73700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4378104190 0x7f43781970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f437dd73700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4378106ac0 0x7f437819e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f437dd73700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437819e6f0 con 0x7f4378106ac0 2023-11-06T01:50:05.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f437dd73700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437819e830 con 0x7f4378101860 2023-11-06T01:50:05.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f437dd73700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437819e9c0 con 0x7f4378104190 2023-11-06T01:50:05.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f43777fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4378101860 0x7f4378196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f43777fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4378101860 0x7f4378196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55682/0 (socket says 172.21.15.53:55682) 2023-11-06T01:50:05.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f43777fe700 1 -- 172.21.15.53:0/634885046 learned_addr learned my addr 172.21.15.53:0/634885046 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:05.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f4377fff700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4378106ac0 0x7f437819e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.919+0000 7f4376ffd700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4378104190 0x7f43781970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f43777fe700 1 -- 172.21.15.53:0/634885046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4378104190 msgr2=0x7f43781970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f43777fe700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4378104190 0x7f43781970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f43777fe700 1 -- 172.21.15.53:0/634885046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4378106ac0 msgr2=0x7f437819e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:05.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f43777fe700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4378106ac0 0x7f437819e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:05.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f43777fe700 1 -- 172.21.15.53:0/634885046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4370009d00 con 0x7f4378101860 2023-11-06T01:50:05.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f4376ffd700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4378104190 0x7f43781970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:50:05.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f43777fe700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4378101860 0x7f4378196b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f436800e9a0 tx=0x7f436800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:05.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f4374ff9700 1 -- 172.21.15.53:0/634885046 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4368009790 con 0x7f4378101860 2023-11-06T01:50:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f4374ff9700 1 -- 172.21.15.53:0/634885046 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4368010430 con 0x7f4378101860 2023-11-06T01:50:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f437dd73700 1 -- 172.21.15.53:0/634885046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437819eca0 con 0x7f4378101860 2023-11-06T01:50:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f4374ff9700 1 -- 172.21.15.53:0/634885046 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4368018680 con 0x7f4378101860 2023-11-06T01:50:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.920+0000 7f437dd73700 1 -- 172.21.15.53:0/634885046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437819f1b0 con 0x7f4378101860 2023-11-06T01:50:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.921+0000 7f437dd73700 1 -- 172.21.15.53:0/634885046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43780fcd70 con 0x7f4378101860 2023-11-06T01:50:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.922+0000 7f4374ff9700 1 -- 172.21.15.53:0/634885046 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f43680105a0 con 0x7f4378101860 2023-11-06T01:50:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.923+0000 7f4374ff9700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43600a0520 0x7f43600a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.923+0000 7f4374ff9700 1 -- 172.21.15.53:0/634885046 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4368014070 con 0x7f4378101860 2023-11-06T01:50:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.923+0000 7f4376ffd700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43600a0520 0x7f43600a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.924+0000 7f4376ffd700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43600a0520 0x7f43600a29e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f436c011490 tx=0x7f436c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:05.926+0000 7f4374ff9700 1 -- 172.21.15.53:0/634885046 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f43680958a0 con 0x7f4378101860 2023-11-06T01:50:06.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.130+0000 7f437dd73700 1 -- 172.21.15.53:0/634885046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f437802ce00 con 0x7f4378101860 2023-11-06T01:50:06.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.131+0000 7f4374ff9700 1 -- 172.21.15.53:0/634885046 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4368095030 con 0x7f4378101860 2023-11-06T01:50:06.129 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-11-06T01:50:06.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 -- 172.21.15.53:0/634885046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43600a0520 msgr2=0x7f43600a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:06.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43600a0520 0x7f43600a29e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f436c011490 tx=0x7f436c009260 comp rx=0 tx=0).stop 2023-11-06T01:50:06.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 -- 172.21.15.53:0/634885046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4378101860 msgr2=0x7f4378196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4378101860 0x7f4378196b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f436800e9a0 tx=0x7f436800ecb0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 -- 172.21.15.53:0/634885046 shutdown_connections 2023-11-06T01:50:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f43600a0520 0x7f43600a29e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4378101860 0x7f4378196b70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4378106ac0 0x7f437819e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 --2- 172.21.15.53:0/634885046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4378104190 0x7f43781970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 -- 172.21.15.53:0/634885046 >> 172.21.15.53:0/634885046 conn(0x7f43780fb1f0 msgr2=0x7f43781024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 -- 172.21.15.53:0/634885046 shutdown_connections 2023-11-06T01:50:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.133+0000 7f437dd73700 1 -- 172.21.15.53:0/634885046 wait complete. 2023-11-06T01:50:06.143 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T01:50:06.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 -- 172.21.15.53:0/2155411025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4054101810 msgr2=0x7f40541080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:06.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 --2- 172.21.15.53:0/2155411025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4054101810 0x7f40541080c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f404c00b0d0 tx=0x7f404c00b3e0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 -- 172.21.15.53:0/2155411025 shutdown_connections 2023-11-06T01:50:06.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 --2- 172.21.15.53:0/2155411025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40540ffc80 0x7f40541000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 --2- 172.21.15.53:0/2155411025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4054101810 0x7f40541080c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 --2- 172.21.15.53:0/2155411025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4054100e70 0x7f40541012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 -- 172.21.15.53:0/2155411025 >> 172.21.15.53:0/2155411025 conn(0x7f40540fb1f0 msgr2=0x7f40540fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:06.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 -- 172.21.15.53:0/2155411025 shutdown_connections 2023-11-06T01:50:06.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 -- 172.21.15.53:0/2155411025 wait complete. 2023-11-06T01:50:06.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 Processor -- start 2023-11-06T01:50:06.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 -- start start 2023-11-06T01:50:06.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40540ffc80 0x7f4054196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:06.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4054100e70 0x7f40541970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:06.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.274+0000 7f405bd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4054101810 0x7f405419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:06.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405bd9b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405419e6f0 con 0x7f4054101810 2023-11-06T01:50:06.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405bd9b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405419e830 con 0x7f4054100e70 2023-11-06T01:50:06.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405bd9b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405419e9c0 con 0x7f40540ffc80 2023-11-06T01:50:06.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405a338700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4054101810 0x7f405419e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:06.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f4059b37700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40540ffc80 0x7f4054196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:06.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f4059b37700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40540ffc80 0x7f4054196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58496/0 (socket says 172.21.15.53:58496) 2023-11-06T01:50:06.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f4059336700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4054100e70 0x7f40541970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:06.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405a338700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4054101810 0x7f405419e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44260/0 (socket says 172.21.15.53:44260) 2023-11-06T01:50:06.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f4059b37700 1 -- 172.21.15.53:0/3165863102 learned_addr learned my addr 172.21.15.53:0/3165863102 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:06.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405a338700 1 -- 172.21.15.53:0/3165863102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40540ffc80 msgr2=0x7f4054196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:06.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405a338700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40540ffc80 0x7f4054196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405a338700 1 -- 172.21.15.53:0/3165863102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4054100e70 msgr2=0x7f40541970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405a338700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4054100e70 0x7f40541970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405a338700 1 -- 172.21.15.53:0/3165863102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c009d00 con 0x7f4054101810 2023-11-06T01:50:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405a338700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4054101810 0x7f405419e120 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f404c00b770 tx=0x7f404c008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f404affd700 1 -- 172.21.15.53:0/3165863102 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c00e040 con 0x7f4054101810 2023-11-06T01:50:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f404affd700 1 -- 172.21.15.53:0/3165863102 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404c01d930 con 0x7f4054101810 2023-11-06T01:50:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405bd9b700 1 -- 172.21.15.53:0/3165863102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405419ec40 con 0x7f4054101810 2023-11-06T01:50:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f404affd700 1 -- 172.21.15.53:0/3165863102 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c01b8f0 con 0x7f4054101810 2023-11-06T01:50:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.275+0000 7f405bd9b700 1 -- 172.21.15.53:0/3165863102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405419f110 con 0x7f4054101810 2023-11-06T01:50:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.276+0000 7f405bd9b700 1 -- 172.21.15.53:0/3165863102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405410b4a0 con 0x7f4054101810 2023-11-06T01:50:06.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.278+0000 7f404affd700 1 -- 172.21.15.53:0/3165863102 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f404c01daa0 con 0x7f4054101810 2023-11-06T01:50:06.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.279+0000 7f404affd700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40400a05a0 0x7f40400a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:06.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.279+0000 7f4059b37700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40400a05a0 0x7f40400a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:06.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.279+0000 7f404affd700 1 -- 172.21.15.53:0/3165863102 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f404c0cd130 con 0x7f4054101810 2023-11-06T01:50:06.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.279+0000 7f4059b37700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40400a05a0 0x7f40400a2a60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4050005950 tx=0x7f4050009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:06.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.282+0000 7f404affd700 1 -- 172.21.15.53:0/3165863102 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f404c096e60 con 0x7f4054101810 2023-11-06T01:50:06.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:06 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3364155795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:50:06.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:06 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3102904535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:50:06.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:06 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/634885046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:50:06.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.504+0000 7f405bd9b700 1 -- 172.21.15.53:0/3165863102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f405419f6c0 con 0x7f4054101810 2023-11-06T01:50:06.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.504+0000 7f404affd700 1 -- 172.21.15.53:0/3165863102 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) v1 ==== 81+0+590 (secure 0 0 0) 0x7f404c09a480 con 0x7f4054101810 2023-11-06T01:50:06.503 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/2774626263 2023-11-07T01:26:25.398163+0000 2023-11-06T01:50:06.503 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/749598229 2023-11-07T01:25:59.898321+0000 2023-11-06T01:50:06.503 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/3877634851 2023-11-07T01:25:59.898321+0000 2023-11-06T01:50:06.503 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/3877634851 2023-11-07T01:25:59.898321+0000 2023-11-06T01:50:06.504 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/2231847638 2023-11-07T01:26:25.398163+0000 2023-11-06T01:50:06.504 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/2187559879 2023-11-07T01:25:59.898321+0000 2023-11-06T01:50:06.504 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/429543130 2023-11-07T01:25:59.898321+0000 2023-11-06T01:50:06.504 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/3127896038 2023-11-07T01:26:25.398163+0000 2023-11-06T01:50:06.504 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/2231847638 2023-11-07T01:26:25.398163+0000 2023-11-06T01:50:06.505 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/2103213255 2023-11-07T01:26:25.398163+0000 2023-11-06T01:50:06.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.506+0000 7f405bd9b700 1 -- 172.21.15.53:0/3165863102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40400a05a0 msgr2=0x7f40400a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:06.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.506+0000 7f405bd9b700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40400a05a0 0x7f40400a2a60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4050005950 tx=0x7f4050009260 comp rx=0 tx=0).stop 2023-11-06T01:50:06.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.506+0000 7f405bd9b700 1 -- 172.21.15.53:0/3165863102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4054101810 msgr2=0x7f405419e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:06.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.506+0000 7f405bd9b700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4054101810 0x7f405419e120 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f404c00b770 tx=0x7f404c008ec0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.507+0000 7f405bd9b700 1 -- 172.21.15.53:0/3165863102 shutdown_connections 2023-11-06T01:50:06.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.507+0000 7f405bd9b700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f40400a05a0 0x7f40400a2a60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.507+0000 7f405bd9b700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4054101810 0x7f405419e120 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.507+0000 7f405bd9b700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40540ffc80 0x7f4054196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.507+0000 7f405bd9b700 1 --2- 172.21.15.53:0/3165863102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4054100e70 0x7f40541970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.507+0000 7f405bd9b700 1 -- 172.21.15.53:0/3165863102 >> 172.21.15.53:0/3165863102 conn(0x7f40540fb1f0 msgr2=0x7f4054067f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.507+0000 7f405bd9b700 1 -- 172.21.15.53:0/3165863102 shutdown_connections 2023-11-06T01:50:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:06.507+0000 7f405bd9b700 1 -- 172.21.15.53:0/3165863102 wait complete. 2023-11-06T01:50:06.508 INFO:teuthology.orchestra.run.smithi053.stderr:listed 10 entries 2023-11-06T01:50:06.516 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:06 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3364155795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:50:06.516 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:06 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3102904535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:50:06.516 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:06 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/634885046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:50:06.517 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T01:50:06.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:50:06.518 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /tmp/cephfs 2023-11-06T01:50:06.653 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:06 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3364155795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-06T01:50:06.653 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:06 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3102904535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-06T01:50:06.654 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:06 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/634885046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-06T01:50:07.146 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-11-06T01:50:07.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:50:07.147 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-11-06T01:50:07.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:50:07.340 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-06T01:50:07.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:07 smithi163 ceph-mon[104416]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-11-06T01:50:07.494 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:07 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/3165863102' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-06T01:50:07.511 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-11-06T01:50:07.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:50:07.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-11-06T01:50:07.592 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:07 smithi053 ceph-mon[99472]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-11-06T01:50:07.593 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:07 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/3165863102' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-06T01:50:07.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:07 smithi062 ceph-mon[103924]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-11-06T01:50:07.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:07 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/3165863102' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-06T01:50:07.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:50:07.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2023-11-06T01:50:07.766 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-06T01:50:07.766 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-06T01:50:07.766 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T01:50:07.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3671 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/8107b7b0d0695faaa65e356f2490317765166e24f5d66d3e9cd4860850f8f32f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/DUSGBQA3BGQ2H3SPABO7VGRGBC:/var/lib/containers/storage/overlay/l/2CKWR4HMJZTXZX26APN7GEWYTZ,upperdir=/var/lib/containers/storage/overlay/8107b7b0d0695faaa65e356f2490317765166e24f5d66d3e9cd4860850f8f32f/diff,workdir=/var/lib/containers/storage/overlay/8107b7b0d0695faaa65e356f2490317765166e24f5d66d3e9cd4860850f8f32f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b9ad6d812e0baff2972a86988c0a3d6e43a7ad0e8a269af1cabb28725b3c93f9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/DUSGBQA3BGQ2H3SPABO7VGRGBC:/var/lib/containers/storage/overlay/l/2CKWR4HMJZTXZX26APN7GEWYTZ,upperdir=/var/lib/containers/storage/overlay/b9ad6d812e0baff2972a86988c0a3d6e43a7ad0e8a269af1cabb28725b3c93f9/diff,workdir=/var/lib/containers/storage/overlay/b9ad6d812e0baff2972a86988c0a3d6e43a7ad0e8a269af1cabb28725b3c93f9/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/ee3b9e203b795a402396994c0b1fa8685fea849c6c352a0d2638f7db0a847260/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/DUSGBQA3BGQ2H3SPABO7VGRGBC:/var/lib/containers/storage/overlay/l/2CKWR4HMJZTXZX26APN7GEWYTZ,upperdir=/var/lib/containers/storage/overlay/ee3b9e203b795a402396994c0b1fa8685fea849c6c352a0d2638f7db0a847260/diff,workdir=/var/lib/containers/storage/overlay/ee3b9e203b795a402396994c0b1fa8685fea849c6c352a0d2638f7db0a847260/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ecd63143bacf6983645ea80b69620d553309a228734a37463e8f3f6f422de43d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/DUSGBQA3BGQ2H3SPABO7VGRGBC:/var/lib/containers/storage/overlay/l/2CKWR4HMJZTXZX26APN7GEWYTZ,upperdir=/var/lib/containers/storage/overlay/ecd63143bacf6983645ea80b69620d553309a228734a37463e8f3f6f422de43d/diff,workdir=/var/lib/containers/storage/overlay/ecd63143bacf6983645ea80b69620d553309a228734a37463e8f3f6f422de43d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/940523fc8538acfad3ab6bf1445af2aa1c31248716371e76e216c8219fd1d0e1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/DUSGBQA3BGQ2H3SPABO7VGRGBC:/var/lib/containers/storage/overlay/l/2CKWR4HMJZTXZX26APN7GEWYTZ,upperdir=/var/lib/containers/storage/overlay/940523fc8538acfad3ab6bf1445af2aa1c31248716371e76e216c8219fd1d0e1/diff,workdir=/var/lib/containers/storage/overlay/940523fc8538acfad3ab6bf1445af2aa1c31248716371e76e216c8219fd1d0e1/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d9298e7f17a51cf32a8dd209ecdf157a52e0df7d98c1991fd623ba19518fcc77/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/DUSGBQA3BGQ2H3SPABO7VGRGBC:/var/lib/containers/storage/overlay/l/2CKWR4HMJZTXZX26APN7GEWYTZ,upperdir=/var/lib/containers/storage/overlay/d9298e7f17a51cf32a8dd209ecdf157a52e0df7d98c1991fd623ba19518fcc77/diff,workdir=/var/lib/containers/storage/overlay/d9298e7f17a51cf32a8dd209ecdf157a52e0df7d98c1991fd623ba19518fcc77/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/dde429841f7b3d5f2a4382eafc45cc8c31bb31804621f35b34411a29a5264de6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/DUSGBQA3BGQ2H3SPABO7VGRGBC:/var/lib/containers/storage/overlay/l/2CKWR4HMJZTXZX26APN7GEWYTZ,upperdir=/var/lib/containers/storage/overlay/dde429841f7b3d5f2a4382eafc45cc8c31bb31804621f35b34411a29a5264de6/diff,workdir=/var/lib/containers/storage/overlay/dde429841f7b3d5f2a4382eafc45cc8c31bb31804621f35b34411a29a5264de6/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1daa9cb1c58572def1a87c593e17314771a7b323f8d738a4464b94a0b4f0b99d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/DUSGBQA3BGQ2H3SPABO7VGRGBC:/var/lib/containers/storage/overlay/l/2CKWR4HMJZTXZX26APN7GEWYTZ,upperdir=/var/lib/containers/storage/overlay/1daa9cb1c58572def1a87c593e17314771a7b323f8d738a4464b94a0b4f0b99d/diff,workdir=/var/lib/containers/storage/overlay/1daa9cb1c58572def1a87c593e17314771a7b323f8d738a4464b94a0b4f0b99d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7656584eb059c5e2023bbec4d4ca26a57ef760b1ffe6f7a2ed33ee4210bfd96b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/DUSGBQA3BGQ2H3SPABO7VGRGBC:/var/lib/containers/storage/overlay/l/2CKWR4HMJZTXZX26APN7GEWYTZ,upperdir=/var/lib/containers/storage/overlay/7656584eb059c5e2023bbec4d4ca26a57ef760b1ffe6f7a2ed33ee4210bfd96b/diff,workdir=/var/lib/containers/storage/overlay/7656584eb059c5e2023bbec4d4ca26a57ef760b1ffe6f7a2ed33ee4210bfd96b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/47ca3174cc6c113f9889dc33f839c609ce5b7cb59b8d95b39d3de269a38e79cc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/DUSGBQA3BGQ2H3SPABO7VGRGBC:/var/lib/containers/storage/overlay/l/2CKWR4HMJZTXZX26APN7GEWYTZ,upperdir=/var/lib/containers/storage/overlay/47ca3174cc6c113f9889dc33f839c609ce5b7cb59b8d95b39d3de269a38e79cc/diff,workdir=/var/lib/containers/storage/overlay/47ca3174cc6c113f9889dc33f839c609ce5b7cb59b8d95b39d3de269a38e79cc/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-11-06T01:50:07.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:50:07.798 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-11-06T01:50:07.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T01:50:07.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2023-11-06T01:50:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2023-11-06T01:50:07.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T01:50:07.928 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-11-06T01:50:07.938 INFO:tasks.cephadm:Teardown begin 2023-11-06T01:50:07.939 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T01:50:07.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:50:08.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.097+0000 7ffab0428700 1 -- 172.21.15.53:0/1713906482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8106a50 msgr2=0x7ffaa8108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.097+0000 7ffab0428700 1 --2- 172.21.15.53:0/1713906482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8106a50 0x7ffaa8108ec0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000b0a0 tx=0x7ffaa000b3b0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.097+0000 7ffab0428700 1 -- 172.21.15.53:0/1713906482 shutdown_connections 2023-11-06T01:50:08.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.097+0000 7ffab0428700 1 --2- 172.21.15.53:0/1713906482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8106a50 0x7ffaa8108ec0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.097+0000 7ffab0428700 1 --2- 172.21.15.53:0/1713906482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffaa81017f0 0x7ffaa8103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.097+0000 7ffab0428700 1 --2- 172.21.15.53:0/1713906482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaa8104120 0x7ffaa8106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.097+0000 7ffab0428700 1 -- 172.21.15.53:0/1713906482 >> 172.21.15.53:0/1713906482 conn(0x7ffaa80fb1c0 msgr2=0x7ffaa80fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:08.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.097+0000 7ffab0428700 1 -- 172.21.15.53:0/1713906482 shutdown_connections 2023-11-06T01:50:08.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.097+0000 7ffab0428700 1 -- 172.21.15.53:0/1713906482 wait complete. 2023-11-06T01:50:08.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffab0428700 1 Processor -- start 2023-11-06T01:50:08.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffab0428700 1 -- start start 2023-11-06T01:50:08.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffab0428700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaa81017f0 0x7ffaa8101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffab0428700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8104120 0x7ffaa80ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffab0428700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffaa8106a50 0x7ffaa80ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffab0428700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa8100480 con 0x7ffaa8104120 2023-11-06T01:50:08.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffab0428700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa81005f0 con 0x7ffaa81017f0 2023-11-06T01:50:08.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffab0428700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa81a0b50 con 0x7ffaa8106a50 2023-11-06T01:50:08.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaad9c3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8104120 0x7ffaa80ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaae1c4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaa81017f0 0x7ffaa8101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaad9c3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8104120 0x7ffaa80ff7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44270/0 (socket says 172.21.15.53:44270) 2023-11-06T01:50:08.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaad9c3700 1 -- 172.21.15.53:0/546596147 learned_addr learned my addr 172.21.15.53:0/546596147 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:08.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaae9c5700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffaa8106a50 0x7ffaa80ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaae1c4700 1 -- 172.21.15.53:0/546596147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffaa8106a50 msgr2=0x7ffaa80ffce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaae1c4700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffaa8106a50 0x7ffaa80ffce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaae1c4700 1 -- 172.21.15.53:0/546596147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8104120 msgr2=0x7ffaa80ff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaae1c4700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8104120 0x7ffaa80ff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaae1c4700 1 -- 172.21.15.53:0/546596147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa0009d00 con 0x7ffaa81017f0 2023-11-06T01:50:08.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaad9c3700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8104120 0x7ffaa80ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:50:08.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.098+0000 7ffaae9c5700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffaa8106a50 0x7ffaa80ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:50:08.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.099+0000 7ffaae1c4700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaa81017f0 0x7ffaa8101160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400e9f0 tx=0x7ffaa400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:08.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.099+0000 7ffa9f7fe700 1 -- 172.21.15.53:0/546596147 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa400c9c0 con 0x7ffaa81017f0 2023-11-06T01:50:08.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.099+0000 7ffab0428700 1 -- 172.21.15.53:0/546596147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa81a0e30 con 0x7ffaa81017f0 2023-11-06T01:50:08.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.099+0000 7ffa9f7fe700 1 -- 172.21.15.53:0/546596147 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaa4010430 con 0x7ffaa81017f0 2023-11-06T01:50:08.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.099+0000 7ffa9f7fe700 1 -- 172.21.15.53:0/546596147 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4018680 con 0x7ffaa81017f0 2023-11-06T01:50:08.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.099+0000 7ffab0428700 1 -- 172.21.15.53:0/546596147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa81a1260 con 0x7ffaa81017f0 2023-11-06T01:50:08.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.100+0000 7ffab0428700 1 -- 172.21.15.53:0/546596147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa80fcd40 con 0x7ffaa81017f0 2023-11-06T01:50:08.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.101+0000 7ffa9f7fe700 1 -- 172.21.15.53:0/546596147 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ffaa4010860 con 0x7ffaa81017f0 2023-11-06T01:50:08.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.102+0000 7ffa9f7fe700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffa940a0570 0x7ffa940a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.102+0000 7ffaad9c3700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffa940a0570 0x7ffa940a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.102+0000 7ffa9f7fe700 1 -- 172.21.15.53:0/546596147 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ffaa4014070 con 0x7ffaa81017f0 2023-11-06T01:50:08.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.102+0000 7ffaad9c3700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffa940a0570 0x7ffa940a2a30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffa98011550 tx=0x7ffa98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:08.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.105+0000 7ffa9f7fe700 1 -- 172.21.15.53:0/546596147 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffaa4095850 con 0x7ffaa81017f0 2023-11-06T01:50:08.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.360+0000 7ffab0428700 1 -- 172.21.15.53:0/546596147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaa802cc70 con 0x7ffaa81017f0 2023-11-06T01:50:08.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.361+0000 7ffa9f7fe700 1 -- 172.21.15.53:0/546596147 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+10818 (secure 0 0 0) 0x7ffaa4098e70 con 0x7ffaa81017f0 2023-11-06T01:50:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:50:08.361 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-06T01:32:26.093194+0000","modified":"2023-11-06T01:49:44.071225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24472,"mds_2":24464,"mds_3":24422,"mds_4":24395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24395":{"gid":24395,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T01:50:08.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 -- 172.21.15.53:0/546596147 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffa940a0570 msgr2=0x7ffa940a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffa940a0570 0x7ffa940a2a30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffa98011550 tx=0x7ffa98009260 comp rx=0 tx=0).stop 2023-11-06T01:50:08.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 -- 172.21.15.53:0/546596147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaa81017f0 msgr2=0x7ffaa8101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaa81017f0 0x7ffaa8101160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400e9f0 tx=0x7ffaa400ed00 comp rx=0 tx=0).stop 2023-11-06T01:50:08.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 -- 172.21.15.53:0/546596147 shutdown_connections 2023-11-06T01:50:08.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7ffa940a0570 0x7ffa940a2a30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaa81017f0 0x7ffaa8101160 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffaa8106a50 0x7ffaa80ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 --2- 172.21.15.53:0/546596147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8104120 0x7ffaa80ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 -- 172.21.15.53:0/546596147 >> 172.21.15.53:0/546596147 conn(0x7ffaa80fb1c0 msgr2=0x7ffaa8102500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:08.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 -- 172.21.15.53:0/546596147 shutdown_connections 2023-11-06T01:50:08.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.364+0000 7ffab0428700 1 -- 172.21.15.53:0/546596147 wait complete. 2023-11-06T01:50:08.365 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 50 2023-11-06T01:50:08.375 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T01:50:08.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.502+0000 7fdebb54f700 1 -- 172.21.15.53:0/2521933798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4102380 msgr2=0x7fdeb41027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.502+0000 7fdebb54f700 1 --2- 172.21.15.53:0/2521933798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4102380 0x7fdeb41027a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009990 tx=0x7fdea8009ca0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.502+0000 7fdebb54f700 1 -- 172.21.15.53:0/2521933798 shutdown_connections 2023-11-06T01:50:08.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.502+0000 7fdebb54f700 1 --2- 172.21.15.53:0/2521933798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb40ff7b0 0x7fdeb4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.502+0000 7fdebb54f700 1 --2- 172.21.15.53:0/2521933798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4102380 0x7fdeb41027a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.502+0000 7fdebb54f700 1 --2- 172.21.15.53:0/2521933798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb40fedf0 0x7fdeb40ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.502+0000 7fdebb54f700 1 -- 172.21.15.53:0/2521933798 >> 172.21.15.53:0/2521933798 conn(0x7fdeb40fa950 msgr2=0x7fdeb40fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:08.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.502+0000 7fdebb54f700 1 -- 172.21.15.53:0/2521933798 shutdown_connections 2023-11-06T01:50:08.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.502+0000 7fdebb54f700 1 -- 172.21.15.53:0/2521933798 wait complete. 2023-11-06T01:50:08.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdebb54f700 1 Processor -- start 2023-11-06T01:50:08.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdebb54f700 1 -- start start 2023-11-06T01:50:08.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdebb54f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb40fedf0 0x7fdeb4196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdebb54f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb40ff7b0 0x7fdeb4196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdebb54f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb4102380 0x7fdeb419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdebb54f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb419e5c0 con 0x7fdeb40fedf0 2023-11-06T01:50:08.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdebb54f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb419e700 con 0x7fdeb4102380 2023-11-06T01:50:08.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdebb54f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb419e890 con 0x7fdeb40ff7b0 2023-11-06T01:50:08.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdeb92eb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb40fedf0 0x7fdeb4196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdeb92eb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb40fedf0 0x7fdeb4196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44282/0 (socket says 172.21.15.53:44282) 2023-11-06T01:50:08.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdeb92eb700 1 -- 172.21.15.53:0/1461739084 learned_addr learned my addr 172.21.15.53:0/1461739084 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:08.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdeb9aec700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb4102380 0x7fdeb419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.503+0000 7fdeb8aea700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb40ff7b0 0x7fdeb4196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.504+0000 7fdeb92eb700 1 -- 172.21.15.53:0/1461739084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb40ff7b0 msgr2=0x7fdeb4196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.504+0000 7fdeb92eb700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb40ff7b0 0x7fdeb4196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.504+0000 7fdeb92eb700 1 -- 172.21.15.53:0/1461739084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb4102380 msgr2=0x7fdeb419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.504+0000 7fdeb92eb700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb4102380 0x7fdeb419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.504+0000 7fdeb92eb700 1 -- 172.21.15.53:0/1461739084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80095f0 con 0x7fdeb40fedf0 2023-11-06T01:50:08.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.504+0000 7fdeb92eb700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb40fedf0 0x7fdeb4196a40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdea8004710 tx=0x7fdea8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:08.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.504+0000 7fdea67fc700 1 -- 172.21.15.53:0/1461739084 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea801d070 con 0x7fdeb40fedf0 2023-11-06T01:50:08.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.504+0000 7fdea67fc700 1 -- 172.21.15.53:0/1461739084 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea8021410 con 0x7fdeb40fedf0 2023-11-06T01:50:08.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.504+0000 7fdebb54f700 1 -- 172.21.15.53:0/1461739084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb419eb10 con 0x7fdeb40fedf0 2023-11-06T01:50:08.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.504+0000 7fdea67fc700 1 -- 172.21.15.53:0/1461739084 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8017650 con 0x7fdeb40fedf0 2023-11-06T01:50:08.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.504+0000 7fdebb54f700 1 -- 172.21.15.53:0/1461739084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb419ef90 con 0x7fdeb40fedf0 2023-11-06T01:50:08.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.505+0000 7fdebb54f700 1 -- 172.21.15.53:0/1461739084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb410c590 con 0x7fdeb40fedf0 2023-11-06T01:50:08.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.506+0000 7fdea67fc700 1 -- 172.21.15.53:0/1461739084 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fdea8021580 con 0x7fdeb40fedf0 2023-11-06T01:50:08.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.507+0000 7fdea67fc700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdea00a05f0 0x7fdea00a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.507+0000 7fdea67fc700 1 -- 172.21.15.53:0/1461739084 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdea80ccf40 con 0x7fdeb40fedf0 2023-11-06T01:50:08.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.507+0000 7fdeb8aea700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdea00a05f0 0x7fdea00a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.507+0000 7fdeb8aea700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdea00a05f0 0x7fdea00a2ab0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4067e60 tx=0x7fdeb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:08.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.510+0000 7fdea67fc700 1 -- 172.21.15.53:0/1461739084 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdea8096b50 con 0x7fdeb40fedf0 2023-11-06T01:50:08.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.759+0000 7fdebb54f700 1 -- 172.21.15.53:0/1461739084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdeb410c870 con 0x7fdeb40fedf0 2023-11-06T01:50:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.760+0000 7fdea67fc700 1 -- 172.21.15.53:0/1461739084 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+10818 (secure 0 0 0) 0x7fdeb410c870 con 0x7fdeb40fedf0 2023-11-06T01:50:08.759 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:50:08.759 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/275572743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":275572743},{"type":"v1","addr":"172.21.15.53:6835","nonce":275572743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/4211608225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":4211608225},{"type":"v1","addr":"172.21.15.53:6837","nonce":4211608225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-06T01:32:26.093194+0000","modified":"2023-11-06T01:49:44.071225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24472,"mds_2":24464,"mds_3":24422,"mds_4":24395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6839/1353368498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1353368498},{"type":"v1","addr":"172.21.15.53:6839","nonce":1353368498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6841/1972292821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":1972292821},{"type":"v1","addr":"172.21.15.53:6841","nonce":1972292821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24395":{"gid":24395,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6833/585581012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":585581012},{"type":"v1","addr":"172.21.15.62:6833","nonce":585581012}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.62:6835/3901057483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3901057483},{"type":"v1","addr":"172.21.15.62:6835","nonce":3901057483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/731722016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":731722016},{"type":"v1","addr":"172.21.15.163:6833","nonce":731722016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/644941927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":644941927},{"type":"v1","addr":"172.21.15.62:6837","nonce":644941927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/392449784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":392449784},{"type":"v1","addr":"172.21.15.163:6835","nonce":392449784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/1997633670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1997633670},{"type":"v1","addr":"172.21.15.62:6839","nonce":1997633670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6837/1759271265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1759271265},{"type":"v1","addr":"172.21.15.163:6837","nonce":1759271265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6839/3554773881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":3554773881},{"type":"v1","addr":"172.21.15.163:6839","nonce":3554773881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T01:50:08.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 -- 172.21.15.53:0/1461739084 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdea00a05f0 msgr2=0x7fdea00a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdea00a05f0 0x7fdea00a2ab0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4067e60 tx=0x7fdeb0009260 comp rx=0 tx=0).stop 2023-11-06T01:50:08.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 -- 172.21.15.53:0/1461739084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb40fedf0 msgr2=0x7fdeb4196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb40fedf0 0x7fdeb4196a40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdea8004710 tx=0x7fdea8009580 comp rx=0 tx=0).stop 2023-11-06T01:50:08.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 -- 172.21.15.53:0/1461739084 shutdown_connections 2023-11-06T01:50:08.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fdea00a05f0 0x7fdea00a2ab0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb4102380 0x7fdeb419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb40fedf0 0x7fdeb4196a40 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 --2- 172.21.15.53:0/1461739084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb40ff7b0 0x7fdeb4196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 -- 172.21.15.53:0/1461739084 >> 172.21.15.53:0/1461739084 conn(0x7fdeb40fa950 msgr2=0x7fdeb40fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:08.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 -- 172.21.15.53:0/1461739084 shutdown_connections 2023-11-06T01:50:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.762+0000 7fdebb54f700 1 -- 172.21.15.53:0/1461739084 wait complete. 2023-11-06T01:50:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 50 2023-11-06T01:50:08.774 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T01:50:08.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.901+0000 7f8a921fa700 1 -- 172.21.15.53:0/3061023462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8c100d90 msgr2=0x7f8a8c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.901+0000 7f8a921fa700 1 --2- 172.21.15.53:0/3061023462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8c100d90 0x7f8a8c1011b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0099c0 tx=0x7f8a7c009cd0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.901+0000 7f8a921fa700 1 -- 172.21.15.53:0/3061023462 shutdown_connections 2023-11-06T01:50:08.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.901+0000 7f8a921fa700 1 --2- 172.21.15.53:0/3061023462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c102920 0x7f8a8c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.901+0000 7f8a921fa700 1 --2- 172.21.15.53:0/3061023462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8c100d90 0x7f8a8c1011b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.901+0000 7f8a921fa700 1 --2- 172.21.15.53:0/3061023462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a8c101f80 0x7f8a8c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.901+0000 7f8a921fa700 1 -- 172.21.15.53:0/3061023462 >> 172.21.15.53:0/3061023462 conn(0x7f8a8c0fc2e0 msgr2=0x7f8a8c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:08.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.901+0000 7f8a921fa700 1 -- 172.21.15.53:0/3061023462 shutdown_connections 2023-11-06T01:50:08.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.901+0000 7f8a921fa700 1 -- 172.21.15.53:0/3061023462 wait complete. 2023-11-06T01:50:08.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a921fa700 1 Processor -- start 2023-11-06T01:50:08.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a921fa700 1 -- start start 2023-11-06T01:50:08.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a921fa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a8c100d90 0x7f8a8c196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a921fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c101f80 0x7f8a8c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a921fa700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8c102920 0x7f8a8c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a921fa700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c19e5d0 con 0x7f8a8c102920 2023-11-06T01:50:08.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a921fa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c19e710 con 0x7f8a8c100d90 2023-11-06T01:50:08.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a921fa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c19e8a0 con 0x7f8a8c101f80 2023-11-06T01:50:08.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a8c100d90 0x7f8a8c196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a8bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8c102920 0x7f8a8c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a8affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c101f80 0x7f8a8c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a8c100d90 0x7f8a8c196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:55734/0 (socket says 172.21.15.53:55734) 2023-11-06T01:50:08.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a8b7fe700 1 -- 172.21.15.53:0/2017362267 learned_addr learned my addr 172.21.15.53:0/2017362267 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:08.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a8b7fe700 1 -- 172.21.15.53:0/2017362267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c101f80 msgr2=0x7f8a8c196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a8b7fe700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c101f80 0x7f8a8c196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.902+0000 7f8a8b7fe700 1 -- 172.21.15.53:0/2017362267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8c102920 msgr2=0x7f8a8c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:08.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.903+0000 7f8a8b7fe700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8c102920 0x7f8a8c19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:08.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.903+0000 7f8a8b7fe700 1 -- 172.21.15.53:0/2017362267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c0095f0 con 0x7f8a8c100d90 2023-11-06T01:50:08.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.903+0000 7f8a8bfff700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8c102920 0x7f8a8c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:50:08.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.903+0000 7f8a8affd700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c101f80 0x7f8a8c196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:50:08.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.903+0000 7f8a8b7fe700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a8c100d90 0x7f8a8c196a50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009990 tx=0x7f8a7c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:08.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.903+0000 7f8a88ff9700 1 -- 172.21.15.53:0/2017362267 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c01d070 con 0x7f8a8c100d90 2023-11-06T01:50:08.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.903+0000 7f8a88ff9700 1 -- 172.21.15.53:0/2017362267 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a7c005030 con 0x7f8a8c100d90 2023-11-06T01:50:08.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.903+0000 7f8a921fa700 1 -- 172.21.15.53:0/2017362267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8c19eb20 con 0x7f8a8c100d90 2023-11-06T01:50:08.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.903+0000 7f8a88ff9700 1 -- 172.21.15.53:0/2017362267 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c004a70 con 0x7f8a8c100d90 2023-11-06T01:50:08.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.903+0000 7f8a921fa700 1 -- 172.21.15.53:0/2017362267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8c19ef40 con 0x7f8a8c100d90 2023-11-06T01:50:08.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.904+0000 7f8a921fa700 1 -- 172.21.15.53:0/2017362267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8c10c5f0 con 0x7f8a8c100d90 2023-11-06T01:50:08.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.905+0000 7f8a88ff9700 1 -- 172.21.15.53:0/2017362267 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f8a7c005410 con 0x7f8a8c100d90 2023-11-06T01:50:08.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.905+0000 7f8a88ff9700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8a780a0570 0x7f8a780a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:08.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.906+0000 7f8a8affd700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8a780a0570 0x7f8a780a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:08.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.906+0000 7f8a88ff9700 1 -- 172.21.15.53:0/2017362267 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8a7c0ccb20 con 0x7f8a8c100d90 2023-11-06T01:50:08.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.906+0000 7f8a8affd700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8a780a0570 0x7f8a780a2a30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8a74011490 tx=0x7f8a74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:08.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:08.909+0000 7f8a88ff9700 1 -- 172.21.15.53:0/2017362267 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8a7c096850 con 0x7f8a8c100d90 2023-11-06T01:50:09.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.112+0000 7f8a921fa700 1 -- 172.21.15.53:0/2017362267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a8c107c40 con 0x7f8a8c100d90 2023-11-06T01:50:09.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.114+0000 7f8a88ff9700 1 -- 172.21.15.53:0/2017362267 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22543 (secure 0 0 0) 0x7f8a7c026090 con 0x7f8a8c100d90 2023-11-06T01:50:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-06T01:50:09.113 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"3f13dbd6-7c43-11ee-8db9-212e2dc638e7","created":"2023-11-06T01:25:25.499519+0000","modified":"2023-11-06T01:33:27.113678+0000","last_up_change":"2023-11-06T01:31:45.402178+0000","last_in_change":"2023-11-06T01:31:27.923708+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T01:28:35.924640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-06T01:32:24.604601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-06T01:32:25.457741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b30174e-feb1-42fd-8b43-0ffd686e01ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6803","nonce":2480021073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6805","nonce":2480021073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6809","nonce":2480021073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2480021073},{"type":"v1","addr":"172.21.15.53:6807","nonce":2480021073}]},"public_addr":"172.21.15.53:6803/2480021073","cluster_addr":"172.21.15.53:6805/2480021073","heartbeat_back_addr":"172.21.15.53:6809/2480021073","heartbeat_front_addr":"172.21.15.53:6807/2480021073","state":["exists","up"]},{"osd":1,"uuid":"73a75b02-34df-4def-b5f6-e29f1190c1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6801","nonce":395697393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6803","nonce":395697393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6807","nonce":395697393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":395697393},{"type":"v1","addr":"172.21.15.62:6805","nonce":395697393}]},"public_addr":"172.21.15.62:6801/395697393","cluster_addr":"172.21.15.62:6803/395697393","heartbeat_back_addr":"172.21.15.62:6807/395697393","heartbeat_front_addr":"172.21.15.62:6805/395697393","state":["exists","up"]},{"osd":2,"uuid":"6650ac65-e6f8-4a01-b2bd-60b0c7995997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6801","nonce":1520683016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6803","nonce":1520683016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6807","nonce":1520683016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1520683016},{"type":"v1","addr":"172.21.15.163:6805","nonce":1520683016}]},"public_addr":"172.21.15.163:6801/1520683016","cluster_addr":"172.21.15.163:6803/1520683016","heartbeat_back_addr":"172.21.15.163:6807/1520683016","heartbeat_front_addr":"172.21.15.163:6805/1520683016","state":["exists","up"]},{"osd":3,"uuid":"e0faf95d-f034-418c-8604-278ed2d8a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6811","nonce":3352913345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6813","nonce":3352913345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6817","nonce":3352913345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3352913345},{"type":"v1","addr":"172.21.15.53:6815","nonce":3352913345}]},"public_addr":"172.21.15.53:6811/3352913345","cluster_addr":"172.21.15.53:6813/3352913345","heartbeat_back_addr":"172.21.15.53:6817/3352913345","heartbeat_front_addr":"172.21.15.53:6815/3352913345","state":["exists","up"]},{"osd":4,"uuid":"f8542e9c-72e2-4551-b1f7-204be77127f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6809","nonce":1956539718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6811","nonce":1956539718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6815","nonce":1956539718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1956539718},{"type":"v1","addr":"172.21.15.62:6813","nonce":1956539718}]},"public_addr":"172.21.15.62:6809/1956539718","cluster_addr":"172.21.15.62:6811/1956539718","heartbeat_back_addr":"172.21.15.62:6815/1956539718","heartbeat_front_addr":"172.21.15.62:6813/1956539718","state":["exists","up"]},{"osd":5,"uuid":"4adb65ab-f410-41a2-b4dc-7e12bcaba8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6809","nonce":464090439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6811","nonce":464090439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6815","nonce":464090439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":464090439},{"type":"v1","addr":"172.21.15.163:6813","nonce":464090439}]},"public_addr":"172.21.15.163:6809/464090439","cluster_addr":"172.21.15.163:6811/464090439","heartbeat_back_addr":"172.21.15.163:6815/464090439","heartbeat_front_addr":"172.21.15.163:6813/464090439","state":["exists","up"]},{"osd":6,"uuid":"2b8502f9-4651-43b4-b212-d37785c4655a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6819","nonce":4140594819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6821","nonce":4140594819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6825","nonce":4140594819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4140594819},{"type":"v1","addr":"172.21.15.53:6823","nonce":4140594819}]},"public_addr":"172.21.15.53:6819/4140594819","cluster_addr":"172.21.15.53:6821/4140594819","heartbeat_back_addr":"172.21.15.53:6825/4140594819","heartbeat_front_addr":"172.21.15.53:6823/4140594819","state":["exists","up"]},{"osd":7,"uuid":"e6ac514c-4385-49cc-b408-f24f49a200ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6817","nonce":1541716587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6819","nonce":1541716587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6823","nonce":1541716587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1541716587},{"type":"v1","addr":"172.21.15.62:6821","nonce":1541716587}]},"public_addr":"172.21.15.62:6817/1541716587","cluster_addr":"172.21.15.62:6819/1541716587","heartbeat_back_addr":"172.21.15.62:6823/1541716587","heartbeat_front_addr":"172.21.15.62:6821/1541716587","state":["exists","up"]},{"osd":8,"uuid":"14d1dc49-9a61-48da-9250-70a70c4a7fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6817","nonce":2361123317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6819","nonce":2361123317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6823","nonce":2361123317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2361123317},{"type":"v1","addr":"172.21.15.163:6821","nonce":2361123317}]},"public_addr":"172.21.15.163:6817/2361123317","cluster_addr":"172.21.15.163:6819/2361123317","heartbeat_back_addr":"172.21.15.163:6823/2361123317","heartbeat_front_addr":"172.21.15.163:6821/2361123317","state":["exists","up"]},{"osd":9,"uuid":"23af8247-4e33-4d08-8467-452c339e1eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6827","nonce":825228850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6829","nonce":825228850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6833","nonce":825228850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":825228850},{"type":"v1","addr":"172.21.15.53:6831","nonce":825228850}]},"public_addr":"172.21.15.53:6827/825228850","cluster_addr":"172.21.15.53:6829/825228850","heartbeat_back_addr":"172.21.15.53:6833/825228850","heartbeat_front_addr":"172.21.15.53:6831/825228850","state":["exists","up"]},{"osd":10,"uuid":"4e90a07b-b1b1-4120-a456-6c8106cc1b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6825","nonce":1978188861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6827","nonce":1978188861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6831","nonce":1978188861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1978188861},{"type":"v1","addr":"172.21.15.62:6829","nonce":1978188861}]},"public_addr":"172.21.15.62:6825/1978188861","cluster_addr":"172.21.15.62:6827/1978188861","heartbeat_back_addr":"172.21.15.62:6831/1978188861","heartbeat_front_addr":"172.21.15.62:6829/1978188861","state":["exists","up"]},{"osd":11,"uuid":"758da1b3-af80-435f-bc06-3c9a3af6e501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6825","nonce":1021137705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6827","nonce":1021137705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6831","nonce":1021137705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1021137705},{"type":"v1","addr":"172.21.15.163:6829","nonce":1021137705}]},"public_addr":"172.21.15.163:6825/1021137705","cluster_addr":"172.21.15.163:6827/1021137705","heartbeat_back_addr":"172.21.15.163:6831/1021137705","heartbeat_front_addr":"172.21.15.163:6829/1021137705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:27:49.385221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:11.419282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:32.767391+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:28:53.995680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:16.021109+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:36.852306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:29:57.124054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:18.762535+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:30:39.345799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:00.011175+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:22.529820+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T01:31:43.667094+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:0/2774626263":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/749598229":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6800/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/3877634851":"2023-11-07T01:25:59.898321+0000","172.21.15.53:6801/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2187559879":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/429543130":"2023-11-07T01:25:59.898321+0000","172.21.15.53:0/3127896038":"2023-11-07T01:26:25.398163+0000","172.21.15.53:6800/2231847638":"2023-11-07T01:26:25.398163+0000","172.21.15.53:0/2103213255":"2023-11-07T01:26:25.398163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T01:50:09.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 -- 172.21.15.53:0/2017362267 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8a780a0570 msgr2=0x7f8a780a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:09.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8a780a0570 0x7f8a780a2a30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8a74011490 tx=0x7f8a74009260 comp rx=0 tx=0).stop 2023-11-06T01:50:09.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 -- 172.21.15.53:0/2017362267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a8c100d90 msgr2=0x7f8a8c196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:09.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a8c100d90 0x7f8a8c196a50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009990 tx=0x7f8a7c00f690 comp rx=0 tx=0).stop 2023-11-06T01:50:09.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 -- 172.21.15.53:0/2017362267 shutdown_connections 2023-11-06T01:50:09.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f8a780a0570 0x7f8a780a2a30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a8c100d90 0x7f8a8c196a50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8c102920 0x7f8a8c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 --2- 172.21.15.53:0/2017362267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c101f80 0x7f8a8c196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 -- 172.21.15.53:0/2017362267 >> 172.21.15.53:0/2017362267 conn(0x7f8a8c0fc2e0 msgr2=0x7f8a8c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:09.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 -- 172.21.15.53:0/2017362267 shutdown_connections 2023-11-06T01:50:09.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.116+0000 7f8a921fa700 1 -- 172.21.15.53:0/2017362267 wait complete. 2023-11-06T01:50:09.128 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T01:50:09.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.257+0000 7f6995fd1700 1 -- 172.21.15.53:0/3770306908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6990101840 msgr2=0x7f69901080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:09.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.257+0000 7f6995fd1700 1 --2- 172.21.15.53:0/3770306908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6990101840 0x7f69901080f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f698800b0d0 tx=0x7f698800b3e0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.257+0000 7f6995fd1700 1 -- 172.21.15.53:0/3770306908 shutdown_connections 2023-11-06T01:50:09.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.257+0000 7f6995fd1700 1 --2- 172.21.15.53:0/3770306908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69900ffcb0 0x7f69901000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.257+0000 7f6995fd1700 1 --2- 172.21.15.53:0/3770306908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6990101840 0x7f69901080f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.257+0000 7f6995fd1700 1 --2- 172.21.15.53:0/3770306908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6990100ea0 0x7f6990101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.257+0000 7f6995fd1700 1 -- 172.21.15.53:0/3770306908 >> 172.21.15.53:0/3770306908 conn(0x7f69900fb220 msgr2=0x7f69900fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:09.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.257+0000 7f6995fd1700 1 -- 172.21.15.53:0/3770306908 shutdown_connections 2023-11-06T01:50:09.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.257+0000 7f6995fd1700 1 -- 172.21.15.53:0/3770306908 wait complete. 2023-11-06T01:50:09.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.257+0000 7f6995fd1700 1 Processor -- start 2023-11-06T01:50:09.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f6995fd1700 1 -- start start 2023-11-06T01:50:09.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f6995fd1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69900ffcb0 0x7f6990196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:09.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f6995fd1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6990100ea0 0x7f69901970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:09.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f6995fd1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6990101840 0x7f699019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:09.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f6995fd1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699019e720 con 0x7f69900ffcb0 2023-11-06T01:50:09.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f6995fd1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699019e860 con 0x7f6990101840 2023-11-06T01:50:09.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f6995fd1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699019e9f0 con 0x7f6990100ea0 2023-11-06T01:50:09.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69900ffcb0 0x7f6990196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:09.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6990100ea0 0x7f69901970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:09.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69900ffcb0 0x7f6990196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44310/0 (socket says 172.21.15.53:44310) 2023-11-06T01:50:09.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6990100ea0 0x7f69901970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.53:58552/0 (socket says 172.21.15.53:58552) 2023-11-06T01:50:09.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698f7fe700 1 -- 172.21.15.53:0/1416092953 learned_addr learned my addr 172.21.15.53:0/1416092953 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:09.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698ffff700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6990101840 0x7f699019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:09.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698effd700 1 -- 172.21.15.53:0/1416092953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6990101840 msgr2=0x7f699019e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:09.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698effd700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6990101840 0x7f699019e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698effd700 1 -- 172.21.15.53:0/1416092953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69900ffcb0 msgr2=0x7f6990196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:09.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698effd700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69900ffcb0 0x7f6990196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698effd700 1 -- 172.21.15.53:0/1416092953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6988009d00 con 0x7f6990100ea0 2023-11-06T01:50:09.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.258+0000 7f698f7fe700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69900ffcb0 0x7f6990196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T01:50:09.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.259+0000 7f698effd700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6990100ea0 0x7f69901970e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f697800e9f0 tx=0x7f697800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:09.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.259+0000 7f698cff9700 1 -- 172.21.15.53:0/1416092953 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697800c9c0 con 0x7f6990100ea0 2023-11-06T01:50:09.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.259+0000 7f6995fd1700 1 -- 172.21.15.53:0/1416092953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699019ecd0 con 0x7f6990100ea0 2023-11-06T01:50:09.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.259+0000 7f698cff9700 1 -- 172.21.15.53:0/1416092953 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6978010430 con 0x7f6990100ea0 2023-11-06T01:50:09.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.259+0000 7f698cff9700 1 -- 172.21.15.53:0/1416092953 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978018680 con 0x7f6990100ea0 2023-11-06T01:50:09.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.259+0000 7f6995fd1700 1 -- 172.21.15.53:0/1416092953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699019f1d0 con 0x7f6990100ea0 2023-11-06T01:50:09.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.260+0000 7f6995fd1700 1 -- 172.21.15.53:0/1416092953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699010b4d0 con 0x7f6990100ea0 2023-11-06T01:50:09.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.261+0000 7f698cff9700 1 -- 172.21.15.53:0/1416092953 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f6978010860 con 0x7f6990100ea0 2023-11-06T01:50:09.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.261+0000 7f698cff9700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f697c0a0520 0x7f697c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:09.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.261+0000 7f698f7fe700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f697c0a0520 0x7f697c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:09.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.261+0000 7f698cff9700 1 -- 172.21.15.53:0/1416092953 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6978014070 con 0x7f6990100ea0 2023-11-06T01:50:09.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.262+0000 7f698f7fe700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f697c0a0520 0x7f697c0a29e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6980005950 tx=0x7f6980009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:09.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.266+0000 7f698cff9700 1 -- 172.21.15.53:0/1416092953 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f69780520e0 con 0x7f6990100ea0 2023-11-06T01:50:09.486 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-mon[104416]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 13 op/s 2023-11-06T01:50:09.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/546596147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:50:09.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1461739084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:50:09.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2017362267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:50:09.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.519+0000 7f6995fd1700 1 -- 172.21.15.53:0/1416092953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f699019f7e0 con 0x7f6990100ea0 2023-11-06T01:50:09.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-mon[99472]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 13 op/s 2023-11-06T01:50:09.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/546596147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:50:09.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1461739084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:50:09.522 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2017362267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:50:09.522 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-a[99448]: 2023-11-06T01:50:09.520+0000 7f0b7baac700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T01:50:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.528+0000 7f698cff9700 1 -- 172.21.15.53:0/1416092953 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6978095660 con 0x7f6990100ea0 2023-11-06T01:50:09.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 -- 172.21.15.53:0/1416092953 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f697c0a0520 msgr2=0x7f697c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:09.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f697c0a0520 0x7f697c0a29e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6980005950 tx=0x7f6980009260 comp rx=0 tx=0).stop 2023-11-06T01:50:09.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 -- 172.21.15.53:0/1416092953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6990100ea0 msgr2=0x7f69901970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:09.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6990100ea0 0x7f69901970e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f697800e9f0 tx=0x7f697800ed00 comp rx=0 tx=0).stop 2023-11-06T01:50:09.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 -- 172.21.15.53:0/1416092953 shutdown_connections 2023-11-06T01:50:09.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f697c0a0520 0x7f697c0a29e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6990101840 0x7f699019e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69900ffcb0 0x7f6990196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 --2- 172.21.15.53:0/1416092953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6990100ea0 0x7f69901970e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 -- 172.21.15.53:0/1416092953 >> 172.21.15.53:0/1416092953 conn(0x7f69900fb220 msgr2=0x7f6990068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 -- 172.21.15.53:0/1416092953 shutdown_connections 2023-11-06T01:50:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.532+0000 7f6995fd1700 1 -- 172.21.15.53:0/1416092953 wait complete. 2023-11-06T01:50:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T01:50:09.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T01:50:09.584 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-mon[103924]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 13 op/s 2023-11-06T01:50:09.584 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/546596147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:50:09.584 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1461739084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-06T01:50:09.585 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2017362267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-06T01:50:09.585 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-h[139660]: ignoring --setuser ceph since I am not root 2023-11-06T01:50:09.585 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-h[139660]: ignoring --setgroup ceph since I am not root 2023-11-06T01:50:09.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.681+0000 7f3cf9146700 1 -- 172.21.15.53:0/3770327725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf4106a60 msgr2=0x7f3cf4108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:09.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.681+0000 7f3cf9146700 1 --2- 172.21.15.53:0/3770327725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf4106a60 0x7f3cf4108ed0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00b0a0 tx=0x7f3cec00b3b0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.681+0000 7f3cf9146700 1 -- 172.21.15.53:0/3770327725 shutdown_connections 2023-11-06T01:50:09.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.681+0000 7f3cf9146700 1 --2- 172.21.15.53:0/3770327725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf4106a60 0x7f3cf4108ed0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.681+0000 7f3cf9146700 1 --2- 172.21.15.53:0/3770327725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cf4101800 0x7f3cf4103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.681+0000 7f3cf9146700 1 --2- 172.21.15.53:0/3770327725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf4104130 0x7f3cf4106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.681+0000 7f3cf9146700 1 -- 172.21.15.53:0/3770327725 >> 172.21.15.53:0/3770327725 conn(0x7f3cf40fb1f0 msgr2=0x7f3cf40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:09.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.681+0000 7f3cf9146700 1 -- 172.21.15.53:0/3770327725 shutdown_connections 2023-11-06T01:50:09.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.681+0000 7f3cf9146700 1 -- 172.21.15.53:0/3770327725 wait complete. 2023-11-06T01:50:09.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.682+0000 7f3cf9146700 1 Processor -- start 2023-11-06T01:50:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.682+0000 7f3cf9146700 1 -- start start 2023-11-06T01:50:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.682+0000 7f3cf9146700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf4101800 0x7f3cf4101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.682+0000 7f3cf9146700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cf4104130 0x7f3cf40ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.682+0000 7f3cf9146700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf4106a60 0x7f3cf40ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.682+0000 7f3cf9146700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4100490 con 0x7f3cf4101800 2023-11-06T01:50:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.682+0000 7f3cf9146700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4100600 con 0x7f3cf4106a60 2023-11-06T01:50:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.682+0000 7f3cf9146700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf41a0b20 con 0x7f3cf4104130 2023-11-06T01:50:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf2d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf4101800 0x7f3cf4101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf2d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf4101800 0x7f3cf4101170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44348/0 (socket says 172.21.15.53:44348) 2023-11-06T01:50:09.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf2d9d700 1 -- 172.21.15.53:0/2973511211 learned_addr learned my addr 172.21.15.53:0/2973511211 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:09.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf259c700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cf4104130 0x7f3cf40ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:09.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf359e700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf4106a60 0x7f3cf40ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:09.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf2d9d700 1 -- 172.21.15.53:0/2973511211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cf4104130 msgr2=0x7f3cf40ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:09.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf2d9d700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cf4104130 0x7f3cf40ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf2d9d700 1 -- 172.21.15.53:0/2973511211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf4106a60 msgr2=0x7f3cf40ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:09.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf2d9d700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf4106a60 0x7f3cf40ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:09.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf2d9d700 1 -- 172.21.15.53:0/2973511211 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec009d00 con 0x7f3cf4101800 2023-11-06T01:50:09.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf359e700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf4106a60 0x7f3cf40ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:50:09.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf259c700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cf4104130 0x7f3cf40ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T01:50:09.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf2d9d700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf4101800 0x7f3cf4101170 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400d980 tx=0x7f3ce400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:09.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cdbfff700 1 -- 172.21.15.53:0/2973511211 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce400bb60 con 0x7f3cf4101800 2023-11-06T01:50:09.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cdbfff700 1 -- 172.21.15.53:0/2973511211 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ce4007640 con 0x7f3cf4101800 2023-11-06T01:50:09.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf9146700 1 -- 172.21.15.53:0/2973511211 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf41a0e00 con 0x7f3cf4101800 2023-11-06T01:50:09.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cdbfff700 1 -- 172.21.15.53:0/2973511211 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4004e90 con 0x7f3cf4101800 2023-11-06T01:50:09.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.683+0000 7f3cf9146700 1 -- 172.21.15.53:0/2973511211 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf41a1230 con 0x7f3cf4101800 2023-11-06T01:50:09.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.684+0000 7f3cf9146700 1 -- 172.21.15.53:0/2973511211 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf40fcd70 con 0x7f3cf4101800 2023-11-06T01:50:09.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.685+0000 7f3cdbfff700 1 -- 172.21.15.53:0/2973511211 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f3ce400bcc0 con 0x7f3cf4101800 2023-11-06T01:50:09.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.686+0000 7f3cdbfff700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3cdc0a0520 0x7f3cdc0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.686+0000 7f3cf259c700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3cdc0a0520 0x7f3cdc0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.686+0000 7f3cdbfff700 1 -- 172.21.15.53:0/2973511211 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8699+0+0 (secure 0 0 0) 0x7f3ce40cbf30 con 0x7f3cf4101800 2023-11-06T01:50:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.687+0000 7f3cf259c700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3cdc0a0520 0x7f3cdc0a29e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8005fd0 tx=0x7f3ce800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.689+0000 7f3cdbfff700 1 -- 172.21.15.53:0/2973511211 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3ce4099390 con 0x7f3cf4101800 2023-11-06T01:50:09.841 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-g[152212]: ignoring --setuser ceph since I am not root 2023-11-06T01:50:09.841 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-g[152212]: ignoring --setgroup ceph since I am not root 2023-11-06T01:50:09.842 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-mds[152236]: -- 172.21.15.53:0/2446406758 <== mon.0 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5568418cf1e0 con 0x5568418a5000 2023-11-06T01:50:09.842 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-g[152212]: starting mds.g at 2023-11-06T01:50:09.842 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-a[152599]: ignoring --setuser ceph since I am not root 2023-11-06T01:50:09.843 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-a[152599]: ignoring --setgroup ceph since I am not root 2023-11-06T01:50:09.843 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-mds[152623]: -- 172.21.15.53:0/2399468449 <== mon.0 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56070771b1e0 con 0x5607076f0c00 2023-11-06T01:50:09.843 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:50:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-a[152599]: starting mds.a at 2023-11-06T01:50:09.892 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-mds[139684]: -- 172.21.15.62:0/1059487834 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556705abb1e0 con 0x556705a90c00 2023-11-06T01:50:09.892 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-h[139660]: starting mds.h at 2023-11-06T01:50:09.893 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-b[140013]: ignoring --setuser ceph since I am not root 2023-11-06T01:50:09.893 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-b[140013]: ignoring --setgroup ceph since I am not root 2023-11-06T01:50:09.893 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-mds[140037]: -- 172.21.15.62:0/815626797 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5577963af1e0 con 0x557796384800 2023-11-06T01:50:09.893 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-b[140013]: starting mds.b at 2023-11-06T01:50:09.894 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-e[139323]: ignoring --setuser ceph since I am not root 2023-11-06T01:50:09.894 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-e[139323]: ignoring --setgroup ceph since I am not root 2023-11-06T01:50:09.894 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-mds[139348]: -- 172.21.15.62:0/4237790890 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564258df11e0 con 0x564258dc6800 2023-11-06T01:50:09.894 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-e[139323]: starting mds.e at 2023-11-06T01:50:09.895 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-k[138982]: ignoring --setuser ceph since I am not root 2023-11-06T01:50:09.895 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-k[138982]: ignoring --setgroup ceph since I am not root 2023-11-06T01:50:09.895 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-mds[139006]: -- 172.21.15.62:0/2310679955 <== mon.0 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5609d509f1e0 con 0x5609d5074800 2023-11-06T01:50:09.895 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:50:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-k[138982]: starting mds.k at 2023-11-06T01:50:09.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:09.939+0000 7f3cf9146700 1 -- 172.21.15.53:0/2973511211 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3cf402cf20 con 0x7f3cf4101800 2023-11-06T01:50:09.987 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-f[140054]: ignoring --setuser ceph since I am not root 2023-11-06T01:50:09.987 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-f[140054]: ignoring --setgroup ceph since I am not root 2023-11-06T01:50:09.987 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-mds[140078]: -- 172.21.15.163:0/3221533298 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5628ca14f1e0 con 0x5628ca124c00 2023-11-06T01:50:09.987 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-f[140054]: starting mds.f at 2023-11-06T01:50:09.988 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-mds[139729]: -- 172.21.15.163:0/3314218230 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fdc45cb1e0 con 0x55fdc45a0800 2023-11-06T01:50:09.988 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-l[139705]: ignoring --setuser ceph since I am not root 2023-11-06T01:50:09.988 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-l[139705]: ignoring --setgroup ceph since I am not root 2023-11-06T01:50:09.988 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-l[139705]: starting mds.l at 2023-11-06T01:50:09.989 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-mds[140418]: -- 172.21.15.163:0/338981355 <== mon.0 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556057cd51e0 con 0x556057caa800 2023-11-06T01:50:09.989 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-c[140394]: ignoring --setuser ceph since I am not root 2023-11-06T01:50:09.989 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-c[140394]: ignoring --setgroup ceph since I am not root 2023-11-06T01:50:09.989 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-c[140394]: starting mds.c at 2023-11-06T01:50:09.990 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-i[140736]: ignoring --setuser ceph since I am not root 2023-11-06T01:50:09.990 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-i[140736]: ignoring --setgroup ceph since I am not root 2023-11-06T01:50:09.990 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-mds[140760]: -- 172.21.15.163:0/2549886900 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a64f2f51e0 con 0x55a64f2cac00 2023-11-06T01:50:09.990 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:50:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-i[140736]: starting mds.i at 2023-11-06T01:50:10.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:10 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/1416092953' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-06T01:50:10.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:10 smithi163 ceph-mon[104416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-06T01:50:10.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:10 smithi163 ceph-mon[104416]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-06T01:50:10.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:10 smithi163 ceph-mon[104416]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T01:50:10.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:10 smithi163 ceph-mon[104416]: osdmap e85: 12 total, 12 up, 12 in 2023-11-06T01:50:10.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:10 smithi163 ceph-mon[104416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-06T01:50:10.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:10 smithi163 ceph-mon[104416]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-11-06T01:50:10.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:10 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2973511211' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-06T01:50:10.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.533+0000 7f3cdbfff700 1 -- 172.21.15.53:0/2973511211 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3ce400de60 con 0x7f3cf4101800 2023-11-06T01:50:10.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.537+0000 7f3cf9146700 1 -- 172.21.15.53:0/2973511211 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3cdc0a0520 msgr2=0x7f3cdc0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:10.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.537+0000 7f3cf9146700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3cdc0a0520 0x7f3cdc0a29e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8005fd0 tx=0x7f3ce800a5e0 comp rx=0 tx=0).stop 2023-11-06T01:50:10.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.537+0000 7f3cf9146700 1 -- 172.21.15.53:0/2973511211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf4101800 msgr2=0x7f3cf4101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:10.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.537+0000 7f3cf9146700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf4101800 0x7f3cf4101170 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400d980 tx=0x7f3ce400dd40 comp rx=0 tx=0).stop 2023-11-06T01:50:10.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.538+0000 7f3cf9146700 1 -- 172.21.15.53:0/2973511211 shutdown_connections 2023-11-06T01:50:10.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.538+0000 7f3cf9146700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7f3cdc0a0520 0x7f3cdc0a29e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:10.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.538+0000 7f3cf9146700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf4106a60 0x7f3cf40ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:10.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.538+0000 7f3cf9146700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf4101800 0x7f3cf4101170 secure :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400d980 tx=0x7f3ce400dd40 comp rx=0 tx=0).stop 2023-11-06T01:50:10.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.538+0000 7f3cf9146700 1 --2- 172.21.15.53:0/2973511211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cf4104130 0x7f3cf40ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:10.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.538+0000 7f3cf9146700 1 -- 172.21.15.53:0/2973511211 >> 172.21.15.53:0/2973511211 conn(0x7f3cf40fb1f0 msgr2=0x7f3cf4102470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:10.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.541+0000 7f3cf9146700 1 -- 172.21.15.53:0/2973511211 shutdown_connections 2023-11-06T01:50:10.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.541+0000 7f3cf9146700 1 -- 172.21.15.53:0/2973511211 wait complete. 2023-11-06T01:50:10.554 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:10 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/1416092953' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-06T01:50:10.554 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:10 smithi053 ceph-mon[99472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-06T01:50:10.554 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:10 smithi053 ceph-mon[99472]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-06T01:50:10.554 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:10 smithi053 ceph-mon[99472]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T01:50:10.555 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:10 smithi053 ceph-mon[99472]: osdmap e85: 12 total, 12 up, 12 in 2023-11-06T01:50:10.555 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:10 smithi053 ceph-mon[99472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-06T01:50:10.555 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:10 smithi053 ceph-mon[99472]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-11-06T01:50:10.555 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:10 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2973511211' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-06T01:50:10.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T01:50:10.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:10 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/1416092953' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-06T01:50:10.642 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:10 smithi062 ceph-mon[103924]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-06T01:50:10.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:10 smithi062 ceph-mon[103924]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-06T01:50:10.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:10 smithi062 ceph-mon[103924]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T01:50:10.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:10 smithi062 ceph-mon[103924]: osdmap e85: 12 total, 12 up, 12 in 2023-11-06T01:50:10.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:10 smithi062 ceph-mon[103924]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-06T01:50:10.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:10 smithi062 ceph-mon[103924]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-11-06T01:50:10.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:10 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2973511211' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-06T01:50:10.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.704+0000 7fbcc3a77700 1 -- 172.21.15.53:0/1447567590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcbc101840 msgr2=0x7fbcbc1080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:10.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.704+0000 7fbcc3a77700 1 --2- 172.21.15.53:0/1447567590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcbc101840 0x7fbcbc1080f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400b0d0 tx=0x7fbcb400b3e0 comp rx=0 tx=0).stop 2023-11-06T01:50:10.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.704+0000 7fbcc3a77700 1 -- 172.21.15.53:0/1447567590 shutdown_connections 2023-11-06T01:50:10.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.704+0000 7fbcc3a77700 1 --2- 172.21.15.53:0/1447567590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcbc0ffcb0 0x7fbcbc1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:10.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.704+0000 7fbcc3a77700 1 --2- 172.21.15.53:0/1447567590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcbc101840 0x7fbcbc1080f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:10.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.704+0000 7fbcc3a77700 1 --2- 172.21.15.53:0/1447567590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc100ea0 0x7fbcbc101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:10.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.704+0000 7fbcc3a77700 1 -- 172.21.15.53:0/1447567590 >> 172.21.15.53:0/1447567590 conn(0x7fbcbc0fb220 msgr2=0x7fbcbc0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:10.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.704+0000 7fbcc3a77700 1 -- 172.21.15.53:0/1447567590 shutdown_connections 2023-11-06T01:50:10.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.704+0000 7fbcc3a77700 1 -- 172.21.15.53:0/1447567590 wait complete. 2023-11-06T01:50:10.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc3a77700 1 Processor -- start 2023-11-06T01:50:10.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc3a77700 1 -- start start 2023-11-06T01:50:10.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc3a77700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcbc0ffcb0 0x7fbcbc069a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:10.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc3a77700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc100ea0 0x7fbcbc19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:10.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc3a77700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcbc101840 0x7fbcbc19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:10.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc3a77700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc19e5e0 con 0x7fbcbc0ffcb0 2023-11-06T01:50:10.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc3a77700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc19e750 con 0x7fbcbc101840 2023-11-06T01:50:10.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc3a77700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc19e910 con 0x7fbcbc100ea0 2023-11-06T01:50:10.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc1813700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcbc0ffcb0 0x7fbcbc069a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:10.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc1813700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcbc0ffcb0 0x7fbcbc069a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44366/0 (socket says 172.21.15.53:44366) 2023-11-06T01:50:10.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc1813700 1 -- 172.21.15.53:0/994308882 learned_addr learned my addr 172.21.15.53:0/994308882 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:10.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.705+0000 7fbcc1012700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc100ea0 0x7fbcbc19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:10.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcc2014700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcbc101840 0x7fbcbc19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:10.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcc1813700 1 -- 172.21.15.53:0/994308882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc100ea0 msgr2=0x7fbcbc19b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:10.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcc1813700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc100ea0 0x7fbcbc19b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:10.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcc1813700 1 -- 172.21.15.53:0/994308882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcbc101840 msgr2=0x7fbcbc19de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:10.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcc1813700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcbc101840 0x7fbcbc19de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:10.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcc1813700 1 -- 172.21.15.53:0/994308882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb4009d00 con 0x7fbcbc0ffcb0 2023-11-06T01:50:10.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcc2014700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcbc101840 0x7fbcbc19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T01:50:10.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcc1813700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcbc0ffcb0 0x7fbcbc069a30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d980 tx=0x7fbcb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:10.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcaeffd700 1 -- 172.21.15.53:0/994308882 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000b950 con 0x7fbcbc0ffcb0 2023-11-06T01:50:10.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcc3a77700 1 -- 172.21.15.53:0/994308882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc19ebf0 con 0x7fbcbc0ffcb0 2023-11-06T01:50:10.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcaeffd700 1 -- 172.21.15.53:0/994308882 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcb0007620 con 0x7fbcbc0ffcb0 2023-11-06T01:50:10.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcaeffd700 1 -- 172.21.15.53:0/994308882 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000f3c0 con 0x7fbcbc0ffcb0 2023-11-06T01:50:10.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.706+0000 7fbcc3a77700 1 -- 172.21.15.53:0/994308882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc19f150 con 0x7fbcbc0ffcb0 2023-11-06T01:50:10.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.707+0000 7fbcc3a77700 1 -- 172.21.15.53:0/994308882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcbc1071c0 con 0x7fbcbc0ffcb0 2023-11-06T01:50:10.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.708+0000 7fbcaeffd700 1 -- 172.21.15.53:0/994308882 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fbcb0010640 con 0x7fbcbc0ffcb0 2023-11-06T01:50:10.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.709+0000 7fbcaeffd700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbca80a0520 0x7fbca80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:10.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.709+0000 7fbcc1012700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbca80a0520 0x7fbca80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:10.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.709+0000 7fbcaeffd700 1 -- 172.21.15.53:0/994308882 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8699+0+0 (secure 0 0 0) 0x7fbcb00cba60 con 0x7fbcbc0ffcb0 2023-11-06T01:50:10.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.709+0000 7fbcc1012700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbca80a0520 0x7fbca80a29e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8005950 tx=0x7fbcb800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:10.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.713+0000 7fbcaeffd700 1 -- 172.21.15.53:0/994308882 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbcb0095430 con 0x7fbcbc0ffcb0 2023-11-06T01:50:10.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:10.916+0000 7fbcc3a77700 1 -- 172.21.15.53:0/994308882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbcbc02cc70 con 0x7fbcbc0ffcb0 2023-11-06T01:50:11.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-11-06T01:50:11.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-06T01:50:11.487 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-06T01:50:11.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: Cluster is now healthy 2023-11-06T01:50:11.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.163:6832/3181242442,v1:172.21.15.163:6834/3181242442] up:boot 2023-11-06T01:50:11.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.163:6836/3382064927,v1:172.21.15.163:6838/3382064927] up:boot 2023-11-06T01:50:11.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.163:6833/2387163723,v1:172.21.15.163:6835/2387163723] up:boot 2023-11-06T01:50:11.488 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.53:6840/4133878425,v1:172.21.15.53:6841/4133878425] up:boot 2023-11-06T01:50:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.53:6838/139725483,v1:172.21.15.53:6839/139725483] up:boot 2023-11-06T01:50:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.62:6832/3745085778,v1:172.21.15.62:6835/3745085778] up:boot 2023-11-06T01:50:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.62:6836/1012048915,v1:172.21.15.62:6839/1012048915] up:boot 2023-11-06T01:50:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.163:6837/122234685,v1:172.21.15.163:6839/122234685] up:boot 2023-11-06T01:50:11.489 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.62:6833/3993523599,v1:172.21.15.62:6837/3993523599] up:boot 2023-11-06T01:50:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: mds.? [v2:172.21.15.62:6834/807817280,v1:172.21.15.62:6838/807817280] up:boot 2023-11-06T01:50:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/2973511211' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-06T01:50:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: fsmap 12 up:standby 2023-11-06T01:50:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-06T01:50:11.490 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-06T01:50:11.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-06T01:50:11.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-06T01:50:11.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-06T01:50:11.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-06T01:50:11.491 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-06T01:50:11.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-06T01:50:11.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-06T01:50:11.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-06T01:50:11.492 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:11 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/994308882' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-06T01:50:11.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.535+0000 7fbcaeffd700 1 -- 172.21.15.53:0/994308882 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbcb0098a50 con 0x7fbcbc0ffcb0 2023-11-06T01:50:11.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 -- 172.21.15.53:0/994308882 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbca80a0520 msgr2=0x7fbca80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:11.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbca80a0520 0x7fbca80a29e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8005950 tx=0x7fbcb800b410 comp rx=0 tx=0).stop 2023-11-06T01:50:11.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 -- 172.21.15.53:0/994308882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcbc0ffcb0 msgr2=0x7fbcbc069a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:11.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcbc0ffcb0 0x7fbcbc069a30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d980 tx=0x7fbcb000dc90 comp rx=0 tx=0).stop 2023-11-06T01:50:11.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 -- 172.21.15.53:0/994308882 shutdown_connections 2023-11-06T01:50:11.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fbca80a0520 0x7fbca80a29e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:11.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcbc101840 0x7fbcbc19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:11.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcbc0ffcb0 0x7fbcbc069a30 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:11.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 --2- 172.21.15.53:0/994308882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc100ea0 0x7fbcbc19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:11.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 -- 172.21.15.53:0/994308882 >> 172.21.15.53:0/994308882 conn(0x7fbcbc0fb220 msgr2=0x7fbcbc104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:11.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 -- 172.21.15.53:0/994308882 shutdown_connections 2023-11-06T01:50:11.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.540+0000 7fbcc3a77700 1 -- 172.21.15.53:0/994308882 wait complete. 2023-11-06T01:50:11.541 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-11-06T01:50:11.554 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-11-06T01:50:11.554 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-06T01:50:11.554 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-06T01:50:11.555 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: Cluster is now healthy 2023-11-06T01:50:11.555 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.163:6832/3181242442,v1:172.21.15.163:6834/3181242442] up:boot 2023-11-06T01:50:11.555 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.163:6836/3382064927,v1:172.21.15.163:6838/3382064927] up:boot 2023-11-06T01:50:11.555 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.163:6833/2387163723,v1:172.21.15.163:6835/2387163723] up:boot 2023-11-06T01:50:11.555 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.53:6840/4133878425,v1:172.21.15.53:6841/4133878425] up:boot 2023-11-06T01:50:11.556 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.53:6838/139725483,v1:172.21.15.53:6839/139725483] up:boot 2023-11-06T01:50:11.556 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.62:6832/3745085778,v1:172.21.15.62:6835/3745085778] up:boot 2023-11-06T01:50:11.556 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.62:6836/1012048915,v1:172.21.15.62:6839/1012048915] up:boot 2023-11-06T01:50:11.556 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.163:6837/122234685,v1:172.21.15.163:6839/122234685] up:boot 2023-11-06T01:50:11.556 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.62:6833/3993523599,v1:172.21.15.62:6837/3993523599] up:boot 2023-11-06T01:50:11.557 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: mds.? [v2:172.21.15.62:6834/807817280,v1:172.21.15.62:6838/807817280] up:boot 2023-11-06T01:50:11.557 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/2973511211' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-06T01:50:11.557 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: fsmap 12 up:standby 2023-11-06T01:50:11.557 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-06T01:50:11.557 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-06T01:50:11.558 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-06T01:50:11.558 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-06T01:50:11.558 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-06T01:50:11.558 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-06T01:50:11.558 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-06T01:50:11.559 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-06T01:50:11.559 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-06T01:50:11.559 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-06T01:50:11.559 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:11 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/994308882' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-06T01:50:11.560 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T01:50:11.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-11-06T01:50:11.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-06T01:50:11.643 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-06T01:50:11.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: Cluster is now healthy 2023-11-06T01:50:11.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.163:6832/3181242442,v1:172.21.15.163:6834/3181242442] up:boot 2023-11-06T01:50:11.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.163:6836/3382064927,v1:172.21.15.163:6838/3382064927] up:boot 2023-11-06T01:50:11.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.163:6833/2387163723,v1:172.21.15.163:6835/2387163723] up:boot 2023-11-06T01:50:11.644 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.53:6840/4133878425,v1:172.21.15.53:6841/4133878425] up:boot 2023-11-06T01:50:11.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.53:6838/139725483,v1:172.21.15.53:6839/139725483] up:boot 2023-11-06T01:50:11.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.62:6832/3745085778,v1:172.21.15.62:6835/3745085778] up:boot 2023-11-06T01:50:11.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.62:6836/1012048915,v1:172.21.15.62:6839/1012048915] up:boot 2023-11-06T01:50:11.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.163:6837/122234685,v1:172.21.15.163:6839/122234685] up:boot 2023-11-06T01:50:11.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.62:6833/3993523599,v1:172.21.15.62:6837/3993523599] up:boot 2023-11-06T01:50:11.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: mds.? [v2:172.21.15.62:6834/807817280,v1:172.21.15.62:6838/807817280] up:boot 2023-11-06T01:50:11.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/2973511211' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-06T01:50:11.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: fsmap 12 up:standby 2023-11-06T01:50:11.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-06T01:50:11.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-06T01:50:11.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-06T01:50:11.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-06T01:50:11.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-06T01:50:11.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-06T01:50:11.647 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-06T01:50:11.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-06T01:50:11.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-06T01:50:11.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='mgr.14152 172.21.15.53:0/3920933554' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-06T01:50:11.648 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:11 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/994308882' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-06T01:50:11.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.699+0000 7fb077fc9700 1 -- 172.21.15.53:0/1656334649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb070106ac0 msgr2=0x7fb070108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:11.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.699+0000 7fb077fc9700 1 --2- 172.21.15.53:0/1656334649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb070106ac0 0x7fb070108ed0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb06800b0a0 tx=0x7fb06800b3b0 comp rx=0 tx=0).stop 2023-11-06T01:50:11.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.699+0000 7fb077fc9700 1 -- 172.21.15.53:0/1656334649 shutdown_connections 2023-11-06T01:50:11.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.699+0000 7fb077fc9700 1 --2- 172.21.15.53:0/1656334649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb070106ac0 0x7fb070108ed0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:11.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.699+0000 7fb077fc9700 1 --2- 172.21.15.53:0/1656334649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb070101860 0x7fb070103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:11.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.699+0000 7fb077fc9700 1 --2- 172.21.15.53:0/1656334649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb070104190 0x7fb070106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:11.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.699+0000 7fb077fc9700 1 -- 172.21.15.53:0/1656334649 >> 172.21.15.53:0/1656334649 conn(0x7fb0700fb1f0 msgr2=0x7fb0700fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:11.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.699+0000 7fb077fc9700 1 -- 172.21.15.53:0/1656334649 shutdown_connections 2023-11-06T01:50:11.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.699+0000 7fb077fc9700 1 -- 172.21.15.53:0/1656334649 wait complete. 2023-11-06T01:50:11.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.699+0000 7fb077fc9700 1 Processor -- start 2023-11-06T01:50:11.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.699+0000 7fb077fc9700 1 -- start start 2023-11-06T01:50:11.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb077fc9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb070101860 0x7fb070196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:11.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb077fc9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb070104190 0x7fb0701970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:11.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb077fc9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb070106ac0 0x7fb07019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:11.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb077fc9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07019e6f0 con 0x7fb070104190 2023-11-06T01:50:11.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb077fc9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07019e830 con 0x7fb070106ac0 2023-11-06T01:50:11.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb077fc9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07019e9c0 con 0x7fb070101860 2023-11-06T01:50:11.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb075564700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb070104190 0x7fb0701970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:11.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb075564700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb070104190 0x7fb0701970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44382/0 (socket says 172.21.15.53:44382) 2023-11-06T01:50:11.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb075564700 1 -- 172.21.15.53:0/687182691 learned_addr learned my addr 172.21.15.53:0/687182691 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-06T01:50:11.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb075d65700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb070101860 0x7fb070196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:11.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb076566700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb070106ac0 0x7fb07019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:11.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb075564700 1 -- 172.21.15.53:0/687182691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb070101860 msgr2=0x7fb070196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:11.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb075564700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb070101860 0x7fb070196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:11.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb075564700 1 -- 172.21.15.53:0/687182691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb070106ac0 msgr2=0x7fb07019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:11.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb075564700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb070106ac0 0x7fb07019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:11.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb075564700 1 -- 172.21.15.53:0/687182691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb068009d00 con 0x7fb070104190 2023-11-06T01:50:11.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb075564700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb070104190 0x7fb0701970b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb06000d950 tx=0x7fb06000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:11.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb066ffd700 1 -- 172.21.15.53:0/687182691 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060009940 con 0x7fb070104190 2023-11-06T01:50:11.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb066ffd700 1 -- 172.21.15.53:0/687182691 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb060010460 con 0x7fb070104190 2023-11-06T01:50:11.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb077fc9700 1 -- 172.21.15.53:0/687182691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07019eca0 con 0x7fb070104190 2023-11-06T01:50:11.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb066ffd700 1 -- 172.21.15.53:0/687182691 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06000f620 con 0x7fb070104190 2023-11-06T01:50:11.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.700+0000 7fb077fc9700 1 -- 172.21.15.53:0/687182691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07019f1b0 con 0x7fb070104190 2023-11-06T01:50:11.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.701+0000 7fb077fc9700 1 -- 172.21.15.53:0/687182691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0700fcd70 con 0x7fb070104190 2023-11-06T01:50:11.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.703+0000 7fb066ffd700 1 -- 172.21.15.53:0/687182691 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fb0600108d0 con 0x7fb070104190 2023-11-06T01:50:11.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.703+0000 7fb066ffd700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb05c0a0520 0x7fb05c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T01:50:11.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.703+0000 7fb066ffd700 1 -- 172.21.15.53:0/687182691 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8145+0+0 (secure 0 0 0) 0x7fb0600cb710 con 0x7fb070104190 2023-11-06T01:50:11.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.703+0000 7fb075d65700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb05c0a0520 0x7fb05c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T01:50:11.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.704+0000 7fb075d65700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb05c0a0520 0x7fb05c0a29e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb06c005950 tx=0x7fb06c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T01:50:11.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.707+0000 7fb066ffd700 1 -- 172.21.15.53:0/687182691 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb060095350 con 0x7fb070104190 2023-11-06T01:50:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:11.907+0000 7fb077fc9700 1 -- 172.21.15.53:0/687182691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb07002cc70 con 0x7fb070104190 2023-11-06T01:50:12.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.556+0000 7fb066ffd700 1 -- 172.21.15.53:0/687182691 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb0600160d0 con 0x7fb070104190 2023-11-06T01:50:12.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.560+0000 7fb077fc9700 1 -- 172.21.15.53:0/687182691 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb05c0a0520 msgr2=0x7fb05c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:12.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.560+0000 7fb077fc9700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb05c0a0520 0x7fb05c0a29e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb06c005950 tx=0x7fb06c00b410 comp rx=0 tx=0).stop 2023-11-06T01:50:12.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.560+0000 7fb077fc9700 1 -- 172.21.15.53:0/687182691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb070104190 msgr2=0x7fb0701970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T01:50:12.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.560+0000 7fb077fc9700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb070104190 0x7fb0701970b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb06000d950 tx=0x7fb06000dc60 comp rx=0 tx=0).stop 2023-11-06T01:50:12.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.563+0000 7fb077fc9700 1 -- 172.21.15.53:0/687182691 shutdown_connections 2023-11-06T01:50:12.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.563+0000 7fb077fc9700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.53:6800/1491433716,v1:172.21.15.53:6801/1491433716] conn(0x7fb05c0a0520 0x7fb05c0a29e0 secure :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb06c005950 tx=0x7fb06c00b410 comp rx=0 tx=0).stop 2023-11-06T01:50:12.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.563+0000 7fb077fc9700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb070106ac0 0x7fb07019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:12.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.563+0000 7fb077fc9700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb070101860 0x7fb070196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T01:50:12.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.563+0000 7fb077fc9700 1 --2- 172.21.15.53:0/687182691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb070104190 0x7fb0701970b0 secure :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb06000d950 tx=0x7fb06000dc60 comp rx=0 tx=0).stop 2023-11-06T01:50:12.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.563+0000 7fb077fc9700 1 -- 172.21.15.53:0/687182691 >> 172.21.15.53:0/687182691 conn(0x7fb0700fb1f0 msgr2=0x7fb0701024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T01:50:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.564+0000 7fb077fc9700 1 -- 172.21.15.53:0/687182691 shutdown_connections 2023-11-06T01:50:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-06T01:50:12.564+0000 7fb077fc9700 1 -- 172.21.15.53:0/687182691 wait complete. 2023-11-06T01:50:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-11-06T01:50:12.581 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-06T01:50:12.616 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-06T01:50:12.674 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-06T01:50:12.727 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-11-06T01:50:12.727 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-06T01:50:12.747 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-06T01:50:12.766 DEBUG:teuthology.orchestra.run.smithi163:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-06T01:50:12.785 INFO:tasks.cephadm:Stopping all daemons... 2023-11-06T01:50:12.786 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-11-06T01:50:12.786 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.a 2023-11-06T01:50:12.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:12 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/994308882' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-06T01:50:12.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:12 smithi053 ceph-mon[99472]: osdmap e86: 12 total, 12 up, 12 in 2023-11-06T01:50:12.815 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:12 smithi053 ceph-mon[99472]: from='client.? 172.21.15.53:0/687182691' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-06T01:50:12.815 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:12 smithi053 ceph-mon[99472]: pgmap v789: 65 pgs: 65 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:50:12.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:12 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/994308882' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-06T01:50:12.892 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:12 smithi062 ceph-mon[103924]: osdmap e86: 12 total, 12 up, 12 in 2023-11-06T01:50:12.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:12 smithi062 ceph-mon[103924]: from='client.? 172.21.15.53:0/687182691' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-06T01:50:12.893 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:12 smithi062 ceph-mon[103924]: pgmap v789: 65 pgs: 65 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:50:12.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:12 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/994308882' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-06T01:50:12.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:12 smithi163 ceph-mon[104416]: osdmap e86: 12 total, 12 up, 12 in 2023-11-06T01:50:12.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:12 smithi163 ceph-mon[104416]: from='client.? 172.21.15.53:0/687182691' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-06T01:50:12.987 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:12 smithi163 ceph-mon[104416]: pgmap v789: 65 pgs: 65 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-06T01:50:13.091 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:12 smithi053 systemd[1]: Stopping Ceph mon.a for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:13.484 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:13 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-a[99448]: 2023-11-06T01:50:13.163+0000 7f0b812b7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:50:13.485 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:13 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-a[99448]: 2023-11-06T01:50:13.163+0000 7f0b812b7700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-11-06T01:50:13.759 INFO:journalctl@ceph.mon.a.smithi053.stdout:Nov 06 01:50:13 smithi053 bash[178717]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-a 2023-11-06T01:50:13.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.a.service' 2023-11-06T01:50:13.895 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:50:13.896 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-11-06T01:50:13.896 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-11-06T01:50:13.896 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.b 2023-11-06T01:50:14.392 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:14 smithi062 systemd[1]: Stopping Ceph mon.b for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:14.858 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-b[103900]: 2023-11-06T01:50:14.428+0000 7f7187651700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:50:14.859 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-b[103900]: 2023-11-06T01:50:14.428+0000 7f7187651700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-11-06T01:50:15.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Nov 06 01:50:14 smithi062 bash[145682]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-b 2023-11-06T01:50:15.168 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.b.service' 2023-11-06T01:50:15.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:50:15.273 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-11-06T01:50:15.273 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-11-06T01:50:15.273 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.c 2023-11-06T01:50:15.679 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:15 smithi163 systemd[1]: Stopping Ceph mon.c for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:15.958 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:15 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-c[104392]: 2023-11-06T01:50:15.680+0000 7fbed4df3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:50:15.958 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:15 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-c[104392]: 2023-11-06T01:50:15.680+0000 7fbed4df3700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-11-06T01:50:16.236 INFO:journalctl@ceph.mon.c.smithi163.stdout:Nov 06 01:50:16 smithi163 bash[146816]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mon-c 2023-11-06T01:50:16.424 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mon.c.service' 2023-11-06T01:50:16.475 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:50:16.476 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-11-06T01:50:16.476 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-11-06T01:50:16.476 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.x 2023-11-06T01:50:16.820 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:50:16 smithi053 systemd[1]: Stopping Ceph mgr.x for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:16.820 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:50:16 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:50:16.820+0000 7fb0aeda0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:50:16.820 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:50:16 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x[99727]: 2023-11-06T01:50:16.820+0000 7fb0aeda0700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-11-06T01:50:17.361 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Nov 06 01:50:17 smithi053 bash[178858]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-x 2023-11-06T01:50:17.367 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.x.service' 2023-11-06T01:50:17.406 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:50:17.406 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-11-06T01:50:17.407 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-11-06T01:50:17.407 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.y 2023-11-06T01:50:17.844 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:50:17 smithi062 systemd[1]: Stopping Ceph mgr.y for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:18.122 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:50:18 smithi062 bash[145839]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-y 2023-11-06T01:50:18.309 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Nov 06 01:50:18 smithi062 systemd[1]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-11-06T01:50:18.313 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.y.service' 2023-11-06T01:50:18.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:50:18.353 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-11-06T01:50:18.353 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-11-06T01:50:18.354 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.z 2023-11-06T01:50:18.737 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:50:18 smithi163 systemd[1]: Stopping Ceph mgr.z for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:19.062 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:50:18 smithi163 bash[146966]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mgr-z 2023-11-06T01:50:19.244 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Nov 06 01:50:19 smithi163 systemd[1]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-11-06T01:50:19.247 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mgr.z.service' 2023-11-06T01:50:19.283 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:50:19.284 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-11-06T01:50:19.284 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-11-06T01:50:19.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.0 2023-11-06T01:50:19.591 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:50:19 smithi053 systemd[1]: Stopping Ceph osd.0 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:20.091 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:50:19 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-0[118800]: 2023-11-06T01:50:19.644+0000 7fb4cf009700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:50:20.091 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:50:19 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-0[118800]: 2023-11-06T01:50:19.644+0000 7fb4cf009700 -1 osd.0 87 *** Got signal Terminated *** 2023-11-06T01:50:20.091 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:50:19 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-0[118800]: 2023-11-06T01:50:19.644+0000 7fb4cf009700 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:50:25.591 INFO:journalctl@ceph.osd.0.smithi053.stdout:Nov 06 01:50:25 smithi053 bash[179001]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-0 2023-11-06T01:50:26.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.0.service' 2023-11-06T01:50:26.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:50:26.856 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-11-06T01:50:26.857 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-11-06T01:50:26.857 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.1 2023-11-06T01:50:27.142 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:50:26 smithi062 systemd[1]: Stopping Ceph osd.1 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:27.642 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:50:27 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-1[109002]: 2023-11-06T01:50:27.226+0000 7f0a9db98700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:50:27.642 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:50:27 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-1[109002]: 2023-11-06T01:50:27.226+0000 7f0a9db98700 -1 osd.1 87 *** Got signal Terminated *** 2023-11-06T01:50:27.642 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:50:27 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-1[109002]: 2023-11-06T01:50:27.226+0000 7f0a9db98700 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:50:33.391 INFO:journalctl@ceph.osd.1.smithi062.stdout:Nov 06 01:50:33 smithi062 bash[145987]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-1 2023-11-06T01:50:34.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.1.service' 2023-11-06T01:50:34.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:50:34.507 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-11-06T01:50:34.507 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-11-06T01:50:34.508 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.2 2023-11-06T01:50:34.868 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:50:34 smithi163 systemd[1]: Stopping Ceph osd.2 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:35.236 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:50:34 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-2[110234]: 2023-11-06T01:50:34.869+0000 7f35fc9cb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:50:35.237 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:50:34 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-2[110234]: 2023-11-06T01:50:34.869+0000 7f35fc9cb700 -1 osd.2 87 *** Got signal Terminated *** 2023-11-06T01:50:35.237 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:50:34 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-2[110234]: 2023-11-06T01:50:34.869+0000 7f35fc9cb700 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:50:40.645 INFO:journalctl@ceph.osd.2.smithi163.stdout:Nov 06 01:50:40 smithi163 bash[147112]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-2 2023-11-06T01:50:41.905 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.2.service' 2023-11-06T01:50:41.946 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:50:41.946 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-11-06T01:50:41.947 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-11-06T01:50:41.947 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.3 2023-11-06T01:50:42.315 INFO:journalctl@ceph.osd.3.smithi053.stdout:Nov 06 01:50:41 smithi053 systemd[1]: Stopping Ceph osd.3 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:42.591 INFO:journalctl@ceph.osd.3.smithi053.stdout:Nov 06 01:50:42 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-3[127479]: 2023-11-06T01:50:42.316+0000 7fee9070d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:50:42.591 INFO:journalctl@ceph.osd.3.smithi053.stdout:Nov 06 01:50:42 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-3[127479]: 2023-11-06T01:50:42.316+0000 7fee9070d700 -1 osd.3 87 *** Got signal Terminated *** 2023-11-06T01:50:42.591 INFO:journalctl@ceph.osd.3.smithi053.stdout:Nov 06 01:50:42 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-3[127479]: 2023-11-06T01:50:42.316+0000 7fee9070d700 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:50:46.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:46 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:46.229+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:46.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:46 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:46.272+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:47.486 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:47 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:47.205+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:47.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:47 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:47.241+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:47.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:47 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:47.231+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:48.240 INFO:journalctl@ceph.osd.3.smithi053.stdout:Nov 06 01:50:47 smithi053 bash[179292]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-3 2023-11-06T01:50:48.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:48 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:48.242+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:48.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:48 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:48.254+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:48.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:48 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:48.248+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:49.291 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.3.service' 2023-11-06T01:50:49.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:50:49.342 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-11-06T01:50:49.343 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-11-06T01:50:49.343 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.4 2023-11-06T01:50:49.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:49 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:49.022+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:50:49.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:49 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:49.236+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:49.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:49 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:49.274+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:49.642 INFO:journalctl@ceph.osd.4.smithi062.stdout:Nov 06 01:50:49 smithi062 systemd[1]: Stopping Ceph osd.4 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:49.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:49 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:49.216+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:50.142 INFO:journalctl@ceph.osd.4.smithi062.stdout:Nov 06 01:50:49 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-4[116880]: 2023-11-06T01:50:49.714+0000 7f52fcf04700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:50:50.142 INFO:journalctl@ceph.osd.4.smithi062.stdout:Nov 06 01:50:49 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-4[116880]: 2023-11-06T01:50:49.714+0000 7f52fcf04700 -1 osd.4 87 *** Got signal Terminated *** 2023-11-06T01:50:50.142 INFO:journalctl@ceph.osd.4.smithi062.stdout:Nov 06 01:50:49 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-4[116880]: 2023-11-06T01:50:49.714+0000 7f52fcf04700 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:50:50.277 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:49 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:49.989+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:50:50.486 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:50 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:50.034+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:50:50.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:50 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:50.198+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:50.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:50 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:50.279+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:50.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:50 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:50.198+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:51.293 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:50 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:50.998+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:50:51.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:51 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:51.025+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:50:51.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:51 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:51.159+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:51.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:51 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:51.295+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:51.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:51 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:51.176+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:52.326 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:51 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:51.952+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:50:52.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:52 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:52.051+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:50:52.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:52 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:52.174+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:52.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:52 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:52.329+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:52.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:52 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:52.199+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:53.329 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:52 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:52.981+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:50:53.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:53 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:53.200+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:53.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:53 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:53.090+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:50:53.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:53 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:53.090+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:28.300000+0000 front 2023-11-06T01:50:28.299952+0000 (oldest deadline 2023-11-06T01:50:52.900682+0000) 2023-11-06T01:50:53.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:53 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:53.332+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:53.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:53 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:53.220+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:54.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:53 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:53.984+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:50:54.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:54 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:54.167+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:54.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:54 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:54.128+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:50:54.488 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:54 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:54.128+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:28.300000+0000 front 2023-11-06T01:50:28.299952+0000 (oldest deadline 2023-11-06T01:50:52.900682+0000) 2023-11-06T01:50:54.488 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:50:54 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:50:54.172+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:50:54.642 INFO:journalctl@ceph.osd.4.smithi062.stdout:Nov 06 01:50:54 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-4[116880]: 2023-11-06T01:50:54.299+0000 7f52f74f6700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:28.721923+0000 front 2023-11-06T01:50:28.721956+0000 (oldest deadline 2023-11-06T01:50:54.022060+0000) 2023-11-06T01:50:54.642 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:50:54 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:50:54.183+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:50:54.643 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:54 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:54.221+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:54.643 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:54 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:54.221+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:50:54.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:54 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:54.368+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:55.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:55 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:55.011+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:50:55.342 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:55 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:55.011+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:50:55.435 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:55 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:55.231+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:55.436 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:55 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:55.231+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:50:55.436 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:50:55 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:50:55.159+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:50:55.486 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:50:55 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:50:55.155+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:50:55.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:55 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:55.184+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:55.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:55 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:55.184+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:50:55.488 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:55 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:55.173+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:50:55.488 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:55 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:55.173+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:28.300000+0000 front 2023-11-06T01:50:28.299952+0000 (oldest deadline 2023-11-06T01:50:52.900682+0000) 2023-11-06T01:50:55.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:55 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:55.404+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:55.892 INFO:journalctl@ceph.osd.4.smithi062.stdout:Nov 06 01:50:55 smithi062 bash[146255]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-4 2023-11-06T01:50:56.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:56 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:56.001+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:50:56.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:56 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:56.001+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:50:56.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:50:56 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:50:56.202+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:50:56.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:56 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:56.194+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:56.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:56 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:56.194+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:50:56.486 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:50:56 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:50:56.137+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:50:56.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:56 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:56.229+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:56.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:56 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:56.229+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:50:56.488 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:56 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:56.173+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:50:56.488 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:56 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:56.173+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:28.300000+0000 front 2023-11-06T01:50:28.299952+0000 (oldest deadline 2023-11-06T01:50:52.900682+0000) 2023-11-06T01:50:56.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.4.service' 2023-11-06T01:50:56.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:50:56.803 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-11-06T01:50:56.803 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-11-06T01:50:56.804 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.5 2023-11-06T01:50:56.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:56 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:56.383+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:56.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:56 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:56.383+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:50:57.096 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:50:57 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:50:57.097+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:50:57.097 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:56 smithi163 systemd[1]: Stopping Ceph osd.5 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:50:57.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:57 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:57.007+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:50:57.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:57 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:57.007+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:50:57.486 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:57 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:57.169+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:50:57.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:57 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:57.169+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:28.300000+0000 front 2023-11-06T01:50:28.299952+0000 (oldest deadline 2023-11-06T01:50:52.900682+0000) 2023-11-06T01:50:57.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:57 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:57.173+0000 7f5123ab6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:50:57.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:57 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:57.173+0000 7f5123ab6700 -1 osd.5 87 *** Got signal Terminated *** 2023-11-06T01:50:57.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:57 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:57.173+0000 7f5123ab6700 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:50:57.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:57 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:57.231+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:57.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:57 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:57.231+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:50:57.642 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:50:57 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:50:57.247+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:50:57.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:57 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:57.178+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:57.643 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:57 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:57.178+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:50:57.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:57 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:57.345+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:57.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:57 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:57.345+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:50:58.330 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:58 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:58.055+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:50:58.330 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:58 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:58.055+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:50:58.486 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:50:58 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:50:58.126+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:50:58.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:58 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:58.254+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:58.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:58 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:58.254+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:50:58.488 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:58 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:58.152+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:50:58.488 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:58 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:58.152+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:28.300000+0000 front 2023-11-06T01:50:28.299952+0000 (oldest deadline 2023-11-06T01:50:52.900682+0000) 2023-11-06T01:50:58.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:58 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:58.332+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:58.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:58 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:58.332+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:50:58.641 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:50:58 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:50:58.198+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:50:58.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:58 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:58.178+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:58.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:58 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:58.178+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:50:59.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:59 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:59.026+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:50:59.342 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:50:59 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:50:59.026+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:50:59.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:50:59 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:50:59.153+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:50:59.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:59 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:59.130+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:50:59.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:50:59 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:50:59.130+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:50:59.487 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:50:59 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:50:59.082+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:50:59.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:59 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:59.183+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:50:59.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:50:59 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:50:59.183+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:28.300000+0000 front 2023-11-06T01:50:28.299952+0000 (oldest deadline 2023-11-06T01:50:52.900682+0000) 2023-11-06T01:50:59.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:59 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:59.265+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:50:59.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:50:59 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:50:59.265+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:50:59.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:59 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:59.374+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:50:59.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:50:59 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:50:59.374+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:00.325 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:00 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:00.073+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:51:00.325 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:00 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:00.073+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:51:00.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:00 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:00.167+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:00.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:00 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:00.100+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:00.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:00 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:00.100+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:00.486 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:00 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:00.036+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:00.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:51:00 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:51:00.220+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:51:00.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:51:00 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:51:00.220+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:28.300000+0000 front 2023-11-06T01:50:28.299952+0000 (oldest deadline 2023-11-06T01:50:52.900682+0000) 2023-11-06T01:51:00.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:00 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:00.224+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:00.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:00 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:00.224+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:00.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:00 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:00.326+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:00.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:00 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:00.326+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:01.261 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:00 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:00.995+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:01.262 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:51:01 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:51:01.237+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.599461+0000 front 2023-11-06T01:50:23.599615+0000 (oldest deadline 2023-11-06T01:50:48.299395+0000) 2023-11-06T01:51:01.262 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:51:01 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5[118280]: 2023-11-06T01:51:01.237+0000 7f511e0a8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:28.300000+0000 front 2023-11-06T01:50:28.299952+0000 (oldest deadline 2023-11-06T01:50:52.900682+0000) 2023-11-06T01:51:01.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:01 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:01.029+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:51:01.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:01 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:01.029+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:51:01.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:01 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:01.029+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:34.887750+0000 front 2023-11-06T01:50:34.887716+0000 (oldest deadline 2023-11-06T01:51:00.787634+0000) 2023-11-06T01:51:01.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:01 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:01.202+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:01.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:01 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:01.051+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:01.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:01 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:01.051+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:01.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:01 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:01.259+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:01.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:01 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:01.259+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:01.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:01 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:01.259+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:01.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:01 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:01.353+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:01.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:01 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:01.353+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:01.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:01 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:01.353+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:02.270 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:01 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:01.996+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:02.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:01 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:01.990+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:51:02.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:01 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:01.990+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:51:02.342 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:01 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:01.990+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:34.887750+0000 front 2023-11-06T01:50:34.887716+0000 (oldest deadline 2023-11-06T01:51:00.787634+0000) 2023-11-06T01:51:02.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:02 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:02.222+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:02.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:02 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:02.098+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:02.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:02 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:02.098+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:02.675 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:02 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:02.272+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:02.675 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:02 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:02.272+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:02.675 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:02 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:02.272+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:02.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:02 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:02.385+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:02.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:02 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:02.385+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:02.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:02 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:02.385+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:03.226 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:02 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:02.952+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:03.226 INFO:journalctl@ceph.osd.5.smithi163.stdout:Nov 06 01:51:02 smithi163 bash[147375]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-5 2023-11-06T01:51:03.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:03 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:03.033+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:51:03.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:03 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:03.033+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:51:03.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:03 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:03.033+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:34.887750+0000 front 2023-11-06T01:50:34.887716+0000 (oldest deadline 2023-11-06T01:51:00.787634+0000) 2023-11-06T01:51:03.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:03 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:03.231+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:03.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:03 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:03.231+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:03.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:03 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:03.074+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:03.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:03 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:03.074+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:03.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:03 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:03.074+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:03.484 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:03 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:03.227+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:03.485 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:03 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:03.227+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:03.485 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:03 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:03.227+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:03.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:03 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:03.424+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:03.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:03 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:03.424+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:03.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:03 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:03.424+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:04.202 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:03 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:03.962+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:04.210 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.5.service' 2023-11-06T01:51:04.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:51:04.252 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-11-06T01:51:04.252 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-11-06T01:51:04.253 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.6 2023-11-06T01:51:04.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:04.050+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:51:04.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:04.050+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:51:04.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:04.050+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:34.887750+0000 front 2023-11-06T01:50:34.887716+0000 (oldest deadline 2023-11-06T01:51:00.787634+0000) 2023-11-06T01:51:04.342 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:04 smithi053 systemd[1]: Stopping Ceph osd.6 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:51:04.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:04 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:04.240+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:04.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:04 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:04.240+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:04.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:04 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:04.089+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:04.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:04 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:04.089+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:04.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:04 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:04.089+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:04.486 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:04 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:04.277+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:04.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:04 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:04.277+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:04.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:04 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:04.277+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:04.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:04.459+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:04.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:04.459+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:04.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:04.459+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:04.842 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:04.613+0000 7f2573fb4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:51:04.842 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:04.613+0000 7f2573fb4700 -1 osd.6 87 *** Got signal Terminated *** 2023-11-06T01:51:04.842 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:04 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:04.613+0000 7f2573fb4700 -1 osd.6 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:51:05.236 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:04 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:04.944+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:05.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:05 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:05.071+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:51:05.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:05 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:05.071+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:51:05.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:05 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:05.071+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:34.887750+0000 front 2023-11-06T01:50:34.887716+0000 (oldest deadline 2023-11-06T01:51:00.787634+0000) 2023-11-06T01:51:05.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:05 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:05.207+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:05.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:05 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:05.207+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:05.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:05 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:05.086+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:05.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:05 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:05.086+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:05.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:05 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:05.086+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:05.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:05 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:05.277+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:05.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:05 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:05.277+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:05.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:05 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:05.277+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:05.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:05 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:05.472+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:05.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:05 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:05.472+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:05.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:05 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:05.472+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:06.236 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:05 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:05.975+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:06.237 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:05 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:05.975+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:06.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:06 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:06.068+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:51:06.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:06 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:06.068+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:51:06.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:06 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:06.068+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:34.887750+0000 front 2023-11-06T01:50:34.887716+0000 (oldest deadline 2023-11-06T01:51:00.787634+0000) 2023-11-06T01:51:06.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:06 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:06.238+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:06.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:06 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:06.238+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:06.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:06 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:06.101+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:06.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:06 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:06.101+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:06.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:06 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:06.101+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:06.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:06 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:06.302+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:06.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:06 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:06.302+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:06.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:06 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:06.302+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:06.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:06 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:06.423+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:06.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:06 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:06.423+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:06.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:06 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:06.423+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:07.236 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:06 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:06.984+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:07.237 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:06 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:06.984+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:07.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:07 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:07.049+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:51:07.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:07 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:07.049+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:51:07.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:07 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:07.049+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:34.887750+0000 front 2023-11-06T01:50:34.887716+0000 (oldest deadline 2023-11-06T01:51:00.787634+0000) 2023-11-06T01:51:07.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:07 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:07.196+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:07.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:07 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:07.196+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:07.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:07 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:07.078+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:07.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:07 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:07.078+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:07.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:07 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:07.078+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:07.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:07 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:07.328+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:07.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:07 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:07.328+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:07.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:07 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:07.328+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:07.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:07 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:07.401+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:07.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:07 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:07.401+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:07.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:07 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:07.401+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:08.236 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:07 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:07.947+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:08.237 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:07 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:07.947+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:08.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:08 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:08.078+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:51:08.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:08 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:08.078+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:51:08.341 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:08 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:08.078+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:34.887750+0000 front 2023-11-06T01:50:34.887716+0000 (oldest deadline 2023-11-06T01:51:00.787634+0000) 2023-11-06T01:51:08.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:08 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:08.183+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:08.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:08 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:08.183+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:08.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:08 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:08.064+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:08.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:08 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:08.064+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:08.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:08 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:08.064+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:08.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:08 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:08.372+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:08.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:08 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:08.372+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:08.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:08 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:08.372+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:08.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:08 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:08.372+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:08.840 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:08 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:08.368+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:08.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:08 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:08.368+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:08.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:08 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:08.368+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:09.236 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:08 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:08.954+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:09.237 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:08 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:08.954+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:09.237 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:08 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:08.954+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:09.332 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:09.029+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:23.384430+0000 front 2023-11-06T01:50:23.384581+0000 (oldest deadline 2023-11-06T01:50:49.284422+0000) 2023-11-06T01:51:09.333 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:09.029+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.486410+0000 front 2023-11-06T01:50:31.486547+0000 (oldest deadline 2023-11-06T01:50:54.386492+0000) 2023-11-06T01:51:09.333 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6[136141]: 2023-11-06T01:51:09.029+0000 7f256fda9700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:34.887750+0000 front 2023-11-06T01:50:34.887716+0000 (oldest deadline 2023-11-06T01:51:00.787634+0000) 2023-11-06T01:51:09.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:09.194+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:09.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:09.194+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:09.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:09.055+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:09.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:09.055+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:09.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:09 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:09.055+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:09.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:09.334+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:09.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:09.334+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:09.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:09 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:09.334+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:09.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:09.370+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:09.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:09.370+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:09.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:09.370+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:09.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:09 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:09.370+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:10.384 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:10 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:10.001+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:10.384 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:10 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:10.001+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:10.384 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:10 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:10.001+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:10.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:10 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:10.190+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:10.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:10 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:10.190+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:10.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:10 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:10.190+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:47.100622+0000 front 2023-11-06T01:50:47.100765+0000 (oldest deadline 2023-11-06T01:51:09.901195+0000) 2023-11-06T01:51:10.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:10 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:10.019+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:10.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:10 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:10.019+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:10.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:10 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:10.019+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:10.591 INFO:journalctl@ceph.osd.6.smithi053.stdout:Nov 06 01:51:10 smithi053 bash[179561]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-6 2023-11-06T01:51:10.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:10 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:10.349+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:10.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:10 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:10.350+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:10.592 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:10 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:10.350+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:10.592 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:10 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:10.350+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:10.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:10 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:10.386+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:10.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:10 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:10.386+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:10.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:10 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:10.386+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:10.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:10 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:10.386+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:11.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:11 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:11.153+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:11.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:11 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:11.153+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:11.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:11 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:11.153+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:47.100622+0000 front 2023-11-06T01:50:47.100765+0000 (oldest deadline 2023-11-06T01:51:09.901195+0000) 2023-11-06T01:51:11.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:11 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:11.038+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:11.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:11 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:11.038+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:11.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:11 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:11.038+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:11.422 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:10 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:10.995+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:11.422 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:10 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:10.995+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:11.422 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:10 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:10.995+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:11.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:11 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:11.330+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:11.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:11 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:11.330+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:11.592 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:11 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:11.330+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:11.592 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:11 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:11.330+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:11.680 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.6.service' 2023-11-06T01:51:11.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:51:11.724 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-11-06T01:51:11.725 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-11-06T01:51:11.725 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.7 2023-11-06T01:51:11.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:11 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:11.424+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:11.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:11 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:11.424+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:11.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:11 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:11.424+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:11.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:11 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:11.424+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:12.052 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:11 smithi062 systemd[1]: Stopping Ceph osd.7 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:51:12.382 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:12 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:12.037+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:12.382 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:12 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:12.037+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:12.383 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:12 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:12.037+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:12.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:12.053+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:12.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:12.053+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:12.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:12.053+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:12.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:12.053+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:12.393 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:12.086+0000 7f061d679700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:51:12.393 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:12.086+0000 7f061d679700 -1 osd.7 87 *** Got signal Terminated *** 2023-11-06T01:51:12.393 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:12.086+0000 7f061d679700 -1 osd.7 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:51:12.393 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:12.142+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:12.394 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:12.142+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:12.394 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:12 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:12.142+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:47.100622+0000 front 2023-11-06T01:50:47.100765+0000 (oldest deadline 2023-11-06T01:51:09.901195+0000) 2023-11-06T01:51:12.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:12 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:12.308+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:12.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:12 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:12.308+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:12.591 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:12 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:12.308+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:12.592 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:12 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:12.308+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:12.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:12 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:12.384+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:12.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:12 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:12.384+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:12.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:12 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:12.384+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:12.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:12 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:12.384+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:13.340 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:12 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:12.993+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:13.341 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:12 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:12.993+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:13.341 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:12 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:12.993+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:13.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:13 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:13.130+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:13.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:13 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:13.130+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:13.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:13 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:13.130+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:47.100622+0000 front 2023-11-06T01:50:47.100765+0000 (oldest deadline 2023-11-06T01:51:09.901195+0000) 2023-11-06T01:51:13.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:13 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:13.061+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:13.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:13 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:13.061+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:13.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:13 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:13.061+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:13.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:13 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:13.061+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:13.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:13 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:13.342+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:13.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:13 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:13.342+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:13.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:13 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:13.342+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:13.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:13 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:13.342+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:13.840 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:13 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:13.344+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:13.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:13 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:13.344+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:13.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:13 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:13.344+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:13.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:13 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:13.344+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:14.359 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:14 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:14.012+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:14.359 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:14 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:14.012+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:14.359 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:14 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:14.012+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:14.393 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:14.148+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:14.394 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:14.148+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:14.394 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:14.148+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:47.100622+0000 front 2023-11-06T01:50:47.100765+0000 (oldest deadline 2023-11-06T01:51:09.901195+0000) 2023-11-06T01:51:14.394 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:14.078+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:14.395 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:14.078+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:14.395 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:14.078+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:14.395 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:14.078+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:14.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:14 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:14.361+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:14.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:14 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:14.361+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:14.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:14 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:14.361+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:14.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:14 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:14.361+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:14.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:14 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:14.374+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:14.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:14 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:14.374+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:14.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:14 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:14.374+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:14.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:14 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:14.374+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:15.376 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:15 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:15.038+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:15.377 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:15 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:15.038+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:15.377 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:15 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:15.038+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:15.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:15 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:15.135+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:15.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:15 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:15.135+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:15.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:15 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:15.135+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:47.100622+0000 front 2023-11-06T01:50:47.100765+0000 (oldest deadline 2023-11-06T01:51:09.901195+0000) 2023-11-06T01:51:15.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:15 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:15.063+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:15.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:15 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:15.063+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:15.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:15 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:15.063+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:15.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:15 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:15.063+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:15.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:15 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:15.378+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:15.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:15 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:15.378+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:15.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:15 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:15.378+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:15.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:15 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:15.378+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:15.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:15 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:15.397+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:15.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:15 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:15.397+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:15.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:15 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:15.397+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:15.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:15 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:15.397+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:16.345 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:16 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:16.072+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:16.345 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:16 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:16.072+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:16.346 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:16 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:16.072+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:16.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:16 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:16.157+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:30.797975+0000 front 2023-11-06T01:50:30.797947+0000 (oldest deadline 2023-11-06T01:50:53.698098+0000) 2023-11-06T01:51:16.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:16 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:16.157+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.699726+0000 front 2023-11-06T01:50:37.699754+0000 (oldest deadline 2023-11-06T01:51:02.399593+0000) 2023-11-06T01:51:16.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:16 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:16.157+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:47.100622+0000 front 2023-11-06T01:50:47.100765+0000 (oldest deadline 2023-11-06T01:51:09.901195+0000) 2023-11-06T01:51:16.392 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:16 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7[125581]: 2023-11-06T01:51:16.157+0000 7f061746a700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.002127+0000 front 2023-11-06T01:50:51.002163+0000 (oldest deadline 2023-11-06T01:51:15.702432+0000) 2023-11-06T01:51:16.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:16 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:16.043+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:16.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:16 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:16.043+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:16.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:16 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:16.043+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:16.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:16 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:16.043+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:16.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:16 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:16.347+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:16.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:16 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:16.347+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:16.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:16 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:16.347+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:16.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:16 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:16.347+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:16.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:16 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:16.390+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:16.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:16 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:16.390+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:16.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:16 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:16.390+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:16.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:16 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:16.390+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:16.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:16 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:16.390+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:17.332 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:17 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:17.027+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:17.333 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:17 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:17.027+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:17.333 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:17 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:17.027+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:17.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:17 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:17.022+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:17.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:17 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:17.022+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:17.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:17 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:17.022+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:17.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:17 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:17.022+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:17.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:17 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:17.022+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:17.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:17 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:17.334+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:17.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:17 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:17.334+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:17.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:17 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:17.334+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:17.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:17 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:17.334+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:17.840 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:17.420+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:17.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:17.420+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:17.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:17.420+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:17.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:17.420+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:17.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:17 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:17.420+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:17.970 INFO:journalctl@ceph.osd.7.smithi062.stdout:Nov 06 01:51:17 smithi062 bash[146524]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-7 2023-11-06T01:51:18.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:18 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:18.041+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:18.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:18 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:18.041+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:18.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:18 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:18.041+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:18.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:18 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:18.041+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:18.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:18 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:18.041+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:18.486 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:18 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:18.313+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:18.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:18 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:18.313+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:18.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:18 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:18.313+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:18.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:18 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:18.313+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:18.487 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:18 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:18.065+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:18.488 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:18 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:18.065+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:18.488 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:18 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:18.065+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:18.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:18 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:18.456+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:18.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:18 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:18.456+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:18.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:18 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:18.456+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:18.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:18 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:18.456+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:18.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:18 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:18.456+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:19.237 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:19 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:19.008+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:19.237 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:19 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:19.008+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:19.237 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:19 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:19.008+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:19.238 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:19 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:19.008+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:19.238 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:19 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:19.008+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:19.245 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.7.service' 2023-11-06T01:51:19.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:51:19.289 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-11-06T01:51:19.290 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-11-06T01:51:19.290 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.8 2023-11-06T01:51:19.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:19 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:19.342+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:19.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:19 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:19.342+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:19.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:19 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:19.342+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:19.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:19 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:19.342+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:19.488 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:19 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:19.109+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:19.489 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:19 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:19.109+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:19.489 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:19 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:19.109+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:19.489 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:19 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:19.109+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:54.301557+0000 front 2023-11-06T01:50:54.301663+0000 (oldest deadline 2023-11-06T01:51:18.401445+0000) 2023-11-06T01:51:19.490 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:19 smithi163 systemd[1]: Stopping Ceph osd.8 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:51:19.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:19 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:19.438+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:19.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:19 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:19.438+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:19.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:19 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:19.438+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:19.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:19 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:19.438+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:19.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:19 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:19.438+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:19.987 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:19 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:19.645+0000 7f2f5a657700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:51:20.055 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:19 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:19.645+0000 7f2f5a657700 -1 osd.8 87 *** Got signal Terminated *** 2023-11-06T01:51:20.055 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:19 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:19.645+0000 7f2f5a657700 -1 osd.8 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:51:20.369 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:20 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:20.066+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:20.370 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:20 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:20.066+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:20.370 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:20 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:20.066+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:20.370 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:20 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:20.066+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:54.301557+0000 front 2023-11-06T01:50:54.301663+0000 (oldest deadline 2023-11-06T01:51:18.401445+0000) 2023-11-06T01:51:20.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:20 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:20.025+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:20.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:20 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:20.025+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:20.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:20 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:20.025+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:20.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:20 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:20.025+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:20.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:20 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:20.025+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:20.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:20 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:20.372+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:20.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:20 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:20.372+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:20.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:20 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:20.372+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:20.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:20 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:20.372+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:20.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:20 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:20.372+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:20.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:20 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:20.391+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:20.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:20 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:20.391+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:20.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:20 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:20.391+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:20.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:20 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:20.391+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:20.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:20 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:20.391+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:21.380 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:21 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:21.106+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:21.381 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:21 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:21.106+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:21.381 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:21 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:21.106+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:21.381 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:21 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:21.106+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:54.301557+0000 front 2023-11-06T01:50:54.301663+0000 (oldest deadline 2023-11-06T01:51:18.401445+0000) 2023-11-06T01:51:21.391 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:21.025+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:21.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:21.025+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:21.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:21.025+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:21.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:21.025+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:21.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:21.025+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:21.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:21 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:21.383+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:21.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:21 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:21.383+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:21.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:21 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:21.383+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:21.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:21 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:21.383+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:21.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:21 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:21.383+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:21.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:21 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:21.381+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:21.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:21 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:21.381+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:21.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:21 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:21.381+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:21.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:21 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:21.381+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:21.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:21 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:21.381+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:22.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:21.983+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:22.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:21.983+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:22.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:21.983+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:22.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:21.983+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:22.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:21 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:21.983+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:22.486 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:22 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:22.137+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:22.487 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:22 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:22.137+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:22.487 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:22 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:22.137+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:22.487 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:22 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:22.137+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:54.301557+0000 front 2023-11-06T01:50:54.301663+0000 (oldest deadline 2023-11-06T01:51:18.401445+0000) 2023-11-06T01:51:22.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:22 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:22.381+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:22.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:22 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:22.381+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:22.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:22 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:22.381+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:22.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:22 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:22.381+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:22.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:22 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:22.381+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:22.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:22 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:22.417+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:22.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:22 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:22.417+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:22.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:22 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:22.417+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:22.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:22 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:22.417+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:22.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:22 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:22.417+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:23.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:22 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:22.975+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:23.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:22 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:22.975+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:23.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:22 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:22.975+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:23.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:22 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:22.975+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:23.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:22 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:22.975+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:23.408 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:23 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:23.143+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:23.409 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:23 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:23.143+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:23.409 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:23 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:23.143+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:23.409 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:23 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:23.143+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:54.301557+0000 front 2023-11-06T01:50:54.301663+0000 (oldest deadline 2023-11-06T01:51:18.401445+0000) 2023-11-06T01:51:23.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:23 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:23.410+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:23.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:23 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:23.410+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:23.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:23 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:23.410+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:23.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:23 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:23.410+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:23.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:23 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:23.410+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:23.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:23 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:23.445+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:23.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:23 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:23.445+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:23.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:23 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:23.445+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:23.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:23 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:23.445+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:23.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:23 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:23.445+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:24.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:23 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:23.955+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:24.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:23 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:23.955+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:24.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:23 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:23.955+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:24.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:23 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:23.955+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:24.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:23 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:23.955+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:24.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:23 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:23.955+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:24.401 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:24 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:24.099+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:27.497022+0000 front 2023-11-06T01:50:27.496803+0000 (oldest deadline 2023-11-06T01:50:53.396932+0000) 2023-11-06T01:51:24.401 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:24 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:24.099+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.798302+0000 front 2023-11-06T01:50:39.798326+0000 (oldest deadline 2023-11-06T01:51:05.698541+0000) 2023-11-06T01:51:24.402 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:24 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:24.099+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:45.699187+0000 front 2023-11-06T01:50:45.699144+0000 (oldest deadline 2023-11-06T01:51:08.599174+0000) 2023-11-06T01:51:24.402 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:24 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8[126153]: 2023-11-06T01:51:24.099+0000 7f2f5644c700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:54.301557+0000 front 2023-11-06T01:50:54.301663+0000 (oldest deadline 2023-11-06T01:51:18.401445+0000) 2023-11-06T01:51:24.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:24 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:24.404+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:24.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:24 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:24.404+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:24.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:24 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:24.404+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:24.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:24 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:24.404+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:24.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:24 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:24.404+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:24.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:24 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:24.489+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:24.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:24 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:24.489+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:24.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:24 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:24.489+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:24.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:24 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:24.489+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:24.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:24 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:24.489+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:25.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:24 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:24.980+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:25.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:24 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:24.980+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:25.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:24 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:24.980+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:25.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:24 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:24.980+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:25.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:24 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:24.980+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:25.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:24 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:24.980+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:25.486 INFO:journalctl@ceph.osd.8.smithi163.stdout:Nov 06 01:51:25 smithi163 bash[147647]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-8 2023-11-06T01:51:25.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:25 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:25.435+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:25.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:25 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:25.435+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:25.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:25 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:25.435+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:25.487 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:25 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:25.435+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:25.488 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:25 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:25.435+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:25.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:25 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:25.504+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:25.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:25 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:25.504+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:25.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:25 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:25.504+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:25.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:25 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:25.504+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:25.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:25 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:25.504+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:25.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:25 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:25.504+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:51:00.916965+0000 front 2023-11-06T01:51:00.916834+0000 (oldest deadline 2023-11-06T01:51:25.016861+0000) 2023-11-06T01:51:26.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:26 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:26.027+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:26.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:26 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:26.027+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:26.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:26 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:26.027+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:26.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:26 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:26.027+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:26.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:26 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:26.027+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:26.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:26 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:26.027+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:26.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:26 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:26.446+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:26.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:26 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:26.446+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:26.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:26 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:26.446+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:26.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:26 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:26.446+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:26.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:26 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:26.446+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:26.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:26 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:26.446+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:26.749 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.8.service' 2023-11-06T01:51:26.790 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:51:26.790 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-11-06T01:51:26.791 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-11-06T01:51:26.791 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.9 2023-11-06T01:51:26.822 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:26 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:26.519+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:26.823 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:26 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:26.519+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:26.823 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:26 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:26.519+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:26.823 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:26 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:26.519+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:26.823 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:26 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:26.519+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:26.824 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:26 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:26.519+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:51:00.916965+0000 front 2023-11-06T01:51:00.916834+0000 (oldest deadline 2023-11-06T01:51:25.016861+0000) 2023-11-06T01:51:27.091 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:26 smithi053 systemd[1]: Stopping Ceph osd.9 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:51:27.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:27 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:27.030+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:27.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:27 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:27.030+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:27.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:27 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:27.030+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:27.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:27 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:27.030+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:27.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:27 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:27.030+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:27.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:27 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:27.030+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:27.542 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:27 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:27.152+0000 7f988c01d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:51:27.543 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:27 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:27.152+0000 7f988c01d700 -1 osd.9 87 *** Got signal Terminated *** 2023-11-06T01:51:27.543 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:27 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:27.152+0000 7f988c01d700 -1 osd.9 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:51:27.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:27 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:27.544+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:27.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:27 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:27.544+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:27.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:27 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:27.544+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:27.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:27 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:27.544+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:27.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:27 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:27.544+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:27.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:27 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:27.544+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:51:00.916965+0000 front 2023-11-06T01:51:00.916834+0000 (oldest deadline 2023-11-06T01:51:25.016861+0000) 2023-11-06T01:51:27.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:27.490+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:27.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:27.490+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:27.988 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:27.490+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:27.988 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:27.490+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:27.988 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:27.490+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:27.988 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:27 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:27.490+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:28.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:28 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:28.075+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:28.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:28 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:28.075+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:28.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:28 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:28.075+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:28.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:28 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:28.075+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:28.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:28 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:28.075+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:28.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:28 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:28.075+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:28.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:28 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:28.442+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:28.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:28 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:28.442+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:28.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:28 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:28.442+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:28.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:28 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:28.442+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:28.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:28 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:28.442+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:28.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:28 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:28.442+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:28.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:28 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:28.498+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:28.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:28 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:28.498+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:28.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:28 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:28.498+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:28.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:28 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:28.498+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:28.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:28 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:28.498+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:28.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:28 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:28.498+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:51:00.916965+0000 front 2023-11-06T01:51:00.916834+0000 (oldest deadline 2023-11-06T01:51:25.016861+0000) 2023-11-06T01:51:29.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:29 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:29.028+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:29.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:29 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:29.028+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:29.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:29 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:29.028+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:29.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:29 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:29.028+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:29.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:29 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:29.028+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:29.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:29 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:29.028+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:29.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:29 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:29.432+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:29.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:29 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:29.432+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:29.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:29 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:29.432+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:29.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:29 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:29.432+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:29.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:29 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:29.432+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:29.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:29 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:29.432+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:29.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:29 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:29.452+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:29.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:29 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:29.452+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:29.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:29 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:29.452+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:29.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:29 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:29.452+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:29.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:29 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:29.452+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:29.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:29 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:29.452+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:51:00.916965+0000 front 2023-11-06T01:51:00.916834+0000 (oldest deadline 2023-11-06T01:51:25.016861+0000) 2023-11-06T01:51:30.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:30 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:30.044+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:30.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:30 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:30.044+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:30.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:30 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:30.044+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:30.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:30 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:30.044+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:30.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:30 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:30.044+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:30.394 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:30 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:30.044+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:30.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:30.453+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:30.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:30.453+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:30.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:30.453+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:30.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:30.453+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:30.739 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:30.453+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:30.739 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:30.453+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:30.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:30 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:30.435+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:30.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:30 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:30.435+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:30.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:30 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:30.435+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:30.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:30 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:30.435+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:30.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:30 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:30.435+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:30.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:30 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:30.435+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:51:00.916965+0000 front 2023-11-06T01:51:00.916834+0000 (oldest deadline 2023-11-06T01:51:25.016861+0000) 2023-11-06T01:51:31.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:31 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:31.084+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:31.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:31 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:31.084+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:31.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:31 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:31.084+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:31.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:31 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:31.084+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:31.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:31 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:31.084+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:31.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:31 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:31.084+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:31.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:31 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:31.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:31.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:31 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:31.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:31.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:31 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:31.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:31.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:31 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:31.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:31.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:31 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:31.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:31.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:31 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:31.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:31.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:31.417+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:19.807819+0000 front 2023-11-06T01:50:19.807794+0000 (oldest deadline 2023-11-06T01:50:45.608587+0000) 2023-11-06T01:51:31.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:31.417+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:32.010388+0000 front 2023-11-06T01:50:32.010362+0000 (oldest deadline 2023-11-06T01:50:55.411043+0000) 2023-11-06T01:51:31.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:31.417+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.712169+0000 front 2023-11-06T01:50:37.712303+0000 (oldest deadline 2023-11-06T01:51:01.212113+0000) 2023-11-06T01:51:31.841 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:31.417+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:44.113237+0000 front 2023-11-06T01:50:44.113300+0000 (oldest deadline 2023-11-06T01:51:09.914004+0000) 2023-11-06T01:51:31.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:31.417+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.015407+0000 front 2023-11-06T01:50:51.015351+0000 (oldest deadline 2023-11-06T01:51:15.715130+0000) 2023-11-06T01:51:31.842 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:31 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9[144126]: 2023-11-06T01:51:31.418+0000 7f988660f700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:51:00.916965+0000 front 2023-11-06T01:51:00.916834+0000 (oldest deadline 2023-11-06T01:51:25.016861+0000) 2023-11-06T01:51:32.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:32 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:32.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:32.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:32 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:32.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:32.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:32 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:32.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:32.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:32 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:32.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:32.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:32 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:32.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:32.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:32 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:32.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:32.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:32 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:32.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2023-11-06T01:51:03.877857+0000 front 2023-11-06T01:51:03.877842+0000 (oldest deadline 2023-11-06T01:51:31.478233+0000) 2023-11-06T01:51:32.986 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:32 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:32.511+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:32.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:32 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:32.511+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:32.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:32 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:32.511+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:32.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:32 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:32.511+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:32.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:32 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:32.511+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:32.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:32 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:32.511+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:33.091 INFO:journalctl@ceph.osd.9.smithi053.stdout:Nov 06 01:51:32 smithi053 bash[179831]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-9 2023-11-06T01:51:33.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:33 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:33.096+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:33.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:33 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:33.096+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:33.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:33 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:33.096+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:33.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:33 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:33.096+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:33.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:33 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:33.096+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:33.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:33 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:33.096+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:33.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:33 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:33.096+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2023-11-06T01:51:03.877857+0000 front 2023-11-06T01:51:03.877842+0000 (oldest deadline 2023-11-06T01:51:31.478233+0000) 2023-11-06T01:51:33.986 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:33 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:33.521+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:33.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:33 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:33.521+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:33.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:33 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:33.521+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:33.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:33 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:33.521+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:33.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:33 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:33.521+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:33.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:33 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:33.521+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:34.103 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.9.service' 2023-11-06T01:51:34.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:51:34.185 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-11-06T01:51:34.186 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-11-06T01:51:34.186 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.10 2023-11-06T01:51:34.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:34.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:34.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:34.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:34.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:34.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:34.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:34.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:34.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:34.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:34.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:34.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:34.394 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:34.079+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2023-11-06T01:51:03.877857+0000 front 2023-11-06T01:51:03.877842+0000 (oldest deadline 2023-11-06T01:51:31.478233+0000) 2023-11-06T01:51:34.394 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:34 smithi062 systemd[1]: Stopping Ceph osd.10 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:51:34.892 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:34.550+0000 7f22ec23f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:51:34.892 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:34.550+0000 7f22ec23f700 -1 osd.10 87 *** Got signal Terminated *** 2023-11-06T01:51:34.892 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:34 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:34.550+0000 7f22ec23f700 -1 osd.10 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:51:34.986 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:34 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:34.500+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:34.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:34 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:34.500+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:34.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:34 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:34.500+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:34.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:34 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:34.500+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:34.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:34 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:34.500+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:34.988 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:34 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:34.500+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:35.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:35 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:35.126+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:35.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:35 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:35.126+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:35.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:35 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:35.126+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:35.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:35 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:35.126+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:35.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:35 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:35.126+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:35.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:35 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:35.126+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:35.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:35 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:35.126+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2023-11-06T01:51:03.877857+0000 front 2023-11-06T01:51:03.877842+0000 (oldest deadline 2023-11-06T01:51:31.478233+0000) 2023-11-06T01:51:35.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:35 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:35.450+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:35.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:35 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:35.450+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:35.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:35 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:35.450+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:35.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:35 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:35.450+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:35.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:35 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:35.450+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:35.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:35 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:35.450+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:36.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:36 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:36.082+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:36.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:36 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:36.082+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:36.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:36 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:36.082+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:36.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:36 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:36.082+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:36.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:36 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:36.082+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:36.394 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:36 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:36.082+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:36.394 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:36 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:36.082+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2023-11-06T01:51:03.877857+0000 front 2023-11-06T01:51:03.877842+0000 (oldest deadline 2023-11-06T01:51:31.478233+0000) 2023-11-06T01:51:36.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:36 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:36.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:36.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:36 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:36.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:36.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:36 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:36.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:36.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:36 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:36.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:36.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:36 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:36.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:36.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:36 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:36.469+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:37.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:37 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:37.088+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:37.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:37 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:37.088+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:37.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:37 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:37.088+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:37.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:37 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:37.088+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:37.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:37 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:37.088+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:37.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:37 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:37.088+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:37.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:37 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:37.088+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2023-11-06T01:51:03.877857+0000 front 2023-11-06T01:51:03.877842+0000 (oldest deadline 2023-11-06T01:51:31.478233+0000) 2023-11-06T01:51:37.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:37 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:37.437+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:37.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:37 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:37.437+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:37.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:37 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:37.437+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:37.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:37 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:37.437+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:37.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:37 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:37.437+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:37.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:37 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:37.437+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:38.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:38 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:38.123+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:38.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:38 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:38.123+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:38.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:38 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:38.123+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:38.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:38 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:38.123+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:38.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:38 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:38.123+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:38.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:38 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:38.123+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:38.393 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:38 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:38.123+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2023-11-06T01:51:03.877857+0000 front 2023-11-06T01:51:03.877842+0000 (oldest deadline 2023-11-06T01:51:31.478233+0000) 2023-11-06T01:51:38.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:38 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:38.413+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:38.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:38 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:38.413+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:38.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:38 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:38.413+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:38.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:38 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:38.413+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:38.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:38 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:38.413+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:38.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:38 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:38.413+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:39.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:39 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:39.163+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:22.569879+0000 front 2023-11-06T01:50:22.569836+0000 (oldest deadline 2023-11-06T01:50:45.469769+0000) 2023-11-06T01:51:39.642 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:39 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:39.163+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:29.972201+0000 front 2023-11-06T01:50:29.972101+0000 (oldest deadline 2023-11-06T01:50:53.472613+0000) 2023-11-06T01:51:39.643 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:39 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:39.163+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:37.474417+0000 front 2023-11-06T01:50:37.474402+0000 (oldest deadline 2023-11-06T01:51:02.174579+0000) 2023-11-06T01:51:39.643 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:39 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:39.163+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.175123+0000 front 2023-11-06T01:50:42.175139+0000 (oldest deadline 2023-11-06T01:51:11.575724+0000) 2023-11-06T01:51:39.643 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:39 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:39.163+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:51.576080+0000 front 2023-11-06T01:50:51.576100+0000 (oldest deadline 2023-11-06T01:51:16.276194+0000) 2023-11-06T01:51:39.643 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:39 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:39.163+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.777339+0000 front 2023-11-06T01:50:59.777352+0000 (oldest deadline 2023-11-06T01:51:23.877243+0000) 2023-11-06T01:51:39.643 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:39 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10[133754]: 2023-11-06T01:51:39.163+0000 7f22e6831700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2023-11-06T01:51:03.877857+0000 front 2023-11-06T01:51:03.877842+0000 (oldest deadline 2023-11-06T01:51:31.478233+0000) 2023-11-06T01:51:39.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:39 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:39.427+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:39.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:39 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:39.427+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:39.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:39 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:39.427+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:39.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:39 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:39.427+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:39.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:39 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:39.427+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:39.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:39 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:39.427+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:40.392 INFO:journalctl@ceph.osd.10.smithi062.stdout:Nov 06 01:51:40 smithi062 bash[146800]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-10 2023-11-06T01:51:40.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:40 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:40.424+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:40.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:40 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:40.424+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:40.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:40 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:40.424+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:40.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:40 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:40.424+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:40.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:40 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:40.424+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:40.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:40 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:40.424+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:40.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:40 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:40.424+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2023-11-06T01:51:16.768429+0000 front 2023-11-06T01:51:16.768445+0000 (oldest deadline 2023-11-06T01:51:39.668297+0000) 2023-11-06T01:51:41.710 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.10.service' 2023-11-06T01:51:41.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:41 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:41.400+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:41.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:41 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:41.400+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:41.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:41 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:41.401+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:41.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:41 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:41.401+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:41.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:41 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:41.401+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:41.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:41 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:41.401+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:41.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:41 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:41.401+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2023-11-06T01:51:16.768429+0000 front 2023-11-06T01:51:16.768445+0000 (oldest deadline 2023-11-06T01:51:39.668297+0000) 2023-11-06T01:51:41.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:51:41.751 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-11-06T01:51:41.752 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-11-06T01:51:41.752 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.11 2023-11-06T01:51:42.106 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:41 smithi163 systemd[1]: Stopping Ceph osd.11 for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:51:42.411 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:42 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:42.107+0000 7f1cd822a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:51:42.411 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:42 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:42.107+0000 7f1cd822a700 -1 osd.11 87 *** Got signal Terminated *** 2023-11-06T01:51:42.412 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:42 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:42.107+0000 7f1cd822a700 -1 osd.11 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-06T01:51:42.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:42 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:42.412+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:42.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:42 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:42.412+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:42.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:42 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:42.412+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:42.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:42 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:42.412+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:42.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:42 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:42.412+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:42.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:42 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:42.412+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:42.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:42 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:42.412+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2023-11-06T01:51:16.768429+0000 front 2023-11-06T01:51:16.768445+0000 (oldest deadline 2023-11-06T01:51:39.668297+0000) 2023-11-06T01:51:43.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:43 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:43.406+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:43.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:43 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:43.406+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:43.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:43 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:43.406+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:43.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:43 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:43.406+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:43.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:43 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:43.406+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:43.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:43 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:43.406+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:43.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:43 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:43.406+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2023-11-06T01:51:16.768429+0000 front 2023-11-06T01:51:16.768445+0000 (oldest deadline 2023-11-06T01:51:39.668297+0000) 2023-11-06T01:51:44.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:44 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:44.398+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:44.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:44 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:44.398+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:44.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:44 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:44.398+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:44.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:44 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:44.398+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:44.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:44 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:44.398+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:44.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:44 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:44.398+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:44.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:44 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:44.398+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2023-11-06T01:51:16.768429+0000 front 2023-11-06T01:51:16.768445+0000 (oldest deadline 2023-11-06T01:51:39.668297+0000) 2023-11-06T01:51:45.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:45 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:45.420+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:45.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:45 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:45.420+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:45.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:45 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:45.420+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:45.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:45 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:45.420+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:45.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:45 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:45.420+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:45.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:45 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:45.420+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:45.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:45 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:45.420+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2023-11-06T01:51:16.768429+0000 front 2023-11-06T01:51:16.768445+0000 (oldest deadline 2023-11-06T01:51:39.668297+0000) 2023-11-06T01:51:46.736 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:46 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:46.376+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2023-11-06T01:50:24.160769+0000 front 2023-11-06T01:50:24.160723+0000 (oldest deadline 2023-11-06T01:50:46.460797+0000) 2023-11-06T01:51:46.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:46 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:46.376+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2023-11-06T01:50:31.161971+0000 front 2023-11-06T01:50:31.162049+0000 (oldest deadline 2023-11-06T01:50:54.662027+0000) 2023-11-06T01:51:46.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:46 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:46.376+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2023-11-06T01:50:39.362805+0000 front 2023-11-06T01:50:39.362840+0000 (oldest deadline 2023-11-06T01:51:01.063099+0000) 2023-11-06T01:51:46.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:46 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:46.376+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2023-11-06T01:50:42.164031+0000 front 2023-11-06T01:50:42.164055+0000 (oldest deadline 2023-11-06T01:51:08.064110+0000) 2023-11-06T01:51:46.737 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:46 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:46.376+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2023-11-06T01:50:53.865911+0000 front 2023-11-06T01:50:53.865967+0000 (oldest deadline 2023-11-06T01:51:19.765936+0000) 2023-11-06T01:51:46.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:46 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:46.376+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2023-11-06T01:50:59.766291+0000 front 2023-11-06T01:50:59.766328+0000 (oldest deadline 2023-11-06T01:51:25.666399+0000) 2023-11-06T01:51:46.738 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:46 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11[134158]: 2023-11-06T01:51:46.376+0000 7f1cd301d700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2023-11-06T01:51:16.768429+0000 front 2023-11-06T01:51:16.768445+0000 (oldest deadline 2023-11-06T01:51:39.668297+0000) 2023-11-06T01:51:47.986 INFO:journalctl@ceph.osd.11.smithi163.stdout:Nov 06 01:51:47 smithi163 bash[147914]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-osd-11 2023-11-06T01:51:49.152 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@osd.11.service' 2023-11-06T01:51:49.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:51:49.193 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-11-06T01:51:49.194 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-11-06T01:51:49.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.a 2023-11-06T01:51:49.548 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:51:49 smithi053 systemd[1]: Stopping Ceph mds.a for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:51:49.841 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:51:49 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-a[152599]: 2023-11-06T01:51:49.550+0000 7f07a90a5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:51:49.841 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:51:49 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-a[152599]: 2023-11-06T01:51:49.550+0000 7f07a90a5700 -1 mds.a *** got signal Terminated *** 2023-11-06T01:51:54.090 INFO:journalctl@ceph.mds.a.smithi053.stdout:Nov 06 01:51:53 smithi053 bash[180105]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-a 2023-11-06T01:51:54.123 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.a.service' 2023-11-06T01:51:54.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:51:54.163 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-11-06T01:51:54.163 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-11-06T01:51:54.164 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.d 2023-11-06T01:51:54.527 INFO:journalctl@ceph.mds.d.smithi053.stdout:Nov 06 01:51:54 smithi053 systemd[1]: Stopping Ceph mds.d for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:51:54.840 INFO:journalctl@ceph.mds.d.smithi053.stdout:Nov 06 01:51:54 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-d[151623]: 2023-11-06T01:51:54.528+0000 7f0b2eeed700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:51:54.841 INFO:journalctl@ceph.mds.d.smithi053.stdout:Nov 06 01:51:54 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-d[151623]: 2023-11-06T01:51:54.528+0000 7f0b2eeed700 -1 mds.d *** got signal Terminated *** 2023-11-06T01:51:59.048 INFO:journalctl@ceph.mds.d.smithi053.stdout:Nov 06 01:51:58 smithi053 bash[180250]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-d 2023-11-06T01:51:59.054 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.d.service' 2023-11-06T01:51:59.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:51:59.090 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-11-06T01:51:59.090 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-11-06T01:51:59.091 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.g 2023-11-06T01:51:59.341 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:51:59 smithi053 systemd[1]: Stopping Ceph mds.g for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:51:59.841 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:51:59 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-g[152212]: 2023-11-06T01:51:59.485+0000 7fb14aad6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:51:59.841 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:51:59 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-g[152212]: 2023-11-06T01:51:59.485+0000 7fb14aad6700 -1 mds.g *** got signal Terminated *** 2023-11-06T01:52:02.039 INFO:journalctl@ceph.mds.g.smithi053.stdout:Nov 06 01:52:01 smithi053 bash[180394]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-g 2023-11-06T01:52:02.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.g.service' 2023-11-06T01:52:02.080 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:52:02.080 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-11-06T01:52:02.081 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-11-06T01:52:02.081 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.j 2023-11-06T01:52:02.341 INFO:journalctl@ceph.mds.j.smithi053.stdout:Nov 06 01:52:02 smithi053 systemd[1]: Stopping Ceph mds.j for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:52:02.841 INFO:journalctl@ceph.mds.j.smithi053.stdout:Nov 06 01:52:02 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-j[151258]: 2023-11-06T01:52:02.444+0000 7f41fdf04700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:52:02.841 INFO:journalctl@ceph.mds.j.smithi053.stdout:Nov 06 01:52:02 smithi053 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-j[151258]: 2023-11-06T01:52:02.444+0000 7f41fdf04700 -1 mds.j *** got signal Terminated *** 2023-11-06T01:52:05.590 INFO:journalctl@ceph.mds.j.smithi053.stdout:Nov 06 01:52:05 smithi053 bash[180543]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-j 2023-11-06T01:52:05.645 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.j.service' 2023-11-06T01:52:05.680 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:52:05.681 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-11-06T01:52:05.681 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-11-06T01:52:05.682 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.b 2023-11-06T01:52:06.054 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:52:05 smithi062 systemd[1]: Stopping Ceph mds.b for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:52:06.391 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:52:06 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-b[140013]: 2023-11-06T01:52:06.055+0000 7f2b1cb14700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:52:06.392 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:52:06 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-b[140013]: 2023-11-06T01:52:06.055+0000 7f2b1cb14700 -1 mds.b *** got signal Terminated *** 2023-11-06T01:52:09.919 INFO:journalctl@ceph.mds.b.smithi062.stdout:Nov 06 01:52:09 smithi062 bash[147062]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-b 2023-11-06T01:52:10.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.b.service' 2023-11-06T01:52:10.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:52:10.151 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-11-06T01:52:10.152 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-11-06T01:52:10.152 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.e 2023-11-06T01:52:10.391 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:52:10 smithi062 systemd[1]: Stopping Ceph mds.e for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:52:10.892 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:52:10 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-e[139323]: 2023-11-06T01:52:10.536+0000 7f5e2a973700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:52:10.892 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:52:10 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-e[139323]: 2023-11-06T01:52:10.536+0000 7f5e2a973700 -1 mds.e *** got signal Terminated *** 2023-11-06T01:52:14.142 INFO:journalctl@ceph.mds.e.smithi062.stdout:Nov 06 01:52:13 smithi062 bash[147205]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-e 2023-11-06T01:52:14.342 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.e.service' 2023-11-06T01:52:14.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:52:14.378 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-11-06T01:52:14.378 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-11-06T01:52:14.378 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.h 2023-11-06T01:52:14.642 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:52:14 smithi062 systemd[1]: Stopping Ceph mds.h for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:52:15.142 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:52:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-h[139660]: 2023-11-06T01:52:14.763+0000 7ff3551e8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:52:15.142 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:52:14 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-h[139660]: 2023-11-06T01:52:14.763+0000 7ff3551e8700 -1 mds.h *** got signal Terminated *** 2023-11-06T01:52:17.929 INFO:journalctl@ceph.mds.h.smithi062.stdout:Nov 06 01:52:17 smithi062 bash[147361]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-h 2023-11-06T01:52:18.128 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.h.service' 2023-11-06T01:52:18.165 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:52:18.166 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-11-06T01:52:18.166 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-11-06T01:52:18.167 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.k 2023-11-06T01:52:18.391 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:52:18 smithi062 systemd[1]: Stopping Ceph mds.k for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:52:18.891 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:52:18 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-k[138982]: 2023-11-06T01:52:18.561+0000 7f1a0226a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:52:18.892 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:52:18 smithi062 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-k[138982]: 2023-11-06T01:52:18.561+0000 7f1a0226a700 -1 mds.k *** got signal Terminated *** 2023-11-06T01:52:21.929 INFO:journalctl@ceph.mds.k.smithi062.stdout:Nov 06 01:52:21 smithi062 bash[147501]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-k 2023-11-06T01:52:22.140 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.k.service' 2023-11-06T01:52:22.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:52:22.177 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-11-06T01:52:22.178 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-11-06T01:52:22.178 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.c 2023-11-06T01:52:22.487 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:52:22 smithi163 systemd[1]: Stopping Ceph mds.c for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:52:22.986 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:52:22 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-c[140394]: 2023-11-06T01:52:22.550+0000 7f49e945a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:52:22.986 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:52:22 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-c[140394]: 2023-11-06T01:52:22.550+0000 7f49e945a700 -1 mds.c *** got signal Terminated *** 2023-11-06T01:52:25.907 INFO:journalctl@ceph.mds.c.smithi163.stdout:Nov 06 01:52:25 smithi163 bash[148193]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-c 2023-11-06T01:52:26.096 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.c.service' 2023-11-06T01:52:26.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:52:26.134 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-11-06T01:52:26.134 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-11-06T01:52:26.134 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.f 2023-11-06T01:52:26.486 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:52:26 smithi163 systemd[1]: Stopping Ceph mds.f for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:52:26.986 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:52:26 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-f[140054]: 2023-11-06T01:52:26.527+0000 7fd337fe6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:52:26.987 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:52:26 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-f[140054]: 2023-11-06T01:52:26.527+0000 7fd337fe6700 -1 mds.f *** got signal Terminated *** 2023-11-06T01:52:30.181 INFO:journalctl@ceph.mds.f.smithi163.stdout:Nov 06 01:52:29 smithi163 bash[148335]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-f 2023-11-06T01:52:30.186 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.f.service' 2023-11-06T01:52:30.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:52:30.220 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-11-06T01:52:30.221 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-11-06T01:52:30.221 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.i 2023-11-06T01:52:30.486 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:52:30 smithi163 systemd[1]: Stopping Ceph mds.i for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:52:30.986 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:52:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-i[140736]: 2023-11-06T01:52:30.597+0000 7fad60084700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:52:30.986 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:52:30 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-i[140736]: 2023-11-06T01:52:30.597+0000 7fad60084700 -1 mds.i *** got signal Terminated *** 2023-11-06T01:52:34.094 INFO:journalctl@ceph.mds.i.smithi163.stdout:Nov 06 01:52:33 smithi163 bash[148475]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-i 2023-11-06T01:52:34.100 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.i.service' 2023-11-06T01:52:34.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:52:34.134 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-11-06T01:52:34.134 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-11-06T01:52:34.135 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.l 2023-11-06T01:52:34.486 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:52:34 smithi163 systemd[1]: Stopping Ceph mds.l for 3f13dbd6-7c43-11ee-8db9-212e2dc638e7... 2023-11-06T01:52:34.986 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:52:34 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-l[139705]: 2023-11-06T01:52:34.509+0000 7feb699d6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-06T01:52:34.986 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:52:34 smithi163 ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-l[139705]: 2023-11-06T01:52:34.509+0000 7feb699d6700 -1 mds.l *** got signal Terminated *** 2023-11-06T01:52:38.043 INFO:journalctl@ceph.mds.l.smithi163.stdout:Nov 06 01:52:37 smithi163 bash[148624]: ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7-mds-l 2023-11-06T01:52:38.155 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f13dbd6-7c43-11ee-8db9-212e2dc638e7@mds.l.service' 2023-11-06T01:52:38.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-06T01:52:38.189 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-11-06T01:52:38.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 --force --keep-logs 2023-11-06T01:52:38.498 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:52:40.136 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 --force --keep-logs 2023-11-06T01:52:40.465 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:52:42.164 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 --force --keep-logs 2023-11-06T01:52:42.480 INFO:teuthology.orchestra.run.smithi163.stdout:Deleting cluster with fsid: 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T01:52:44.094 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-06T01:52:44.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-06T01:52:44.170 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-06T01:52:44.202 INFO:tasks.cephadm:Archiving crash dumps... 2023-11-06T01:52:44.205 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641/remote/smithi053/crash 2023-11-06T01:52:44.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/crash -- . 2023-11-06T01:52:44.259 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-06T01:52:44.259 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2023-11-06T01:52:44.261 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641/remote/smithi062/crash 2023-11-06T01:52:44.262 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/crash -- . 2023-11-06T01:52:44.308 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-06T01:52:44.308 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2023-11-06T01:52:44.310 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641/remote/smithi163/crash 2023-11-06T01:52:44.311 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/crash -- . 2023-11-06T01:52:44.360 INFO:teuthology.orchestra.run.smithi163.stderr:tar: /var/lib/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-06T01:52:44.361 INFO:teuthology.orchestra.run.smithi163.stderr:tar: Error is not recoverable: exiting now 2023-11-06T01:52:44.362 INFO:tasks.cephadm:Checking cluster log for badness... 2023-11-06T01:52:44.362 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-11-06T01:52:44.400 INFO:teuthology.orchestra.run.smithi053.stderr:grep: /var/log/ceph/3f13dbd6-7c43-11ee-8db9-212e2dc638e7/ceph.log: No such file or directory 2023-11-06T01:52:44.402 INFO:tasks.cephadm:Compressing logs... 2023-11-06T01:52:44.403 DEBUG:teuthology.orchestra.run.smithi053:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T01:52:44.447 DEBUG:teuthology.orchestra.run.smithi062:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T01:52:44.449 DEBUG:teuthology.orchestra.run.smithi163:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T01:52:44.475 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-06T01:52:44.491 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-06T01:52:44.493 INFO:teuthology.orchestra.run.smithi163.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-06T01:58:37.659 INFO:tasks.cephadm:Archiving logs... 2023-11-06T01:58:37.660 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641/remote/smithi053/log 2023-11-06T01:58:37.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-06T01:59:11.896 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641/remote/smithi062/log 2023-11-06T01:59:11.897 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-06T01:59:49.877 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641/remote/smithi163/log 2023-11-06T01:59:49.879 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-06T02:00:25.740 INFO:tasks.cephadm:Removing cluster... 2023-11-06T02:00:25.740 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 --force 2023-11-06T02:00:26.056 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T02:00:26.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 --force 2023-11-06T02:00:27.128 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T02:00:27.921 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 --force 2023-11-06T02:00:28.225 INFO:teuthology.orchestra.run.smithi163.stdout:Deleting cluster with fsid: 3f13dbd6-7c43-11ee-8db9-212e2dc638e7 2023-11-06T02:00:29.009 INFO:tasks.cephadm:Removing cephadm ... 2023-11-06T02:00:29.009 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-06T02:00:29.032 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-06T02:00:29.063 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-06T02:00:29.080 INFO:tasks.cephadm:Teardown complete 2023-11-06T02:00:29.081 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-06T02:00:29.092 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-06T02:00:29.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-06T02:00:29.095 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-06T02:00:29.105 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-06T02:00:29.241 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-06T02:00:29.252 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-06T02:00:29.253 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T02:00:29.256 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T02:00:29.259 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T02:00:29.273 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2023-11-06T02:00:29.274 INFO:teuthology.orchestra.run.smithi163.stderr:bash: ntpq: command not found 2023-11-06T02:00:29.275 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2023-11-06T02:00:29.279 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-06T02:00:29.279 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2023-11-06T02:00:29.279 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +846us[ +846us] +/- 78ms 2023-11-06T02:00:29.279 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv02.front.sepia.ceph.com 3 6 377 16 -391us[ -389us] +/- 79ms 2023-11-06T02:00:29.280 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 -557us[ -555us] +/- 90ms 2023-11-06T02:00:29.280 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T02:00:29.293 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-06T02:00:29.294 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2023-11-06T02:00:29.294 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +895us[ +895us] +/- 78ms 2023-11-06T02:00:29.294 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -333us[ -408us] +/- 48ms 2023-11-06T02:00:29.294 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -497us[ -497us] +/- 90ms 2023-11-06T02:00:29.295 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T02:00:29.322 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-06T02:00:29.322 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2023-11-06T02:00:29.322 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +903us[ +903us] +/- 78ms 2023-11-06T02:00:29.322 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -330us[ -387us] +/- 48ms 2023-11-06T02:00:29.323 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -500us[ -558us] +/- 90ms 2023-11-06T02:00:29.323 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T02:00:29.324 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-06T02:00:29.339 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-06T02:00:29.339 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-06T02:00:29.366 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-06T02:00:29.479 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-06T02:00:29.621 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-06T02:00:29.745 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-06T02:00:29.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:00:29.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-06T02:00:29.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:00:29.821 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-06T02:00:29.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:00:29.854 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-06T02:00:29.866 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-06T02:00:29.909 INFO:teuthology.task.internal:Duration was 2916.397389 seconds 2023-11-06T02:00:29.910 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-06T02:00:29.945 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-06T02:00:29.946 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-06T02:00:29.949 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-06T02:00:29.951 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-06T02:00:30.000 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-06T02:00:30.040 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-06T02:00:30.045 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-06T02:00:30.281 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-06T02:00:30.281 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2023-11-06T02:00:30.281 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-06T02:00:30.310 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2023-11-06T02:00:30.311 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-06T02:00:30.341 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2023-11-06T02:00:30.342 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-06T02:00:30.375 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-06T02:00:30.375 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T02:00:30.378 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T02:00:30.384 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T02:00:30.471 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-06T02:00:30.471 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-06T02:00:30.474 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-06T02:00:30.477 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-06T02:00:30.619 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-06T02:00:30.631 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-06T02:00:30.632 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-06T02:00:30.659 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-06T02:00:30.693 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-06T02:00:30.761 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-06T02:00:30.773 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-06T02:00:30.776 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-06T02:00:30.778 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-06T02:00:30.800 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2023-11-06T02:00:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2023-11-06T02:00:30.814 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2023-11-06T02:00:30.854 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-06T02:00:30.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:00:30.873 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-06T02:00:30.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:00:30.911 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-06T02:00:30.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:00:30.928 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-06T02:00:30.940 INFO:teuthology.task.internal:Transferring archived files... 2023-11-06T02:00:30.941 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641/remote/smithi053 2023-11-06T02:00:30.941 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-06T02:00:30.986 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641/remote/smithi062 2023-11-06T02:00:30.986 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-06T02:00:31.029 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641/remote/smithi163 2023-11-06T02:00:31.029 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-06T02:00:31.070 INFO:teuthology.task.internal:Removing archive directory... 2023-11-06T02:00:31.071 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-06T02:00:31.073 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-06T02:00:31.076 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-06T02:00:31.120 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-06T02:00:31.133 INFO:teuthology.task.internal:Not uploading archives. 2023-11-06T02:00:31.134 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-06T02:00:31.159 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-06T02:00:31.159 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-06T02:00:31.162 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-06T02:00:31.165 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-06T02:00:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 6 02:00 /home/ubuntu/cephtest 2023-11-06T02:00:31.179 INFO:teuthology.orchestra.run.smithi062.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 6 02:00 /home/ubuntu/cephtest 2023-11-06T02:00:31.181 INFO:teuthology.orchestra.run.smithi163.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 6 02:00 /home/ubuntu/cephtest 2023-11-06T02:00:31.182 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-06T02:00:31.193 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-06T02:00:31.259 INFO:teuthology.nuke:Checking targets against current locks 2023-11-06T02:00:31.281 DEBUG:teuthology.nuke:shortname: smithi053 2023-11-06T02:00:31.282 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T02:00:31.304 DEBUG:teuthology.nuke:shortname: smithi062 2023-11-06T02:00:31.305 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T02:00:31.310 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 01:05:43.993804', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-06T02:00:31.331 DEBUG:teuthology.nuke:shortname: smithi163 2023-11-06T02:00:31.331 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T02:00:31.335 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 01:05:43.995175', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-06T02:00:31.370 INFO:teuthology.orchestra.console.smithi053:Power off 2023-11-06T02:00:31.371 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-06T02:00:31.380 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 01:05:43.996383', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-06T02:00:31.394 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2023-11-06T02:00:31.394 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:00:31.410 INFO:teuthology.orchestra.console.smithi062:Power off 2023-11-06T02:00:31.411 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-06T02:00:31.424 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-11-06T02:00:31.442 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2023-11-06T02:00:31.442 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:00:31.456 INFO:teuthology.orchestra.console.smithi163:Power off 2023-11-06T02:00:31.457 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-06T02:00:31.467 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-11-06T02:00:31.479 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2023-11-06T02:00:31.480 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:00:31.501 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2023-11-06T02:00:35.425 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:00:35.469 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:00:35.502 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:00:35.535 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-11-06T02:00:35.578 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-11-06T02:00:35.613 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2023-11-06T02:00:39.541 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:00:39.580 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:00:39.615 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:00:39.650 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-11-06T02:00:39.689 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-11-06T02:00:39.726 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2023-11-06T02:00:39.751 INFO:teuthology.orchestra.console.smithi053:Power off completed 2023-11-06T02:00:39.827 INFO:teuthology.orchestra.console.smithi163:Power off completed 2023-11-06T02:00:43.690 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:00:43.801 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2023-11-06T02:00:43.902 INFO:teuthology.orchestra.console.smithi062:Power off completed 2023-11-06T02:00:44.007 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} duration: 2916.3973891735077 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2023-11-06T02:00:44.007 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-06T02:00:44.095 INFO:teuthology.run:FAIL